Tuesday 6 December 2016

Getting Credentials into Jenkins scripts

So you need a password or key to be hidden but used inside a Jenkins script.

The Credentials Binding Plugin is what you need to use. This can inject these as environmental variables in your Jenkins job scripts:


This way you can then use them later in a script to clone a repository without giving out your password to a Jenkins slave, like so:


git clone https://$SECRET@bitbucket/scm/project/repository.git

This works with files for ssh keys as well, so you can now script your password and keys into your scripts. A service account with access only to what it needs to would be the best way to support this and you have those access credentials as safe as your jenkins master is.,



Friday 25 November 2016

Triggering bamboo, with only specific file change in source control

Another small issue while working on bamboo today, I wanted to only trigger a bamboo plan from a specific subset of a SVN or GIT repository, so we can only trigger full test builds when we change real code and not on documentation updates.

So you think this would be easy right ? Not exactly.

There is a menu option in Bamboo under the repositories menu as follows :



So input a small change like the above for change just on XML files. Change the files in the test repository and boom, nothing.

So it turns out after some digging that Bamboo only matches on the full file path, some examples are here


After knowing this it becomes a little easier to get right, but it's not obviously clear and as a user there is no trigger log immediately available it's not clear how to proceed, hopefully this clears things up.

Tuesday 27 September 2016

Silent Install Pycharm with shortcut


So I was having an Issue where an installer was not creating a shortcut on the desktop, turns out it was a installer issue but due to the fact we SCCM to deploy we can run a simple powershell script to solve this issue.

Below is the script, that needs to be run as admin.


$InstallPath = "${env:ProgramFiles(x86)}\JetBrains\PyCharm Community Edition 5.0.4\"
$InstallPath
.\pycharm-community-5.0.4.exe /S /D=$InstallPath
# Create a Shortcut with Windows PowerShell
$TargetFile = "$InstallPath\bin\pycharm.exe"
$ShortcutFile = "$env:Public\Desktop\Pycharm.lnk"
$WScriptShell = New-Object -ComObject WScript.Shell
$Shortcut = $WScriptShell.CreateShortcut($ShortcutFile)
$Shortcut.TargetPath = $TargetFile
$Shortcut.Save()


UPDATE 10/07/2017:

I have found the real reason why this is required, when using silent install the installer will always default to non-admin and as such does not do the following:

-          No shortcut gets created in the Public\Desktop
-          No Start Menu entries are created
-          The software is not listed in the “Programs and Features” list in Control Panel
-          No registry key is added to the HKLM:\..\Uninstall branch
-          The uninstall registry key is created in the HKCU branch

this is tracked here:
IDEA-146215

Tuesday 12 April 2016

SSH Keys and Windows, basic no passphrase

For most people GIT access is controlled via username and password, this is fine when working only a small project, when rarely doing push and pull from your remote repos online. However the next step is to use an SSH key.

This mean that you share some information between your PC and the remote server, these are called keys, for the uninitiated. Using the Git for Windows installer and OpenSSH you can generate and configure an SSH key for the connection.

Disclaimer - This is not the most secure method but is meant to be a stepping stone to passphrase based SSH with an agent.

To create an SSHKey look to use the ssh-keygen application that comes with GIT for windows.
The SSH-keygen application is normally stored in the C:\Program Files\Git\usr\bin\ and a key can be generated with the following command
"ssh-keygen" -t rsa -f bitbucket_rsa -N ""
Details:
-t type RSA
-f filename butbucket_rsa
-N passphrase

The command above creates a key pair in the current directory called bitbucket_rsa and bitbucket_rsa .pub with no passphrase. You will need to copy the bibucket_ras to the ~.ssh folder in your user directory.
To make the PC use the private key (bitbucket_rsa) when making the connection, you can use the config file in the ~/.ssh directory. This file is formatted in the following way.
host bitbucket.blah.com
IdentityFile ~/.ssh/bitbucket_rsa
Details
host then the URL of the server that you are making the connection to.
Identify file this is the location of the private key file.

To add a ppublic key into your bitbucket account you need to do the following:
  • Sign in to bitbucket
  • click the top right hand profile button
  • click the manage account button
  • click SSH keys
  • click on the add key button
  • copy and paste the contents of the bitbucket_rsa.pub into the text box and then click add key.
Your key is now on the server.

When you make the first connection over command line to the server, you will be asked if you accept the host key the server and a finger print will be shown much like below:



[bitbucket.com]:3456,[0.0.0.123]:9090 ssh-rsa RRRB3NzaC1yc2EAAAADAQABAAABAQCch1FyAnxrvKPw7AkhVyfR9FOJNVzIAUfgxlhHDR7rafYgmJkSwUCZtsYKRv6wNUEeRHh5EU/yYr/2zn0+5l2A/AG6X95SwrBBSzpeelFo2hoIjIZmnHv93k5VK0OXfpAfKqBRDMY5s1tVcdyKa6qCirHydyYbViM3Ob3mBhrhY/w87jXZqaMLQlNUbMfH6iGdaVBNeqgqSPeclgHge7kwSoIukMY0pqiQAinS0bb3aPzl4dwNCJBrghV1eYKGHL7/ESX+Mj8PrsJDzWv1IxG6Ey5jjcY+zw/386sKY2QbmB55m91lg50LfEZP/1OorbLBxDqThAW5maU7J27knGaR
Once verified this will be contained in a knownhosts file in the .ssh directory, the same location as the config file and the private key.

Now you should have a very simple SSH connection available.

Wednesday 24 February 2016

Atlassian SourceTree 1.8

So today I received an e-mail about the new SourceTree release from Atlassian, since I really like Atlassian products (can you probably tell from other posts on Bamboo) and have been using sourcetree for a while, I was excited to check out the improvements.



What I got was a a complete UI change that has ruined the experience of SourceTree completely, a few of the notes that I and many other haven't liked about it.

Icons
Flat, bland, and near identical icons, means you need to read the text rather than looking at the icon, which I think is the point of an icon, to be visual.


Read Pane

Why oh why is the read pane done in this way, the text is centred. With a white background.
This just is crazy and looks very weird, not sure why this has been done this way.



There is a lot of vocal dislike of this, present here


Monday 11 January 2016

Bamboo and Squish session 1 vs session 0

Bamboo and Squish 
using different sessions

I am looking at integrating squish and bamboo for GUI testing using QT. Squish is a good choice of tool for this as the program is written in python and Squish can run scripts in python and so a good deal of knowledge can be carried over. However these are some caveats to this approach, the main one I shall address today is that the service and the user GUI run under different sessions and are therefore hard to integrate.

Windows uses different sessions for services, as Bamboo only runs as a service, it runs in session 0, so if a application is started in that session it will not have a GUI. This creates an issue for GUI testing. Squish helpfully gives an API that the squishrunner can use to solve this:

 startapplication("AUT")  

This tells the squish server to start the configured AUT, this will be created in the same session as the squish server which needs to be a user session so that a GUI is created. As you can see that's an other caveat of the process, in that the server needs to be in a user session and so it cannot be started by the bamboo service. Therefore we need to use the auto log-on feature of windows documentation on the windows site here which are summarised below:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon]
"AutoAdminLogon"="1"
"AutoLogonCount"=dword:99999999
"DefaultDomainName"="domain.com"
"DefaultPassword"="blahblah"
"DefaultUserName"="squish"

Once logged on we can use a startup script (simple bat file in the startup folder) to run the squishserver and we have a squish server running in a user session that can talk to a squishrunner running in session 0 with the services.

There is another issue with

 startapplication("AUT")  

It starts the application from scratch, for an application with a large startup time this can be quite annoying and time consuming for each test that you need to run a better way is using the startaut application to start the application in an "attachable state" from the first instance. Therefore I wrote a little wrapper for this application so that the AUT is started from the python script in an attachable way. To start it in the user context I compiled the python using py2exe and pass it the attachable parameters...

 """  
   This will create the start aut wrapper for squish and bamboo  
   using py2exe it will start the AUT on the port given.  
 """  
 from __future__ import print_function  
 __author__ = 'jdengel'  
 #######################################  
 # main program starts here  
 import os  
 import sys  
 import subprocess  
 import time  
 def is_number(string_to_test):  
   """  
   :param string_to_test: checks string is a int  
   :return: BOOL  
   """  
   try:  
     int(string_to_test)  
     return True  
   except ValueError:  
     return False  
 def validate_args():  
   """  
   validate the arguments  
   1. number port  
   2. AUT to start  
   3. location of the startaut from squish   
   :return:  
   """  
   if len(sys.argv) != 4:  
     print("incorrect number of paramter given\n"  
        "expected:\n"  
        "port, app, location of startaut")  
     return 2  
   if is_number(sys.argv[1]) is False:  
     print("Port is not a number {0}".format(sys.argv[1]))  
     return 3  
   if os.path.isfile(sys.argv[2]) is False:  
     print("file to start does not exist, this is an issue {0}".format(sys.argv[2]))  
     return 4  
   if os.path.isfile(sys.argv[3]) is False:  
     print("start aut application does not exist {0}".format(sys.argv[3]))  
     return 5  
   return 0  
 RET_VALUE = validate_args()  
 if RET_VALUE is not 0:  
   sys.exit(RET_VALUE)  
 # open the AUT on the right port  
 try:  
   subprocess.Popen([sys.argv[3], '--port={}'.format(sys.argv[1]), sys.argv[2]])  
   print("Starting application to test on port {0}", sys.argv[1])  
 except SystemError, exp:  
   print("Fatal error - {0}".format(exp.message))  
   sys.exit(8)  
 print("finishing START_AU application")  

This allows the squish scripts to start the application in an attachable state and therefore streamline the process of testing since an application might take a long time to startup.

The only thing to be careful of it that you reset the application back to a known state after the test has finished. This can be done using the init and cleanup functions in the squish python script.

So now you can streamline your testing.

Wednesday 6 January 2016

Restart Windows machine remotely.

Let's face it sometimes you have to restart a machine.

But it's a VM, or you remote into the machine so you don't have access to the machine itself.

You need a way to restart a machine remotely and easily without having to remote in each time you want to do it.

You need the shutdown command that's in the command prompt. It takes the following arguments.

ParameterFunction
-sShuts down the computer.
-rReboots the computer.
-fForce-close all running applications.
-m \\ComputerSpecifies a specific computer on the network that you want to shut down or reboot. Absent this parameter, the command affects only the local PC on which it is run.
-t xxSets a delay (xx seconds) before the specified operation commences.
-c "message"Add your own text inside the quotations to provide remote users with a message about why and when their PC will shut down or reboot.
/aAborts a shutdown or restart if used during the (-t) delay period.
/hHibernates the computer.
/?Displays the full help document with all commands.

Now imagine that you need to restart a bamboo agent cluster as you updated the server and it requires a restart of the agents.

No problem as you can whip up a bat file to go through the cluster restarting the agents. 

shutdown -f-r -m \\computer1
shutdown -f -r -m \\computer2

Now you have a simple script to restart a bank of computers without any trouble, no logging in or remotely accessing the machines.