You have just completed an awesomely complex change to your shinny new webapp! After running all your unit tests things are in the green and looking clean.
Very satisfied at the quality of your work you fire up the application to verify that everything is still working as advertised. Below is what greets you on the root path of your app
We have all been here at some time or another! What happened! Perhaps it was not your code that broke it! Maybe the issue originated from another part of your organisation, or maybe it came from somewhere on the "inter-webs".
Its time to look at the underlying problem however ..... testing web user interfaces is hard! Its time consuming and difficult to get right. Manual clicks, much typing, cross referencing client specifications etc, surely there must be an easier way. At the end of the day we DO need to test our user interfaces!>
Automated Web UI Testing
Thankfully UI testing today can be Automated, running real browsers in real end to end functional tests, to ensure our results meet (and continue to meet) expectations.
For the sake of brevity and clarity in this demonstration we will focus on testing an existing endpoint. It is considered common place to find functional tests included as part of a wider build pipeline, which may consist of such steps as:
- Build
- Unit Test
- Deploy to Test Environment
- Perform Functional Tests
- Deploy to Production
In this article we will be focusing on the functional testing component of this pipeline. We will proceed on the assumption that your code has already been, built unit tested and deployed to a Functional Test environment. Today we will :
- Add Automated UI testing to an existing endpoint google.com
- Configure ContinuaCI to automatically build our project, and perform the tests
Software Requirements:
Step1: Prepare a Selenium endpoint
Firstly we will prepare for our UI tests by setting up a Selenium server. Selenium is a browser automation framework which will be used to 'remote control' a real browser. This machine will be designated for performing the UI tests against, preferably this will be a machine separate from your ContinuaCI server.
Log into the machine you have chosen for the Selenium server with administrator privileges
Download and install Mozilla Firefox (getfirefox.com), this will be the browser that we target as part of this example, however Selenium can target lots of other browsers. For a full breakdown please see the docs page: .
Download Selenium Server (docs.seleniumhq.org/download), at the time of writing the latest version is 2.41.0.
Place it into a permanent location of you choosing, in our example ("C:\Program Files (x86)\SeleniumServer")
Download NSSM (nssm.cc/download), unzip it and place into a permanent location of you choosing "C:\Program Files (x86)\nssm-2.22\"
Ensure that port 4444 is set to allow traffic (this is the default communicationsport for Selenium)
Open a console and run the following commands:
"C:\Program Files (x86)\nssm-2.22\win64\nssm.exe" install Selenium-Server "java" "-jar \"C:\Program Files (x86)\SeleniumServer\selenium-server-standalone-2.41.0.jar\""
net start Selenium-Server
In order to uninstall the Selenium server service, the following commands can be run:
net stop Selenium-Server
"C:\Program Files (x86)\nssm-2.22\win64\nssm.exe" remove Selenium-Server
Step2: Create a test class and add it to source control
Create a new class library project in Visual Studio, lets call it 'tests'
Open the Nuget Package Manager Console window (tools menu-> library package manager -> package manager console), select the test project as the default project and run the following script:
Install-Package Selenium.Automation.Framework
Install-Package Selenium.WebDriver
Install-Package Selenium.Support
Install-Package NUnit
Install-Package NUnit.Runners
Create a new class called within the tests project (lets call it tests) and place the below code (Note: line 23 should be changed with location to the Selenium-Server we setup in the previous step):
using System;
using System.Text;
using NUnit.Framework;
using OpenQA.Selenium.Firefox;
using OpenQA.Selenium;
using OpenQA.Selenium.Remote;
using OpenQA.Selenium.Support.UI;
namespace SeleniumTests
{
[TestFixture]
public class test
{
private RemoteWebDriver driver;
[SetUp]
public void SetupTest()
{
// Look for an environment variable
string server = null;
server = System.Environment.GetEnvironmentVariable("SELENIUM_SERVER");
if (server == null)
{
server = "https:// *** PUT THE NAME OF YOUR SERVER HERE ***:4444/wd/hub";
}
// Remote testing
driver = new RemoteWebDriver(new Uri(server), DesiredCapabilities.Firefox());
}
[TearDown]
public void TeardownTest()
{
try
{
driver.Quit();
}
catch (Exception)
{
// Ignore errors if unable to close the browser
}
}
[Test]
public void FirstSeleniumTest()
{
driver.Navigate().GoToUrl("https://www.google.com/");
IWebElement query = driver.FindElement(By.Name("q"));
query.SendKeys("a test");
Assert.AreEqual(driver.Title, "Google");
}
}
}
Step3: Test the test!
Build the solution
Open NUnit build runner (by default this is located at ~\packages\NUnit.Runners.2.6.3\tools\nunit.exe) , Select file -> Open Project, and locate the tests dll that you have build in the previous step
click the run button
~ 15 seconds or so you should have one green test!
So what just happened? Behind the scenes an instance of firefox was opened (on the Selenium Server), perform a simple google search query and undertook a simple Nunit assertion has verified the name of the window was equal to "Google", very cool!.
Now lets make the test fail, go ahead and change line 78, lets say "zzGoogle", build, and rerun the test. We now have a failing test. Go ahead and change it back so that we have a single passing test.
Create a source control repository
In this example, we're using mercurial
open a command prompt at ~\
type "hg init"
add a .hgignore file into the directory. Forconveniencewe have prepared one for youhere
type "hg add"
type "hg commit -m "initial commit""
Step 4: Setting up Automated UI testing in ContinuaCI
Navigate to the ContinuaCI web interface
Create a project
Open ContinuaCI
Select "Create Project" from the top tasks dropdown menu
Name the project something memerable; In our case: "pete sel test 1"
Click the "Save & Complete Wizard" button
Create a configuration for this project
Click "Create a Configuration"
Name the config something memorable; in our case "sel-testconfig-1"
Click save & Continue
Click the 'Enable now' link at the bottom of the page to enable this configuration
Point to our Repository
under the section "Configuration Repositories", select the "Create" link
Name the repository "test_repo"
Select "Mercurial" from the "type" dropdown list
Select the Mercurial" tab from the top of the dialogue box
Enter the repository location under "source path" in our case '\\machinename\c$\sel-blog-final'
Click validate to ensure all is well
Click save, your repository is now ready to go!
Add actions to our build
Click on the Stages tab
We will add a nuget restore action, click on the "Nuget" section from the categories on the left
Drag and drop the action "Nuget Restore" onto the design surface
Enter the location of the solution file: "$Source.test_repo$\tests.sln"
Click Save
Build our tests
Click on the "Build runners" category from the categories on the left hand menu
Drag and drop a Visual Studio action onto the design surface (note that the same outcome can be achieved here with an MSBuild action).
Enter the name of the solution file: "$Source.test_repo$\tests.sln"
Specify that this should be a 'Release' build under the configuration option
Click save
Setup ContinuaCI to run our Nunit tests
Select the 'unit testing' category from the left hand menu
Drag and drop an NUnit action onto the design surface
Name our action 'run UI tests'
Within the files: option, specify the name of the tests project '$Source.test_repo$\tests\tests.csproj'
Within the Project Configuration section specify 'Release'
Specify which version of NUnit
In order to provide greater configuration flexibility we can pass in the location of our Selenium server to the tests at runtime. This is done within the 'Environments' tab. In our case the location of the Selenium server ishttps://SELSERVER:4444/wd/hub.
Click Save
Click save and complete Wizard
We are now ready to build!
Start a build immediately by clicking the top right hand side fast forward icon
A build will start, and complete!
When viewing the build log (this can be done by clicking on the green build number, then selecting the log tab) we can see that our UI tests have been run successfully. They are also visible within the 'Unit Tests' tab which displays further metrics around the tests.
Step 5: Getting more advanced
Lets try a slightly more advanced example. This time we will examine a common use case. A physical visual inspection test needs to be conducted before a release can progress in the pipeline.
Place the following code within our test class.
using System;
using System.Text;
using NUnit.Framework;
using OpenQA.Selenium.Firefox;
using OpenQA.Selenium;
using OpenQA.Selenium.Remote;
using OpenQA.Selenium.Support.UI;
namespace SeleniumTests
{
[TestFixture]
public class test
{
private RemoteWebDriver driver;
[SetUp]
public void SetupTest()
{
// Look for an environment variable
string server = null;
server = System.Environment.GetEnvironmentVariable("SELENIUM_SERVER");
if (server == null)
{
server = "https:// *** PUT THE NAME OF YOUR SERVER HERE ***:4444/wd/hub";
}
// Remote testing
driver = new RemoteWebDriver(new Uri(server), DesiredCapabilities.Firefox());
}
[TearDown]
public void TeardownTest()
{
try
{
driver.Quit();
}
catch (Exception)
{
// Ignore errors if unable to close the browser
}
}
[Test]
public void FirstSeleniumTest()
{
driver.Navigate().GoToUrl("https://www.google.com/");
IWebElement query = driver.FindElement(By.Name("q"));
query.SendKeys("a test");
Assert.AreEqual(driver.Title, "Google");
}
[Test]
public void MySecondSeleniumTest()
{
// Navigate to google
driver.Navigate().GoToUrl("https://www.google.com/");
IWebElement query = driver.FindElement(By.Name("q"));
// Write a query into the window
query.SendKeys("a test");
// wait at maximum ten seconds for results to display
var wait = new WebDriverWait(driver, TimeSpan.FromSeconds(10));
IWebElement myDynamicElement = wait.Until< IWebElement >((d) =>
{
return d.FindElement(By.Id("ires"));
});
// take a screenshot of the result for visual verification
var fileName = TestContext.CurrentContext.Test.Name + "-" + string.Format("{0:yyyyMMddHHmmss}", DateTime.Now) + ".png";
driver.GetScreenshot().SaveAsFile(fileName, System.Drawing.Imaging.ImageFormat.Png);
// perform an code assertion
Assert.AreEqual(driver.Title, "Google");
}
}
}
Build, and run the test.
In this example we added an additional test to perform a google search, wait at maximum 10 seconds for results to display, take a screenshot (stored it to disk), and perform an NUnit assertion. The screenshot output from the test can be made available as an artifact within Continua!
Firstly lets commit our changes; "hg commit -m "added a more advanced test""
Open the configuration in Continua CI (clicking the pencil icon)
Navigate to the stages section
Double click on the stage name (which will bring up the edit stage Dialogue box)
Click on the Workspace rules tab
Add the following line to the bottom of the text area: "/ < $Source.test_repo$/tests/bin/Release/**.png". This will tell Continua to return any .png files that we produced from this test back to the ContinuaCI Server.
Click on the artifacts tab.
Add the following line : **.png" This will enable any .png files within the workspace to be picked up and displayed within the Artifacts tab.
**.png
Click save
Click Save & Complete Wizard
Start a new build
Sweet! A screenshot of our test was produced, and can be seen within the Artifacts tab!
Clicking on 'View' will display the image:
We have put the sourcecode of this article up onGithub.
Please subscribe and comment! We are very excited to see what you guys come up with on Continua, happy testing!
Some additional considerations:
- The user which the Selenium service runs under should have correct privileges
- The machine designated as the Selenium server may require access to the internet if your webapp has upstream dependencies (eg third party API's like github)