How do I test a user interface

TDD: Testing of graphical interfaces (GUI)


Nowadays, software-based application systems are mostly operated via graphical user interfaces (GUI). The intuitive handling and thus easier operation compared to applications that are controlled via command inputs is the reason for this development. In order to do justice to quality assurance, besides testing the individual software components, testing the user interface is also necessary. The scope of a GUI, with a multitude of elements such as buttons, input fields or menus and their functionality, usually quickly becomes enormous.

In software development, the process of testing a product that uses a graphical user interface is therefore of great importance. To ensure that the functionality meets its written specifications, the developer of an application must develop suitable test cases. All possible user actions should be covered and it should be ensured that the desired user effect occurs and leads the user to the goal.

A qualitative set of test cases is decisive, which tests the full scope of the functionality of a graphical user interface. A manual test session makes little sense for reasons of time and costs. Testing is usually carried out automatically with the aid of tools. The extent to which automatic test processing makes sense and where the limits are will be explained later. At the end of the elaboration there is a short conclusion.

Determination of a test framework

The design and behavior of a GUI, which can be seen as a user interface between man and machine, lead to the following consideration.

  • What is user-friendly design? Testing of user friendliness through possible support from psychological counseling.
  • How does the surface behave with changing users and various interactions. The experience of a user should also flow into the problem determination and the dynamics of the interface should be recorded.
  • Different hardware can influence the behavior of a GUI (graphics card, monitor).
  • Testing the graphical user interface with various test tools.

This results in a possible test framework that contains a list of all test cases. This allows a comparison of the given behavior with the behavior during the test execution. The test shows whether the software meets the requirements or not.

General information about user interface testing

Testing the interaction between the software and the user is the core aspect of a GUI test session. The test can be divided into two areas. A sub-area deals with the representation of the GUI with regard to the specified design and checks how it appears to the user and whether the specifications are adhered to. The criteria are, on the one hand, the clarity and, on the other hand, the uniformity of the entire user interface. Graphic properties of the interface elements such as the alignment of text modules, the size and color of the text improve readability and provide an overview. Redundant labeling of buttons with the same functionality and compliance with their labeling convention can also be tested.

The second part deals with the functionality behind the user elements of a GUI and tests the function and behavior of the respective elements. It should be ensured that the implemented function is executed correctly as required and that the expected behavior is communicated to the user.

Since the implementation of a user interface is event-driven, the user has a relatively high degree of freedom when interacting with interface elements. Accordingly, all possible user combinations that can be executed on the GUI should be taken into account in the test coverage. This applies to "intended" user actions and in particular to "not intended" user actions that lead to exceptional situations or incorrect behavior. Here the software must be able to handle such a situation. With an extensive GUI, it is usually possible to reach a certain status in several ways. The software must react to user paths via keyboard, mouse or menu entries as specified. However, it is almost impossible to cover all variants of the inputs with test cases.

It is not always desirable to automate all testing activities. There are tests that are much easier to perform manually. Test cases that are particularly difficult to automate may not be worthwhile. Other points such as tests that are rarely carried out or those where the test can be verified very easily by humans are also included. Not to be forgotten are test cases that require physical interaction.

Acceptance test

An acceptance test is the testing of the software supplied by the customer. This therefore expects a functioning system. According to the black box method, which does not require any knowledge of the internal functioning of the software product to be tested, the customer only checks the behavior of the software in the case of specified actions such as: user input or limit values ​​during data acquisition.

End-to-end test

In an end-to-end test, individual or all components of a process are involved and are tested with end-to-end test cases.

Regression test

The dynamics in the development phase of a software project call for regression tests. They are used to repeat test cases, even during operation. The aim is to identify the effects of parts of an application that have already been tested, which can arise when the software is modified. Because of their repetitive nature, it makes sense to automate regression tests.

Procedure for test case generation

Basically, it is possible to create test cases manually by programming and to have them executed automatically. A very common method, however, is the capture-replay method, which is supported by the GUI test tools discussed here. According to this approach, all user inputs and interactions with the user interface are recorded in the background and incorporated into the test script. Checkpoints can be created during recording. They are used to generate a target / actual comparison in order to check the behavior of the surface. The script can be played as often as required and thus supports the automated test session. It is therefore very suitable for regression tests. In order for checkpoints to make sense at all, it is necessary to define a pool of data. Test scripts that access such a data pool follow the principle of data-driven test runs (data-driven test). The test script fetches the comparison data from this pool and checks it against the input data of the GUI. The test script uses all or randomly selected comparison data. By decoupling from the actual test script, it is possible to create new test cases very quickly by changing or expanding the data pool. This achieves a certain degree of flexibility. Certain, repetitive user actions can be linked with keywords. According to the keyword-driven test, these actions are packed into procedures. This is also a frequently used method for optimizing test-driven development.

Test tools

The following list shows a limited selection of test tools, so-called CAST tools (Computer Aided Software Testing Tool), which allow test cases to be automated with regard to graphical interfaces.


Selenium is an open source framework from ThoughtWorks for testing web interfaces. It was released as free software under the Apache 2.0 license. With this tool it is possible to generate automated tests. By recording user actions (user story), test cases can be repeated as often as required. It is installed as a plug-in for the Firefox browser.

As part of Selenium, HTML can be used via the integrated development environment (IDE) as a language for test cases in order to write test scripts.

Selenium Remote Control (SRC) is a test tool for designing test scripts that run against a web application and finally there is Selenium Grid for managing larger test suites or different test environments.

Selenium IDE

Selenium IDE allows you to generate test cases based on HTML very easily and quickly. In addition, the code can be edited directly in the IDE. This code can be exported into other high-level languages.

The integrated development environment enables test suites to be managed using batch jobs. Such test suites consist of individual HTML test cases, which are then played automatically. The following code example from a Selenium tutorial shows the syntax of a composite test session.

<html><head><title>Test Suite Function Tests - Priority 1</title></head><body><table><tr><td><b>Suite Of Tests</b></td></tr><tr><td><ahref="./Login.html">Login</a></td></tr><tr><td><ahref="./SearchValues.html">Test Searching for Values</a></td></tr><tr><td><ahref="./SaveValues.html">Test Save</a></td></tr></table></body></html>

Selenium Remote Control (SRC)

Selenium Remote Control serves as a server for starting and stopping web browsers. Selenium RC receives and interprets the Selenium commands. Then they are carried out. SRC also acts as a representative for HTTP requests and returns the results to the test program via http get / post. SRC provides a client library (API) for programming languages ​​such as Java, C #, Ruby, Perl or Php. This gives a maximum of flexibility for developing the test logic. To bypass the focus on Firefox, SRC can be used to send generated test cases to common browsers such as Internet Explorer, Safari, Chrome or Opera. To ensure functionality, Selenium uses Javascript to control the respective browser. The following diagram shows the SRC architecture.

Source: - SRC architecture

Generate a test case

Selenium supports capture replay. A small example shows how Selenium generates a simple web test case using this method. The Selenium IDE is used for this. Since this tool is integrated into Firefox, creating a test case is relatively easy and convenient. The Selenium IDE offers all the necessary functions to use capture replay. Via the menu item Tools-> Selenium IDE in the menu bar of Firefox you can reach the tool you need for the test creation.

The popup houses various buttons that are used to control the recording. The red record button is responsible for creating the test case. This starts the recording of the user action. For this simple test, the Google homepage is called up via the URL field of the Selenium IDE. The keyword “selenium” is entered in the search box on the Google site and sent off. The Google results page shows a hit. Another click on the red Record button of the Selenium IDE ends the recording of the actions carried out. This creates a simple test framework and Selenium shows the following result.

  • Source: Generated commands

The "Table" tab contains the commands generated by Selenium. Exactly in the order in which the user actions were carried out. The test framework can now be repeated as often as required using the play function of the IDE. All recorded actions are automatically carried out in the same way as they were previously generated when they were created. While the recording is running, Selenium marks all successfully passed events in green and writes all processes in a log section. If all commands are marked in green, the run is successful. The IDE also offers a “Command” dropdown field which allows additional commands to be attached to the test framework. For example, it is possible to build in "assertions" to generate test points. Select the corresponding line in the command section of the IDE and choose the command from the “Command” drop-down menu. In this example the instruction "assertTextPresent". The "Target" field awaits the entry of the string to be checked. This turns the test framework into a test case. If the string is found when it is run again, the test is successful. If the string was not found, the command line is displayed in red in the IDE. An error is thrown in the log section and the test run was negative. Selenium offers the possibility to export the generated command code.

Exported Java file from the example, which was generated by Selenium.

package com.example.tests; import com.thoughtworks.selenium. *; import java.util.regex.Pattern; public class NewTest extends SeleneseTestCase {public void setUp () throws Exception {setUp ("", "* firefox"); } public void testNew () throws Exception { ("/"); selenium.type ("q", "selenium rc"); ("btnG"); selenium.waitForPageToLoad ("300000"); assertTrue (selenium.isTextPresent ("Results * for Selenium rc")); }}

Selenium renders additional JavaScript code into the web pages, which in turn assumes that this tool exists while it is running. The commands consist of instructions that the browser should process. For this purpose, Selenium uses the JavaScript capabilities of the browser. Selenium checks the application against its specification without knowing the internal structure of the code. Errors can therefore be uncovered that prevent the program from being executed correctly from a technical point of view. Texts, links and HTML elements can be checked automatically and all Javascript events can be fired via Selenium. This allows the automation of almost all user stories.

Selenium and PHPUnit

For the creation and execution of test cases it is necessary that the Selenium server is running. This server, developed in Java, can be started by simply specifying the following command on the command line.

java -jar selenium-server.jar

This starts a Jetty web server and Selenium Remote Control is available. In order to establish communication between the server and the test program, a client driver is required, depending on the preferred programming language.

PHPUnit is an open source framework for testing PHP scripts. It is based on the xUnit concept and is suitable for automated test runs. The test cases generated by the Selenium IDE can be started from within the PHPUnit framework. The Selenium-SRC plays an important role here, as it allows the Selenium commands to be executed remotely via HTTP requests in the browser. A PHP client is used as a translator between PHPUnit and Selenium. The procedure is demonstrated using an example (from a Selenium tutorial).

The connection setup is implemented in the GUITest class to establish communication with the Selenium SRC. This is done in the setUp () method. In the derived class PHPUnit_Extensoins_SeleniumTestCase, among other things, the runSelenese () method is defined. This method expects the path to the Selenium test cases.

runSelenese (path to the Selenium test cases);

In this way, the HTML test case generated by Selenium by default can be started.

In general, PHPUnit_Extensions_SeleniumTestCase sends an HTTP call to the Selenium SRC and processes its response in the PHPUnit assert instructions. For example, it can be checked whether a defined text is available. It is also possible to test whether a certain DOM element can be found on the web page to be tested.

What can't Selenium test?

Besides the ease of use, everything that JavaScript cannot be tested cannot be tested. It is therefore not possible to switch tabs in the browser, close popups or click away message boxes. Selenium does not record these actions. Settings in the browser cannot be changed either.

IBM Rational Functional Tester

The Rational Functional Tester (RFT) enables test suites to be created under Windows and Linux. It is a professional tool for automatic GUI tests, functional and regression tests. The RFT is one of the market leaders in the field of commercial GUI test tools. The tool supports the scripting languages ​​Java under Eclipse and Microsoft Visual Basic in the .NET Framework. During installation, it can be integrated into the development environment of Eclipse or Microsoft Visual Studio if required. As with Selenium, test cases can be created using the capture replay function.


Mapping of the RFT in the Eclipse IDE

Generate a test case

Test cases are generated as Java or Visual Basic code. The automatic scripting of the RFT is used to record the user action. All actions of the user that are carried out on the surface are recorded in the background. As an example (from a tutorial), the procedure is demonstrated using a login interface of a web application. A simple test case should check whether the login dialog behaves as expected when entering the login data. While the recording is running, the focus is placed on the input field with a mouse click and the user data is entered. In this example the character string “admin” for the user field and “123” for the password field. After clicking on Login, the login dialog is closed and it is checked whether the expected feedback in the form of the message “You are logged in” corresponds to the behavior.

Illustration of a fictitious login mask

The RFT creates the following source text from this user action:

public class TestLoginSite extends TestLoginSiteHelper {public void testMain (Object [] args) {startApp ("http: // localhost: 1203 / WebApp / login.aspx"); // start application // Document: Login: http: // localhost: 1203 / WebApp / login.aspx text_user (). Click (atPoint (36,13)); // Enter login data browser_htmlBrowser (document_anmeldung (), DEFAULT_FLAGS). inputKeys ("admin"); text_pass (). click (atPoint (32,10)); // Enter login data browser_htmlBrowser (document_anmeldung (), DEFAULT_FLAGS). inputChars ("123"); button_anmeldensubmit (). click (); // click button // Document: Main page: http: // localhost: 1203 / WebApp / main.aspx html_loginMessage (). PerformTest (LoginMessage_textVP ()); // Checkpoint: Comparison of the output with the target value browser_htmlBrowser (document_haupseite (), MAY_EXIT) .close (); // close browser}}

All instances that are returned as test objects come from the base class TestObject. From the code it can be seen that with the call of the text_user () Method a test object is returned, which refers to the user text field with the corresponding coordinates. The method text_user () is in this example in the derived and additionally generated class TestLoginSiteHelper implemented. The same goes for the method text_pass (). Information is stored in an internal GUI map in order to link the test object with the corresponding GUI element. This information is obtained from the method click (atPoint (36,13)) taken, which in turn from the subclass GuiTestObject originates. In the line with the instruction,

html_loginMessage (). performTest (LoginMessage_textVP ());

is a checkpoint LoginMessage_textVP () anchored. This checkpoint contains the information that the string “You are logged on” is of the “Text” type. This is done with the property of the test object html_loginMessage () compared. The checkpoint was previously set when recording with the script creator. This is done very simply by selecting the corresponding GUI element and entering the setpoints. To support the test point entries, the RFT has a test point assistant ready. For querying properties and testing probe points are the basic methods getProperty () respectively performTest () from the base class TestObject responsible. After the test has been carried out, the RFT creates a test protocol in which all relevant information on the test points, warnings and errors that have been passed are listed. The test script created can be used over and over again. Furthermore, the test coverage can be further improved by programmatically adding to the scripts.


The test tools discussed here are without question a great help when it comes to testing graphical user interfaces. They support automated test execution where it makes sense. However, they also require a certain amount of effort in creating test cases. Especially during the development of an application, regression tests may have to be adapted again and again due to frequent modifications.

Ultimately, it is always the person who discovers the first error in the software. Because first a test case has to be created that also works. When user actions are recorded, errors are usually found and corrected promptly. Even if the evaluation of the user-friendliness is required, the human cannot be replaced and shows the limits of the tool-supported test session. The test must be carried out manually here.




--Detlef Kretschmer 10:24 am, May 13th. 2010 (CEST)
Email: [email protected]