Working with NUnit tests

Instructions

We use the open source application NUnit for our tests.

• Download the latest version from https://launchpad.net/nunitv2/ Choose the latest version and NOT the one for .NET v1.1. At the time of writing the download choice is NUnit-2.6.0.12051.msi.
• Install this file on your PC using the default options
• Build the entire Open Petra solution
  • either by running nant generateSolution once; then running nant compile will compile the tests for OpenPetra (and re-running it will recompile the tests)
  • or by doing the same thing using the Developer's Assistant
• You will find the resulting tests in dlls in the directory delivery/bin/ along with all the dlls that make up the Open Petra run-time.

Now you are ready to start testing...

• Open NUnit (which you installed as part of the msi above) from the start menu
• Choose File | Open Project to open the dll that you want to test, eg. Tests.Common.dll.
• You can now run a single test, or run all tests at once, depending on which item in the tree you select
  • Note: run the 32 bit version of NUnit if you are on Win 7 (found in %ProgramFiles(x86)% and usually has "x86" in the executable name).

If all your test(s) pass, you will see a nice green bar across the top of NUnit. If any test fails the bar will be red. You will (usually) get helpful pointers to the part of the test that failed.

Important Note: If you are running a simple test or a server test you probably don't need to have the server console running at all. If you are running a client test, you will definitely have to start the server console before you run the test. If you forget to do this, the 'error' message that you get when the test fails (as it will) is not very helpful and unlikely to remind you to start the server - until you have seen this unhelpful message for the twentieth time!

Available Tests

• We have quite simple function tests, eg. in csharp\ICT\Testing\Common (which tests a lot of the functionality of Ict.Common)
• We have tests for string operations (TStringHelper), Database operations, CSV&XML Input/Output, etc.
• We have tests to prove the correct operation of the server and database
• We have tests to prove the correct operation of several of the client screens
• We have a test to validate 'printed' output
  • The test in csharp\ICT\Testing\Common\Printing does not use the NUnit framework, but allows the user to see how an HTML file gets printed to paper or PDF.

File System Organisation for Tests

Test DLLs are organised in our folder structure beneath csharp\ICT\Testing. Most tests tests are in a lib sub-folder which is then further categorised by module feature (MCommon, MFinance, MPartner etc). You should further separate out specific server tests from client tests.

Anatomy of a Test Class

Start by reading the NUnit help, as well as looking at some existing examples of Open Petra tests. You will soon discover that test methods carry attributes such as [SetUp], [Test] etc. In fact all your public methods will need an attribute. This is for setting up the testing environment, cleaning it up afterwards, defining which method is a test and so on. You will also probably have a few private helper methods for doing repetitive tasks in your code.

Each [Test] method needs to set up its data, perform the action to be tested and then probe the result(s) using one or several of the 'Assert' overloads such as Assert.IsTrue() or Assert.AreEqual(). A good test method will have a few lines of code to set up the test - by making a call into the class being tested - and then a number of assert lines that test for successful execution. It is also a good practice to separate out the data initialisation part into a private helper method so that it is easy to read what the test is doing from the code.

Writing New Tests

When you are writing a new test you need to think about the following.

• Can this test be added as a new test class (as a new .cs file) to an existing test DLL?
• If you decide that there are a large number of tests to write on a new subject, then you will probably decide to start a new project DLL. In that case, where will the new project live in our Testing hierarchy?
• What data will the new test need to work with (if any)? All our tests so far use the 'demo' database, which starts with a limited amount of content. That is normally a good thing for testing as it allows you to set up the starting condition that you want. However you will need to think whether your test needs to be sure to start with a particular table in an empty state or whether you can start with any prior data condition and whether your test can delete any data that you add.
• Having decided what the working data is going to look like, it is quite a good idea to create a testData.cs file (perhaps as a partial class of the main test class) and define the data handling methods that you will make use of in that file. This helps to keep your test file cleaner. See for example csharp\ICT\Testing\lib\MFinance\ExchangeRates\CorporateDataSet.cs which is a partial class of csharp\ICT\Testing\lib\MFinance\ExchangeRates\CorporateRate.test.cs.
• Plan carefully exactly what tests need to be carried out in order to prove the functionality of the target code being tested. Think about how to make sure that the maximum amount of the target code can be forced to run. This will mean arranging to run a nice test that succeeds, but also to set up conditions where the target code is made to run with 'bad' input conditions and prove that it handles those correctly.

You may well be able to copy an existing .cs file and use it as the basis of your new test class. If you need to start a new project (DLL) then

• decide where it will be located in the existing file structure and create a new folder for it.
• add a typical test code file to the folder (copy an existing one and then delete all the content except maybe the Setup and TearDown parts).
• set the namespace to match the location you have chosen.
• include at least the #using statements indicated below

Now using Nant or the Developer's Assistant, generate the full solution (if you are already up to date this can be with a minimal compile). Load the Open Petra Testing solution. You will have a new project created for you with some initial project references.

As you write your tests you will likely need to add further project references. You can work out what these references are from the target code. You then can use the IDE to add a reference to your test project, but make sure to add a #using statement as well. Then you can be sure that when you re-generate the whole solution and the projects are re-created the references will all be correct.

Here are the minimum required references for a server project:

using System;
using System.Data;
using NUnit.Framework;
using Ict.Testing.NUnitPetraServer;
using Ict.Common.Data;
using Ict.Common.DB;
using Ict.Common.Remoting.Server;
using Ict.Common.Remoting.Shared;
using Ict.Petra.Server.App.Core;

Here are the likely candidates for a client test

using System;
using System.Windows.Forms;

using NUnit.Extensions.Forms;
using NUnit.Framework;

using Ict.Testing.NUnitForms;
using Ict.Testing.NUnitPetraClient;
using Ict.Testing.NUnitTools;

using Ict.Common;
using Ict.Common.IO;
using Ict.Common.Controls;
using Ict.Petra.Client.CommonControls;
using Ict.Petra.Client.CommonForms;

The [Test] methods have to be able to run in any order. If they alter the database they have to set the database correctly for the test and then (possibly) delete any changes they have made.

Server Tests

Background

If you only want to write a server test, you don't need to understand the content of this background section. You can skip directly to #Authoring Server Tests

We have a full server wrapped up in a dll, in csharp\ICT\Testing\NUnitPetraServer, Ict.Testing.NUnitServer.dll. This server allows a user to login, and then access the webconnector functions, Database access, and basically everything else inside the server.

Authoring Server Tests

In some respects, server tests are easier to write than client tests. We have already seen that usually it is not even necessary to run the server console separately in order to execute a server test. The test itself is probably simpler to write and the number of separate 'Asserts' required to prove success is probably fewer.

You are likely setting up the database to contain known data and asserting that you can retrieve the correct data set from it. Or you are confirming that given a specific data set you can successfully update the database without errors or conflicts.

Existing Server Test Samples

• The project in csharp\ICT\Testing\MFinance\server\Gift has a simple test for DataAccess and for a WebConnector.
• The project in csharp\ICT\Testing\MFinance\server\CrossLedger has a simple test for adding and removing ledgers of various types and testing the server method: GetAvailableLedgers().

Client Forms Tests

Background

If you only want to write a client test, you don't need to understand the content of this background section. You can skip directly to #Authoring Client Forms Tests

We make use of NUnitForms (http://nunitforms.sourceforge.net/). The project is alive, even though the last release is a couple of years old. We have our compiled dll of the latest VCS snapshot in our own VCS, in csharp\ThirdParty\NUnit.

We have made small modifications to NUnitForms so that it works for our generated forms with all the random Layout control names blurring the path to controls. The modifications are maintained in csharp\ThirdParty\NUnit\src.

NUnitForms is an extension to NUnit to enable NUnit to run tests on Windows Forms and access control properties. You do not need to install NUnitForms. NUnit can automatically access the DLL.

Our DLL Ict.Testing.NUnitClient.dll (project in csharp\ICT\Testing\NUnitPetraClient) is a full OpenPetra client, allows a demo user to login to the separately running server, and then open OpenPetra screens as if it was a normal user. There is currently no mock-up server.

Authoring Client Forms Tests

Basically, these tests are integration tests. Each test will likely call the Show method on the screen being tested. You can then access each individual control on the screen and check its value, change its value, push buttons and generally interact exactly as you would if you were using the screen yourself. You can even cope with pop-up dialogs and check their content and click one of the dialog buttons. For each type of control on the screen there is a Tester object. So for example 'ButtonTester'. Objects exist for all the standard Windows controls and Open Petra has created objects for its own specialised controls like auto-populated combo boxes and Petra Date controls. Every Tester object has a Properties property which returns the actual object itself. Here are some examples of code you might use.

If you want to simulate clicking a button -

ButtonTester btnNew = new ButtonTester("btnNew");
btnNew.Click();

If you want to access an actual control's properties, or to set its value -

TCmbAutoPopulated cmbFromCurrency = (new TCmbAutoPopulatedTester("cmbDetailFromCurrencyCode")).Properties;
Assert.AreEqual("EUR", cmbFromCurrency.GetSelectedString());

This is slightly more readable than

TCmbAutoPopulatedTester cmbFromCurrency = new TCmbAutoPopulatedTester("cmbDetailFromCurrencyCode");
Assert.AreEqual("EUR", cmbFromCurrency.Properties.GetSelectedString());

especially when there are many asserts; it also looks more exactly like the code that is being tested.

For the grid you may well decide to create both a 'tester' and a grid object, since you need the tester to SelectRow and an object to check the number of rows, for instance. Also bear in mind that the tester for a grid is a paged grid, which is a super-class of a standard grid, but allows a different set of properties and methods. You can cast a paged grid to a standard grid though. Most of our screens use standard grids. Here is some sample code.

TSgrdDataGrid grdDetails = (TSgrdDataGrid)(new TSgrdDataGridPagedTester("grdDetails")).Properties;
Assert.AreEqual(10, grdDetails.Rows.Count);

TSgrdDataGridPagedTester grdTester = new TSgrdDataGridPagedTester("grdDetails");
grdTester.SelectRow(1);

Data Access From Client Forms Tests

You will likely need to load and save data to the database within your test method. There is a standard base class (SerialisableDataSet) that you can use for this purpose. It has a simple LoadAll() and SaveChanges() method.

All you need to do is to include your variant of the class code here inside your own test class. (Change the Table to suit your test table)

        private class FMainDS : SerialisableDS
        {
            public static ADailyExchangeRateTable ADailyExchangeRate;

            public static void LoadAll()
            {
                ADailyExchangeRate = new ADailyExchangeRateTable();
                SerialisableDS.LoadAll(ADailyExchangeRate, ADailyExchangeRateTable.GetTableDBName());
            }

            public static bool SaveChanges()
            {
                TTypedDataTable TableChanges = ADailyExchangeRate.GetChangesTyped();
                
                return SerialisableDS.SaveChanges(ADailyExchangeRate, TableChanges, ADailyExchangeRateTable.GetTableDBName());
            }
        }

Now the LoadAll() and SaveChanges() methods can be called within your [Test] methods. Also the coding style is similar to that of most screens in that you work with a constructs such as FMainDS.TableName.Rows.Count.

Running Client Forms Tests

You should by now have realised that running a client forms test is no different from running a simple test or a server test. Load the test DLL into NUnit, select the test (or all of them) and click Run. But don't forget to start the server first! You will find that compiling your test requires that the server is NOT running but running your test requires that it is. So you will become quite used to the workflow of starting the server, running the test, stopping the server, editing the code and round again...

Existing Client Test Samples

• The project in csharp\ICT\Testing\MFinance\GLForm creates GL Batches, posts them, and does some checks.
• The project in csharp\ICT\Testing\MFinance\ExchangeRates runs very extensive tests on both the corporate and daily exchange rate screens. It also uses the serialisable Data Set described above. It tests user interaction, button clicking, validation, dialogs, import of data and many other features and is a good (though extensive) example to look at.

Debugging

Just a final note about debugging your test. Most of the time you will not need to debug your test code because your 'Asserts' will give you enough information about what is going on. Furthermore you can use Console.WriteLine() in your test code and the output will appear in NUnit on the 'Text Output' tab. However it is perfectly possible to single step debug through your test code by 'attaching' your IDE to NUnit-agent.exe (not NUnit.exe) in the list of running processes. Then when you run the particular test in NUnit you will hit your breakpoint.