General

Requirements

Tests execution should be controllable through nant
Tests execution should be done by build server
For writing tests it should be possible to execute separate test cases

How do we want to get test peripherals?

Write test data into database
- Set up new, additional test data for each test run
- Delete database and set up a database snapshot
Write mock ups, so that database is not needed for the test
Mixed way of test data in database and mock up

Intuitive feeling: Test data in database by adding test data for each test run, perhaps mixed with some mock ups

kind of tests we want to write

Tests, which are testing the server code
- by calling the server functionality directly (in the same process)
- by connecting as client to the server
Tests, the client code without UI
Tests, which are testing by interacting with the UI?

Usage in the past

Calls a function directly
- no database
  - Test of TFileDiffMerge
  - Test for parts of ICT.Common
  - Pseudo-Test of some controls
  - yml writer test
- with database
  - rudimentary sequencer test and db constraint test

Dialogs automated and asserts rudimentary only on dialogs
- With database and server
  - reset database and Import Data through nant
    - Import/Export Gift batches
    - GL Tests
  - generates data through GUI with minimal random string
    - GL Forms
  - Expects certain data exists / does not exists
    - MFinance.server.gift
    - Partner import (assert on the number of imported partners only)
A first concept of some dialog tester classes which can be generated automatically by the yaml files

How to write a test

Sit down and think, what functionality you have implemented
Write down the different use cases - each use case generates more than one test
Write down, what needs to be tested. There are always following cases:
1. Good case in middle range of the possible values
2. Good case next to the limits of the possible values in all directions
3. Bad Case in middle range of the not allowed values
4. Bad Case next to the limits of the not allowed values
Write down, on what outcome you should assert (Remember: We are doing white box tests and look INTO the software!)
1. in the database
2. in the state of the server (if in the same process)
3. in the state of the client (if client is used)
Write down, which test data you need and how you could generate it
Implement the test. Do the test in three stages:
Testing through Client GUI should be avoided, if possible. It is better to change the client, so that this is possible

Always ensure, that you on later stages you do not re-test earlier stages. Sometimes it is sufficient to show, that the function is correctly called!
1. Test without DB, whenever possible
2. Test inside the server process
3. Test like a client without GUI
Get your test working

Need to look into

Interesting concepts

http://blog.benhall.me.uk/2010/07/improving-testability-with-castle.html Dependency injection even for configuration

xUnit like frameworks

http://www.nunit.org/
- ChristianK uses the version of NUnit that is integrated in SharpDevelop 3.2, as well as the standalone NUnit application. The SharpDevelop-bundled version works well and is nicely integrated into the IDE, but doesn't offer the full feature set of the standalone NUnit application. Combined with the integrated NCover application it is very handy, though!
  - Caveat: The test project needs to be a Commandline or Windows Form Project for it to work (DLL projects don't work).
http://gallio.org/ ( http://www.mbunit.com/ is part of it)
- Mirror concept (Interface to reflection for accessing private properties and methods)
- http://stackoverflow.com/questions/3678783/mbunit-vs-nunit
- http://stackoverflow.com/questions/261139/nunit-vs-mbunit-vs-mstest-vs-xunit-net
http://www.csunit.org/
http://dotunit.sourceforge.net/
https://launchpad.net/nunit-3.0/ (Still beta version)
http://xunit.codeplex.com/

Mock Up

(Ok, Thias a mock up can be used to implement a non existing partner module if the routines are required in the financial modul. But I recommend a discussion "mock up yes or no" before we decide to select one of them. I think a mock up is a good idea if you are in the design phase of one module while you are realizing the other. But here? Wolfganguhr 06:45, 12 May 2011 (UTC))

Validates test quality

http://ncover.sourceforge.net/
- ChristianK uses the version of NCover that is integrated in SharpDevelop 3.2. It works well and is nicely integrated. Combined with the integrated NUnit application it is very handy!

Thias we have actually decided not to run tests with full code coverage ... Wolfganguhr 06:48, 12 May 2011 (UTC)

- - Caveat: The test project needs to be a Commandline or Windows Form Project for it to work (DLL projects don't work).
http://nester.sourceforge.net/

Integration test framework

http://qaliber.net/testbuilder.php
http://blogs.msdn.com/b/ericgu/archive/2007/10/09/unit-testing-through-the-ui.aspx (Overview how to access forms)
http://nunitforms.sourceforge.net/
http://watin.org/ http://aleksjones.com/blog/?p=44

N/A

http://code.google.com/p/autofac/ (Library for Inversion of Control / Dependency Injection)
http://roaster.codeplex.com/ (nunit for compact framework)
http://dbunit-net.sourceforge.net/ (Write data to database and remove it afterwards)

May be that is something that will solve our problems. In our test cases we allways have to integreate test data depending on the situation and even wrong test data in order to create an error message. Actually this is a sql file created by the data base and imported by nant. But this only works for one database and the actual realization is something tricky.