Historically, testing software was something you did at the end of development to see if everything worked. It still is, but the aim is ‘acceptance testing’ - is the customer happy, contracts fulfilled? Problem was that with inevitable time pressures the testing usually ended up being perfunctory and shoddy, buggy software was the result. Today developers test as we go.
Unit testing with JUnit
I’ve already hinted there are different kinds of testing with different aims. Unit testing is about testing each distinct component of the software in as isolated a way as possible. I’m using the popular JUnit (version 5) to manage and automatically run those tests so that with a click in my Netbeans IDE those tests run and report which fail and which pass. Then you fix the fails.
Tests do need to be written and debugged and manually checked they are working. That is time spent not actually developing the product, so generally I’m not writing a new test for each possibility. It’s a judgement call, how much testing to create. Currently [1] I’ve got 90 java source files, with 57 unit tests which take about 12 seconds to all run and report. It becomes an easy process - edit code, run tests, create new tests if useful, move on. Rather than an unpopular phase of software development, unit testing is a practice that allows quite radical code changes with high confidence you’ve not broken something.
Integration testing with JUnit
Tests that check how major parts of the code work together are integration tests, they are typically harder to write and take longer to run. There is no clear distinction with unit tests, but they usually involve setting up a surrounding environment such as a prepared database or a running game in a specific state. Currently I’ve 4 integration tests (testing major subsystems of the game) taking 4 seconds to run, which at the moment is still trivial time-wise.
Separating these tests from the unit tests proved a bit tricky. Add a new maven plugin (maven-failsafe-plugin) into the POM and name the test files with an “…IT.java” format. The default JUnit tests can be run, the integration tests are ignored (because they don’t end with “Test.java”) but can still be run independently. Bit clunky but it works, check the POM in the repository.
Log capture
One problem for integration testing is how do you check the right things have happened? For example has the database now got the correct data in it? Remember these tests run ‘outside’ your code, they won’t have special access to the database. That would be a security nightmare waiting to happen.
Installing the log-captor maven plugin allows for the programs logfile output to be tested. So with lots of very low level (trace) logging statements in the code to report what’s happening inside the code, the external based testing can confirm things went as planned. When running the code for real, just switch off the trace logging - the Setting up logging with log4j and slf4j article discusses logging in more detail.
Together these test mechanisms can become very sophisticated with the code being automatically and repeatedly built, tested and reports being sent to managers, fails creating bug reports for developers. For a simple one-developer project like this that’s all overkill. But getting testing in place means better code, less time spent tracing weird bugs and more liberty to modify code.
[1] there’s going to be much, much more.