TinyOS 2.x Automated Unit Testing / TUnit

TUnit

TUnit is a unit testing framework built for low power embedded RF devices, particularly those built using TinyOS 2.x.

TUnit differs from other unit testing frameworks in the following ways:

TUnit tests are executed across many different types of embedded platforms.
Unit tests are run on one or multiple hardware platforms at a time, communicating over RF
Tests can specify hardware configurations they should execute on, allowing TUnit to filter out tests that do not apply to the hardware platforms currently being tested.
RF and physical issues are mitigated in the testing by keeping all wireless devices within close range of one another.
TUnit performs both Testing and Characterization of embedded software over time, allowing developers to see how changes to the code affects performance.
TUnit is designed to be used both publically and privately.
- Publically to improve the integrity of code built by the TinyOS community (as seen here).
- Privately to ensure proprietary software built using TinyOS works as expected.

Build Process

Our automated build system runs the TUnit Java application inside of Ant, which is automated by CruiseControl. Because of the nature of the testing, a single test run may take hours.

The TinyOS 2.x baseline and TinyOS 2.x Contribs CVS directories are checked out from SourceForge.
TUnit is compiled fresh and begins. All hardware is checked and verified for its existence.
All tests located in the publically accessible tinyos-2.x-contribs/tunit/tests directory are compiled using the latest baseline code. Test suites can be filtered out in test runs depending on the hardware being tested in the current test run.
The embedded hardware executes the tests, asserting results and statistics along the way. The Java side of TUnit stores results in XML files and creates charts using JFreeChart.
Ant's JUnitReport task creates the Test Report HTML from the tests' XML results.
Another Java application post-processes the HTML reports, adding in the charts and statistics.
Wireless lava lamps are toggled to indicate pass or fail of the last TUnit execution, which you see on our webcam.
Developers (will be) alerted to problems through a mailing list.

Documentation

Documentation on TUnit can be found here.
Take a look at the presentation and quick start / reference to see details on TUnit and how to build your own tests.

Adding tests to this system

We need the TinyOS community to help supply tests! We're offering just a framework and an automated system to run tests, but we need other people to help contribute tests, especially for their own software.

That said, we don't want just anybody contributing tests. For the security of our systems running these tests, we would like to know who's got access. To add your own TinyOS 2.x unit tests to the build process, send an email to dmm at rincon dot com with your sourceforge username. You'll get access.

Then, simply upload your test to the appropriate directory in tinyos-2.x-contribs/tunit/tests. Before upload, be sure to run your tests through a local copy of TUnit to verify they work.

Each test should have its own directory and a suite.properties file, which tells TUnit about the hardware the test applies to. Look to other test directories for an example of a suite.properties file.

Testing isn't limited to the TinyOS baseline. Tests can be uploaded for anything available in tinyos-2.x-contribs as well.

After uploading your tests, you'll be able to see the progress on this website as they run through the system.

Hardware Combinations Being Tested

TUnit will apply unit tests to various combinations of hardware. To do this, Rincon Research Corp. has contributed the following hardware for public TinyOS 2.x unit testing:

All hardware is connected via USB.

Note that we don't have a huge testbed setup, because after all, this is unit testing. And unit testing is supposed to be small. That doesn't mean we couldn't run on a testbed. If someone wants to do that, that would be a good thing.

Two motes are enough to cover the majority of unit test cases. More motes than that help in characterization - i.e. asking the question "How fair is this MAC layer in sharing the channel?" vs. "Ensure the MAC layer is sharing the channel fairly."

If you would like to donate hardware to this public TinyOS 2.x test bed, we would love to support you. Again, two motes of some platform is typically what's needed for effective unit testing. Please contact us (below) so we can work out any details.