Embedded Systems Code Testing Tools
What do you mean it’s impossible to test that?
Backstory
Sample source code to be tested.
I have considerable experience with Agile software development practices and Test-Driven Development (TDD) techniques. At the time of this work, members of the embedded software community had long dismissed these as impractical, unhelpful, or impossible in firmware development.
Like-minded colleagues and I took this as a dare and have proven our detractors wrong. In the process of applying TDD to embedded software we created three key open source tools and frameworks to allow real, easy, thorough unit testing.
Work
Sample unit test and mock code of sample source above. Note this sample may not reflect conventions of up-to-date tools.
Test-Driven Development with unit testing is most easily accomplished with automated test runners. Automated test runners are most easily managed by using consistent testing interfaces. Consistent test interfaces lend themselves to developing frameworks. As such, we created Unity, CMock, and Ceedling for use with the C programming language. We also published a site Throw the Switch for mad scientist embedded developers interested in taking over the world with their well-tested code.
Unity is a lightweight xUnit-style test framework for C written in C. CMock is a framework for mocking C interfaces (i.e. function signatures) for interaction-based testing; it is written in Ruby to inspect C interfaces and generate C code mocks from them. Ceedling is a command-line build environment for scanning the source under test, generating code using the other tools, building and executing the tests, and reporting results.
Results
These tools are publicly available and are in very active use. We have successfully used them both on-target and in cross-compiled and desktop simulation environments. NASA has used these tools on several projects including aspects of Mars landers. NIST uses these tools for the development and maintenance of the national timekeeping infrastructure. Abbott Medical has used these tools for medical device firmware development.
As an example of the real world value of these tools, a large embedded project that made use of these tools concluded development with nearly 6,000 automated unit tests and 95% test coverage. Said project was very large, requiring 5 developers and 20 months of development. It made it to final QA testing with less than 50 defects — far, far less than industry average for a project of such size.