A few years ago, I was asked to perform the audit of an embedded device. This device was used in professional fleet management applications, to upload various data to a central application server: vehicle location and speed, logic states of a few digital inputs, etc. The technical archictecture was based on a microcontroller and a GSM/GPRS/GPS module. An over-the-air (OTA) update function had been implemented, so that remote software update was possible. Nothing really complex, nor revolutionary.
Several thousand vehicles had been equipped. But quite rapidly, up to one third of installed devices stopped working. And the most stressful point was that the OTA update was no more functional. Consequently, every non-functioning device had to be removed from the vehicle, reprogrammed, and reinstalled.
To start my investigations, I asked for two elements: source code and circuit diagram. After a few hours wandering through various source files, I got a first global feeling. Source code had been written by someone who had no experience at all in writing embedded software. For instance, digital input signals were polled, instead of being processed in an interrupt-driven way. This had led the developer to write around ten times the number of code lines usually required to process a digital input. And, far worse, it could happen that short pulses were not detected...
What is interesting is that just a few days before, I had been told that the software had been written by a senior software engineer. I was really surprised, because of the discrepancy between code quality and alleged experience of code author. So I asked for some more information about the developer. And I rapidly understood: developer's experience was in the field of web development, not embedded development. He was a very good web developer, and was totally ignorant of the embedded world.
This is the salient point of this project case: delegate the right task to the right people.
The developer had been assigned to the project for two reasons: he was available, and his seniority was perceived as being of the right level. But the project leader was so inexperienced that he had not yet discovered that software domain is a huge domain, with many distinct subdomains.
And the above maxim could be applied as well to the the manager who had nominated the project leader...
By the way, looking at the circuit diagram, I additionally understood why the OTA mechanism was not reliable. We'll go through this in a next article.