Lots of things to talk about, plus a story about debugging, because I’ve not talked much about what working looks like.

Not a lot to report from Selworthy: post-deploy, we’re beginning to work out what’s next, corralling future plans, and I think now is a good time to re-examine process. It also looks like I have a few days of sysadminning ahead of me – never my favourite task, but doing just enough to hand it off to somebody else at least means it’s a finite task.

I continued typing away at Cleeve. Only a few big projects left to write up, but there are definitely some second drafts to come.

A couple of phonecalls and meetings mean that there are two new things on the horizon and inches from actualisation. Let’s call them Gummershow (gummers-how, not gummer-show) and Lowick for now. Should have more about these next week.

And over on Longcrag, I continued hacking away at some embedded code and learning a few things on the way. Let’s talk about how that’s going.

My development setup looks like this:

At the top is the Silicon Labs development board. At the bottom is my test harness: rather than having a messy breadboard, I designed and built a simple circuitboard containing my test hardware – a button, a switch, some jack sockets and LEDs, and a pinheader to connect them all up. That can just be put straight into my breadboard as an intermediate between it and the pinouts of the development board. Much easier to work with. (Why isn’t it a ‘shield’? a) It’s cheaper – PCBs are priced by size but also b) I don’t know which pins I want to work with yet. This lets me choose and change my mind).

The code is all running on the Silabs board, and I’m able to debug it over the USB line. Proper watching of expressions and variables here – a huge quality-of-life improvement over printing strings to a serial port.

I had the code working end-to-end last week, and had been compiling it in the ‘debug’ profile. I swapped over to the ‘release’ profile to see what would happen, and the answer is: a bunch of things broke in unexpected ways. After eyeballing my code, I altered my approach to debouncing, but no dice.

I decided to stop using my eyeballs and start using the right tool for the job – so I fired up the oscilloscope. One pin in particular, which ought to light an LED for 50ms, was blinking imperceptibly. I used the scope to time it. In ‘debug’, it was blinking for 50ms; in ‘release’, it was blinking on for 390µs or so. Not what I had in mind.

I continued poking to no avail; I ended up wiring a pin into the oscilloscope and using it to debug all manner of variables. The system clock was running correctly; all the timing code was behaving correctly; evidently the error was somewhere else.

I decided to look at something else – another output that ought to toggle on button presses and would, unreliable. I discovered that it was not staying lit but bouncing, imperceptibly – a tiny square wave of pulses before it settled into a state. And the scope told me those pulses were 390µs wide.

And that was the key that cracked it.

The bounce indicated that the variables was altering, but far too fast – it was being changed in the main loop, when it really ought to have been changed in the system tick. That was a mistake – when I corrected it, things started working correctly.

The real key to understanding the problem, though, was the duration – 390µs – being exactly the same as the burst on the other LED. It turned out that in the ‘debug’ profile, when the board runs slower owing to all the other things it’s doing, the main loop was taking more than 1ms, and so the problem never emerged. In the debug profile, the main loop was taking less than 1ms – probably around 390µs, in fact – and the problem reared its head.

It took what felt like an absurd amount of time to solve this problem – about a day and a half, I reckon, over this week and the previous one. But when I solved it, I felt satisfied: I’d gone from ‘trying everything’ to real diagnosis – and debugging software on an oscilloscope. (It also reminded me about the value for the right tool for the job. Sometimes, I wonder if the scope is a bit of an indulgence – and every time it turns out to be exactly what I need, I am super grateful for it).

Embedded code, eh. I’m hitting a point with this where I think it might be time to think about building a full prototype. Erk.