Latest work case studies, summer 2019 edition11 July 2019
The case studies on this website were getting a little stale. No more! I’ve just published lots of new case studies of individual projects over the past three years.
The big headline that I’m most keen to talk about is a long, detailed writeup of my work on CaptionHub - a project I worked on for 3.5 years, known in this feed as Selworthy. CaptionHub is an online tool for collaboratively captioning and subtitling video. I served as technical lead and pathfinder, taking the initial idea - the “what if?” - to a prototype and beyond into a shipping product, whilst the team grew and the product acquired clients. The write-up is detailed not just because of the length time I worked on the project, but because of the way the product changed as it developed and grew in scale. It’s a project that shows the breadth of my capabilities well, and the finished product is something I’m very proud of.
But there’s lots more in there too. Highlights include: an open source tool for musicians; teaching on the Hyper Island MA; building a digital musicbox; creating a Twitter bot for an installation at the Wellcome Collection.
The write-ups all include extended thought on process, and, of course, link back to the relevant weeknotes that I wrote during the process.
You can find out more at the updated projects list.
I’m currently looking for new projects to work on: technical leadership, early stage exploration, communication of ideas, are all areas I’m keen to continue in. Topic areas I’m particularly interested in include building tools for creatives and professionals, the bridge between the physical and digital, and audio and video. I’ve written more about my capabilities here.
Weeks 165-1663 February 2016
Twinklr is moving on very nicely. At the end of this fortnight, I wrapped up a first pass of rewriting the existing code. The music box now happily plays in a wide variety of scales, with any note selectable as root: that makes it much easier to play in key with other musicians, or to improvise around existing pieces of music. It already makes the whole thing much more instrumenty.
I also worked together a simple save/load mechanism with eight memories: the instrument stores not only the position of notes, but also the scale, root, and length of the stave, maing it easy to restore a state you once had.
And finally, at the end of this three-week period, I rewrote my code for spitting MIDI out of the device. There’s now a simple-to-use MIDI preference panel, making it easy to reload available inputs, alter the length of notes that you’ll send out, and a ‘panic’ button to reset all output. It’s really coming together.
Selworthy is being tightened into place for a new deployment: that means shoring up some minor issues, and lots of little improvements based upon user feedback. I’m also overhauling the code to make it easier to bring other developers into the project.
I spent a quick hour of lunch assembling the latest revision of the Walton PCB and we finally have a fully functioning version. I’m thinking about replacing one of the components – there’s some mechanical tolerance I’m not happy with – but now the whole shebang is working, it’s worth working out what to do next with the project: is it something to keep to myself? To open source? To offer for purchase as a kit? I might talk to some people about it shortly.
And finally, I spent some time tinkering with some electronics as a spin-off from Twinklr: exploring an aspect of its interactions and translating them to a slightly different environment. That involved first just breadboarding up some components to discover how they worked; this went surprisingly well, and I think I’m going to keep pursuing this angle of the project in odd moments.
Weeks 161-1624 January 2016
Last weeknotes for 2015!
The end of the year isn’t always quiet for me: I’m often pounding on until the end; may as well make use of all that time. And in some ways, this year was no different. There was various client work to wrap up, and remaining time to get Twinklr underway.
I added one new feature to Selworthy, altering the domain model to make it easier for the standalone version to handle multiple clients with different needs. Didn’t take long, but will definitely make it more useful even for internal usage. The tool is beginning to see internal usage in anger, which is great: the client are finding important issues much faster as a result, but are also seeing marked productivity gains that we hoped it’d lead to. As a developer, that’s exciting and rewarding.
I went over to Brixton to install Roshill for Oubliette: getting the electronics from the workbench into the room and the final enclosures. Mike and Dave gave me a good hand with this and, after some hairy installation moments, we ended up with a working puzzle. A quick board of pull-down resistors I’d built up beforehand made all the difference.
New PCBs for Walton arrived. I’d hoped these would fix the bug and in one sense, they did – but another bug still remained. I’ve now sussed what it is, though, and the only way around it is Ordering The Right Part. So I’ve got some 3PDT switches coming from Germany – along with some more equipment for the workbench – and we should have a final revision in early January.
I got inducted on the laser cutter at Makerversity, and put it to use. First, I built a test front panel for Walton; the good news is my new workflow for doing so works, and everything fitted. The engraving still needs some work, but I think it’s definitely going to be a goer for prototypes.
I also spent some time beginning to try fabricating some useful odds and sods for Twinklr. At the end of the year, I designed a test jig in Fusion 360 – in 3D – and then worked out how to bring it down to cuttable flat pieces, which I then produced. A big leap for me: CAD is not my natural environment, but now I can see how that workflow might work, and I know that one more thing in the project will be doable.
I rewrote the Twinklr software. Already? Well, it turned out that the Raspberry Pi wasn’t hugely happy with endless 800*480 Canvas rendering, so I rebuilt the code to render in SVG. This had lots of neat side effects: it makes a few new feature we’re interested in much easier, and it begins the ‘tidying’ of the code to make it easier to work on. It also makes a few other things easier – in particular, animation – which is always good.
On a less successful note, I spent a long while wrestling with Raspbian Jessie and USB audio, and am coming to the conclusion that before I spend any longer on it, I’m going to start again on Wheezy. The Pi is a frustratingly… idiosyncratic beast. I’m considering other options for Twinklr – not very seriously, given how suitable the Pi+Touchscreen combo is – but a day of messing around with Linux audio config really did make “not very” become a bit more seriously.
As ever, more Twinklr notes – and scrapbooking – are on the Twinklr Notebook.
And that was 2015. Yearnotes to come likely, but in the meantime: happy new year.
Weeks 159-16016 December 2015
The penultimate weeknotes of the year – and late they are, owing to end-of-year busyness. But nontheless, what happened?
Selworthy continued with its lick of polish: I started tidying up some UI interactions, and tidying some features. I also got slightly sidetracked with an issue with Redis on one of our staging boxes, which ate some time, but was swiftly resolved thanks to the investment I’d put into Ansible earlier in the year. So that was a relief.
Roshill, the puzzle I’m building for Oubliette, moved forward quite a lot: we integrated the mechanical components (primarily stepper motors) and I debugged most of the electronic issues – much of which came down to needing some pull-downs, and ensuring the security of various connections. It’s nearly completed, and in the final weeks of the year we’ll install it in the space.
I spent a couple of days working on exploring some data and sketching in code with George over at Good, Form and Spectacle. An interesting few days; some careful exploration of a relatively small dataset led to insight from even the smallest of our interventions, and it was great to work with George again.
MV Works kicked off with the chance to meet the rest of the cohort and get a feel for the space and the project. Fun to meet so many people properly, and various plans have already started forming. I ordered some initial components to test my new hardware plans with, and also began planning an overhaul of the code. Oh, and I kicked off a Tumblr notebook for the project, to act as both a scrapbook of things we’re interested in and a place to store work in progress. I’ll no doubt continue to write weeknotes around it, but it’s also nice to just have a rolling scrapbook to post to.
Tangentially, I had an interesting Skype conversation with Dean Taylor. Dean wanted to sound out ideas around interaction design and sound production, and we ended up having a really interesting chat about all manner of things – but in particular, what I’ve taken to calling instrumentness; it was really handy as a way of framing my thoughts for Twinklr a little.
And finally: a studio move to fit around all that. I’m no longer in East London, but based out of Makerversity in Somerset House, just off the Strand. (I’m still with studiomates PAN, though). It’s a great space, and has wonderful facilities and people; looking forward to spending time there.
Joining the MV Works 2015-16 cohort4 December 2015
I’m pleased to announce Richard Birkin and I have been selected as part of the mv.works 2015-2016 cohort. We’ve received funding to work until April 2016 on a new iteration of Twinklr – our physical/digital music box.
We’re going to spend the time iterating on the hardware, software, and enclosure, adding functionality and hopefully making the build more repeatable. Along the way, we’ll be documenting our progress, and exploring the opportunities the object – the instrument – affords.
We’re really excited – this news comes just as we wrap up the first phase of exploratory work. It’s a hugely exciting opportunity – the space to design an instrument for performance and composition, to build an object to create with, and to develop this idea that’s been percolating slowly. Of course, we’ll share all our progress along the way. It’s going to be good.
Weeks 157-15824 November 2015
Two more weeks have pasted, which means two more weeks of contiguous weeknotes to write up.
I finished my first pass at Roshill – a puzzle I’m building for Oubliette Escape Rooms. Dave provided a second frontpanel for it, after the first had a few holes cut too small, and this one was much better. My new approach to wiring paid off, and by the end of Week 157 I’d wired the puzzle together and written most of the microcontoller code. I also met the mechanical engineer working on another critical part of the prop, and we worked out how we could piece our work together. There’s a second pass of work to come, but everything’s in a really good state, and it’s nicely maintainable – very easy to swap out individual components if they fail, thanks to an awful lot of spade connectors I crimped on.
My PCBs I ordered for Walton arrived, so I set about assembling one. The good news is that the PCB broadly functions correctly: I’d read the datasheets correctly, and the custom parts I’d had to make from scratch in EAGLE were all entirely correct. However, there was a significant flaw in the circuit (mainly to do with how negative voltage behaves) and so I’m going to need to take a second pass; I made my learning smile and moved on. Fortunately, some debugging (ably assisted with insight from James) solved the main issue.
I also took the opportunity to lasercut another frontpanel for that project. This too had some issues, largely down to my design process. I’ve resolved those by finding a workflow to get from EAGLE to SVG – namely, using the handy gerber2svg, which does exactly what you might think, and has the added bonus of being a command-lne tool. That’s all helped me have 100% confidence in the frontpanels from here on out. By the end of week 158, I had a second design of the PCB with the error corrected that was just about ready to send off for manufacture – although I have a sneaky feeling some last-minute tweaks (or even functionality) might be added before then.
Selworthy is gearing up to be handed over to new developers and deployed; I’ve been reviewing how some of its dependencies are packaged, and building out a secondary command-line tool to support it.
Most excitingly, I went up to Richard’s studio in Belper for a day of playing with Twinklr, filming and photographing it to document the end of the first phase of the project. It was by all measures a success; we got done what needed doing, but, more importantly, we lost ourselves in it. It was a joy to watch Richard just playing this instrument we’d built. After we had fixed a few minor issues and demonstrated basic functionality, we then plugged Twinklr into an analogue synthesizer I’d taken up, and thence into a long effects chain. And then I just watched as Richard explored and composed, writing patterns and melodies on Twinklr that played out on the synth, and the time just passed. A really good couple of days; I’m looking forward to sharing this project more properly with you in the immediate future.
Weeks 154-15526 October 2015
A fragmented fortnight, but lots of things moving in a variety of directions:
- Selworthy is in a holding pattern whilst we wait for a few pieces beyond my control to fall into place.
- Twinklr is pretty much at the end of its initial phase – some painting, staining, and filming to do – but there’s a possible future for it that I’ve been doing some work applying for.
- Roshill and Walton have had me staring into the maw of electronic component retailers: how many thousands of flick-switches are available? Now, it turns out, I know. I’ve been testing a variety of switches for Roshill, and Walton’s seen me staring at EAGLE, moving many things around a PCB as I test a variety of ideas.
- Lots of meetings – potential future work, catching up with peers; all very positive, and lots of things to chew over.
- And, to cap it all, I spent a pile of week 155 just being ill at home; much-needed bed rest made me functional again pretty fast, and saved the studio my sniffling.
Walton and Roshill will perhaps become more evident over time, but for now, they’re going to keep me ticking over. So whilst it was an interesting fortnight for me, there’s not a vast amount to report publicly. Hopefully that’ll change soon!
Week 152-1535 October 2015
Week 152 saw my return to the studio after a week off, which was an excellent tonic after a busy August.
I spent much of the fortnight building a standalone environment for Selworthy. It was initially configured for an internal environment provided by one of its end-users, but my client wanted their own install to be able to demonstrate it and use it themselves.
I did this with Ansible. It’s a tool I didn’t know well at the beginning of the fortnight, but by the end, it had already proved its value several times over.
Ansible is a tool to help provision servers. It allows me to write small rules to describe the configuration of a server component, that will be idempotent – I can run them as many times as I’d like, and they either bring the server up to that configuration, or do nothing.
To begin with, it felt quite slow: prodding at syntax until the behaviour I want has been applied. I built the environment by building and destroying servers repeatedly, continuing to test that it’d work from a standing start. And each time I do that, the fact that all previous steps just work immediately becomes a huge advantage.
The other advantage is that Selworthy actually consists of two separate application – the main application, and a media-encoding tool – which have fairly similar system requirements. I built the Ansible playbooks for the media encoder first – which took a bit under a week of work. Then, however, it took about an hour to provision the main application server – because so much of the work had already been done and could be reapplied. It’s useful to be able to provision boxes so fast, and I’m definitely going to be continuing to use Ansible on future projects. The value isn’t just in the repeatability: it’s also in the ease of sharing it with other potential developers – guaranteeing that they can build the same environments as you. I’m not a great sysadmin, and it’s never going to be the main focus of my work, but the more controlled the environment I’m deploying to, the more confident I can be in it – and that’s a huge advantage.
By the end of the work, I could build the entire environment for Scribe in about ten minutes, from a single command, and then deploy each application to it in another single command. I was pretty pleased with that.
Around that work, I fitted a variety of other small pieces of work. I had an excellent chat with Holly Gramazio over coffee one morning, where she told me how Now Play This had gone, and we had a long conversation about the curation of games. It’s always valuable as a freelancer to be able to chat to colleagues, especially those at the edges of some of the spaces I work in, and Holly’s always a delight to talk with; nice to get out of the software for a morning, and hear about other friends' successes.
I spent an afternoon at the Polyphonic Playground Lab, watching how musicians worked with the playground, and thinking about ways to potentially involve a much wider audience as part of the project. It was also a good opportunity to think about the intersection of music, sound and interaction design, which I’m thinking about a lot at the moment. That overlapped with the TouchOSC controller for Panoramical that I built the weekend before, as a small personal piece of tinkering.
Finally, I spent some time chatting with Richard about bringing Twinklr into land. We’ve got a plan of how to do that, which should fit around our life in the next few weeks, and an idea of where we’d like to take it next. As part of that exploration, I kicked off an attempt to rebuild it in Unity. As ever when porting code from something you know well to something less well, it can feel painfully slow at times, and as a result, I’ve only got code for drawing staves at the moment – but it’s a beginning, and something to build on. I’ve been thinking about a Unity build simply to make it easier to iterate on the real-time aspects of the code, and especially to consider more complex interactions that will become harder and harder to write in the browser.
Oh – and I took the opportunity, nearly six months on, to finally write up Rubato, another project Richard and I worked on together. As usual, project write-ups are always slower than I anticipate (and I still carve out a fair chunk of time for them), but I’m pleased with how that turned out.
Weeks 148-1491 September 2015
Still got my head down, which means we’re still on fortnightnotes for the time being.
Weeks 148 and 149 are the usual story: a variety of interesting meetings, keeping the Twinklr ball kicked up into the air, and taking Selworthy well past the inner marker on its landing path.
Twinklr’s firmly in Richard’s hands – he sent me some photos of a rough jig for the enclosure, which definitely felt right to me, so I’m looking forward to seeing how that moves on. I spent some time thinking about a particular possible spin-off for the project, which was an exciting change of direction.
I spent much of the fortnight deploying Selworthy into its new environment. That’s involved integrating with external authentication providers, collaborating with other developers on the configuration of its environment, and debugging issues with particular versions of libraries and packages.
Needless to say, all quite challenging. There’s a lot of context switching, between server administration, deployment, researching new versions of packages, and then writing application code to reflect those changes. Each context shift takes me a while to recover from, and so I’ve been working on my tenacity.
By the end of the fortnight, many, many moving pieces were settling into their final positions, with only the last bit of fettling left to go. Sometimes, this is what the work looks like – just pushing everything a little bit further forward.
Weeks 146-14718 August 2015
A busy fortnight, bringing Twinklr to life, and pushing Selworthy along its landing approach.
Selworthy continues on its flight path, roughly in line with our plans and projections. The current work is to re-architect the project a little, so that it can live entirely inside a firewalled internal environment. Initially, we built it using web-based services, and so we’re slowly removing those. The main work there is building a secondary application to handle media encoding and processing, replacing Transloadit.
As a result, I’ve spent much of the past few weeks neck-deep in Sidekiq and Redis, to build asynchronous queues; in Vagrant, building up virtual machines to host the new architecture (in order both co get a better understanding of the live environment, and to take advantage of XSendfile); overhauling lots of tests, and finding out about the finer details of CentOS (as I’m usually a Ubuntu user). Lots to take on board.
And sometimes, that means diverging from the shortest route between two paths. The port to Vagrant began when I needed to serve the app through a real web server; that felt like a detour, and the days spent on it felt like a digression from my critical path. When I’d finished the work, though, it turned out I’d not only achieved the goal I was setting out to, but also derisked some later work. It felt like slowing down; in fact, it saved me time. So I’m reminding myself to trust my own competency: if I’ve made a decision, it’s probably a sensible one, and right now, sensible decisions trump fast ones.
By the end of these two weeks, the project is in a really good place: the new architecture works, is tested, and I’m more confident in it. I have a good map in my head for what’s coming next, too. So: onwards!
Meanwhile, Twinklr made some great progress.
The PCB manufacture I’d been exploring in the past couple of weeks didn’t turn out so well: due to various misunderstandings with the way Fritzing labels files, and some misunderstandings with drillbits, the home-made PCBs weren’t particularly satisfactory. On the plus side: I know they’re possible to build. However, I’ve explored getting some simple boards made – from the same layouts – with Ragworm, and look forward to seeing how they turn out. So despite my own PCB manufacture process not working out so well, it’s been useful to wrap my head around the design process.
Richard came down for an afternoon in Week 147. He brought with him the crank handle, attached to a rotary encoder, and quickly held together in a prototype jig. After some staring at a data-sheet, we wired the encoder into my circuitry, plugged into the tablet, and fired up Twinklr.
What happened next was we lost quite a lot of time in the best possible way. We spent a while just playing with the sequencer, tweaking and adjusting the code to fit the feel of the handle, and seeing what suited it. We spent some time sketching and discussing what the final enclosure will look like – something beautiful and very much like an instrument; I spent a while looking at pictures of the old RCA theremin (in which the instrument was also its own music stand, and designed like a piece of furniture).
I’m also using the Chrome Web MIDI API to let Twinklr control other instruments: we’re interested in how other musicians or performers might adapt it with their own work. I wired it into my Microbrute and we spent a good while seeing how the crank-handle sequencer felt attached to the analogue synthesizer; it turned out to be a great fit for a performative, tactile synth.
It was great to see the project coming together, even if we only spent an afternoon together. Very pleasing to feel it working, and to begin to see the opportunities and edges for what to do next with it. The next immediate step, though, is building it up, and I’ve given Richard all the electronics to explore enclosing it. Good stuff.