Posts tagged as aiux

I was working up to the wire in 2024; my main client project wrapped up on December 20th. It was an intense sprint finish to the year, and since then, after some genuine rest, I’ve mainly been getting my feet back under the desk, and slowly trying to bring the shape of 2025 into focus.

What was I up to in the past six-ish months?

Google Deepmind - further prototyping / exploration

I returned to the AIUX team at Deepmind for another stint on the project we’d been working on (see: Summer 2024 Worknotes), taking us from September through to the end of the year. This was meaty, intense, and involving, but I’m pleased to where it got to.

This time, the focus was in two areas: replatforming the project onto some better foundations, and then using those new foundations to facilitate exploring some new concepts.

This went pretty well, and I got to spend some good time with Drizzle (now in a much better place than when I first used it a long while ago; rather delightful to work with) and pgvector. Highly enjoyable joining vector-based searches onto a relational model, all in the same database.

And I think that’s all I can say about that. Really good stuff.

Poem/1 - further firmware development

I continued occasionally work on the firmware development of Poem/1 with Matt Webb. The firmware was in a good place, and my work focused on formatting and fettling the rendering code, and iterating on some of the core code we’d worked on earlier in the year.

16nx

In September, I released a completely new version of the open-source MIDI controller I make. It has no new functionality. Instead, it solves the more pressing problem that you could no longer make a 16n, because the development board it was based on is no longer available. I ported everything to RP2040, and designed it around a single board with all componentry pre-assembled on it. I’ve got a case study of this coming next week, examining what I did and why, but for now, the updated 16nx site has more details on it. Another full-stack hardware project, with equal parts CAD, schematic design, layout, and C++.

Ongoing consulting

I kicked off some consulting work on a geospatial games project, currently acting as a technical advisor (particularly focusing on tooling around geodata), and this should continue into 2025. The way we’re working at the moment is with longer meetings/workshops based around rich briefs: the client assembles their brief / queries as a team, I prep some responses, present back to them, and that opens up future discussion. It’s a nice shape of work, focusing on expertise rather than delivery, and makes a good balance with some of the more hands-on tasks I have going on.

---

Coming up

2025 is, currently, quiet; slightly deliberately so, but I’m eyeing up what’s next. I’m back teaching for a morning a week at CCI from mid-February; I have further brief consultancy with the games project on the slate; there are few other possibilities on the horizon. Plus, after a crunchy end-of-year on the delivery front, I’ve got a lot of admin to do, and ongoing personal research.

But that’s all “small stuff”.

Like clockwork, here is where I say “I’m always looking for what’s next”. You can email me if you have some ideas. Things I’m into: R&D, prototyping, zero-to-one, answering the question “what if”, interactions between hardware and software, making tools for others to use, applying mundane technology to interesting problems. Perhaps that’s a fit for your work. If so: get in touch.

Coming up for air.

What happened is: I lined up the Next Thing (as mentioned at the end of last quarter’s worknotes), and then it promptly proceeded to entirely consume my time and brain for the next quarter.

Which is good, from an income-and-labour perspective, but was somewhat to the detriment of the content strategy here, where I’d hoped to be able to write smaller, spikier pieces of content between the studio updates.

So a goal for this quarter: making sure I don’t let client work overwhelm the routines I’d like to establish.

What’s been going on for the past few months?

Google Research/Deepmind AIUX - prototyping and exploration

The previous Next Thing.

Just as the quarter was beginning, I lined up a short, intense prototyping project with the crew at AIUX that I’d previously worked with. They began the project within Google Research, but by mid-project had been reorganised into Google Deepmind.

This made almost no difference to our work, though it did make the “ooh” people make when you tell them who your current client is go up a semitone more than usual (before you tell them you can’t tell them anything else owing to confidentiality).

I spent a couple of months with a motley crew of internal and external folks, pulling together an interesting, full-stack working prototype. Designing interactions, poking technology to see what it feels like, working out how to communicate the new ideas that a piece of software makes possible.

I think all I can say about what we were up to is “stuff to do with vector embeddings”.

The stakeholder feedback at the end of the project was really enthusiastic, and there’s talk of further work on this brief. Very much the right project at the right time.

Creative Computing Institute - end of term

June marked the end of term at CCI, where I wrapped up my fifth year of teaching Sound and Image Processing to first year undergraduates - and which means shortly after I’ve breathed out from the end of class, I have to mark 31 portfolios. It’s a lot to get through, but I’m always surprised and delighted by the places some students will take me, and it’s those moments that keep me going as I work through reading and executing a lot of Processing.

When I tell people I’ve been marking, they often ask about the impact of ChatGPT on my results. All I can say is: yes, we are educating in the age of ChatGPT now. (Other LLMs are available; I’m using ChatGPT as shorthand for LLM chatbots you ask for answers).

We don’t have a problem with students using LLMs; they’re a tool like any other, and for students less confident in a second (or third+) language, or students newer to programming, they can be a confidence-booster. We ask students to cite them just like any other source. (I usually teach citation by suggesting you quote as little as possible, and then, if you are uncomfortable revealing the size of your quotation, perhaps that’s an indication for you to quote less.)

Despite this, I believe I had many students who used ChatGPT and didn’t cite it. You can’t prove it, but there’s a… vibe? Smell? It never feels like plagiarism or copying; it’s more like a number of students all sharing the same tutor, whose personal style and predilections keeps appearing in their work. The programming equivalent of delvish, perhaps. (I had lots of (int) style casting this year, for instance, which seemed like a tic given I’d never really introduced it, and Processing’s built-in int() function is probably more intuitive).

But there are also thoughtful uses of these tools. I’ve had students submit entire transcripts of their conversations with ChatGPT, for instance, which has been good: they showed they had the knowledge necessary to ask meaningful questions and build on them, and showed how they refined their own knowledge in the dialogue. I’ve also had students describe processes - one talked about submitting his code to ChatGPT and then asking how would you improve this? which is an interesting approach to virtual pairing/refactoring, and shows a good degree of insight - not to mention curiosity.

With marking done, I rounded out the year by going to the student festival, the end-of-year show of work. This was the second year where students I’d taught in their first undergraduate year were graduating. It was hugely enjoyable and rewarding to see how they have developed, and how they are expressing their ideas and interests in creative computation a few years down the line.

Lunar - hardware/software refinements

I’ve just kicked off a very short engagement with Lunar - some further refinements to the LED interaction tools work that we did. Some new hardware, and a revision to both firmware and browser tooling has made for a busy week back leaping between C++ and Typescript.

What’s next?

The next few months are coming into focus now. August looks a little less frantic, with some smaller projects slotting into the schedule alongside a deliberate amount of Slack, and there are rumblings of larger work in September that could keep me busy for several months.

Meanwhile there’s admin to be done, and some plates to keep spinning, as well as making sure I return to some Process Notes here. And so: onwards!