Posts tagged as abberley

A busy couple of weeks. These were the last two weeks in the run up to the delivery of the alpha of Swarmize. That meant lots and lots of small things – all the things you tend to remember in the final few weeks.

Firstly, completing and polishing the documentation. This wasn’t a last-minute thing: it’s been something I spent a while on. As well as some decent enough READMEs throughout the git repository – enough to help other Guardian developers – I also focused on delivering a set of case studies.

You can view them all on the Swarmize site. They cover basic usage and form embedding, advanced-usage (with the storage and retrieval APIs), and a real-world case study. I’m particularly pleased with the use of animated GIFs to explain interactive process – not endless instructions on how to perform a mouse gesture, nor a slow video with interminable narration. I think they’re a really useful fit for documentation.

I tested the API which, after I’d sketched it in Ruby, Graham built a more robust version of in Scala – and removed a few wonky features. That all went pretty smoothly. I also spent a while testing the embeddable forms in a variety of browsers, and learning a lot about <a href="https://developer.mozilla.org/en-US/docs/Web/API/Window.postMessage">postMessage</a> as a result of trying to build a smoother experience in mobile browsers.

A few new features also snuck their way in. Dangerous, at the last minute, but they were relatively straightforward to implement and definitely turned out to be worthwhile additions.

And, of course, we spent some time at the Guardian offices demoing it and explaining it. We also got a particularly nice write-up and interview on journalism.co.uk.

I’m hoping to write up my own project page on Swarmize shortly, which will touch a bit more on the process and my involvement (as well as providing a clearer answer than these weeknotes to What It Is).

Some good news regarding Burton: it got some funding. Not a vast amount, but enough to achieve what Richard and I would like to – so that’s going to be a focus before the end of the year. Really looking forward to it.

And, finally, I spent a good while working on my talk for Web Directions South. I think that’s all come together reasonably well. Some of these Things are not like the others is an expansion of “A Lamppost Is A Thing Too“, and covers Hello Lamppost, Columba, and various thinking about designing connected objects and services, expanded into a longer, more wide-ranging 45 minute session.

I hope it goes down well. I’ll no doubt have more to say in week 107 – which is going to be spent in Australia.

Week 103 was, I think, a rather good week.

Swarmize/Abberley is moving into its final month before hitting Alpha. The last piece of the puzzle is some kind of API to extract data from the system. We can already put data in, and we’ve got the tools to download and explore it, but what we really need now is a way for developers to extract data programatically: a retrieval API. One we’ve got that, the whole end-to-end process is sketched out.

To that end, I started sketching out an API as a tiny Sinatra application, that Graham will hopefully port to Scala shortly. Within a day or so of work, I had a simple API that allowed exploring of results, overviews of field counts, as well as outputting GeoJSON FeatureCollections given an appropriate location field to pivot around. I’ve started calling this type of code spelunking: diving into a thing to see how it feels, to learn by doing, and trip over myself soon enough that I understand the real demands of the system.

I also sketched out several example applications in flat HTML and node.js-backed client-server apps to illustrate how the API could be used. It’s not enough to just write that code, though: it also needs to be documented. A lot of the week was spent writing clear, concise API documentation, and I’m going to be tidying up all the documentation over the coming weeks. We’re really focusing on everything that will help The Guardian use Swarmize in anger once the alpha is complete.

Pitches for funding for Burton have gone in. I sat down with Richard, who was passing through London on Monday, and talked over various ideas, which were all exciting and productive, and so we hammered some things out in Google Docs – or, at least, Richard hammered most of it out and I offered input where appropriate. Crossing fingers there, but I think we’ll find a way to do something.

And on Friday morning, I brought the film producers I’m mentoring into the studio to sit down and do some drawing. Not a lot, I’ll admit, but it was good to just start showing the process of using your hands to think with and to slow yourself down – forcing you to reason with how much you can fit on a screen, and precisely what needs to be present for a user to interact with. It was useful, and I’m looking forward to seeing how it’s shaped their thinking next week.

Back to work after a week off.

That meant diving hard into Abberley/Swarmize. We’d run a test of it around the Scottish Independence Referendum, and now I was working out how to re-implement various features so they’d hold up under the strain of a 90k+ dataset; something that Swarmize is likely to have to deal with.

This meant lots of back and forth, but by Friday night, I’d sussed out how to deliver very large CSV files efficiently – and cache them on Cloudfront for the majority of users.

I also overhauled lots of the real-time updates in the app. These are still fairly straightforward: fundamentally, it’s a traditional HTML-based application, but it’s useful for editors to have live updates.

My homegrown caching routines got thrown out in favour of properly implementing a Backbone-based solution. Backbone has always made conceptual sense, but I’ve never seen a way to implement it into my own work. Thanks to a really useful chat with Nat, something large in my head clicked and I finally saw how to use it in a way I was comfortable with. This led to a big rewrite, and now various graphs and tables are updating themselves in a sensible, sane way.

Other work on it this week: spending some time with Graham overhauling our deployment routines, so rather than reduplicating code, we have a single deploy script that (through configuration) can deploy any component; lots of fixes to code and infrastructure.

The flexibility and general scope of my work on Swarmize has been great, but there are times when it makes me feel pulled in a variety of directions – and right now, that’s hitting a little hard. Fortunately, I ended the week with a long chat with Matt, who’s sponsoring the project, and he helped shape what the next four weeks – to Alpha – looks like in my head. That’s helped set a brief for what we’re doing, and I’m looking forward to writing some burndown lists.

Thursday saw my regular mentoring session for CreateInnovate, where we moved some ideas forward, but it became clear that week 3 would be spent with some pens, drawing: getting thinking out of the head and onto paper, to make it clearer.

I also spent a few hours this week sketching, thinking, and prodding the feasibility of a potential new pitch with a friend I’ve wanted to work with for a while. It feels strong enough that it’s got a codename – Burton – and I’m hoping we’ll find a way to bring it into the world before the year’s out.

Week 100 pulled me in three directions, all of which needed a fair amount of attention to complete before a holiday in week 101.

On the Wednesday, I went to Watershed’s Making the City Playable conference, and delivered a slightly tweaked version of Driftwood. I had lots of great conversations, especially during the Open Sessions, and it was a pleasure to catch up with several friends and colleagues I’d not seen for a while. Sadly, I couldn’t stay for the second day.

I spent Thursday wrapping up Ruardean, delivering wireframes, slides, and a technical brief for the digital component of the theatre show. Hopefully it builds on what Bohdan, Ben and I discussed the week before, and will act as firm foundation for what’s to come. They’re running a Scratch at BAC next week, and they’ll talk a bit about the digital component on the Friday night. (I’m away for week 101, hence shipping early).

And there rest of the week was Swarmize/Abberley. I began the week writing siege-like scripts to bombard Swarmize with results, to get a feel for how the site functioned with a more realistic volume of data. This quickly led to some front-end changes, and also provided the foundation for building a first prototype of time-series graphs.

We integrated a Cloudfront CDN set-up, in order to cache stylesheets and (more importantly) CSV files for completed swarms. I overhauled some out-dated backend code, and worked with Graham on a shared piece of data that affected both the Scala and Ruby apps.

Sadly, the Swarm we were going to run in public during week 101 fell through – but at the very end of the week, another opportunity presented itself, and we focused on rigging things up to make it work. We’re really keen to test Swarmize with live data, and as the first live test runs through week 101, it’ll give us a better of understanding of what to design around – and what the editors work with it need.

A good week 100: lots of balls in the air, and now they need to be caught appropriately in the coming weeks.

Week 87 brought some much-needed focus to my calendar for the next few months.

The big news was the confirmation of Lewesdon. This is another project with the Guardian. I’ll be able to talk more about that shortly, but it’s going to be an interesting few months wrangling data and building tools that talk to one another. I’m looking forward to working with the rest of the team, if only for the challenge!

I also confirmed a few other much smaller projects – workshops and the like. They fit neatly around Lewesdon, and will make for a nice change of pace. I also confirmed a speaking engagement towards the end of the year.

And I kicked off a new back-burner project, currently for myself, codenamed Abberley. It’s a very tiny poke at an idea for a product, and the main thing that’s interesting is not the functionality but a few design patterns. I’ve shown a tiny draft of it to people and had some encouragement, so I’m going to keep chipping away in a few meetings.

There was also a final burst of work on Housedon.

Busy, and a bit scattered, then, but the result of the past couple of weeks since returning from SF is that the next few months are much clearer. That feels like a good result, and I’m looking forward to the first few days of sketching a clearer picture of Lewesdon next week.