thoughtwisps One commit at a time

Hello and welcome to thoughtwisps! This is a personal collection of notes and thoughts on software engineering, machine learning and the technology industry and community. For my professional website, please see race-conditions. Thank you for visiting!

more services, more logging, more problems

The better part of the day was spent, in what one can only describe, as end to end JSON message tracing hell. I am currently working on a platform where several individual components communicate via JSON messages. These messages are produced, ingested, reprocessed and placed back onto messaging streams for downstream consumers. Each components runs as an independent service. Early in the week, we detected several problems in one of the end to end flows of data and started trying to find out what things could be done to unb0rk the process. The resolution of the bug is not terribly interesting (mostly some fixing in a few places in the Python glue code that keeps the whole thing together), but as an experience, debugging an end to end flow in a microservice-y type of environment is definitely something everyone should do at least once (perhaps even if your stack is a monolith). You quickly learn to appreciate the importance of writing good (informative, not spammy) log messages and having a log aggregator that allows you to filter by regex, keyword and date range. I can’t even imagine what it’s like to trace a single event across 10s or 100s of microservices.

from 0 to internet trash mob in 280 chars

I had a bad interaction with someone I consider a role model, so I guess this is salty Sunday thoughts, but here goes anyway.

Goodbye for now, women-in-tech Twitter community. One blog post and a whole community is getting torn apart by 280 character hot takes. When did we lose the ability to talk about things like adults? Why can we not take criticism and disagreement and use it to become better as a community? I guess when we figured out that snide subtweets generate more engagement than direct conversation.

It’s been a good run.


Two people, two senior leaders in technology whose work I follow and respect, recently had an inflamed public exchange on Twitter, which made my heart sink. The exchange was about a piece of advice that one of them had published for women in technology - ‘women in tech’ as the movement is usually known. I read the post, not once, but several times. I didn’t agree with all of it. I thought some of it was problematic, the other part practical. It seemed like advice that actually acknowledges that the real working world is not always a nice, fair, just place and I appreciated it for its candor. But advice is just that: advice. It’s not a how-to manual or a guaranteed way to succeed in an industry. It’s often based on the author’s own experiences or the experiences of those she has worked with or mentored. It won’t work for everyone and every life situation.

Although the media is infatuated with the narrative of the genius, Harvard drop-out techie founder, who loves hoodies, raw water and waxing lyrical about the upcoming tech brotopia, there are other stories out there, other ways of being in tech and being successful. There is no one guaranteed way to make it as a ‘woman in tech’- three words which are problematic, because we are people before we are a gender, especially before we are a society’s idea of how our gender should act and behave in the world. This is not to say that gender does not affect our lives - it does, sometimes with extremely devastating consequences.

I suppose what I’m trying to say is that every woman’s tech career will be different and there is no advice that will work for everyone.

I wanted to share some of these half-baked thoughts on the platform and initially retweeted a few posts and commented, but after a while I gave up and deleted the lot. Twitter is ideal for half-baked steaming hot takes, but this is one fire I don’t want to throw more kindling on. I don’t know what value I can add beyond restating what other people have already said. It reminds of the time another prominent developer (who happens to be a woman) criticised some women in tech groups. There was lots of unnecessary hand-wringing and hot takes. But also a few well-measured responses. If we as ‘women in tech’ can’t take a critical look at our movement and its aims and can’t tolerate scrutiny from people who don’t agree with our goals, then what are we really doing.

Also, I’m just tired of talking about this. Actions speak louder than tweets or words. So that’s what I’m going to focus on.

notes on a conversation with the nyt

The acronym - NYT - means ‘now’ in Finnish and that’s how I’ve come to think of this digital place - the ‘now’. It’s usually the first thing my trigger happy muscle memory punches into the URL bar once the morning caffeine kickstarts the neurons and sometimes the last thing I scroll before I call it a day in the digital universe.

A long time ago, on a Finnish forum, someone lauded the visual design of the NYT website - the old school page that looks like someone simply lifted the front page of a newspaper and crammed it into bits of JS, CSS and HTML- one of the best visual designs ever. The poor guy was flamewared out of the thread. I do remember thinking how boring all that black and white was. Why not add some geocities colour schemes while you’re at it?

Of course, what did I know. At this point, the most advanced web development my friends and I were doing was pasting source code snippets from other websites to create virtual kennels and stables -remember those? Yeah. This makes me sound like some relic in the museum of the internet and indeed, at times I do feel like one. In fact, I still remember the time before the search engine monopolies, because for school assignments my classmates and I collected lists of various search engines and what was best for what type of searching. I still remember that Vivisimo - the search engine aggregator was a thing.

But back to the black and white. Tonight, I attended a WWC London meetup to hear from the tech and digital teams at the New York Times. The crowd was treated to visually stunning presentations from the T Brand team and to some powerful (black and white - just like truth ought to be?) marketing campaigns that were launched last year and this year. Two years that the microhistorians of my time will surely pin as the two most troubling times to run a major US newspaper. Among the presentations was a walkthrough of an investigation into a phishing campaign, an interview with the NYT interactive team about coding live websites to track results from the Olympics and a conversation with the NYT CTO Nick Rockwell.

Someone from the audience asked about ‘boring vs exciting technologies’, which was a nod to the famous Etsy blog post that makes the case for building tech stacks out of conventional, well tested technologies that have known unknowns. Rockwell answers that the NYT team does a bit of both, but that letting smart, capable people experiment with new tech is a key in keeping them engaged. I can see the merits in the views of Rockwell and the Etsy engineers who advocate for boring tech. It’s fun to play with new languages and frameworks, but sometimes I feel that the tech industry as a whole is too in love with what Martin Thompson, in his GOTO Copenhagen 2017 keynote ‘Engineering You’, calls ‘intellectual masturbation’ - trying out new ways of solving the same problems without going back to the basics. Sometimes this results in people trying to cajole every single problem a customer or a client has into something that can be engineered using the developer’s favourite framework without properly understanding what the original problem was. First look at the problem, its constraints, its end users and operators, then choose the tool that optimizes for a good experience for as many of these stakeholders as possible. In the end it’s always about making tradeoffs, but I know that I would trade ‘having fun with a new framework’ for ‘I know exactly how this thing I developed behaves when prod stack is on fire’ any day, especially the day when I am on early morning support.

2017 - in review

I am here where my time began. Not exactly (that place no longer exists, not in the way it did when I was young), but close enough. Up until this year, I had a firm conviction that I could map out my time from start to point X (some yet undefined pinnacle of whatever weird thing is considered success in our society) and then follow that map, carefully executing each step from point A to point B until I reach X. It didn’t happen that way. I didn’t make it through a lot of things, the trajectory that I so optimistically drew as upward slanting, tanked. I have failed, a lot (which is not to say, all failure is bad!). And this year, I have failed more than ever and learned more than ever. In fact, even the X I set out for myself has shifted. So here I am, with my failures and perhaps a few small successes, saying goodbye to 2017 and welcoming 2018 and all its challenges!


London is cold and quiet and most days the fog from the Thames is a milky white shroud. I’ve parked work on some side projects and am focusing on deploying a tricky component on AWS for a client. I’m also searching for “Austin, TX weather”, because I have been fortunate enough to receive a Cloud Native Foundation Diversity Scholarship to attend KubeCon and CloudNativeCon in Austin! Thank you very much to Ms. Wendi West from the Linux Foundation who was patient in helping me navigate travel arrangements and a huge thank you to Erica von Buelow (engineer at CoreOs), Kris Nova (developer at Heptio and also, badass mountaineer - just see her Twitter feed!), Jessie Frazelle (container engineer at Microsoft and all things Linux) and Michelle Noorali (Kubernetes exper from Microsoft) who made it possible for 103 diversity scholarship attendees to fly to Austin for a great week! I learned a lot, met a lot of interesting people, learned about containers, orchestration, what it means to be Cloud Native and lots more, which I’m saving for a proper blogpost.


One month into my new role as a independent software engineer. New team, new way of working. I’m still learning the ins and outs of collaborating with a fully distributed team. Thus far I can say that working remotely suits me very well. I, along with many of my developer colleagues, find open offices distracting. I appreciate that there are legitimate cases where teams are more productive when all barriers for communication are removed, for example ops and customer facing teams, but for feature teams, this often just does not work. The whole month passes in a bit of a blur - there are hardly any notes in my diaries.


I started my first ever job as an independent software engineer, which was a big (albeit slightly unplanned) milestone in my development career. Along with getting used to working fully remotely and configuring all of the ropes for my new consulting company, I’m procrastinating and preparing to give a talk at PyCon UK 2017. The topic is machine learning security - a newly formed field that blends aspects of infosec and ML engineering. I discover lots of gems while making my slides: self-driving cars that can be confused by salt circles, neural network image classifiers that misclassify pandas as gibbons and June, the oven that is connected to a GPU so it can use machine learning to find the optimal temperature for your cupcakes. I make the trip to Cardiff and spend a day holed into my hotel room practicing my talk over and over again. In the evening, the evening of a great boxing match in the Cardiff stadium, the streets are the colour of flashing bright blue and red and the shadows of hundreds of people walking across the streets to see the match. I walk past them and the castle and look into the glass eyes of the stone animal statues.


I attend Container Camp London and learn a bit about container security from Andrew Martin and Ben Hall using Katakoda. I realise that things move quickly in the infra & ops space. The difference in talks between this year and last year is staggering: all of a sudden a whole new vocabulary has evolved for discussing platform architecture - service mesh, container orchestration, managing state and secrets and so forth. Jess Fraz and her amicontained tool gets a shoutout in the closing hacking containers finale - all jokes and super long and frightening bash scripts courtesy of Andrew Martin and Ben Hall. I think I might have gone home and tried some of the techniques on, but unsuccessfully.

Rome and all of its delights, but most of all the canoli and the coffee. I finally saw the Ecstasy of St. Teresa at the Santa Maria della Vittoria and the Pieta. On a group tour of the Vatican, I stood next to an elderly American lady who was visiting Rome with her extended family. In the Sistine chapel, ushered in by the neverending flow of the crowds and shushed several times by a policeman standing by the microphone, we smiled at each other and looked at the frescoes. I remember the heat of the summer that seeped through the doors and the windows and made the air inside heavy or perhaps it was the collective warmth from the bodies of hundreds of tourists gathered inside. The sussuration of hundreds of people wrinkles the silence and I wonder, what is it like to inhabit this space alone.


I’m not in a good place and trying to reconcile my decision to leave a development job I had wanted for a long time with the challenges that I faced. The one good thing about the whole situation is that I can finally enjoy the sun and the Thames.


I read a lot.


I go to Borrowdale and attempt to hike Skiddaw. My map reading skills being what they are, I end up on the summit of Little Man. I stand next to the trig point and puzzle over the map while the fog and clouds swirl around me and conclude that I must have reached Skiddaw only to descend and realise that the summit is further away. In the end, I make an attempt at Skiddaw, but give up due to poor visibility. Until next time, Skiddaw! I descend past the summit and Sale How to the Skiddaw YHA hostel and then follow the Cumbria Way into the valley. The cloud cover disperses and the most brilliant beautiful sunlight illuminates the valley and the mountain streams running through the veins of the mountain. I’ve never felt so alive and at peace.


Working and doubting some of my choices.


I summit Snowdon using the Llanberis Way.


A brief visit home to enjoy the snow and then back to work.


Promotions. I recall a Hacker News comment about prestige that Dan Luu has kindly preserved for posterity.