April 13, 2019

Reckless spending.

Buttondown has a new Twitter handle, which also marks my first (and, if I were to bet on it, last) foray into social media account purchasing.

I had had @buttondownemail for a while and it was, you know, fine. The success or failure of an enterprise is not determined by the concision of its social media identifier, you know? But, because I am foolish, I wanted @buttondown, and had a lingering to-do item to try and track down the owner.

The account appeared to be in use, but it was hard to tell: there was a name ("D. Laackeman"), a location ("Vermont"), and a handful of followers and following behind a protected account. I did some sleuthing, found someone who matched the profile on LinkedIn, and slid into their proverbial DMs.

We agreed on a price of $1,000, which took the form of a tax-deductible donation to the charity of the seller's choice. This was pretty much ideal: I get a bit of aloe on the sting of spending a thousand dollars on a vanity handle.

The sticker price reminded me of another incident of foolish spending that I didn't have the immediate acuity to recognize as such:

Spending two thousand dollars on various email enrichment APIs over the course of 2018 was...rougher. That was me spending money because I didn't think about what users actually wanted or about unit economics. At least with a Twitter handle, I can be a little more honest with myself about how truly vacuous the whole thing is.

Traffic spikes.

I got to spend my morning dealing with Heroku scale-ups. A couple miscellaneous notes:

• I received a Pingdom alert that Buttondown was getting the "hug of death" precisely as soon as someone emailed me about it which is...fortuitous, if not unhelpful.
• I am very thankful for Heroku making it easy (and satisfying!) to ratchet up my server count from 3 to 30 with a flick of the wrist.
• I don't have a good sense of what Buttondown's true top-end capabilities are, and that makes things all the more painful when they fall over. Overscaling is easy, so I err on that side, but I need to do some sort of stress test against traffic. (I need metrics in general, too, which is fair; the baseline ones provided by Heroku aren't terribly useful.)
• The newsletter that caused it is hilarious. I am informed by my hipper friends that MSCHF is "a thing".

Performance anxiety.

I spent my morning reading a former colleague's paean to performance:

I’ve really come to appreciate that performance isn’t just some property of a tool independent from its functionality or its feature set. Performance — in particular, being notably fast — is a feature in and of its own right, which fundamentally alters how a tool is used and perceived.

One of the recurring pieces of feedback and appreciation we got from Stripe engineers following the Sorbet rollout was how fast the tool was. In a world of slow tooling1 developers genuinely appreciated a tool that felt fast and responsive.

I think the value of performance is sometimes well-understood intellectually — many engineers know of and talk about the perceptual thresholds for response time or the impact of latency on conversion rates — but it’s rarely appreciated on a really visceral level, and also is often given lip service more so than real investment. It feels popular to bemoan slow software these days, and yet it also feels rare that a team is doing something about it — our tools still keep getting slower. My experiences lead me to the strong belief that while our tools do make it increasingly hard to write fast software, it’s very much still possible, and very much still worth it.

Buttondown is....not quite fast enough. It's faster than a lot of other similar tools, but there are operations that simply take too much wall-clock time or are otherwise flaky in performance. I think about sending emails and validating subscribers as the two performance bottlenecks and they are both getting worse, not better!

The first step is, like I said, having actual metrics around this so I can track the change over time, but when given the choice between instrumentation or polish I tend to default towards the latter. There are some easy ways where I'm pinning performance, like unit tests making sure certain operations don't exceed a certain database load:

    def test_list_num_queries(self):
        FakeData.subscribers(10, newsletter=self.newsletter)
        with self.assertNumQueries(5):
            self.client.get(self.url)

But this is hardly sufficient. Time to, amongst other things, set performance budgets.