Sequential Exeuction, Multiprocessing, and Multithreading IO-Bound Tasks in Python

Python makes concurrency easy. It took less than an hour to add multiprocessing to my blog engine, First Crack, and I have used it often since. Everyone likes to call premature optimization the root of all evil, but architecting programs for concurrent execution from the start has saved me hundreds of hours in large data capture and processing projects. Color me a Knuth skeptic. This article compares sequential execution, multiprocessing, and multithreading for IO-Bound tasks in Python, with simple code samples along the way.

Good times create weak men

Nikita Prokopov, on the growing complexity of software and the shrinking number of people who understand it:

“In programming, we are developing abstractions at an alarming rate. When enough of those are stacked, it becomes impossible to figure out or control what’s going on down the stack. ... Docker and Electron are the most hyped new technologies of the last five years. Both are not about improving things, figuring out complexity or reducing it. Both are just compromised attempts to hide accumulated complexity from developers because it became impossible to deal with.”

I saw this in college, when even Computer Science students favored complex and bloated frameworks over their own code. I make a concerted effort to counter it in my own projects: First Crack, the custom blog engine behind this site, uses vanilla Python 2 or 3 and has no dependencies. If you decide to code, you don’t have to start with C — but you should learn it. I resisted this during college, but two years into my career, I have come around: the lower you go, the better you will understand things higher in the stack. With greater understanding comes an increased ability to influence those things, which, in turn, will make you better at your job.

Michal Charemza brought up another good point in Mini HTTP guide for developers:

“Frameworks often hide/abstract parts of HTTP away. I think this is often a bit of a shame: it hides what’s possible with HTTP, and so can lead to effects on engineering decisions.”

By learning to use frameworks instead of the tools and protocols they implement, developers not only miss out on foundational knowledge that will help them become better at their job, but also hamstring themselves to the subset of features the frameworks’ creators’ felt important enough to enable. Expanding a project beyond that expected use case will require diving into those low-level tools and protocol—-which, again, you do not have to start with, but you should learn at some point.

Trump's Surprising Embrace of Industrial Policy to Fight China

I took some time to update Keeping Up with Current Events and My Evening Reads over the holidays, which involved adding Matt Stoller’s excellent newsletter BIG to the list of sources that help me stay informed. His most recent issue, Trump’s Surprising Embrace of Industrial Policy to Fight China, is a prime example of the shrewd analysis that makes it so interesting. This issue is a great way to start the new year, and new subscribers will find his roundup of 2019’s best posts at its beginning helpful, too.

What Happens When Your Career Becomes Your Whole Identity

Janna Koretz, writing for Harvard Business Review, on an interesting phenomenon called enmeshment. Working in a profession known to encourage and reward workaholism, I see this often — and work to counter it where possible.

Why are RV's so poorly made?

Andrew Zaleski digs into the factors contributing to the RV market’s notorious lack of quality. I thought a lot about getting an RV instead of my first house, but this pushed me away from buying an RV then and ensured that when I do decide to go that route, I will build it myself.

Updates to My Morning and Evening Reads

I spent enough time Internet-less this holiday season that I had a chance to catch up on a few administrative tasks. One of those involved updating Keeping Up with Current Events and My Evening Reads to reflect new sites, feeds, and newsletters I now follow, as well as those I no longer read. I work hard to educate myself and stay informed, and I encourage you to check those posts out as a great starting point to doing the same for yourself.

This Page is Designed to Last

Jeff Huang, echoing a point I made in Own Your Platform, in his piece on the importance of building websites that stand the test of time:

“I’ve recommended my students to push websites to Heroku, and publish portfolios on Wix. Yet every platform with irreplaceable content dies off some day. Geocities, LiveJournal,, now Yahoo Groups. One day, Medium, Twitter, and even hosting services like GitHub Pages will be plundered then discarded when they can no longer grow or cannot find a working business model.”

And then, later, he echoes a point I made in How to Own Your Platform, about the vulnerable position relying on trendy third-party frameworks puts you in:

“... a growing set of libraries and frameworks are making the web more sophisticated but also more complex. First came jquery, then bootstrap, npm, angular, grunt, webpack, and more. If you are a web developer who is keeping up with the latest, then that’s not a problem. But if not, maybe you are an embedded systems programmer or startup CTO or enterprise Java developer or chemistry PhD student, sure you could probably figure out how to set up some web server and toolchain, but will you keep this up year after year, decade after decade? Probably not, and when the next year when you encounter a package dependency problem or figure out how to regenerate your html files, you might just throw your hands up and zip up the files to deal with ‘later’. Even simple technology stacks like static site generators (e.g., Jekyll) require a workflow and will stop working at some point. You fall into npm dependency hell, and forget the command to package a release.”

Run your website, and own the production stack. Jeff has some some good ideas, but do not paper over poor back-end design with good front-end work. Own your platform, so that you can ensure both will stand the test of time.

'No One Wins if War Extends Into Space'

“We’re at an inflection point today in the idea of space being developed as its own domain. When you look at the threats that we have today, there’s some similarity to what we had in the ‘80s. That waned in the ’90s and that led us to the idea of space as an uncontested area, so we designed space architecture based off the fact that we wouldn’t really be threatened in space. Today you’ve got a pretty much more aggressive threat. So that space area of operations is the area that we focus on for protecting and defending critical assets as we look at the growing threat from our adversaries.”

Replace “space” with “cyberspace”, and BG James could have said this twenty years ago about the fifth domain.

Why Teslas Are Bad At Towing

Jason Fenske does a nice job explaining why the Cybertruck will not tow well: high efficiency gives electric vehicles competitive ranges despite their batteries’ low energy capacity, but also means that even a light, 5,000 pound trailer will slash that figure. In distance = energy / force, a small numerator (low capacity) causes range to drop off much faster as the denominator (towed weight) increases. See the blue graph below, which depicts range (the y-axis) as a function of towed weight (the x-axis). Traditional vehicles, although less efficient, have vast energy stores; in their case, a high numerator means the denominator has much less of an impact as it increases. See this in the green graph.

Force graph with a high versus low numerator

As I said about the Cybertruck last week, “it’s not a very good sportscar, and it’s not a very good truck.” I still think electric vehicles will revolutionize the overland space, but it will take years for the batteries to get there.

Lightning Round

I often see articles that I would like to talk about, but that I do not feel warrant a whole post here. Twitter is the obvious choice for quips like these, but I have little love for the service, prefer to keep all my work on my own platform, and feel that nuanced opinions should not come in 280 character bursts. Occasional “lightning rounds” will fill that gap. Together, a handful of these micro posts will make a full one, giving me somewhere to share things that used to just slip through the cracks.

First Crack Release Notes, November 2019

A few days late, again, but here it is: First Crack’s release notes for November, 2019. Again in October, like in September, I spent most of my dev time on an Instapaper-like read later service. I use it every day, and plan to release it. I did get a couple things done, though; once again, I did not neglect First Crack entirely.

Thoughts on Tesla's Cybertruck

I like to think about things. To riff off Austin Kleon, I want to become a professional writer so that I can be a professional thinker. For now, though, as I have said many times before, I write to clarify, condense, and record my thoughts. I have sat down today to talk about Tesla’s new Cybertruck — not because the Internet needs one more hot take, but rather to pursue those goals.

How to Encourage a Blogging Culture at Your Company

I like this post a lot, and the idea even more: Noah Gibbs lays out some good reasons to encourage a culture of blogging within an organization, and solid strategies for getting started. My boss floated an idea like this a few weeks ago, and while I had my doubts at first, time — and this piece — have changed my mind. More to follow.

Hot Takes on Tesla's Cybertruck

I spent the morning reading hot takes on Tesla’s new Cybertruck. I have some more thinking to do, but Jason Torchinsky wrote a great post on the truck’s design, and Kim Reynolds did a nice job approaching the topic from a manufacturing standpoint. Andrew Collins also has a nice comparison of the Cybertruck versus some of its competitors, which is beats on all fronts.

You Can't Lift Because You're Not Trying

I have a few things to say here, so let me say them all up front. Your diet doesn’t matter. Your training plan doesn’t matter. The reason you will never reach your goal is because you don’t want to. You don’t work hard enough to achieve it, and nowhere near hard enough for anything like a diet or training plan to make a difference. Those factors come into play at such a high level, one so far from where you are now, that all the time and effort you pour into these areas is wasted. Now let me explain.

The Wothahellizat

I found this incredible rig on Pinterest, of all places. Inspiration for the next version of mine.

Pipe to/from the clipboard in Bash script

At home, I can send a link from any device to a local web server that saves it for later. Then, when I have time to read, I can see all my saved articles, move them into folders, and refer back to them as needed. At work, though, those links go into a text file. When I get home, I find the text file, open it, copy the list, clear the file, close it, and then send the list to the server. pbcopy makes that process just a little easier. Now, cat ingest.txt > pbcophy & echo "" > ingest.txt puts the contents of ingest.txt on my clipboard and then clears the file, so I can add the links through the web interface with just a bit less hassle.

Start before you think you're ready

“Me, personally, I became a professional writer so I could be a professional reader.”

I love that line. The rest of Austin Kleon’s post is good, but that line is great.