A Brief Framework 16 Review
Let me save you some time: the Framework laptop is great, and you should buy it. The rest of this post will delve1 into the why, but that’s the “so what.”
My Setup
I last wrote about my setup in May of 2019, almost five years ago. While some of those specifics have stayed the same, much has changed since then, and so I decided to write a new version of that article today. For ease of comparison over time, this post will—-for the most part — follow the same format as in 2019.
Laziness is the Root of All Evil
In a class a few days ago, the instructor trotted out Donald Knuth’s famous quote about premature optimization. Of course, as nearly every other instructor has ever done, he cherry-picked the catchy middle part: “Premature optimization is the root of all evil.” He did not, however, share the context around it. Here’s the full quote:
The Danger of Metrics
I got an email a few weeks ago about my article Operational Metrics: The Next Step in the Evolution of Defensive Cyberspace Operations1 from a researcher studying the same problem. We talked about my article, some of the other writing I have published on this topic, and and then something I briefly touched on in my SOC Metrics series: the dangers of metrics.
Artificial Intelligence Reading List
Thanks in no small part to OpenAI’s ChatGPT, the past year has seen an explosion in interest in artificial intelligence in general, and in large language models in particular. That democratization of access turned this niche research area into a common topic of conversation, and has led to a lot of fascinating writing on the subject. Although certainly not comprehensive, this article collects some of my favorite articles, papers, and resources into a single reading list.
You Don't Need AI, You Need an Algorithm
A few weeks ago, I participated in a working group tasked with identifying areas where artificial intelligence and machine learning could improve network utilization. What, specifically, “improve” and “utilization” meant were some of the first questions we addressed. During that conversation, though, one of the participants made an insightful observation that the group promptly ignored. He told a brief story about how someone had asked him for a networking device that could aggregate multiple uplinks and then “use AI to choose the best one.” When he offered a simple WAN failover solution, a technology that has existed for decades and comes built-in to open source firewalls like pfSense, commercial appliances from the likes of Cisco, and every modern mobile phone, the customer told him, “No.” Like our working group, that customer person had a solution (artificial intelligence) driving their requirements (an AI-based WAN failover), not the other way around.
Introducing Colors
Over a year ago I read Matthew Strom’s article How to pick the least wrong colors, where he explored the fascinating intricacies of designing good color schemes. Here, “good” meant not just whether the colors looked nice, but also how distinguishable they were from each other and whether or not they accomodated individuals with conditions like protanopia, for example, who cannot see red light. He came up with a great solution and posted the NodeJS code to GitHub. Since I do most of my work in Python, though, I had few chances to take advantage of his research — until now. My latest project, Colors, helps to generate large sets of attractive and accessible color schemes. The current version of this project is, for the most part, a direct NodeJS-to-Python port of Matthew’s code from 2022 with a few minor changes, but I look forward to building on it and keeping pace with Matthew. Colors is currently available as a Jupyter notebook and a regular Python script.
SANS Recommendations for Defensive Cyber Analysts
I keep fielding this question in private, so I finally decided to answer it in public: “What SANS courses should I take?” Although I have a much longer answer to give about training in general, this article answers that specific question based on my own personal experience, having taken several SANS courses over the years.
ChatGPT Wrote my Shell Prompt
I have, for years now, wanted a cool shell prompt — something worth showing off on r/linux or even r/unixporn. Not enough to dig into the weeds and figure out how to write one myself, but enough that it came up every once in a while. Then, the other day, I had a great idea: why not just let ChatGPT make it for me?
The Baseline Fallacy in Defensive Cyber Operations
Both civilian cybersecurity practitioners and military defensive cyber operations personnel continue to push a similar idea, that to identify malicious activity one must “just get a baseline and then flag anything different.” If only! I call this the Baseline Fallacy:
Collection, Transport, and Presentation: The Three Wicked Problems Inhibiting Data-Driven Decision-Making in the Army
Military leaders have sought hard data to drive their decisions for decades, perhaps most famously beginning with Secretary of Defense Robert McNamara’s so-called Whiz Kids’’ in the 1960s. As retrospective analysis of those decisions made clear, however, data alone does not good decisions make. Errors in collection, transmission, and presentation decimated the efficacy of this initiative.1 The Vietnam War is a cautionary tale in data-driven decision-making gone wrong, an important reminder that modernity’s insatiable need for more data is no more a silver bullet today than it was sixty years ago.
Git is My Hammer; Everything is a Nail
A few days ago, I mentioned that I use Git to synchronize work between several different devices and across several different platforms. This setup has served me well, and so today I want to talk about it in detail.
Changing History with git-filter-repo
I have used git to synchronize work between several different devices and across several different platforms for quite some time now. Over the years, though, inconsistencies have crept in. The name attached to my commit messages, for example, might be “Zac” on one device, “Zachary Szewczyk” on another, and “Zac Szewczyk” when committing via GitLab’s web editor. The same applied to the email address attached to those messages. Elijah Newren’s git-filter-repo project made it a breeze to fix this.
Cybersecurity Tools & Resources
My book Handbook for Defensive Cyberspace Operations contains an extensive chapter on tools and resources for cybersecurity analysts. As a closed project, however, this knowledge has had little impact outside of my niche community. This post contains a version of that chapter suitable for distribution to the public.
Passing CISSP
Like college degrees before them, professional certifications seem to be waning in popularity in the cybersecurity industry. Perhaps as a way to mitigate the well-documented workforce shortage, some companies have gradually begun to account for competence and experience, too, rather than onerous credentials alone — and in some cases, base their hiring decisions solely on those criteria. This is a localized phenomenon, though, exclusively limited to the private sector, and even there primarily limited to smaller firms. In the public sector, at many large firms, and particularly at large firms that work with the government, formal credentials such as a degree and professional certifications remain not only an important factor, but in many cases the only factor. This led me to take (ISC)<sup>2</sup>’s Certified Information Systems Security Professional (CISSP) exam earlier this summer. Even as a Cyber Operations Officer, credentials like this one carry great weight. I passed on my first attempt, so in this article I want to share my preparation strategy.
Individual versus Program SOC Metrics
The other day, a member of SANS’s GIAC Advisory Board asked a question about Mean Time to Detection, or MTTD. In the ensuing discussion, another member cautioned against confusing key performance indicators used to assess individual efficiency, to then serve as the basis for penalties and rewards, with measures of program efficiency. Holding individuals accountable for MTTD, for example, made little sense when that detection relied on factors outside of their control such as effective data collection and efficient aggregation into a centralized platform, each of which were likely to be managed by a separate department. I alluded to the challenge of implementing metrics in part one, but did not deal with this issue specifically. This bonus article provides specific recommendations for applying SOC metrics to individuals versus an entire program.
There is no iceberg
From David Heinemeier Hansson:
It’s human nature to assume there’s a good reason for why things are the way they are. And that this reason is either benign, based on careful deliberation, or malignant, derived from malice or incompetence. But this is a false dichotomy that often steers us away from the simpler answer: Nobody thought about this at all.
Given an absence of information, many tend to fill those gaps with generous assumptions. At the beginning of Russia’s botched invasion of Ukraine, national security pundits considered it some sort of 4D chess game; time proved otherwise. No one knows what they’re doing, “It’s OK though — that’s not where your problems are coming from. Rather, your problems are coming from the fact that you think other people know what they’re doing.”
Amateurs Talk Detection. Professionals Talk Collection.
I went to a presentation from a large cybersecurity firm the other day. The salesmen — and they were all salesmen, as even a few post-presentation questions made clear — focused on the intelligence their company produced, but knew little about the rest of their company’s intelligence cycle. As a consumer of their products, though, I consider knowledge of the process that created them critical. After an hour on finished intelligence, and in particular a slide that touted a “globally deployed sensor grid”, I wanted to know more about that production process in general, and about their collection specifically. Unfortunately, when asked, the salesmen offered a handful of handwavy “We have millions of sensors across the globe”, and, “We ingest billions of events per day” statements, but little beyond that.
Operational Metrics: The Next Step in the Evolution of Defensive Cyberspace Operations
As a senior First Lieutenant, I volunteered to teach new Second Lieutenants about defensive cyberspace operations. Over the course of a year I spent more than twelve hours with three different Cyber Basic Officer Leaders Course (BOLC) classes; each time, they asked the same question: what did your missions ac- complish? Each time, I struggled for an answer.
SOC Metrics, Part III: Measures of Effectiveness
In part one of the SOC Metrics series, I introduced the idea that success requires the right person doing the right things in the right ways. That article also described several foundational metrics, ways to measure the SOC’s ability to produce meaningful results. Part two then focused on measures of performance (MOPs), which assess whether or not (and to what degree) the SOC is doing the right things. This article delves into measures of effectiveness, the last step in defining useful SOC metrics.
SOC Metrics, Part II: Measures of Performance
In part one of the SOC Metrics series, I introduced the idea that success requires the right person doing the right things in the right ways. Measures of performance (MOPs) assess whether or not (and to what degree) the SOC is doing the right things, and measures of effectiveness (MOEs) assess whether or not (and to what degree) the SOC is doing them in the right ways. MOPs and MOEs rely on foundational metrics to produce meaningful results, though, and so I started with them in part one. This article delves into measures of performance, the next key step in defining useful SOC metrics.
SOC Metrics, Part I: Foundational Metrics
I prefer a hands-off approach to management. Describe a goal, give me the freedom and resources to achieve it, and I will. The success of that approach, which the Army calls “mission command”, ultimately depends on the right person doing the right things in the right ways. Unfortunately, many focus on that first criterion but neglect the second and the third. In many cases, that leads to failure. In the final days of my last job, in a security operations center (SOC), I realized that this explained many of our systemic problems. We had the right people, they just did the right things in the wrong ways or — in some cases — did the wrong things altogether. This was not an inherent consequence of that hands-off approach, of mission command, but rather a necessary consequence of its partial implementation: we lacked measures of performance (MOPs), to assess whether or not (and to what degree) we were doing the right things, and measures of effectiveness (MOEs), to assess whether or not (and to what degree) we were doing them in the right ways. In their absence the right things and the right ways became inconsistent and subjective, and so did our success. I wrote this series to fix that.
XeTeX on Windows
I switched from Ubuntu to Windows 10 a few days ago. After eight years of macOS and then a year of Linux, I wanted to circle back to Windows.1 Since my latest writing project relies on XeTeX, a lesser-known typesetting engine within the already obscure TeX world, I expected some trouble recreating my workflow in Windows; I was not disappointed.2 This post explains the steps I went through to create a functional TeX writing environment in Windows 10.
Command versus Leadership
The Army has a strange relationship with words. On the one hand, precision means everything: the term “seize” means something very different to a ground forces commander than the term “secure”. On the other hand, “leader” has become an umbrella term for both the entire force in general and the select few in command positions specifically. This has little impact on non-commissioned and warrant officers who — for the most part — will never command a unit; “leadership”, to them, means leadership: influence to accomplish a mission. For officers, however, the impact of this imprecision is significant.
Bookmarks
I frequently encounter people eager to learn or interested in understanding current events but that do not know where to start. Many lack the experience to know what they should study. Others have become so disillusioned by the hyper-partisan twenty-four-hour news cycle that they just ignore current events altogether. While I have addressed those challenges in Personal Development and the two-part series Keeping Up with Current Events and My Evening Reads, respectively, those aggressively curated lists leave out many useful sources that did not make the cut. I just don’t have time to keep up with everything. This article highlights those sources as well as several other valuable resources. While I may not visit these websites often, these are my first stops when I must seek out trustworthy information on a range of topics. This article also links to several useful tools both as a way for me to keep track of them and as a way to highlight them to others.
Threat Intelligence & Threat Research
Network defenders often struggle to obtain timely, accurate, relevant, predictive, and tailored intelligence about the enemy and other aspects of the operational environment. This document attempts to address that difficulty by detailing a single, curated list of reputable, open sources of threat data, information, and intelligence. This list combines traditional sources such as cybersecurity organizations that publish detailed reports (think finished intelligence) with non-traditional ones like social media that provide feeds more akin to information or raw data. While many others exist, I tried to stick to those with a high signal to noise ratio. That ruled out some decent sources, but I consider that a worthy trade off. Keeping up with this entire list would require a full-time job as-is, so while you may choose differently, choose wisely.
Advice to New Lieutenants
As I close in on my promotion to captain, mentorship has become an increasingly important part of my job. Many new lieutenants find the Basic Officer Leader Course just that — basic — and so they report eager to develop personally, professionally, and militarily. My article Personal Development addresses that first domain in terms of improving one’s personal, professional, and military knowledge, but neglects other areas. After several conversations with new lieutenants eager for professional development as officers and military development as warfighters, I decided to turn some of those discussion points into a post here. As a Cyberspace Operations Officer, most of this advice will target other 17-series soldiers, although some of it may help new officers in other branches. This advice is also not specific to officers, and in some cases may apply to enlisted soldiers as well.
Your First Week as a New Lieutenant
Aside from maintaining my article on personal development, I have shared little about this profession of arms. As I close in on my promotion to captain, though, mentorship has become an increasingly important part of my job. After several similar conversations with new officers, I decided to answer a common question here: “What do I do as a new lieutenant?” While the answer to that question will change from branch to branch and from unit to unit, I decided to write about some advice that should generalize well. If you follow this advice during your first week, you will put yourself in a good spot at the beginning of your assignment and set you up for success in the long run.
Build Systems, and Habits
If I like an article, I look into its author. Most good writers write well and often. This helps me find independent websites, like Ben Kuhn’s blog. In his 2018 piece, Stop Trying, Ben talks about steps he took to better his life: tracking time with RescueTime, and waking up with automatic lights. His productivity went up, and he gets out of bed on time now — because these changes required “literally zero maintenance”. He tried to make productivity and wakefulness a habit, but did not succeed until he built systems around those two goals. From Stop Trying:
Staying Focused with a Simple Method
Leo Babauta, in Staying Focused with a Simple Method:
“When you notice yourself avoiding something hard or uncertain ... the method is to turn towards it. Turn towards what you’re avoiding. Open to the discomfort, embrace it as training and growth. Bring curiosity. Do it even when you don’t feel like it. This is the training. The simple method makes it easier. Take it on, and see what happens.”
As I have said before, and will keep on saying, there is no shortcut, life hack, or trick; grind.
"Be yourself" is terrible advice
One of my favorite mentors used to tell me this: “Be yourself, unless yourself is a shitbag; then be someone else.”
Attribution
Not long after I found an old piece on attribution from Thinkst Applied Research, Cisco’s Talos Intelligence group posted an article on the same topic. Attribution is fraught, as both pieces explain. IP geolocation is woefully inaccurate, many of the tactics, techniques, and procedures tied to specific actors are either so common that they apply to several groups or so general that they might as well, and the proliferation of viable tool sets has made even this metric unreliable. At best, we can attribute activity with a degree of certainty — but never complete confidence, and you should be circumspect of anyone who does.