Personal Software

I find myself more and more disappointed in the state of most software, and I’m writing more and more tools for myself. I enumerate the kinds of bugs I experience that make software feel low-quality, then I discuss some examples of tools I’ve built, and I finish by discussing languages that I think offer the most support for building the kinds of tools I’m describing.

Software is Crap

In my (admittedly limited, extremely narrow) experience, most software is crap. It’s amazing that we can communicate wirelessly and work remotely and share information easily! But the dedication to quality in most software is poor, and the overall effect for me is to make software feel like crap.

Let me give some examples.

Spotify

The desktop Spotify app takes 10+ seconds to fully open on my brand new Macbook Pro with an M1 Pro chip. By fully open, I mean all album art loads, main pages are completely filled, etc. This is after opening/closing it less than thirty seconds before, so I don’t think it’s because there are invalid cached files. Even Microsoft teams takes less than 4 seconds (but there are other issues with Teams).

Spotify also continues to recommend podcasts, and most recently, audiobooks to my account. I don’t want to listen to podcasts or audiobooks on Spotify. If I wanted to listen to podcasts, I would do so on podcast app. Same goes for audiobooks. If I didn’t pay for Spotify, this wouldn’t make me unhappy, because, as the saying goes, “if you’re not paying for it, you’re the product”. But I pay for this product!

If I queue songs on Spotify desktop while listening on my phone, the queue sometimes disappears.

For a while, when I opened an album on Spotify iOS, if I swiped back, then my scroll position would reset to the top of the screen (this one almost had me back to Apple Music).

Then, because Spotify changed their web API structure (well within their rights as a private company), spotify-tui broke. Unfortunately, the maintainer is busy with other projects (again, not something I am complaining about). But somehow I am sad that a bug-free product just grew a bug.

Microsoft Teams

Microsoft Teams is one of the worst pieces of software I have ever needed to use, especially on macOS. Everything is slow. There isn’t a single page that doesn’t take its sweet time to render.

You can only have page open at a time, so if you wanted to see a chat and your calendar at the same time, you’re out of luck. (This also means you have to switch between pages; this wouldn’t be so bad if the app was snappy; see above.)

I find the information density very low; when I discovered the dense chat setting, it made a huge quality of life difference. If only there were a similar setting for the main channels.¹

The search implementation is really, really awful. There is no sort and are no patterns for more specific search²

Sony WH-1000XM4 Headphones

My headphones use my phone’s ringer volume, rather than the media volume. When I start listening to music, the volume is all the way up because I leave my ringer as loud as possible. Then when I adjust the volume down, it adjusts my ringer volume as well, so my alarms all very quiet after disconnecting.

git

Git’s UX is garbage. This isn’t a bug, and I know maintainers are working on improving this situation, and needing to maintain backward compatibility is hard. But the UX is pretty bad. Everything else is amazing though!

For more examples, check out Dan Luu’s one week of bugs.

Tools I Develop

With that in mind, I end up spending a lot of time working on my own solutions to make my computer use as smooth as possible.

I will discuss some examples so it’s clear what kind of tools I want to build.

todo

todo is an 80 line bash script that opens my editor in ~/Documents/todo with a file with the current day. If the previous day’s file exists, it copies it into today’s file. It’s extremely simple and extremely practical.

wiki

wiki is a 130 line fish script that makes it easy to add, find and edit documents in ~/Documents/wiki. It also includes a basic tag system. I use it all the time for daily logs on projects as well as snippets and other pieces of knowledge I don’t want to use.

Airpods.app

Airpods.app was a simple AppleScript wrapper I built around a BluetoothConnector which let you connect to Airpods really quickly.

Slow YouTube

Slow YouTube was a client-side only website built in Elm that only showed the channels you wanted to subscribe to, and only the three most recent videos. This prevented any sort of distractions or recommended videos. There’s an instance up at samuelstevens.me/elm-slow-youtube if you have an API key you want to use.

Quiet Hacker News

Quiet HN is an Elm app that shows the top 30 links from Hacker News without any comments. I use this site every day.

pycodesearch

pycodesearch is a command line search tool that lets you use regular expression semantic search over Python codebases. You can find all matches for hi!.*sorry in all string literals, for example.

This is just a small list of examples. There are dozens of shell functions, scripts and web apps that I develop for one-off use, which sometimes are used all the time (todo, wiki) and sometimes are used once and then never again (I’m looking at you, pycodesearch).

Tradeoffs

Given that I am going to write a lot of small scripts/tools to make my life better, what’s the best way to do it?

Getting Started

The most important factor is how easy it is to get an initial version working. Most of the time, I develop a tool because I am just fed up and want something working right this very second. Scripting languages, interpreted languages with interactive debuggers, languages that I personally know very well, languages with large standard libraries that don’t need a lot of imports are all winners here. For example, Python and fish are both winners for getting something done quickly (see pycodesearch or wiki for examples in each of these languages).

Returning to Development

Another important factor is how easy it is to come back and make edits after a long time. Often I will get a version 1 finished, use it and adapt my workflow to the tool, then realize a new feature would be neat. But I haven’t worked on the original tool in sometimes months! So languages that improve ease of returning to development are also better for this kind of software.

How would a language improve ease of returning to development? I actually think of the language as encouraging me to make it easy for myself, either through cultural norms or hard constraints.

For example, Elm’s type sytem makes it very easy to return to development. If you look at the commit history for (elm-slow-youtube)[https://github.com/samuelstevens/elm-slow-youtube/commits/master], you can see that I stopped working on it from August 8th until November 2nd. I explicitly remember coming back to the code and realizing how easy it was to figure out what was going on (admittedly, it is a particularly simple app) because of the types.

As a counterexample, look at bash. Google’s style guidelines for bash explicitly tell you to not write anything serious in bash:

If you are writing a script that is more than 100 lines long, or that uses non-straightforward control flow logic, you should rewrite it in a more structured language now. Bear in mind that scripts grow. Rewrite your script early to avoid a more time-consuming rewrite at a later date.

As one more positive example, consider Go’s culture of testing. The standard library’s test package is pretty good, so if you are building something without external packages, you can still write tests (unlike Python, where pytest is the only sane option). Having both unit and integration tests help a ton with coming back to a project. For example, when developing relic, I use existing tests to ensure there are no breaking behavior changes. If a language encourages the writing of tests through culture, language support, etc., I will be more likely to use them when working on personal projects.

Performance

A third factor is performance, especially start-up speed. The majority of tools I write are thin wrappers around my editor (see todo for a great example). If I have to start up an entire VM just to open my editor, that’s going to be a waste of time. Whenever possible, I want my tools to be lightning quick. Performance doesn’t matter for most applications, except when you get such high-performance software that you can completely upgrade your workflow. While developing relic, I realized that being able to see patterns in hyperparameter configs just by querying with relic ls meant I was making significantly better decisions about what options to pursue, and I was doing it more often than I would have if I had to fire up a Jupyter notebook.

I want to discover those new workflows that dramatically improve my ability to run ML experiments, track progress, develop new insights, etc. If a 20% slowdown in software performance prevents me from doing that, that’s not a 20% loss in progress, it could be closer to 100%.

500ms vs 600ms doesn’t make a big difference. But 200ms vs 20ms does.

Speed matters when it can affect your workflow.

Distribution

The last relevant factor is how easy it is to run on different environments. Python is great except you need a python interpreter installed (and it needs to be the right version) and you need all dependencies installed. Then you need to make it available in your path, and that installed interpreter and dependencies can’t conflict with your other virtual environments…fine, I’ll just write a bash script instead.

These are the benefits of Rust and Go–I don’t use any systems without a relatively modern libc, so I don’t need to get into the whole static-vs-dynamically-linked debate. I just don’t want to deal with node_modules/ or venv/ in my $HOME.

Options

There are four main options available to me, depending on what kind of tool I want to build. Then there are some other options that might be worth the learning curve if I discover glaring deficiencies in my already known languages.

This is also a Part II of my initial debate on programming langauges, but specific to writing personal tools.

Contendors

These are the options I reach for every single time when I need to write a tool. They (mostly) embody the traits I mentioned earlier.

Fish

Fish is better than bash for writing scripts. I rewrote my wiki script in fish in a morning, and it didn’t include sed or awk once. All the options are nice and human readable, the built-ins are very well documented. It’s a shell, so it’s great for manipulating some other commmands (ripgrep, fzf and fd) and it has very low startup time. It doesn’t have any support for unit testing (it’s still a shell langauge), but it’s easier to read than bash and doesn’t care about being POSIX compatible (which is fine by me).

Go

Go (cross-)compiles to single binaries, has great test support, has a great standard library (but you can use packages easily because it compiles to a single binary), has strong performance and is easy to write (but you do have to write a lot of it). The downside is that you have to write a lot of it, you can’t have unused variables, you need to set up a go.mod to start, etc., which makes it harder to get started. But if you develop version 0 in an easy language, then develop your workflow around version 0, then decide you need a real version 1, Go is probably a good choice.

Python

I write enough Python that, for example, I can add an argument parser without consulting documentation. That alone makes Python useful for getting shit done because I can just sit down and crank it out. If I need a script for an existing Python project, I will write it in Python because I know I’ll have an interpreter available. If I want to write a standalone program, I will rarely use Python because of the distribution hassle.

Elm

Elm is the only sane way to write web interfaces. Given the choice, I would re-write TicketBay as a web app in Elm every day of the week. It takes a little bit to get started, but if I’m writing a whole web interface then I might as well write it once instead of writing it in html/css/js, then typescript, then react, then elm. Plus the build system doesn’t exist, whereas I have to relearn webpack/babel/esbuild/(whatever the JavaScript community is doing) every single time I use them.

Worth the Learning Curve?

Should I learn Rust, Zig, Scopes, Odin, Janet, Nim, Dart or D?

All of these languages would replace Go in my personal dev use. They all compile to native executables. Some notable differences:

• Rust, Zig, Scopes and Odin don’t have garbage collection.
• Rust and Scopes use a borrow checker.
• Janet is a Lisp-like language.

Conclusion

Writing this post helped me clarify my position on what software I should use to write personal software. The answer is:

• fish for shell scripts/wrappers around $EDITOR
• Go for more serious CLI utilities
• Python for tools inside existing Python projects
• Elm for web UIs.

Some questions that are still unanswered:

• What should I use for writing desktop app(let)s? Things with a single-page GUI that edit some configs or move some files, etc.
• What should I use for terminal-based UIs? Maybe Go is best (like whatscli). There is also textual in Python, but then we have the distribution issues associated with Python.
• Should I be writing these tools in languages without garbage collection for maximum speed? Since they are often very simple, I wouldn’t have to worry about the complexity of managing lots of objects myself.
• Should I look into Janet? Lisps are, you know, Lisp. Apparently something magical happens (but I don’t get it yet).

Appendix: $EDITOR & Plugins

None of this even considers the application I use the most: (Neo)Vim. Given that NeoVim added Lua to make plugins easier to write, why don’t I write plugins for my editor? Those have to be some of the highest impact tools I can write, but learning how to interact with NeoVim just feels so hard.

Given that kakoune has a more consistent UX, should I use that? I would love to use helix but there is no soft-wrapping (issue).

[Relevant link] [Source]

Sam Stevens, 2024