Software tooling

Learning a new programming language is hard. For the obvious reasons, but also because of tooling. (This post is mostly going to be a rant, ignore it if you're looking for practical information.)

I think nothing is more frustrating when learning a new language than not being sure where errors come from. Do you need to install another library that you're building against? Do you need to add paths to your environment? Are you passing incorrect flags to the compiler? Is your project definition missing some configuration?

And it goes on: What IDEs have good integration? Is the language or its community opinionated on project layout, documentation, testing? If not, are there still any best practices to follow? All this makes it hard to effectively use a new language at first, even if the language's syntax and semantics are familiar from the start. But all this should be trivial to take care of.

When I started programming (with Java), it was (as far as I could tell) the time of Ant build scripts. In its functionality, Ant is similar to Make, in that it defines rules for building specific parts of a project, and dependencies between rules to deduce which rules should fire and in which order. Unlike Make, Ant is Java based and its build files are XML (thus making Ant ugly to read or write).

When I heard about Maven, it was a huge revelation. Not because it made expressing build processes so much easier (or because it was less ugly - Maven still used XML build files), but because unlike Ant, Maven included what I consider the most important part of language infrastructure: dependency management! Tell Maven what libraries your project depends on, and it will download them and configure all paths so they're found. Nice!

So there's dependency management, there's building your project, and also importantly, there's installing the tools in the first place. I usually don't want to pollute my operating system by installing stuff when I have no idea what I'm actually doing.

Let's leave Java behind and look at Python. One major problem is the split between Python 2 and 3. They are largely incompatible and Python 3 has lots of cool stuff, but some software still requires Python 2. On the plus side, there are pyenv and virtualenv which let you install Python without root and Python libraries for specific projects, respectively.

Normally, your Python package will have a file setup.py which contains your project's description: name, version, dependencies, etc. As this is a Python file, you could think this is meant to contain build instruction, but that's not really the case. The file should contain a single call to setup(...), which uses the data provided in its parameters in a fairly declarative way. Further, in "modern" usage, you will likely use pip to handle dependencies. For example, pip install -e ./myproject will find ./myproject/setup.py and install the package (as editable, -e) as well as all missing dependencies.

So pip handles dependencies. What about building? Well, Python is interpreted, so normally not much. To make a "distribution" of your package, you can call python setup.py sdist and/or python setup.py bdist_wheel to make a source or binary distribution.

In JavaScript, the situation is more confusing. There are tools like bower, grunt, gulp, browserify, webpack, npm, yarn, etc. that all enter the picture, and tomorrow there will be even more. Also, there's three major ways to execute JavaScript: in a Browser, in Node and in Internet Explorer. But despite that, if you specify a package.json and use e.g. npm, you have a pretty good chance of distributing your packages to your users. (Does it show that I have not used JS as much as Python?^^)

My last look will be at Rust, because it's cool and is in a rather special situation: Rust is new, and its build tool, cargo, was developed from the beginning with the language and the rest of the tooling. It handles dependencies, building code and documentation, testing and publishing packages all in one. For installing Rust, there is the rustup tool, which fits approximately pyenv's profile: it lets you install and choose different versions of Rust. This is especially important as Rust has a six-week release cycle and a nightly channel that allows to use new unstable features.

Cargo is not required to build Rust, you can use the compiler rustc on its own as well, but usually there's no reason to go that way. That means that Rust is a rather homogenous environment (as long as you're not using unstable features) and you can focus on getting your code to compile at all; the rest of your environment will most likely work out of the box.