Concept: Reimagining Cygwin’s package manager

Synopsis: Cygwin is a POSIX-like/Linux-like environment available for Windows; we only wish that its package manager (non-initiates, read: “app store”) were a bit more like apt-get/Synaptic et al.

For those of us who have become reliant on unixish tools for any sort of productivity, the venerable Cygwin is the special sauce that makes Windows feasible for anything, and that’s important everywhere you’re stuck with Windows (e.g. your mid-size business employer that relies on the Microsoft stack). Additionally, Cygwin was a large part of how I learned to use unixy environments in the first place^[1], and might still be for a number of users.

A package manager is a concept with which users of any actual Linux distribution will be familiar. For those who use smartphones, it’s the very basis for what’s now known as an “app store” or “market”: If you have a program you haven’t installed but would like to run, rather than going to the product’s website and downloading an installer (as is traditional on Windows and other platforms), you locate their product in the platform’s repository and hit install from there. The product itself is provided as a package, which contains most of the same data as an installer but is a little lighter (see below). The advantages of package managers over traditional installers include:

• Since the package manager exists to do the job, a package need not include the program logic that unpacks it, making the download size somewhat smaller.
• The package can specify to your package manager what prerequisite packages (e.g. for other programs or libraries) may need to be installed for it to work, freeing the user from downloading, (and the developer from packaging) prerequisites the user probably already has installed.
• The experience of installing a program from the package manager is uniform from one program to another. It is generally as simple as possible; if the package doesn’t absolutely require some piece of configuration information before being installed (most don’t), you select a package, say OK to any dependencies, hit Apply, and then let it do its downloading and unpacking work. Once the box disappears, that program is ready.
• The experience of uninstalling a program is similarly uniform. You unselect a package, then the process proceeds as with installing.

For those more comfortable with the command line, Linux distros have command-line versions of their package managers that do the same job. For example, if the GIMP happens not to be installed on an APT-based distro (like Debian or Ubuntu) it’s very likely that sudo apt-get install gimp will do exactly what I need it do.

Predictably, Cygwin has its own package manager, which is in fact the same program used to install Cygwin in the first place: setup.exe.

For a certain class of users and/or use cases, setup.exe does its job really well. As indicated by its name, it’s better for a first-time install than anything else. Perhaps its most important features are its size (under 700K in the pictured version) and its lack of external dependencies, both properties becoming of an installer program. It’s written in C++ and compiled on the MinGW toolchain^[2], meaning it’s written as a Windows program rather than a Cygwin program, which caters to both of those ends.

But, and I do mean this in the nicest and most respectful way, setup.exe kinda sucks as package managers go. Here are some of the lowlights and my proposed solutions:

• As an installer, it puts way too much of the decision-making process up front. You get a wizard page to choose your local package cache directory, one for proxy setup, one for which repository (“mirror”) you want to use, and then the huge one for package selection. An “express” mode would be great: Either detect my proxy settings or don’t ask for them until connecting by conventional means fails. Choose the mirror that seems e.g. closest to me. Start downloading the packages that are part of the minimal system. Perhaps let me choose the other packages I’d like to install while that download happens; it does take a while.
• As a package manager, it acts way too much like an installer. If I’m running the program to install one new package, it’s sad that I have to approve my previous selections for local cache, proxy setup, etc. a second time have to pass -M to the installer to skip repeating my previous selections. Take me directly to the package selection screen with just an option that is GUI-accessible to go back and change those things.
• The dependencies aren’t calculated until the page after package selection. Synaptic has the option (enabled by default) to show you the dependency changes as you select a package, and that’s pretty useful.
• Only one repository can be used at a time. This is painful when you consider that Cygwin Ports (cf. “universe” in Ubuntu) is implemented as a separate, complementary repository, often with dependencies in the primary repository. In e.g. Debian-based systems, repositories can co-exist. Ubuntu takes it a step further with PPAs, which allow third-party developers to package and deploy their software in their own special-purpose repositories, making messy manual installation of third-party packages unnecessary.
• Properly, once the platform is installed, starting the package manager means executing its command (e.g. apt-get) from your preferred interface on that platform. That’s not straightforward with setup.exe because setup.exe does not install setup.exe on your platform. Those with the foresight to have stashed it will have to locate that copy; the rest will need to re-download it from cygwin.com.
• There is no real command-line interface to setup.exe.
• It accepts command-line parameters, but doesn’t provide the functionality of e.g. apt-get.
• apt-cyg does a good job of filling this gaping hole, but it does lack certain features (e.g. doesn’t check digests [edit: it does in fact check the MD5s, but doesn’t check the signature on the list file itself]).

I think the installer and the package manager should be two (perhaps more) programs; they have separate goals that are better served separately. The former needs to be small and light but not necessarily full featured, while the other needs to do more but can be a bit heavier.

A really straightforward approach would be to factor out an installer that, without extensive configuration, installs just enough of the Cygwin platform to run a complete package manager. This installer wouldn’t be complicated and could easily be smaller than setup.exe. Without the full-blown package/version selection step, it could be implemented with something like NSIS or WiX. For admins and other nitpickers, it wouldn’t be all that hard to change the installer script to change what packages are installed by that initial installer, or to fix a default repository. Otherwise, that first step needs to be as short as possible in order to make the next step seem like part of it.

The next step, of course, is the complete package manager. Personally, I’d enjoy a port of APT to Cygwin, partly because it works well in Ubuntu, and partly because there’s a host of other tools that work with it, such as Synaptic (which could work nicely with the Windows port of GTK+). But that’s not perfectly straightforward because Cygwin uses its own package system. The system itself is not fundamentally too different from APT, but it is different in several details. Now, porting APT to other package managers isn’t anything that hasn’t been done before (for a while there was an APT for RPM), but it would involve a lot of digging around in C++ and require the effort of maintaining an entire active port.

I’ve been trying to figure out exactly what the right approach might be, this post in fact being part of that process. I wonder now if the right approach, or at least a good bootstrap, might be to

1. Port apt-get (and other tools as necessary) directly to Cygwin with little to no alteration. (Some alteration is likely to be necessary.)
2. Develop a translation proxy, perhaps as a local HTTP server, that would alter paths and perhaps in fact convert Cygwin packages (each being .tar.bz2 archives containing installable files plus, in parallel, supporting metadata) into Debian-style packages (.deb packages, which are two optionally-compressed .tar archives, one metadata and one payload, glued back-to-back in a more obscure .ar archive).
• While we’re at it, let’s implement it in a more approachable language than C++. Perhaps Ruby or Python. This isn’t an application that requires super speed, but super maintainability would be nice.

The way I see it, there are two directions it could go from there.

In one outcome, the above development is a bootstrap for porting APT to the Cygwin package system, by way of a complete separate port or by somehow implementing a pluggable backend for multiple package systems.

In the other, the above development is a bootstrap for converting Cygwin at large over to using .deb repositories. The repository format simulated by the real-time proxy at first would be deployed for real on Cygwin mirrors. In fact, code from the proxy program could be used at the server to convert an entire mirror from one format to the other.

I like the latter scenario, even if it would involve convincing more people, just because it wouldn’t require any actual development beyond the conversion proxy. Maintaining a converter proxy doesn’t sound like nearly as big a job as maintaining a fully distinct port of APT. Plus, the conversion period could be arbitrarily lengthy because the proxy would work throughout the meantime. An additional pro to this is the possibility of adding a PPA-like system for third-party devs. Because APT supports multiple repositories at once, Cygwin, Cygwin Ports, and PPAs would co-exist in harmony. Because of the proxy, Cygwin and Cygwin Ports could even be in the current format while PPAs are implemented directly in APT form.

That might be worth investigating, eh?

1. ^[1]To drop some history: Back in the dark times of 2000 or so, GParted wasn’t yet a thing and virtual machines weren’t quite feasible for much, so installing Linux meant having to gingerly repartition your drive, hoping that nothing got ruined, and setting up a bootloader for dual boot. If you didn’t set up your networking just so on the first try, you’d have to boot back into Windows to look up a solution (often repeated ten or fifty times). Around that time, NTFS support in Linux wasn’t so great either; it was read-only at best, so any significant sharing of data between systems would require a FAT partition. The situation wasn’t significantly better until, let’s say, 2007 or so, when ntfs-3g brought in read-write NTFS support and Ubuntu’s mission for a stable desktop Linux had gained some ground. (I hopped onto the Ubuntu bandwagon for keeps around 2006, at which point I had a PowerBook to fall back to in dire straits.) Cygwin, being not an OS but a compatibility layer, allowed casual tinkerers and power users alike to avoid much of the mess.↩
2. ^[2]Both Cygwin and MinGW provide GCC for Windows, but Cygwin-targeting code requires Cygwin’s unix-like simulation layer while MinGW targets the Windows API directly.↩