Title photo
frugal technology, simple living and guerrilla large-appliance repair
Mon, 17 Apr 2017

I reinstalled the Windows Subsystem for Linux, got a new Ubuntu LTS and now have my scripts back

After a Windows 10 update hosed my laptop and took the Ubuntu/Bash command line (aka the Windows Subsystem for Linux) and all of my scripts along with it, I restored Windows 10 from the laptop's backup partition, and activated the Windows Subsystem for Linux (aka WSL).

This time around I got Ubuntu 16.04 rather than 14.04, which is overall better because there are some really, really old packages in 14.04, including a super-old nodejs. Unfortunately, the old unison in 14.04 matched what is on my server (and unison versions across computers must match, or they don't work).

Luckily I was able to download a 14.04 package from the Ubuntu archive and install it in the 16.04-powered WSL. I restored my scripts, including one I made that is very WSL-specific: It takes all of the files in a Windows directory (usually images, sometimes text documents, but it could be anything), copies them into a working directory in the WSL and uses chmod to change their permissions to 644. That way I can download images while in the web browsers of the Windows world, create text files, working on all of them with Windows tools, and then transfer those files into the Linux side, where I can sync them to the server's filesystem with unison.

Aside: It's not impossible to get a Unix-style ssh program that works from the Windows command line, but it's not at all easy, either. That makes the Windows version of Unison less than useful for working with remote servers.

Now I have scripts in the WSL to:

  • Transfer and apply proper permissions to image and text files from Windows to the WSL
  • Update this Ode blog using unison to sync files and then reindex the blog via Ode's Indexette
  • Create [a static blog archive[(http://stevenrosenberg.net/documents/archive.html) by using a custom Ode theme and a query to return all posts, using curl to bring the html down to the laptop, then copying it into the local Ode filesystem, and then syncing with the server via unison.

I have a feeling I've written about most of these scripts before, and if/when I find those entries, I will link them here. If not (or if there have been updates), I will write them up in the near future.

Why Unison? Unison is a file-synchronization tool. While files can be synced from one system to another with rsync, which I use for backups, the situation with this Ode blog is different. Anthor way to synchronize two filesystem is to use git, the version-control tool.

What unison enables me to do is make changes locally, or on the server, and then reconcile those changes across both systems. So if I write or edit a post on my local filesystem, or make any kind of change on the server, running unison ensures that I have the latest files (and versions of files) on both filesystems. If I used rsync, making changes on the server but running rsync on the client wouldn't work. Git would be great, except that it only reconciles changes in the filesystem that have been checked in. Changes on the server are generally not checked in, and even if I scripted that on the server, Ode (through its Indexette and EditEdit addins) itself makes changes to the filesystem and doesn't check them in. So git wouldn't work.

I came up with unison because it's the easiest. Another alternative csync2 looks a lot harder to figure out. But I do recommend csync2 if you're doing something heavy-duty with more than 2 servers.

When I started looking for this kind of tool, I knew what I needed was a kind of Dropbox for servers. I'm sure there are people who have hacked Dropbox to work on a non-GUI server. Actually that would be a pretty good solution.

The difference with unison is that you have to "consciously" run it to sync the two filesystems. You could run it as a cron job, or somehow set it up as a daemon (which might be how Dropbox works), but for the purposes of this particular blog, syncing when needed works fine.

Using the WSL has provided me the opportunity for the first time in quite a while, to set up unison. It's a great thing to run unison -batch and have the entire blog filesystem copied to an empty directory on my laptop in about a minute. (And then any changes I make on either laptop or server can be synced with another unison -batch, or just unison for a more interactive session. Plus, never underestimate software you can install yourself, on your own computers, and use as you wish. I pay for my shared-hosting service, but otherwise I run whatever software I wish without paying any monthly fees for any of it.

Are there other ways to keep two or more filesystems in sync? I'd sure like to know if there were.