Migrated VPS

black server racks on a room

When I started hosting this website on DigitalOcean about 9 years ago, the version of Ubuntu that was all the rage was 14.04 LTS. So I started my hosting journey with that. Pretty soon though, 16.04 came along and since I was ever active on my server, I upgraded to that using nothing more than a few apt update commands. Since then, other than a few forced efforts to secure the OS and install what I needed for experimentation, I didn’t do much to upgrade the underlying software.

So it happened that, when at the beginning of the year I tried to upgrade from PHP 7.3 to 7.4 (a process which failed), I was made aware of the fact that the chasm between where my software stack is and where it ought to be is rather large. I tried running a straightforward upgrade from 16.04 to 20.04. The blocker was mysql. Apparently, no matter what third party repos I tried, the upgrade from what I was running to whatever’s the current just wasn’t possible. Well, it may be possible, but it would not be easy. The recommended path, on multiple websites, forums, and blogs, was to just fire up a new VPS and migrate my websites and services manually. Daunting.

When I learnt of this, I realized that the amount of time and effort it would take was too much for me to give at that moment. Family needs and other projects held precedence. Right now, I wouldn’t say those needs have abated, just that I’ve adjusted to both those asks, and I’ve given myself enough time and another factor for this migration – money. DigitalOcean is a nice provider in that they’ll only charge me for what I use through the number of days that I use it. I know this is sort of the norm everywhere now, but it’s a nice-to-have and a nice-to-mention nevertheless. Instead of doing the entire migration within the span of a few hours, tiring myself, and increasing the odds of a failed migration, I spread the entire project over the last few days. I moved my other WordPress install first, the one whose failure wouldn’t affect me directly and personally. It’s a side project that we’ve gotten side-tracked from. I’d be totally fine if it craps out.

Moving WordPress seemed daunting, until I realized that I have a tool that can make it extremely easy. I’ve been backing up this website to Dropbox using UpdraftPlus for the longest time. It’s fast, easy, and totally a background process which has not needed my input since I set it up. I checked it out and sure enough, it’s got a pretty straightforward restore process too, included in the free version of the plugin. Of course, they offer paid tools for much easier migration. But I reckoned the free one has got to work just as well. UpdraftsPlus creates a bunch of separate zip files for the database, uploads, themes, plugins, and “other”. All you have to do to migrate is to create a fresh install of WordPress, install the plugin and drop the files into the interface and then hit restore.

This blog’s backup comes in at about 750 MB, while the other site is about 160 MB. I did the latter first, and since it stayed up just fine over the last few days, while for the first time in my life I ran two VPS in parallel in DigitalOcean, I ported over this blog as well as the other applications and sites which I wanted to keep. It ended up being a good housekeeping too, since most of the active nginx sites were not doing anywhere and thus were liable to be security issues. Plus, it gave me a chance to really start from scratch.

Over the years, I let the older VPS grow organically and get cluttered as all in-use systems do. When I was attacked by a script kiddie trying to get into this site and wreak havoc (at which they partially succeeded), I installed fail2ban and went aggressive with it, to the point where I got locked out of SSH quite a few times and had to recover via console. I installed multiple versions of node to run shortlived telegram bots or expressJS apps. I installed numpy to create a webUI for an experiment my brother wanted to run. I also created a series of scripts to run via cron – to periodically free up space and memory, to pull in data and recycle logs.

All of this had become a sore point for me anyways. The services running on the VPS often went down. The APIs responded only half the time. The downtime was somewhat acceptable till it wasn’t.

So this new VPS, well, I’ll run it as clean as I can for as long as I can. Of course, I’ll get hit by something or the other and I’ll have to respond with better security measures. But I wasn’t running any firewall before and ubuntu 20.04 seems to be running ufw by default, which is nice. I was also able to update PHP from v7.3 all the way to v8.0, which is nice, but came with it’s own set of challenges. One function in WordPress and another in a homegrown bookmarking tool were failing since they don’t work in PHP 8.0, so I had to spend some time figuring that out. But it’s good to have the latest software and to hope I’ll keep things updated better this time around.

All in all, a good experience. My old VPS is now sitting in shutdown mode. I’ll let it sit for a couple weeks, while I test out the new system and see if I forgot to move some settings or such. I know it’ll cost me almost twice as much for the month to run both machines in parallel, but it’s worth the peace of mind I’m getting.

Plus, this migration got me in touch with some projects I’d forgotten! I regularly use my liveblog, but completely forgot about “SomeDay”, a bookmark/linkblog of articles I didn’t finish reading and hope to, some day. It’s got an RSS feed and all, so maybe you can find something in there that you might want to read, today.

Links to everything currently hosted on my new VPS –

this blog


scratch.nikhco.in – a minimal writing tool with local browser storage and ability to start a TogetherJS session to collaborate with others in real time.


someday.nitinkhanna.com – I haven’t read these articles yet. Maybe you should try?

Blog Experiments

I did two things this week regarding my blog –

  1. I read a lot on Instapaper, mostly non-fiction articles. I make a lot of notes and highlights and all of those come over to this blog. Why? Well, at some point I thought it would be a good idea to write articles based on my readings. It’s also a way of preserving all of those thoughts in case Instapaper some day goes kaput. But the fact that I have all of this text sitting in my blog, counting against my word count, and not contributing to my readership has been irritating me ever since I started the practice.

    A few days ago, I setup a new blog on WordPress – https://nitinsnotes.home.blog/ with the objective of posting everything there instead of here. If I can build a readership for the ideas and quotes I publish there, I figured, I can bring over the readers back to this blog eventually and grow the kind of things I write about.

    There’s only one problem – I read a lot of varied topics, but the one I write notes most about is politics. I’ve never been comfortable airing my views on politics. It was never taught to me to be overtly political, and the environment I’m in now doesn’t allow for many public mistakes. Whether this is a perceived threat or a real one, I do not care to find out.

    So, within a day of creating the blog, I’ve abandoned it. All my comments are still coming to this blog and hiding in plain site – they are only visible to logged in users. So if you’re curious as to what I write about, ask me and I’ll create an account for you on my blog and let you in. Otherwise, I’m happy writing those thoughts for myself for now.
  2. The other thing that happened was that I noticed that my blogs were running into some technical difficulties. I was not able to update plugins or open MySQL in the browser to take a look at it. Turns out, my VPS thinks it’s run out of space, despite the fact that I recently updates from a 20 GB node to a 50 GB one. I noticed that the /var/mail folder was choked up with thousands of files, and the ibdata1 file has overgrown. I cleared up the former with a nice ‘find -delete’ command, and for the former, I’ve got a script that takes the backup of all my blogs, deletes the ibdata1 file, and reups the backups to bring everything back online. In the end, it tells me how much space it saved me.

    The last time I ran this, maybe last year or so, I regained about 5 GB. So I ran it again. Turns out, I’ve updated my MySQL version somewhere in between and the thing completely broke, without giving me back my two blogs! Gulp!

    Luckily, I read through the script and recovered my blogs, without losing much uptime or any data. But this sort of thing is exactly what scares me. I’ve got scripts that take backups regularly, but it never feels enough.

    Regardless, has anyone else dealt with large ibdata1 files? What can I do about that? Also, I still don’t know why my system thought it’s run out of space. Maybe the sheer number of files in /var/mail? Due to the assumed lack of space, MySQL crashed and wouldn’t launch back up, until I deleted the mail folder’s contents. So I’m not sure I want to be in this situation again!

Running Compass on Vultr


Recently, I came across a tweet by Aaron Parecki, where he talked about a lifelogging app he built (and recently released) which tracks our location constantly.

I’ve been using Moves on-and-off over the years and partly due to it being now owned by Facebook, and partly because it’s a very crashy app (first time works fine, doesn’t open ever after that and stops tracking properly soon after; I assume the developer is now working on some darker features for the Facebook apps and so doesn’t spend as much time on his own creation), I’ve never been satisfied with Moves.

So, I downloaded Aaron’s Overland GPS Tracker app (free!) and set it up. The app is rather bare and the functionality is not well explained (within it). But it’s free, open source, a one-man job, and in line with the vision for indie dev, so it’s up to us to figure things out. I asked a few questions, got pointed to the settings explainer here. Well worth a read if you download the app.

The next step of the app was to install a remote server which ingests the data and makes it human readable and useful. As Aaron explains, the quest is to answer the question – “where was I at blah date at blah time?” The app’s official homepage recommends one of two servers to send the data to – a service called Icecondor and a server Aaron wrote called Compass. Compass looks nicer than Icecondor, is self-hosted, and I’ve been itching to play with Vultr.com‘s SSD Cloud, which competes with DigitalOcean in pricing and resources. So, here’s a walk-through for getting yourself setup with Vultr, installing Compass, and setting it up with Overland GPS to start tracking your location as creepily as Facebook and Google do it! 🙂


Vultr is a nice competitor to DigitalOcean. At $2.50/mo for their cheapest VPS, it’s half the price of what DigitalOcean offers ($5/mo for the same RAM, storage, and CPU, but DO offers twice the bandwidth and, well, is trusted more). There had to be a caveat, right?

I signed up and the first thing I was told to do was to add money to the account. I had the option of not adding any cash and just attaching my credit card, but I’m going to end up using Vultr for something or the other, so I threw $10 at them (shut-up-and-take-my-money style!).

Then, they told me I can deploy a new server! I picked Seattle as my server location, Ubuntu 17.10 as my poison (which was probably a bad idea; more on that later), and scrolled down to the Server pricing. The $10/mo server was pre-selected for me and the $2.50 option was grayed out! (Seriously though, they should give names to these tiers. It’s silly to keep referring to the price.)

I googled around a bit and found out that they keep disabling the cheapest tier (they call it “Temporarily Sold Out”) as a sort of bait-and-switch model to drive new users to the more expensive options. But that sounds somewhat bullshit. If this was truly the behavior, I’d like my money back. But, and I’m glad I did this, I went back and started clicking around to look for solutions. It came in the form of New York! Turns out, they try to drive users to lesser used data centers while everyone who’s trying to set things up actually tries to use the “Silicon Valley” data center (seriously? Who the heck put a data center there???)

New York and Miami currently have open $2.50/mo tiers (ugh, that naming is so needed! I guess I’ll call it the Micro tier and the next one Mini), and networking is not a problem for me (who cares if a little more bandwidth is needed to get this non-time-sensitive data to New York and back), so I picked New York and threw my hat in the ring.

The server came up within… minutes? (Seriously, it was fast!) and I had an IP address to point to! Yay! But, what’s the password? The usual Ubuntu password didn’t work and I looked around at their docs and there wasn’t much to go by (Vultr’s docs aren’t as awesome as DigitalOcean’s. They’re good, just not there yet. They have a documentation bounty program if you’re interested, dear reader.) Then I checked the email which I would have received on server activation. It said that the password is on the dashboard (silly me!).

As I said before, Vultr’s documentation isn’t great, so I followed a mix of Vultr’s LEMP install here and DO’s LEMP stack installation instructions here. I installed PHP 7.1 with FPM (which, I must admit, was a little leap-of-faith because I wasn’t sure Aaron’s code would work without throwing up legacy issues, which it didn’t) and skipped most of the tweaking that Vultr recommends (YMMV).


Then, I copied over the Compass files (from here) and started following the Setup. The first issue was the .env file. There’s a few settings in there which are confusing, so here’s what I did –

BASE_URL -> This is your website. It uses HTTPS. More on that below.

STORAGE_DIR -> This is the data directory which is supposed to store your incoming data. Oddly enough, it doesn’t. When you use the application, the GUI prompts you to make a ‘database’ (it should be called a ‘project’ Aaron). This database makes its own folder in the Compass directory, so this variable invariably doesn’t get used. Set it anyways.

APP_KEY -> This confused me a bit. I don’t think this is a password. But I set it to something like a password. It’s a 32 char string, so have fun setting it up.

DB_CONNECTION -> Set this all up as you would any other MySQL application. Use the WordPress tutorial by DigitalOcean as a hint of what to do.

DEFAULT_AUTH_ENDPOINT -> This was one of the more confusing things I saw. Was the idea that this was some generic authorization? To figure out, I found Aaron’s own Compass website and tried to login. Turns out Aaron uses a very neat authorization process. There’s no password. All you do is tell which Indie authorization website you want to use to authenticate who you are and it’ll allow you to login. Specifying this URL will mean that if you can login to that other website, you can login to this website. The default is set to ‘https://indieauth.com/auth’. If you let this remain, it’ll mean that anyone who has an indie auth login anywhere will be able to create an account on your Compass server and potentially use it for their own data. So, I authenticated myself into Aaron’s server and now I have an account there! Of course, I don’t recommend this. I changed this Endpoint to my withKnown.com site. That way, only people who can login to my withKnown site can login to my Compass server. Who can login to my withKnown server? Only me. 🙂

There’s a piece of the puzzle which needs addressing. APP_DEBUG is set to true right now. So whenever there’s an error, Compass spits out the entire MySQL connection string, including password, as well as very important system information out to anyone to see. I suspect that once you’re done setting up this server and you trust it, you should follow the Laravel process of ‘migrating’ the application from dev mode to production. This will help secure your application.


After this, I moved on to running Composer to install all the dependencies which I needed for Compass. Here’s all the issues I faced there –

“Composer not installed” – Install using

"apt install composer"

“danielstjules/stringy 1.10.0 requires ext-mbstring” –

"apt install php7.1-mbstring"

“phpunit/phpunit 4.8.21 requires ext-dom” –

"apt install phpunit"

“zip extension and unzip command are both missing” –

"apt install zip unzip"

Now, you can run ‘composer install’ and it’ll work.



I recommend using nginx. You’ve got a small server and you don’t want Apache to drown the memory, so just use nginx.

Aaron’s config for nginx were clear, but not helpful, because it doesn’t go with the usual nginx config floating around tutorials. So here’s mine (relevant portions only) –

index index.php index.html index.htm;
root /var/www/nitinkhanna/html/compass/public;

location / { 
    try_files $uri /index.php?$args; 
location /index.php { 
    include snippets/fastcgi-php.conf;
    fastcgi_pass unix:/var/run/php/php7.1-fpm.sock;    
    fastcgi_param   SCRIPT_FILENAME $document_root$fastcgi_script_name;
location ~ \.php$ {
    include snippets/fastcgi-php.conf; 
    fastcgi_pass unix:/var/run/php/php7.1-fpm.sock;

At this point, I thought I was done. But then, when I tried to open the site, I ran into some very nice errors in the application. First of all, notice the root. The root of the application is not the compass folder itself, but the public folder inside it. This is not mentioned anywhere in the documentation and was well worth twenty minutes of “what the heck?” and then some. But you have it on good authority that this is what you’re supposed to do.

Secondly, the application wasn’t done making me install stuff. So I also had to install curl –

apt install php-curl

Then, I wanted to digress a little and make my life a little more difficult (or easy, depending on who you ask). Aaron’s own Compass server uses Let’s Encrypt based SSL. I’ve always wanted to secure my own sites using SSL, but I’m lazy. For this, I thought, why not!

I found the CertBot instructions for installing with nginx and Ubuntu here. They’re pretty straightforward, with a small error that I ran into – Cloudflare. I use Cloudflare as my DNS, security, loadbalancer, God of Small Things. Cloudflare provides SSL. It’s literally a one click. When you add a new A record to your domain (such as compass.p3k.io), it adds DNS and security itself by routing traffic through Cloudflare’s network. CertBot doesn’t work with that. CertBot needs direct access to the server. So, I had to disable Cloudflare’s lovely protection for my subdomain and let certbot do it’s job. It did so. It automatically modified the nginx config to accept HTTPS-only connections and to route all traffic to HTTPS. I was even able to setup crontab to auto-renew certs –

43 6 * * * certbot renew --post-hook "service nginx restart"

After this, you run the job queue commands as listed by Aaron and you should technically have a running website. But there’s a catch, as there always is. This server that I’ve got is not a ‘mini’. It’s a ‘micro’. 512 MB RAM is not enough to run MySQL, Ubuntu 17.10, nginx, php-fpm, and actually run an application on top of that. So, I ran into a very cryptic error –

SQLSTATE[HY000] [2002] No such file or directory 

At this point, I had the application running and I was able to visit the site and all, but try to login and it threw this error. The php artisan command also started throwing this error (by the way, you’re supposed to run the ‘php artisan queue:listen’ command in the background for this server. Follow the instructions here to set up supervisord to do so). Most people on StackOverflow seemed to think that if you replace ‘localhost’ with ‘’ in the app’s settings, it’ll start working again. But that didn’t help. Finally, someone recommended (not in real-time. I’ve only once ever in my life used StackOverflow in real-time to get answers to a question) restarting MySQL. Well duh.

Oh? MySQL won’t restart. Why???

It was this community question on DigitalOcean that gave me the answer I was looking for – I had run out of RAM. Turns out, 512 MB is just enough to play with a server, but not enough to run it for reals. Nonsense. Let’s just add a swap!

I used this excellent and very easy DO tutorial to add swap to my VPS. Notice the shade it throws at you for trying to use swap on SSDs. They specifically say that it doesn’t recommend using swap for DO “or any other provider that utilizes SSD storage” and that this degrades hardware performance for you and “your neighbors”. DO recommends upgrading your instance so it has more RAM instead of using swap. We don’t listen.

Added swap and voila! It’s working! MySQL fires up and the app stops throwing silly errors! I ran htop all night on the instance to monitor for Memory and Swap use and it works just fine! At last, we can login!



OK, we logged in using our designated Indie Auth website! Now what? You’re staring at the blank screen that recommends you create a database. Do it. You give it a fancy name and it spits out a bunch of configuration. Now what? First of all, change the Timezone in the settings to where you are. It’s set to UTC right now, but for me, it’s PST. Also, use

dpkg-reconfigure tzdata

in your Ubuntu command line to change the timezone of your server to where you are. Remember, my server is in New York. But I told it that its timezone is America/Los Angeles. Because.

OK! You’re good to go! You can throw some data at this server! Head over to the Overland GPS app and add this endpoint to it. Only, what’s the endpoint? I added just my compass server’s URL and that didn’t seem to work. Then I looked at the app screenshots and there it was –


That’s your Receiver endpoint! But, where should I find this? In your Compass ‘database’ settings, You’ve got a read token and a write token. Next to the write token is a link which says “show API endpoint”. Click it and out pops another line which shows you the above. Simply copy this and magically move it to your phone (I WhatsApp myself these things) and you can plug it into the app and start sending data! The first time you plug it in, the app will collect all the data you’ve accumulated till then (I had some 25000 points of data to transmit) and smoothly move everything to the server (Aaron really has done a great job with the app). After that, it’ll move the data in batches the size of which you can specify (God knows why).

But. You’ll see some odd things. For example, in the afternoon, the server’s map changed the date over to the next data (I suspect this is because my server was still on UTC time. Running the tzdata command above should solve this). Also, whenever there’s no data (or the data hasn’t loaded yet), the map points to Portland. I get that Aaron is from there, but I think we should be able to configure this (Seattle, woooo!) because it’s a little jarring. Finally, this will teach you how bad your GPS data is anyways. Most of the time, the map has me squarely in the water, or swimming out and coming back, or has me cross the I-90 bridge by, well, not crossing the bridge but swimming along it). But, that’s just the world we live in.


  1. Why does this server need MySQL? The Compass documentation says that the data is stored in flat files. Then is the MySQL database only used for temporary storage of data before it’s processed and saved to flat files?
  2. Is HTTPS a requirement of the server or a nice-to-have? I am not sure about this and I just took the safer route.
  3. The app, in debug mode, spits out way too much information which it shouldn’t. I’d like clear instructions on migrating it off debug mode.
  4. Did I decipher the meaning of DEFAULT_AUTH_ENDPOINT correctly? Not sure. Also, Aaron, if you’re reading this – what do I do with my login on your Compass server? Can you allow people to store their data on there, just for visualization (and wiped every night so as not to flood your server).
  5. I still don’t know what the best configuration is for the app (battery-use to tracking). If you’ve got pointers, throw them in the comments below!