Some quality of life improvements on my iPhone

When iOS 15 dropped, I noticed that it added a feature that Shortcuts could run on their own, without user approval every time. This is a pretty major change to the way they were working before, and allows for some truly good automation.

A few months ago, I created a folder in my Photos app called Wallpapers and added subfolders called Morning and Evening. I created automation that runs at Sunrise and Sunset and sets a random wallpaper from the folders as the lockscreen wallpaper. It’s a nice way to update my lockscreen frequently.

Over time though, I got bored of the same few wallpapers, so I’ve created two more automations – these go out to source.unsplash.com and pull wallpapers using simple search terms.

Unsplash has run their free Source endpoint for a long time and even though it’s technically deprecated, they don’t prevent it’s use if you know what you’re doing. The search terms I use are –

https://source.unsplash.com/1080×1920/?Morning and

https://source.unsplash.com/1080×1920/?Sunset

Note that if you put the search term as “Evening”, it leads to some particularly Non-Family Friendly results.

So now, I’ve got 4 automations – on Mondays, Wednesdays, and Fridays, I set Morning and Evening wallpapers from my local folder. On the rest of the days, I let Unsplash send me some nice wallpapers for my phone twice a day.

The best part of this is that the wallpapers from Unsplash don’t get downloaded to my phone and clutter my photos. They directly get used as wallpapers.


The other quality of life improvement I’ve made is webapps!

At some point, I found this shortcut which lets you create a fullpage standalone browser app icon on your iOS homescreen for any URL or website you pass to it.

I had just installed Amazon Luna and rocketcrab as webapps using Safari’s Add to Homescreen feature some time before that, and really like how they come off almost as proper apps (as good an app as Amazon can make, and they make some spectacularly terrible apps).

When you try to turn a website into a webapp but it doesn’t support this feature, it opens in a new tab in Safari, which takes away from the feeling of a standalone app. But the shortcut above solves that problem!

It creates a webapp using a configuration profile, which you then have to go into the settings app to accept. It’s an unsigned profile, so the risk is all yours. But you can look at what the Shortcut is doing and let me know if there are any security concerns.

One caveat – the shortcut asks for an icon image. You better have one ready when you’re using the shortcut and it has to be more than 128×128 pixel. I tried an image that was 64×64 and the icon just turned out blank.

Since I discovered this, I’ve gone on somewhat of a binge. I made webapps (or Web Clips, as iOS calls them) of three webbooks I’m reading on and off (these aren’t available as ebooks in any way). I also often have to check up on my GitHub Actions runs of a particular secret project, so I made a webapp of that direct URL. I made one of my blog, so I can easily go into the admin section and make edits to my posts in the Gutenberg editor (which still doesn’t have proper support in WordPress iOS apps). The only one I haven’t made (and thus opens in Safari) is solitaired.com and that’s basically because I got lazy. I’ll make it one of these days.


From the time I started writing this post, I made another improvement.

I don’t really like Wallpapers cluttering my photos app. Over time, they make a mess, the good ones used to get lost when I moved phones, and overall, it’s a lot of pain to manage them in the Photos app, which needs a long overdue overhaul, Apple.

I figured out that I can make a shortcut that actually picks a random file from a folder in the Files app. So I moved both the Morning and Evening folders to the iCloud Drive and now I can add any good wallpapers I find on my desktop to my phone too! 🙂

I like when things fall into place nicely like this 😀

Cover art is from emoji.supply, which is a ridiculously awesome source of emoji based wallpapers!

Really enjoying Call Of Duty: Mobile nowadays

I’ve been playing COD:M on my iPhone since the last few… months, and I’ve been really enjoying it. It’s a no-fuss game, with graphics settings to match even my older iPhone 7, and really good development cycles and fast moving seasons (storylines and season passes).

They’ve done a bit of work to gamify the interface though, with lots of pre-game notifications and pop ups that try to convince you to buy the season pass or individual weapons and characters. It’s just enough to not irritate me and feels not more than a hurdle to cross and get to the actual gameplay.

I’ve been getting good at the game. So good, in fact, that in the last five days I’ve gotten banned twice. Both times, I was playing really, really well, and came in first on the scoreboard, so I assumed that one or more players from the other team reported me as probably using a cheat.

I’m not, but that’s a sort of rite of passage of playing FPS games well – you get accused of cheating. Problem is, the first time I was banned for an hour and the second time, for a day! Now I’m worried that if I play too well and get reported again, I might lose access for even longer! And in typical fashion, there is no recourse in-app to contact the moderators when banned.

I wouldn’t say I’m addicted to the game, but who am I kidding. I’m afraid to lose access and this might affect how well I play in the future. But hey, every time I got banned, I ended up spending time on coding on side projects, which is awesome!

All said and done, I’m really enjoying Call Of Duty Mobile and I’d recommend it to everyone! It’s a free download.

Adventures in NOT buying things

pexels-photo-2942361.jpeg

I’ve been thinking about external storage for the last few days, for our iOS devices. When we bought my wife’s iPhone XS Max, we made the mistake of going for the 64 GB option. Pretty soon, tired of a filled-to-the-brim phone, she opted for Apple’s 50GB iCloud solution, priced at $1/mo, to both backup her photos and to shut up Apple’s continuous prompts about a full iCloud.

This solution has been serving her well. Somehow, her photo storage needs have landed at about a 100 GB, which sits well between her phone and the cloud.

But more and more, I’ve been thinking that I want to get rid of the dollar a month charge. For that, the obvious way would be to have daily backups and cleanup, but the question becomes, “to what?”

Dropbox seems like an obvious choice. So does OneDrive. But there’s something irksome about cloud storage. It feels like a gambit – these cloud providers want more of your money, and getting us hooked on Dropbox’s initial awesomeness and then baiting-and-switching to the shitty version of the company they’ve become leaves just an odd taste in my mouth.

So I started thinking of some sort of hardware solution. Many companies have come and gone (see pogoplug), but there’s a product from a few years ago that instantly popped into my mind – the SanDisk iXpand flash drive. This is a little widget that connects to your iPhone through a lighting connector and sucks out all your photos. Compared to when I first saw it, the pricing seems affordable now – 256 GB sets you back $60. The device is actually pretty neat because the other end is a USB-A port, so you can plug it into your computer when it’s time to backup your backups.

But then I started thinking – maybe 256 GB is enough, but the lightning port certainly is not. What if I move to Android one day? Or Apple dumps this port for a USB-C in the future? That’s what freezes me – the what-ifs. Instead of living (and spending) in the now, I worry that my choices might be proven wrong in the future.

So I started looking for wireless storage devices, the kind that can connect through wifi and an app, and work with a majority of devices. First hit – LaCie FUEL 1TB – for $136 on Amazon. Holy crabapples! Twice the price and 4x the storage? Ridiculous! The second result? WD 4TB My Passport Wireless Pro for $190. Oof. I need to do more research! What if I opt for the 4 TB and just around the corner (on the second page of the search results) is a 12 TB one for just a bit more? Storage is a strange world.

These external storage options aren’t without their issues though – sometimes their apps haven’t been updated in a few years, meaning they don’t support new features or even new iOS versions. Most of the apps I looked at (WD My Cloud Home being one) don’t seem to support background uploading. Google Photos and Dropbox can upload your photos to the cloud when charging, but WD has trouble uploading to the HDD sitting next to your phone. Cool.

So, once again, I’m frozen. I know the iXpand is not the best solution. The market has moved on, there is more storage available for a better price, and the future-proofing aspect of using wireless just makes sense.

But there’s one more weird thing at the back of my mind – why fix something that isn’t broken? If my wife’s current storage needs are met at $12/year, then why spend upwards of a hundred dollars to solve it in a worse way (if background uploads don’t work). If I just tell myself that I’m paying $1/month for “external storage”, I’m a much happier person. Aren’t I?

Do you, dear reader, use any external wireless mobile storage? Which one? Are you happy with it? What quirks does it have?

Refreshing my RSS feeds list

Welp, I’ve done it this time. I was fiddling with some settings in my current feed reader of choice – Fiery Feeds – and I hit a sync button that’s meant to download everything from iCloud and rebuild the database. Turns out, iCloud is, as usual, not good at actually saving important data. Part of this is my fault. I have had some 14,000 unread items in there, and about 900 feeds. Sync would often time out and almost never complete.

So I lost all my feeds. As I stared at it dumbly, waiting for the feeds to come back, a calm came over me. This is what inbox zero feels like. When, after multiple forced syncs later, nothing happened, I was relieved.

I thought about it. The last OPML export I have is from December of 2019. I’ve added maybe 20 feeds since then, which are now lost. If I import the OPML, I’ll get back my starred items and general state, but I’ll not get back the calm.

So, I’ve decided to do an overhaul of my feeds. I know a lot of sites I’ve subscribed to either don’t exist any more, or haven’t updated in a while. So it’s time to shed the load.

Working through this large an OPML file is a chore. First, I tried to do it manually. Too much work. Then I tried to find tools to help. I found a six year old github repo to find dead feeds. It found a few, but mostly got it wrong. Instead, I’ve imported the OPML to my Firefox LiveMarks extension. It’s not the perfect solution, but at least I’m able to go through the list faster and cull it satisfactorily.

Other than the feeds that are dead, I’m also striving to shed some weight. At some point, I subbed to some GTD and Productivity feeds. Deleted those. It’s no longer my area of interest. Older still are feeds related to Network Engineering. It’s what my MS is in, but it’s no longer my main area of concern. So I’ve removed those. I’m also removing webcomics that haven’t been updated since mid-2019. There are quite a few of those. Frankly, it’s fine if the authors want to take a break. I, too, don’t update my blog often. But there are other ways for me to discover their content. Tapas and Instagram are doing a good job, so I’m going to lean on those for my comic needs. This doesn’t mean all webcomics are going away from my feeds. On the contrary, I’m keeping most of them, specially long-running stories that I follow keenly, like Gaia comic, and Slack Wyrm. But others are out.

At some point I also subscribed to a lot of programming related blogs. Those are nice navel-gazing, but ultimately worthless to me. I’m not a programmer, I’m a scripter. I’m not into deep programming concepts even on the languages that are my bread, butter, and jam – python and JavaScript. So for me to sub to serious computer scientists and programmers was a mistake then, and is a mistake now. It’s not that I won’t glean something off them, just that I don’t need to, right now.

This is tough work, but it’ll be worth it in the end. Recently, I found out that a friend has a very strict gate on who she follows on Instagram. She has a roster of 99 people and whenever she has to follow someone new, she forces herself to remove one person from the list. I’ve never, ever removed a feed from my list. This is the same list I’ve been carrying around since my first RSS feed reader – Fever – and some items are even carried forward from Google Reader. I’ve always thought that at best, the feeds that die are not much extra weight than some processing cycles, and at worst, the items I don’t read get deleted at the end of my 15 days, one month, two months, 90 days limit. That moving limit is part of the cause of all this trouble I’m in.

But the largest forcing function is my feed reader. Fiery Feeds is an awesome piece of software and Lukas Burgstaller is an exceptional dev, and a highly responsive support person. But I made a conscious choice at one point to move away from all server-side RSS feed services and use Fiery Feeds’ native, on-device accounts. I’m paying for the app because I love and want to support it, so I might as well use the biggest feature Lukas has introduced. But this on-device, synced-via-iCloud system has its drawbacks, and this means that I can’t be an ignorant buffoon about my feeds any more. I have to shed, cull, strip, whatever you want to call it.

One very interesting thing I’ve done over time is to use kill-the-newsletter.com to the best of its abilities. I do not like newsletters, but there’s a LOT of content that’s going to email newsletters exclusively nowadays, and that sucks. Kill The Newsletter converts these emails to RSS feed items. It’s not a perfect solution, specially since it’s a bit of a blackbox, but it works just fine for now and it’s FOSS, so I’m happy. So, these are a guilty pleasure I’m not getting rid of. We’ll see how this decision pans out. Maybe I’ll have to figure out a way to merge all newsletters into one RSS feed. Or use a dedicated app to read newsletters on my iPhone. There are a few of those out there now.

All in all, this is an exercise in refreshing and rethinking what I consume online. Hopefully, it’ll lead to a better feed reading experience for me.

Reeder 4 is here?!

I’ve been experimenting with Tiny Tiny RSS for a couple of weeks now, to figure out if I’m done with Fever RSS. Part of it is that Fever runs on a Digital Ocean VPS and I’d like to reduce the load on that server as much as possible.

I run TT-RSS on my home PC and get access to it using ZeroTier wherever I am. My app of choice for it on the move is Fiery Feeds. It’s a beautiful app with an amazing, rich feature set. But the main reason I use it is because Reeder doesn’t natively support TT-RSS.

But, I’m learning that I’m set in my ways. the TT-RSS web app feels just as dated as Fever, and there are some features in Fever that I really enjoyed, which I miss in TT-RSS. Since I’m looking at two dated, self-hosted RSS readers, why not go with the one I’ve used more?

Also, Fiery Feeds is gorgeous, but it’s not Reeder. Reeder is familiar and cozy. It’s got matured syncing and great UX. Fiery Feeds comes close, but there are some things which are just irksome, such as the way it opens up articles as a pop up. The cadence of Reeder’s panes is just beautiful to work with.

Today I learnt that Reeder 4 has been out since April 25th. At an absurd cross-device cost of $5, it’s a steal for all the features Reeder gives you.

But here’s the flip side – I only use Fever (and now TT-RSS) and while TT-RSS has a plug-in for Fever, I’m more inclined to just use Fever. Which means I have no space for using any other paid or free RSS reader service. So I’m really restricted in what I’m doing with Reeder. Further, though I’ve started using Instapaper as my read later solution, I’ve never really understood the point of using Reeder for Instapaper. It’s fine to skim through, but wouldn’t you rather use the Instapaper apps to read the content? For me, the highlighting and notes functionality of Instapaper is essential. How does one ignore that to read content on Reeder?

Lastly, there’s the demise of Mercury. Mercury has been my savior in this mess of RSS feeds. My TT-RSS installation has a Mercury plugin that cleans up and extracts content from the feeds. It really just works, and since they’ve made it open source and unavailable as an API, Reeder is bound to suffer. Reeder 4 is now experimenting with something called Bionic Reading and it’s a hit or miss. We’ll see.

So here it is –

  1. I’m tired of TT-RSS and untired of Fever (for now).
  2. I love Reeder 3 and Reeder 4 is ridiculously cheap for an upgrade. I have an older iPad Pro, so Reeder 3 works really well for me. But buying 4 means supporting Rizzi in the amazing work he’s doing.

  3. Fiery Feeds is awesome and pretty and useful, but I don’t pay for the subscription, so I don’t get to use the fancy automatic folders that it creates.

  4. RIP Mercury

  5. I really want to get my RSS feed reader off the VPS and onto my local setup, secured and made accessible through ZeroTier.

How many people reading this have already bought Reeder 4?

How is it?

Have you used Bionic Reading?

Which RSS reader service do you use it with?

Do you read Instapaper articles on Reeder 4? How is that experience?

Reuters takes offense at hacked apps in iOS

It is unclear how much revenue the pirate distributors are siphoning away from Apple and legitimate app makers.

Source: Software pirates use Apple tech to put hacked apps on iPhones | Reuters

It’s taken a long time and another massive Facebook privacy scandal for the news media to discover this underbelly of hacked apps chugging along happily due to Apple’s Enterprise Apps program.

I’ve used one on and off – Instagram++

I must say, it’s a liberating experience – I see no ads on Instagram, I see no random “Suggested Friends to Follow” crap.

I had to resort to this because my Instagram experience is vastly worse off than my wife’s and my friends’. I see, on average, 3x more ads on Instagram than others around me. How many ads does my wife see? None.

So to my mind, using Instagram++ makes perfect sense. If I can hack my way to a better UX, why shouldn’t I? It’s the same as using an adblocker.

I don’t support piracy of services. There’s no legit reason to not pay for Spotify.

As for hacked games, well, cheats and hacks have always existed, and will continue to exist, despite the alarmed voice of this Reuters article.

Also, the article got one thing wrong – I’ve observed Apple kick out the Enterprise cert almost once a month, sometimes two or three times a month. They seem to make it sound as if Reuters alerting Apple was the only thing that forced Apple into action.

They’re very much aware of the problem and can’t or won’t do much about it. Talking about it as if it’s the end of the App Store is just noise.

As for how much revenue these services generate? Not close to enough. They do seem to have a comfortable existence, and so might be able to get around Apple’s 2FA proposal by just buying a bunch of phone numbers in China. But do they run a massive profit? You bet that if they did, Apple would be all over them.

This is the same as the jailbreak community in some senses – only a small percentage of users are actually trusting these services not to misuse the extensive powers that Enterprise certs give them. Out of that small percentage, a further small percent is paying for it.

It’s sad that large companies like Facebook pulling the shit that they do often also bring to light little players that are just trying to provide a good service to users.

Now, the technical aspect of this – Instagram++ is available online for download as an IPA if you want to use your own developer account. If you don’t have a dev account, Apple now allows side-loading, but it is a cumbersome process that expires after 7 days. Apple’s earlier sideloading used to be 30 days. When Apple made it free for everyone to sideload (not just if you’re a $99/year paying developer), they reduced the time frame of the cert to 7 days, which in my mind is a total d*ck move.

If Apple really wants to combat Enterprise cert misuse while letting users do whatever they want with their systems, they can just legitimize sideloading and let me choose when my cert would expire, but Apple isn’t that generous.

Till a good solution presents itself, services like TweakBox, Tutu, and AppValley will continue to operate by hook or by crook. So be it.

The Original iPad mini and Apple’s fluid vision

It is meaningless, unless your tablet also includes sandpaper, so that the user can sand down their fingers to around one-quarter of the present size. Apple’s done extensive user-testing on touch interfaces over many years, and we really understand this stuff. There are clear limits of how close you can physically place elements on a touch screen before users cannot reliably tap, flick, or pinch them.

Source: A Look Back at the Original iPad mini – MacStories

 

It seems like Steve Jobs and Apple understood that you can’t place things too close inside the screen, but forgot that you can’t place the screen and the edge too close either, because it’ll cause hours of headaches by unwanted swipes, taps, and hard presses. The Apple of today thinks bezels are bad and it is wrong. Steve Jobs might have said the above, but he’s also the one constantly touting that they made their devices thinner, which reduces battery life and also the ‘holdability’ of mobile devices.

(Proof of the holdability issue – When was the last time you held your iPhone 7 or above naked, without a case, and felt confident that you’ll not drop it? It’s been months for me and when I did it last night, it felt alien. It seems Apple has outsourced the job of holdability to the cases that we inevitably put on our sold-a-kidney-for devices.)

I love my original iPad Mini and still use it. It’s a very well built device. The iPads of today make me feel like Apple just wants to make the jump to touch ‘computers’ instead of trying to keep the iPad what it is – a touchscreen tablet that feels different than anything else on the market.

The difference between a touch computer and a touch tablet? The former, you keep on your desk and work on using a keyboard (think Surface). The latter, your kids hold while they’re watching YouTube videos on in the car.

But this gives Apple a great new diversification strategy – do you want an iPad to work or an iPad to play?

Till now, they’ve kept these two together. But maybe, bowing to market forces, they’ll break these two use cases apart and give us two iPads that do very different things. That’ll require Apple to stop treating the iPad like it’s just the overgrown brother of the iPhone. Let’s see if they do that.

p.s. With iOS 12, if Apple is truly committed to making software releases that don’t completely destroy older devices, that’s also relevant to corporate uses. Companies don’t keep updating everyone’s hardware every two years ‘because the software got old’. So if Apple wants an iPad on every office desk (as they should), they really need to get their software updates game right, which they seem to be on a path to.

p.p.s I was going to call the title “Apple’s faltering vision” (because clickbait!) but Apple’s vision is rather fluid. If they see a market segment responding well, they go after that, instead of doubling down on losing segments like some other companies do.

Security vs Usability

I’ve come to a point where I do **not** update apps, plugins, software in general. I know that’s a regressive approach to safety, but safety can’t keep trumping usability all the time.

Source: My comment on Stephen’s Notebook

 

Every few days, I have a conversation about security vs usability somewhere. With my iPad Mini, I blindly trusted Apple to do the right thing and they’ve screwed me over. It’s a beloved device, destroyed completely by iOS 9.

So I’ve basically given up on this bullshit harp that companies sing of ‘security’ to shove software updates down our throats. Sometimes it’s their stupidity, and sometimes it’s just them being sinister. The new Microsoft is the old Microsoft. The benevolent Apple is an insidious Apple. Don’t get me started on Facebook, twitter, and Google. Gmail is just the latest casualty of our overzealous overlords.

Yes, security is a big problem. Yes, it needs constant vigilance. But just like national defense budgets, one key phrase doesn’t allow organizations to completely railroad people’s expectations, asks, hopes, and in this case, UX.

If you’re concerned that by not updating software, you’re living on the edge, restrict the things you do on that device, while keeping other devices that are completely updated and secured. Use only frequently updated third party browsers instead of the default options. Read up on the latest security scares on the Internet and just be aware of the situations you can get into. But most importantly – back up. Make frequent backups of things you care about. I don’t care if it’s as much as letting iCloud run its course every night, and Google Photos siphoning off your pics. Just do it so that if you brick your device, or get hacked, you’re not set back a hundred years.

99% of security is just keeping your eyes open.

How do you like them upgrades?

Every few days, my iPhone politely but firmly nudges me to ‘downgrade’ my iOS from iOS 10 to iOS 11. I say downgrade because that’s what iOS 11 is to me – a crappy OS that was shoved out with half baked ideas which work well for the latest and greatest iPhone, but not at all for any other device Apple supposedly still supports. Getting rid of that prompt requires careful jumping through a confusing menu that makes it too easy to accept a “sure go ahead with this change at night when no one is watching” option. Most of the time, I am able to do just that. But last night, in a haze of trying to actually use my phone, I must have hit the wrong button, because when I woke up, my phone had restarted and was magically on iOS 11.4.1. Yay.

Before I talk about iOS 11, I just want to say why I didn’t want to get on it –

  1. It’s terribly built – simple features such as the ability to close apps quickly (in a few years time, Apple will reveal that just like their battery nonsense, closing apps DOES actually increase the speed of the phone, as empirically witnessed by a Bajillion people), the ability to turn off the wifi completely through the Control Center, the ability to actually use the phone for half an hour without draining the battery completely (my wife got on iOS 11 as soon as it released and she had the worst experience possible with that OS) were nice to have in iOS 10.
  2. I won’t be able to use all my apps – Apple, with iOS 11, waged a war on 32 bit apps. Now, most apps (99.9% I’d say) were smart about it and went 64bit, but I still have 4 apps on my phone, two of which I was using every few days till yesterday, which are 32 bit. So long Stress Baal and Sunstroke. You will be sorely missed.
  3. It will most certainly screw up my Apple Watch – I have a Series 0 (zero) Apple Watch. When will I buy the new one? Probably not for another few years. It’s a watch. It’s somewhat smart and lets me see messages and cut phone calls, but that’s about it. Do I need LTE? If AT&T pays me $15/mo instead of charging it from me, I might. But one minute into using the new OS, I was told to update my Watch from version 3.2.3 to 4.3.2 and told that if I do not, the phone will force unpair my watch and reset it. Thanks Obama. I exited the Watch app on my phone and plan on opening it at some point in the future. My watch is no longer getting notifications and isn’t able to send heart rate data to the phone (so much for Apple’s “we’re helping you take care of your health” crap. If the data collection is conditional, it’s not really helpful, is it?). But I know that watchOS 4 will screw up the watch, the third party apps, the battery usage. Basically, this is Apple’s way of making you buy a new watch. NO.

Now, coming to iOS 11. I immediately noticed that most apps seem to work differently – Google Maps had some new and interesting UI changes, Egg Inc had AR, the photos app had an irritating number of new features it had to tell me about before it let me use the app, the screenshots were showing up at the bottom (which is nice), etc.

Oh wait, backup. AR. That gleaming, new, awesome technology that’s changing the world! Yeah, I used it. For about 30 seconds. Then I was done.

Literally the only thing I could imagine using AR for – Egg, Inc. With that, my AR experience has ended. Well done, Apple.

Incidentally, I only recently watched this rather interesting video about how Apple will eventually launch AR glasses and they will be more successful than Google’s half-ass attempt because, well, Apple. It’s worth a watch 🙂 –

The rest of the stuff, is as I expected – meh. The app switcher can now close apps (yay!). The wifi stupidity that Apple propagated with iOS 11 is still there (so it’s always going to drain your battery no matter what). The animations and speed of launching apps is meh. Apple really wanted to make you feel something different, and well, I feel it, but I don’t care for it. It’s more a disruption than a nice addition. Plus, if you close an app that sits at the top of the screen vs at the bottom, the animation helps you see where the app is ‘going to’, but that’s really a rather stupid thing to care for Apple. I say that because I’m sure anyone who has as many apps as I do uses the search bar to get to apps instead.

Oh, yeah, that might be the silver lining – in iOS 10, I would swipe down, type out the name of an app I want, and the phone would just sit there, like a dunce, unsure of what I want it to do. Something was really borked in the code there and sometimes the search would work perfectly and other times it would go completely for a toss. Hopefully, that experience will be more consistent with iOS 11. If not, I’ll know that Apple did not even bother improving the Siri search code underneath and just dressed it in iOS 11 style. Typical Apple. Let’s see.

I’m no Luddite. I like experimenting with new stuff. But I really was hoping to go directly from iOS 10 to iOS 12. When iOS 12 drops, it’ll most likely not support my Series 0 watch. But at least it’s purported to be better than this monstrosity Apple threw our way. It’s OK to skip an OS, it’s OK to turn off auto-upgrades and auto-updates and watch your ‘to update’ App Store list burgeon to 197 apps. It’s OK to let the latest and greatest go while developers work on hardening releases. We all do it in some sphere of our lives. It’s just that my sphere was the one I’m staring at the most during my day – my phone. I want it to be consistent, familiar, and with less fluff. Sometimes people stick to a particular iPhone for a lot longer than they can, because they like the form factor and the materials used. Well, iOS 10 was that for me. But now my phone has moved past it. Time to adopt the new and shiny and see what changes this brings. Hopefully some nice AR filters.

Running Compass on Vultr

Intro

Recently, I came across a tweet by Aaron Parecki, where he talked about a lifelogging app he built (and recently released) which tracks our location constantly.

I’ve been using Moves on-and-off over the years and partly due to it being now owned by Facebook, and partly because it’s a very crashy app (first time works fine, doesn’t open ever after that and stops tracking properly soon after; I assume the developer is now working on some darker features for the Facebook apps and so doesn’t spend as much time on his own creation), I’ve never been satisfied with Moves.

So, I downloaded Aaron’s Overland GPS Tracker app (free!) and set it up. The app is rather bare and the functionality is not well explained (within it). But it’s free, open source, a one-man job, and in line with the vision for indie dev, so it’s up to us to figure things out. I asked a few questions, got pointed to the settings explainer here. Well worth a read if you download the app.

The next step of the app was to install a remote server which ingests the data and makes it human readable and useful. As Aaron explains, the quest is to answer the question – “where was I at blah date at blah time?” The app’s official homepage recommends one of two servers to send the data to – a service called Icecondor and a server Aaron wrote called Compass. Compass looks nicer than Icecondor, is self-hosted, and I’ve been itching to play with Vultr.com‘s SSD Cloud, which competes with DigitalOcean in pricing and resources. So, here’s a walk-through for getting yourself setup with Vultr, installing Compass, and setting it up with Overland GPS to start tracking your location as creepily as Facebook and Google do it! 🙂

Vultr

Vultr is a nice competitor to DigitalOcean. At $2.50/mo for their cheapest VPS, it’s half the price of what DigitalOcean offers ($5/mo for the same RAM, storage, and CPU, but DO offers twice the bandwidth and, well, is trusted more). There had to be a caveat, right?

I signed up and the first thing I was told to do was to add money to the account. I had the option of not adding any cash and just attaching my credit card, but I’m going to end up using Vultr for something or the other, so I threw $10 at them (shut-up-and-take-my-money style!).

Then, they told me I can deploy a new server! I picked Seattle as my server location, Ubuntu 17.10 as my poison (which was probably a bad idea; more on that later), and scrolled down to the Server pricing. The $10/mo server was pre-selected for me and the $2.50 option was grayed out! (Seriously though, they should give names to these tiers. It’s silly to keep referring to the price.)

I googled around a bit and found out that they keep disabling the cheapest tier (they call it “Temporarily Sold Out”) as a sort of bait-and-switch model to drive new users to the more expensive options. But that sounds somewhat bullshit. If this was truly the behavior, I’d like my money back. But, and I’m glad I did this, I went back and started clicking around to look for solutions. It came in the form of New York! Turns out, they try to drive users to lesser used data centers while everyone who’s trying to set things up actually tries to use the “Silicon Valley” data center (seriously? Who the heck put a data center there???)

New York and Miami currently have open $2.50/mo tiers (ugh, that naming is so needed! I guess I’ll call it the Micro tier and the next one Mini), and networking is not a problem for me (who cares if a little more bandwidth is needed to get this non-time-sensitive data to New York and back), so I picked New York and threw my hat in the ring.

The server came up within… minutes? (Seriously, it was fast!) and I had an IP address to point to! Yay! But, what’s the password? The usual Ubuntu password didn’t work and I looked around at their docs and there wasn’t much to go by (Vultr’s docs aren’t as awesome as DigitalOcean’s. They’re good, just not there yet. They have a documentation bounty program if you’re interested, dear reader.) Then I checked the email which I would have received on server activation. It said that the password is on the dashboard (silly me!).

As I said before, Vultr’s documentation isn’t great, so I followed a mix of Vultr’s LEMP install here and DO’s LEMP stack installation instructions here. I installed PHP 7.1 with FPM (which, I must admit, was a little leap-of-faith because I wasn’t sure Aaron’s code would work without throwing up legacy issues, which it didn’t) and skipped most of the tweaking that Vultr recommends (YMMV).

Compass

Then, I copied over the Compass files (from here) and started following the Setup. The first issue was the .env file. There’s a few settings in there which are confusing, so here’s what I did –

BASE_URL -> This is your website. It uses HTTPS. More on that below.

STORAGE_DIR -> This is the data directory which is supposed to store your incoming data. Oddly enough, it doesn’t. When you use the application, the GUI prompts you to make a ‘database’ (it should be called a ‘project’ Aaron). This database makes its own folder in the Compass directory, so this variable invariably doesn’t get used. Set it anyways.

APP_KEY -> This confused me a bit. I don’t think this is a password. But I set it to something like a password. It’s a 32 char string, so have fun setting it up.

DB_CONNECTION -> Set this all up as you would any other MySQL application. Use the WordPress tutorial by DigitalOcean as a hint of what to do.

DEFAULT_AUTH_ENDPOINT -> This was one of the more confusing things I saw. Was the idea that this was some generic authorization? To figure out, I found Aaron’s own Compass website and tried to login. Turns out Aaron uses a very neat authorization process. There’s no password. All you do is tell which Indie authorization website you want to use to authenticate who you are and it’ll allow you to login. Specifying this URL will mean that if you can login to that other website, you can login to this website. The default is set to ‘https://indieauth.com/auth’. If you let this remain, it’ll mean that anyone who has an indie auth login anywhere will be able to create an account on your Compass server and potentially use it for their own data. So, I authenticated myself into Aaron’s server and now I have an account there! Of course, I don’t recommend this. I changed this Endpoint to my withKnown.com site. That way, only people who can login to my withKnown site can login to my Compass server. Who can login to my withKnown server? Only me. 🙂

There’s a piece of the puzzle which needs addressing. APP_DEBUG is set to true right now. So whenever there’s an error, Compass spits out the entire MySQL connection string, including password, as well as very important system information out to anyone to see. I suspect that once you’re done setting up this server and you trust it, you should follow the Laravel process of ‘migrating’ the application from dev mode to production. This will help secure your application.

 

After this, I moved on to running Composer to install all the dependencies which I needed for Compass. Here’s all the issues I faced there –

“Composer not installed” – Install using

"apt install composer"

“danielstjules/stringy 1.10.0 requires ext-mbstring” –

"apt install php7.1-mbstring"

“phpunit/phpunit 4.8.21 requires ext-dom” –

"apt install phpunit"

“zip extension and unzip command are both missing” –

"apt install zip unzip"

Now, you can run ‘composer install’ and it’ll work.

 

nginx

I recommend using nginx. You’ve got a small server and you don’t want Apache to drown the memory, so just use nginx.

Aaron’s config for nginx were clear, but not helpful, because it doesn’t go with the usual nginx config floating around tutorials. So here’s mine (relevant portions only) –

index index.php index.html index.htm;
root /var/www/nitinkhanna/html/compass/public;

location / { 
    try_files $uri /index.php?$args; 
}
location /index.php { 
    include snippets/fastcgi-php.conf;
    fastcgi_pass unix:/var/run/php/php7.1-fpm.sock;    
    fastcgi_param   SCRIPT_FILENAME $document_root$fastcgi_script_name;
}
location ~ \.php$ {
    include snippets/fastcgi-php.conf; 
    fastcgi_pass unix:/var/run/php/php7.1-fpm.sock;
}

At this point, I thought I was done. But then, when I tried to open the site, I ran into some very nice errors in the application. First of all, notice the root. The root of the application is not the compass folder itself, but the public folder inside it. This is not mentioned anywhere in the documentation and was well worth twenty minutes of “what the heck?” and then some. But you have it on good authority that this is what you’re supposed to do.

Secondly, the application wasn’t done making me install stuff. So I also had to install curl –

apt install php-curl

Then, I wanted to digress a little and make my life a little more difficult (or easy, depending on who you ask). Aaron’s own Compass server uses Let’s Encrypt based SSL. I’ve always wanted to secure my own sites using SSL, but I’m lazy. For this, I thought, why not!

I found the CertBot instructions for installing with nginx and Ubuntu here. They’re pretty straightforward, with a small error that I ran into – Cloudflare. I use Cloudflare as my DNS, security, loadbalancer, God of Small Things. Cloudflare provides SSL. It’s literally a one click. When you add a new A record to your domain (such as compass.p3k.io), it adds DNS and security itself by routing traffic through Cloudflare’s network. CertBot doesn’t work with that. CertBot needs direct access to the server. So, I had to disable Cloudflare’s lovely protection for my subdomain and let certbot do it’s job. It did so. It automatically modified the nginx config to accept HTTPS-only connections and to route all traffic to HTTPS. I was even able to setup crontab to auto-renew certs –

43 6 * * * certbot renew --post-hook "service nginx restart"

After this, you run the job queue commands as listed by Aaron and you should technically have a running website. But there’s a catch, as there always is. This server that I’ve got is not a ‘mini’. It’s a ‘micro’. 512 MB RAM is not enough to run MySQL, Ubuntu 17.10, nginx, php-fpm, and actually run an application on top of that. So, I ran into a very cryptic error –

[PDOException]                                    
SQLSTATE[HY000] [2002] No such file or directory 

At this point, I had the application running and I was able to visit the site and all, but try to login and it threw this error. The php artisan command also started throwing this error (by the way, you’re supposed to run the ‘php artisan queue:listen’ command in the background for this server. Follow the instructions here to set up supervisord to do so). Most people on StackOverflow seemed to think that if you replace ‘localhost’ with ‘127.0.0.1’ in the app’s settings, it’ll start working again. But that didn’t help. Finally, someone recommended (not in real-time. I’ve only once ever in my life used StackOverflow in real-time to get answers to a question) restarting MySQL. Well duh.

Oh? MySQL won’t restart. Why???

It was this community question on DigitalOcean that gave me the answer I was looking for – I had run out of RAM. Turns out, 512 MB is just enough to play with a server, but not enough to run it for reals. Nonsense. Let’s just add a swap!

I used this excellent and very easy DO tutorial to add swap to my VPS. Notice the shade it throws at you for trying to use swap on SSDs. They specifically say that it doesn’t recommend using swap for DO “or any other provider that utilizes SSD storage” and that this degrades hardware performance for you and “your neighbors”. DO recommends upgrading your instance so it has more RAM instead of using swap. We don’t listen.

Added swap and voila! It’s working! MySQL fires up and the app stops throwing silly errors! I ran htop all night on the instance to monitor for Memory and Swap use and it works just fine! At last, we can login!

 

Overland

OK, we logged in using our designated Indie Auth website! Now what? You’re staring at the blank screen that recommends you create a database. Do it. You give it a fancy name and it spits out a bunch of configuration. Now what? First of all, change the Timezone in the settings to where you are. It’s set to UTC right now, but for me, it’s PST. Also, use

dpkg-reconfigure tzdata

in your Ubuntu command line to change the timezone of your server to where you are. Remember, my server is in New York. But I told it that its timezone is America/Los Angeles. Because.

OK! You’re good to go! You can throw some data at this server! Head over to the Overland GPS app and add this endpoint to it. Only, what’s the endpoint? I added just my compass server’s URL and that didn’t seem to work. Then I looked at the app screenshots and there it was –

https://compass.p3k.io/api/input?token=E6ncEYWxT...

That’s your Receiver endpoint! But, where should I find this? In your Compass ‘database’ settings, You’ve got a read token and a write token. Next to the write token is a link which says “show API endpoint”. Click it and out pops another line which shows you the above. Simply copy this and magically move it to your phone (I WhatsApp myself these things) and you can plug it into the app and start sending data! The first time you plug it in, the app will collect all the data you’ve accumulated till then (I had some 25000 points of data to transmit) and smoothly move everything to the server (Aaron really has done a great job with the app). After that, it’ll move the data in batches the size of which you can specify (God knows why).

But. You’ll see some odd things. For example, in the afternoon, the server’s map changed the date over to the next data (I suspect this is because my server was still on UTC time. Running the tzdata command above should solve this). Also, whenever there’s no data (or the data hasn’t loaded yet), the map points to Portland. I get that Aaron is from there, but I think we should be able to configure this (Seattle, woooo!) because it’s a little jarring. Finally, this will teach you how bad your GPS data is anyways. Most of the time, the map has me squarely in the water, or swimming out and coming back, or has me cross the I-90 bridge by, well, not crossing the bridge but swimming along it). But, that’s just the world we live in.

 

Questions/Issues
  1. Why does this server need MySQL? The Compass documentation says that the data is stored in flat files. Then is the MySQL database only used for temporary storage of data before it’s processed and saved to flat files?
  2. Is HTTPS a requirement of the server or a nice-to-have? I am not sure about this and I just took the safer route.
  3. The app, in debug mode, spits out way too much information which it shouldn’t. I’d like clear instructions on migrating it off debug mode.
  4. Did I decipher the meaning of DEFAULT_AUTH_ENDPOINT correctly? Not sure. Also, Aaron, if you’re reading this – what do I do with my login on your Compass server? Can you allow people to store their data on there, just for visualization (and wiped every night so as not to flood your server).
  5. I still don’t know what the best configuration is for the app (battery-use to tracking). If you’ve got pointers, throw them in the comments below!