Giving up on WordPress embeds

Back in 2012, I wrote a post called Conversations as a future of blogging. Well, I say I wrote, but actually I must have had a conversation on this up-and-coming platform called branch.com with a few people I invited to it from twitter. Using their tools, I embedded the conversation to my blog on the link above. I thought the embed would last forever, because a) where is branch going to go, and b) WordPress usually gracefully downgrades embeds, right?

Nope. branch.com died three years after launch. They gave us a period of time to request backups of our conversations. I remember doing that, but I never received one in my email. Perhaps they expected me to come back to their site to download the files? I asked all the other people involved in the conversation and none of them cared to request a backup of the copy. This goes back to my posts here about the futility of exporting your data from online services, even though in this case, a full backup was exactly what I needed to recover my blog post.

I was further wrong in that WordPress does no such thing with embeds. In the case of officially supported embeds, I’ve seen the system do this once before and I thought that was standard behavior, but in the case of link embeds, there’s no clear way for it to be possible.

Recently, I introduced a random post finder to my blog. With it, I’ve been discovering a lot of great and terrible posts I’ve written over time. It’s my way of getting closer to my blog.

But it has also left me reeling from all the broken links and embeds that I trusted to work forever. The branch.com embed, images and links I’ve linked to, PDF files I thought would never go away. Heck, even Facebook CDN stuff has disappeared and that company doesn’t let go of any data!

Recently, Automattic introduced inline GIFs from giphy through Jetpack. The model they’re following is pretty neat – they shipped the Gutenberg editor within WordPress 5, and have been extending it using Jetpack with blocks that allow various kinds of content, including GIFs. It shows the scope that Gutenberg has in the future.

Coming back to embeds, Giphy, the company Automattic is leaning on, came into existence in 2013, and I suspect will be out the door before its 10 year mark. Such is the way of life on the Internet. All of these are fads and fads can raise millions, but they eventually all die. Automattic will simply pull out the block from Jetpack and replace it with something else, but we users will be left with broken links and missing context on our blogs.

When the block was introduced some time ago, I played with it and added some GIFs to a blog post. It’s a lot of fun to express ourselves visually. But if branch.com is any indication, embeds come back to bite us later on.

Therefore, I’m getting off embeds. I’d rather download the GIF and upload it to my media. I’d rather take a screenshot of a webpage than to iframe it and hope it sticks around a few years from now. Jetpack already has a massive CDN operation behind them, so you’re never really serving your content directly from your site if they can help it. So there’s no need to worry about storage and bandwidth issues.

As I go through my site, rediscovering old posts, I’ll keep coming across these embeds. Whenever I have context, I’ll try to replace it with relevant information. But, as in the case of branch.com, I will just put a note that explains what happened there, some general thoughts on the topic, and move on. Once bitten by embeds, twice shy now.

A year with Facebook

A year ago, I decided to change my relationship with Facebook. I decided to be more active on the network, but not in the way Facebook would want me to be – commenting, liking, browsing, and clicking more.

I wanted to use Facebook to put out my thoughts more. So I actively started blogging more and putting it all on Facebook, a practice I had stopped for a while because I wasn’t getting anything out of it. I did another thing – something I’ve often been told off for, but I just wanted to experiment with – I connected my twitter account with Facebook. The benefit? All my tweets (and quote tweets, which is a little silly) started getting posted to Facebook. This meant that each passing, silly thought, which we often toss into the void, became instantly visible to my real life friends.

In a way, I did do all the things that would be considered an increase in Facebook activity – I have spent the past year listening to a podcast called Philosophize This! The podcast has an accompanying community on Facebook. Though I didn’t interact with the community much, I did become a part of it. I also found a community relating to an app I use a lot – Day One. The community also chugs along, though I’ve not derived as much value from it as I would like.

I also started using Facebook a lot more. There was a time when I would gleefully count the stupid notification counter on the Facebook website approach 99. I call it stupid not because I have prejudice against it. I like notifications. They’re an excellent approach to garnering attention. But somewhere along the way, Facebook decided that I am not a worthy enough user of their service and they downgraded my experience. They made the counter stupid by pushing every little activity to it. Things which belong in the newsfeed – someone posted something, someone liked someone else’s post, someone had a birthday – were suddenly in my notifications. But at the beginning of the year, I decided to be more proactive, hoping that the algorithm would notice this and rid me of the stupid notifications and only give me the smart ones. I’ll let you know that the algorithm is not smart. It never did recognize my contribution and that portion of the experiment quickly bombed. Now I don’t care what the notification counter says. Whenever it irks me, I click it to reset it and ignore the notifications. (They’ve added even more notifications now – friend suggestions, community posts; heck they’ve even added Facebook notifications to the Instagram app, because why ruin just one social network when you can ruin two?)

I even went ahead and actively started using Instagram. I thought, maybe one Facebook property will feed into the algorithm of the other? See above regarding algorithm smartness.

But the last thing, that of posting more, I did religiously. After my initial December 20th, 2017 post, I’ve posted 25 public posts on my blog, a marked increase over the 13 posts I made in 2017. The plan was that all of the posts would be posted to Facebook and the ensuing conversations, controversy, and opinion would all happen in Facebook. After all, only if I contribute more to the platform, will I reap the rewards of the happiness that are supposed to come from it.

I also definitely did not delete any (well, most) of the tweets that got pushed from twitter to Facebook. I don’t like posting about political stuff openly. It’s like religion, everyone has one, and it’s best kept personal. But some tweets do get out once in a while. I believe I deleted those from Facebook. About 70 tweets made it to Facebook before disaster struck.

In the words of Hillary Clinton,

What Happened?

Well, the year started off nicely. Posting to Facebook is certainly a good way to garner attention. Friends who often forget that I have a blog were reading my posts and sometimes even clicking through to come to my actual website to check it out. The fact that Facebook discards in-text HTML, thus removing all URL references from a post both helps and hinders. It removes all context, but it also means that astute readers realized they had to click through.

I don’t have a lot of unknowns on my Facebook account. I do have a bunch of acquaintances, and people I haven’t met in years. I’m not a particularly social person irl. But everyone on there is someone I know or knew once. So it’s not like I was able to appeal to the masses and drive ‘traffic’ to my blogs. What I did achieve is a meager amount of conversation – a few likes and comments per post.

This extended to both types of posts. Folks who had never heard me express things about the random topics I post about on twitter and other microblogs, suddenly had access to my thoughts. Some reacted like idiots, some had positive or negative comments, and some just hit like and moved on.

All this stopped on August 1st. The declaration came in the form of a blog post by Facebook on their developer portal on April 24th. It was hidden between a bunch of other deprecated APIs, which I’m sure broke a lot of other things for other people. At the time, a huge noise rose, specially in the WordPress world about this. A lot of blogs depended on this API to post to Facebook using either the Jetpack plugin or the dlvr.it service (or other, similar services). Matt Mullenweg commented on the change, hoping that Facebook will reverse their decision and re-embrace the open web, to which this decision shuts the doors. But that’s not Facebook’s way. I reckon they heard him once in 2017, so they’re done listening to him for a decade.

I didn’t bother with finding workarounds to this problem. Smarter and more dedicated people than me would have found ways if there were any. Regardless, I wrote a blogpost on August 2nd and manually posted it to my Facebook profile on August 11th. This was my last cross-post from twitter or my blog to Facebook. It did not get any likes or comments.

According to some people, removing this API is important in helping fight the corruption that was revealed in the Cambridge Analytica scandal. But from what I can see, removing the ability for content to come in through legitimate sources is certainly not the way to go if you want to increase trust in your system. This was just a random move by Facebook, which is running around in headless chicken mode right now. It would be better if it were actually headless right now though, because the current head is part of the pattern of problems that Facebook manifests in this world.

Regardless, my year-long experiment ended mid-year.

The outcome of this experiment was this – I fell in love with the written word again. I also fell in love with my blog again. Though I now have newfound respect for a few things – first of all, I’m glad that my twitter is no longer connected to Facebook. The stream of consciousness that goes into twitter is not at all suited to Facebook, even though it should be, and for a majority of the world this has been a learning curve. Rants and raves belong to the place where outrage is common. You put it on Facebook and you alienate friends and get fired from jobs. While none of that happened to me, the effect was clear – people who I’ve never bothered to talk to my every day thoughts about were suddenly talking to me about them whenever I met them. This was… awkward. So I’m glad it’s no longer happening.

The second thing I’m glad of is discovering a rather important aspect of WordPress – private blog posts. While I’d like to talk about this more in another post, the overview is this – when you see 24 published posts for the year of 2018, I see 58. My process used to be that I would write a post and just leave it in drafts if it didn’t feel ‘complete’. This was wholly unsatisfying. Now, I privately publish my posts, giving them a timestamp that helps me date my thoughts. I also believe deeply in the concept of the blog as an Outboard brain as once proposed by Cory Doctorow. Though not as vibrant and well published as his blog, Boing Boing, my blog is my space, and having things published and showing up on the home page of my site when I’m logged in means I get to think about those things more.

What happened on the Facebook end of things? I noticed that the folks who interacted most with my posts were the same over and over – friends in the US who share my time zone, and some in India who I frequently interact with on Facebook. But what happened when the posts stopped? Nothing.

No one noticed. No one pinged me and asked me what was wrong with my blog and my tweets. Part of this is just the way the internet operates. Even with the extensive RSS setup I have, where I follow a lot of amazing blogs, if one slows down, I don’t have an easy way to figure it out. Time spent on the internet gets filled up by whatever is available.

The other half of this, I blame on Facebook. Their algorithm has become too smart for themselves. A willing user such as I should be able to push my posts to my friends without acting like an SMB and paying them money. In the same breath that they turned off the wall feeds, they promoted creating a separate page for one’s blog. This is a bad approach. For Facebook, it makes perfect sense – they can easily show hundreds of thousands new pages being created within the year, with all that untapped potential for paid promotions. That money will never come. A blogger such as I would rather trust the open web as a source of feedback and views than Facebook, whose track record for respecting ad spend is poor if not terrible. Facebook is a hungry beast, always looking for its next fix.

I’m tired of being Facebook’s fix. I don’t care for it any more. I have had an intense love for it as a platform at one time. I’ve been in awe of the leadership at one point. But now the spells are broken. 2018 was a journey, both public and private, in trying to see where Facebook goes. For me, it’s led itself to a dead end.

postscript – I opened Facebook recently, after perhaps a month, and a few things jumped at me. First of all, Facebook wished me for being with them since ten years. I think that’s serendipity. No social network online has a good life of more than a decade. Facebook should be no exception. While the company has morphed and plundered and established itself as the place to go to steal access user data, it should know that its main platform is tired and done for. I will slowly stop visiting and interacting with it. I know a lot of people have done this in 2018, but I still have derived some utility from it, so I’m sure it’ll feel somewhat bad to do so. On a new device I setup recently, I specifically made it a point to uninstall Facebook (it came preinstalled for some reason), while I did install Instagram. I know this is counter intuitive, but this is a signal from me to the company that it’s time to retire your aging platform or at least break it up instead of amalgamating into it. Facebook’s ugly attempts at driving people back towards their main property are so transparent that they should accept that it’s time.

The second thing I noticed was that Facebook had killed off an ugly experiment it has forced me to be a part of since two years – the Facebook marketplace and Video tabs. The main app has had these tabs since the beginning of 2017 for me (ymmv) and I never used them. I’ve looked forward to the day Facebook does *one* smart thing and recognizes that users would like an experience that’s suited to their needs instead of Facebook’s. By the way, for a brief time last year, when I discovered the Facebook groups app, my daily activity on Facebook actually increased, because I was able to get an ad-free, clean, groups-only experience of Facebook. Then FB killed off that app. So it goes. I’m glad that Facebook has removed its craigslist clone from my Facebook experience, but I didn’t celebrate it the day I saw it. Too little, too late.

Some thoughts on WordPress 5 and Gutenberg

Ok, this is me trying out Gutenberg after it’s full GA release. Let’s see how well it works. This entire post has been written on Gutenberg on Firefox on Windows, saved, privately published, and then edited on Gutenberg on Firefox on Mac, and published publicly. Yay.

Hmmm. There are some interesting quirks. The private publishing thing is available, so potentially there’s scope for the private posts plugin to be updated for Gutenberg. I like this plugin because every post is private by default and that gives me the freedom to publish immediately and edit later.

On a rock, undecided.

There’s an oddity here that might be useful to most people – when you start editing, the menu on the right shifts from Document to Block, so you can quickly change the Block settings if you want. That’s nice and all, but the switch is irritating to me. Maybe in a while I won’t even notice.

Other than that, it’s definitely performing better than it was last time I played with it, when it was in beta. That time, it just completely soured my experience because it kept crapping out on me. But this time, it seems stable and I can actually type a sentence without being constantly kicked out of the editor.

This is a title. Yeah, I know.

LOL. I just noticed that Gutenberg has support for drop capping. I don’t think I’ve ever had it before in my blogs. Interesting!

Is Gutenberg supposed to be useful for longform writing too? I don’t think longformers care about inline images. Also, the whole moving text up or down thing doesn’t make much sense to me. It’s just a weird concept. Maybe it’s useful to speech writers or essay writers – they need to present ideas in coherent ways, with each paragraph a complete idea. So technically, they could massively benefit from being able to move ideas around quickly for the overall coherence and flow of the thing they’re writing.

Wait, does Gutenberg not have autosave? The Classic editor does. It does it every time you stop writing for a significant and noticeable amount of time. But Gutenberg just seems to sit there.

I hit the gear icon and the right side menu disappeared. That’s good. More screen space to focus on writing, even though all the writing is happening within this one central column.

I wonder if Gutenberg would be useful to Instagram poets. Does this allow you to place text anywhere on the page? That might make a very pretty ‘flow’ thing that would work beautifully! If it doesn’t exist, someone should make it!

I really like Unsplash for pictures now. It’s not always on point, but there are some gorgeous pics out there! The Instant Images plugin is also nice – it doesn’t play with Gutenberg, but it sits outside and so it’s easy to add an image and then come to Gutenberg on an already open post and just click on the image block to pull the latest images. That seems to work well. My main problem with the plugin is that it’s got a max image size. They’re just trying to foster consistency, I think. But for an image that’s 5000×4000, to bring it down to 1600×1200 max size is a little irritating. But it does the sizing well actually. No graininess there! (Except maybe the graininess introduced by my theme)

Inline images in Gutenberg aren’t perfect. They don’t do everything as advertised, which will put more pressure on theme devs, I think. For example, the three images in my post till now don’t quite align the way I see them inside the editor. Weird. I wonder how they’ll look if I exit the editor and come back?

Embeds are separately supported now, as a block for each one of them. Nice. Good exposure to functionality. Earlier it used to be – use this shortcut and put the url in there in this format, and then do this incantation to call upon that demon to embed stuff on your blog. Now, it’s just there. Might actually cause an increase in link embedding across WordPresslandia. Maybe. Let’s see. I noticed after publishing that the embed doesn’t look the same inside and outside. Jeez. I think the embeds are a feature of Jetpack and that needs to be further updated to work properly with Gutenberg?

Instagram embedding did not work. Maybe because my blog isn’t https? I dunno. It’s fine.

I wonder if Automattic is tracking Gutenberg installs and usage? They should. It’s pretty good to highlight usage in the first week, first month etc.

edit: When Gutenberg opens in edit view (or maybe this is only on Mac), the currently being edited block is highlighted while others are faded out. That’s nice for focus, but weird for reading and revising

edit to that edit: I realized this is called Spotlight mode. Now, why is Spotlight mode active on my Mac and not on Windows? Don’t tell me there’s small JS differences which the devs have not reconciled yet.

edit: Gutenberg is NOT a happy camper on the Mac. I can move the central edit column horizontally in a not-so-good way. Separately, the dropcap paragraph, when it moves into edit mode, removes the dropcap in a very ugly way.

Verdict: I’m keeping Gutenberg on for a while. A few more posts it in. Let’s see. I already have the Classic Editor installed and I might just go back to that if I don’t see a lot of value in Gutenberg, or if I see a lot of noise in it.

That’s all folks!


Now and Then

A few years ago, a rather interesting idea came to my attention. In an effort to better inform their readers about their day-to-day activities, some bloggers decided to create a ‘now’ page. This page would be frequently updated with what the blogger was doing right now (today, this week, or this month) and would be a great way to update people and also push bloggers to interact with their blog more frequently.

That idea started with Derek Sivers and is immortalized on the nownownow.com website.

I too made a /now page. You can see it here or from the main menu on my site.

Recently, I came across the idea of an ‘on this day’ page here and here. I found it interesting enough to want to implement it on my blog too. Note that this is different than an archive page, which WordPress already has, because that just lists blog posts as a list and is a great way to find a particular post, but not a great way to discover or reminisce.

I call it the /then page. You can see mine here. Most days, it’ll be blank, while some days will show 3-4 posts over the years. That’s just how my creativity ebbs and flows, and how my day job keeps me busy on certain times of the year more than others.

I hope to use the now and then pages frequently, to discover old posts and rethink ideas, and explain to the world and to myself what I’m doing. If you come to the then page and see it is blank, drop me a line and remind me to blog more often! 🙂

If you want to discuss how I made it, drop me a line here and I’ll flesh out the basics in the commends. Most of the code came from Colin Walker.

To: The Awl, Subject: Before you die

long live the awl

Hey Sylvia and Team,

I knew about The Awl only when Sylvia was heading it. I discovered it last year or so and bothered to send in a few typos along the way. I loved the writing, I loved the off-current-affairs topics, I loved the esoteric posts. I loved Fran Hoepfner’s writing as well as music recommendations, specially since they pointed squarely to playlists on Spotify. Here’s a list of five posts that I have bookmarked. (There were probably more, but I lost some data from my RSS reader one day and didn’t bother to go looking for that)

Now The Awl is dying. People are celebrating this amazing blog/site/publishing platform. They’re talking about what a grand experiment it was, they’re talking about how it began and how it seemed to forever be bootstrapped (this is one of my failings – I do not bother with initial intent, so I don’t know when something will turn on me. I should be more careful of this). They’re relishing in the long list of people who ‘graduated’ from The Awl and the platform it gave to the many who now work elsewhere. They’re toasting to the feeling of having a personal blog but with an editor and getting paid to write on it.

Well I’m a little pissed. I discovered The Awl too late. I supported it too late. I want to discover more writers (and I sure am not going to use Medium to do it) and read more content and send more letters to the Editor. But they’re going away. At the end of the month, no less! I may be an infrequent reader of the site, but I don’t remember seeing any discussion with the readers, or any indication that it’s going under. All I see now are articles written in other publications by people who moved on from The Awl and know its inner workings. They’re waxing poetic about how there were never any investors, how The Awl always had a blog-y feel instead of a publication, how the end was inevitable because 2008-2013 was the time when small ad networks supported small blogs, but we all knew this was going to end some day.

Well, it did and frankly, if you’re in the business of publishing online, you should know one thing – ads are dead. Everyone, when they get a new computer, follow this format –

  • Open default browser
  • Download Google Chrome
  • Install Adblock

That’s the way this works now. If you don’t know that, well, sorry. If The Awl thought that life in the Adworld was tough because the site wasn’t targeted enough, here’s a tip –

2012 may have been the age of small ad networks, but 2018 is the age of ASKING YOUR USERS FOR SUPPORT.

Seriously, if writing apps (like Ulysses and Bear) can move to a subscription model, if The New York Times can politely remind people to donate, if the Guardian can ask for as little as $1, the least you could have done was to mid-2017 start a darn Patreon page. If nothing else, you’d see who out of the thousands of people who visit your pages found your content worth supporting.

But I get it, you’ve thrown your hat in the ring. You’ve decided that the thing you knew – ads – is no longer going to keep the virtual doors open and so it’s time to move on. So be it. I’ve followed my favorites on twitter and Instagram and hopefully will find some meaningful voices on Medium (ugh) and indie blogs (have you heard? they’re making a come-back. ping me, anyone, who needs help setting up a WordPress blog for ~$3/mo). In the meanwhile, I’ll be looking for ways to start my own Awl, in memory of this interesting ‘experiment’.

The Awl is dead, but its marks remain.

 

 

Photo by maritimeantiques

8 years of blogging

I missed a rather important anniversary over the weekend. I just noticed that WordPress wished me a few days ago for being with them since 8 years. Of course, I’ve been writing since long before that, but most of my writing was read only by my family and the greatest achievement of my writing then was when my parents published my writing in a small book which they presented to me on my birthday. With blogging, I was still being read mostly by my family, but online and I had a sense of achievement in that I was hitting publish every time I completed a blog post, thereby putting it out there for everyone to read, if they so chose to.

My earliest blog is on wordpress.com here. I have tried a variety of platforms over time but WordPress just seems to be the right one for me. Of course, I left that blog some years ago and came to nitinkhanna.com (by way of blog.nitinkhanna.com) and self-hosted WordPress. But all that matters is that on-and-off, here-and-there, I was writing and I was publishing. I seem to have been able to average a post per month or so, though please don’t hold me to that standard (my last post here was more than a month ago and I’ve not had much to write in that intermediate time). But I am proud that I have a cumulative 1,37,361 words published on my blog (with some 66 posts sitting in drafts) Continue reading

Balloons, or how tech companies need to stop and take stock

Balloons is, concurrently, a ‘fine WordPress theme’ and a ‘whimsical’ one. It is also a theme that caught my attention when I was browsing for WordPress themes recently. Let me be clear – I was not browsing for themes for my own site. I was browsing for themes for our nikhco.in domain, which looks to be in need of a refresh.

But Balloons caught my eye. Why? I’m not sure. Maybe it was the large number of balloons that are front and center at the head of the theme. Maybe it was the oddly small typography, which could look great if it were a few font sizes bigger. But as soon as I saw it, it caught my attention. I started thinking about how I would modify it to suit my needs and change some things I’d definitely get irritated at. I hate when theme authors fixate on certain social network links but not others or add an unneeded sidebar to the theme. But then, I stopped and took a step back.

This planning and plotting I was up to, was it needed? Was it a useful change to my site? Was this theme better than my current theme? I have put many hours into editing my current theme, “Independent Publisher”, to make it look the way I wanted it to look. So should I be putting those same hours again, so soon, into a completely new theme with completely new issues I’d have to fix? I like the challenge, but is the effort valuable? Have I received negative feedback on my theme? Has someone told me that it’s not good the way it looks or maybe it fundamentally conflicts with the content? I write on a variety of subjects – code, fiction, politics, observations about the world, and movie reviews, among others. So it’s been hard to find a theme that fits all that content. Thus, over the years, I’ve experimented with many themes, many plugins and formats to elicit some kind of a reaction from my otherwise passive readership.

I was talking to my brother recently and we were talking about how LinkedIn has the habit of trying new things with their site. I understand the impulse. It’s all about constantly evolving. You have a product, you want to make it better. There’s also the business case for it. For startups and fledgling companies alike, there’s a market to capture and industries to disrupt. Thus, the need for experimentation drives them to keep trying to do new things. If a company working on a professional social network can also act as a Rolodex and be the go-to resource for industry news, that’s better for their business.

But my brother’s point was valid too – you’ve got a product. You’ve released it to the general public. You’re working on minor improvements all the time. Let. it. sit.

There’s oftentimes no need to add that new feature to your current site. If you want to experiment, make a separate platform or a new app to try things. Put it under your label, call it “LinkedIn Connect” or “Facebook Paper”. But don’t try to shove new ideas down the throats of your current users. Let them get used to the current system. Let them complain and argue the merits and demerits of it. Let them give you real feedback and then act on it. At the end of the cycle, if the new idea is that popular, roll it into your current system. Integrate your changes. But don’t start out with the assumption that people will be OK with a constantly changing platform. Most of the time, there’s no need for that.

We talked about all the other companies out there too, including giants such as Google, Cisco and HP. Those who sit on their laurels get surprised by a leaner, smarter company coming along to steal their market share. But those who continually reinvent just to keep the rust off, lose their focus and their customers. If you’ve got a radical improvement to your product, go for it. But make sure you’ve got a second set of eyes telling you that the new is actually better than the old, not just newer than the old.

So, as I looked at Balloons, I silently sighed. There was no need for it. No one is telling me that my tech posts look bad in the new theme. My most popular post ever “Installing Fever on AppFog” still gets visited a few times a week even though it’s years old now. People still read through it on a theme that’s better suited to fiction than tech tutorials and no one seems to mind. Older posts about code are still visited and no one cares if the font is larger than needed.

I bookmarked the theme and closed the tab. One day perhaps, I’ll dust it off and show it to someone and ask if it would make for a better theme for my blog. Until then, my site looks good and I’ve decided what to do with it – Let. It. Sit.


Authors Note – I wrote and edited this post on Hemingwayapp. It’s an amazing editor. It points out sentences that are hard to read, phrases that can be simpler, and the use of adverbs and passive voice. It helped me get rid of all the instances of passive voice in this text. The makers, the Long brothers, have come up with a new Beta version that you should check out. The New Yorker has taken notice of the app too, among other news media. You can read about their coverage here. This article got a grade of 6 on the app, which is not at all bad!

Fixing Jetpack’s Stats module

Despite the hate that Jetpack gets for being a bloatware plugin, it is one of my favorite and the first step whenever I setup a new WordPress install. However, Jetpack does have a few irritating habits that I cannot overlook. One of these is the stats module. The module actually does pretty well, posting data to the wordpress.com dashboard and making it easy for me to quickly glance at the number of visitors I’ve had for the day.

However, every so often the module craps out and logs a large number of visits from crawlers, bots and spiders as legitimate hits, since those are not in the official list of crawlers, bot and spiders to look out for. To fix this, I went out to look for the list and to add to it. One quick GitHub code search later, I found that the file class-jetpack-user-agent.php is responsible for hosting the list of non-humans to look out for. What I found inside was actually a pretty comprehensive list of software, but one that definitely needed extending.

If you want to do what I did, find the file in your WP installation at –
/wp-content/plugins/jetpack/class.jetpack-user-agent.php

Inside the file, look for the following array variable –
$bot_agents

You’ll see that the array already contains common bots like alexa, googlebot, baiduspider and so on. However, I deepdived (meaning did a sublime text search) into my access.log files and found some more. To extend the array, simply look for the last element (which should be yammybot) and extend it as follows –
'yammybot', 'ahrefsbot', 'pingdom.com_bot', 'kraken', 'yandexbot', 'twitterbot', 'tweetmemebot', 'openhosebot', 'queryseekerspider', 'linkdexbot', 'grokkit-crawler', 'livelapbot', 'germcrawler', 'domaintunocrawler', 'grapeshotcrawler', 'cloudflare-alwaysonline',

Note that you want to leave in the last comma, and you want all the entries in lower case. This doesn’t actually matter, because the PHP function that does the string compare is case-insensitive, but it just looks neater. You’ll also notice that I’ve added the precise names of the bots, like ‘grokkit-crawler’ and ‘clousflare-alwaysonline’ but you can be less specific and save yourself some pain. This will, however, affect your final stats outcome.

Notes –

  1. Some of the bots are pretty interesting. I saw tweetmemebot, which is from a company called datasift, which seems to be in the business of trawling all social networks for interesting links and providing meaningful insights into them. Another was twitterbot. Why the heck does twitter need to send out a bot? We submit our links to it willingly! Also interesting were livelapbot, germcrawler and kraken. I have no idea why they’re looking at my site.
  2. Although Jetpack does not have a comprehensive list of bots, it still does a pretty good job. I found the main culprit of the stats mess in my case. Turns out, CloudFlare, in an effort to provide their AlwaysOnline service (which is enabled for my site), looks at all our pages frequently and this doesn’t sit well with Jetpack. I hope this tweak will fix this now.
  3. Although this fix is currently in place, every time the Jetpack plugin gets updated, all these entries will disappear. That’s why this blog post is both a tutorial for you all and a reminder and diary entry for me to make this change every time I run a Jetpack update. However, if someone can tell me a way to permanently extend Jetpack, or if someone can reach out to the Jetpack team (hey Nitin, why don’t you file a GitHub issue against this?) it’ll be awesome and I’ll be super thankful!

Update – I was trying to be hip and did a fork of Jetpack and GitHub, made the changes and then tried to make a pull request. Turns out, I don’t know how to do that, so I opened an issue instead. It sits here.
 

Word of the day: rubric

According to TheFreeDictionary, rubric means a title, class or category. It’s also used when referring to a subheading or the full title of a file/post or page. Neiman Journalism Lab used it as follows –

The Brief, a tailored summary of business and international news under the rubric of “Your world right now.”

Source: Maybe the homepage is alive after all: Quartz is trying a new twist on the traditional website front door » Nieman Journalism Lab

Continue reading

How I Follow Blogs on the Open Internet

Colin Devroe’s post about Fred Wilson’s post about how hard it is to follow blogs on the Open Internet is interesting to me.

Ok, before we go any further, yes, this is very meta. Yes, I could have written this entire thing as comments on Colin’s blog (no, it doesn’t support comments) or Fred’s blog (has nice disqus comments) but I didn’t because that’s the point of blogging. I can write this ‘commentary’ on my blog. Sort of like Greek philosophers writing entire books just discussing each other’s books. Very meta indeed. Continue reading