“Which way to Svoboda?”

 
1 Say Thanks!
Don't
move!

I was reading a BBC news report of how, recently, pro-Russian sites are popping up in the Czech web sphere, which could allude to some serious USSR-style propaganda. The article referenced the 1968 Prague Spring, which was when the then Czechoslovakia government tried to establish reforms which would lead to freedoms to the press and private sector, the division of Czechoslovakia into Czech Republic and Slovakia and a general upliftment of the people who were suffering cruelly under the rule of the Soviet Bloc. Needless to say, Soviet Russia didn’t take kindly to this and, along with their friends of the Warsaw Pact (Poland, Hungary, Bulgaria and East Germany) attacked Czechoslovakia to take back control.

Of course, they won. Even with the way things were under the USSR, they had tanks, weaponry and manpower and Czechoslovakia had, well, a leader who told his people not to resist. But resist they did. Without the necessary means to win the war, they resisted in the only other noble way – confound the heck out of the enemy. In the most peaceful way, road and street signs across the country were painted over or removed so as to completely confuse the incoming force.

The result was hilarious. Supposedly, one could see troops stopped in rural areas trying to study maps and making sense of how every village they’d visited was called either Dubček or Svoboda (which means freedom). Road signs were painted over, except those that led to Moscow. The result of that was that an invading force from Poland spent a day roaming around before being routed out of the country, empty-handed.

Now, these reports come from Wikipedia and further from two separate sources, but I’d say you should take them with a grain of salt regarding their veracity. However, the point to understand is that in those days, it was possible to confound an incoming force by the sheer ingenuity of changing your road signs and hiding all the maps. Of course, today’s military will simply whip out their iPhones and tell you where to invade next. But this episode lends importance to the idea that with the accumulation of power so dependent on finding your enemy, it is important to also control the means of finding the enemy in the first place. This is obviously the reason why countries like Russia, China and India as well as the EU are working to create their own version of the GPS system (which, mind you, is owned by the US Government).

Clearly, in tomorrow’s war, one of the first efforts will be to either block the enemy’s signals, thereby preventing them from finding our accurate locations. The other, more radical one, would be to try to shoot down their navigation satellites, a scenario that has given birth to more science fiction movies than we care to admit.

But, going back to that wondrous time when people still had to use maps and ask for directions from locals, I must say that it is remarkable that someone thought of the simple idea that perhaps one way of stalling the enemy is to paint over the signs which will tell them how to get to the capital. That’s your trivia for the day.

Balloons, or how tech companies need to stop and take stock

balloons-featured
 
0 Say Thanks!
Don't
move!

Balloons is, concurrently, a ‘fine WordPress theme’ and a ‘whimsical’ one. It is also a theme that caught my attention when I was browsing for WordPress themes recently. Let me be clear – I was not browsing for themes for my own site. I was browsing for themes for our nikhco.in domain, which looks to be in need of a refresh.

But Balloons caught my eye. Why? I’m not sure. Maybe it was the large number of balloons that are front and center at the head of the theme. Maybe it was the oddly small typography, which could look great if it were a few font sizes bigger. But as soon as I saw it, it caught my attention. I started thinking about how I would modify it to suit my needs and change some things I’d definitely get irritated at. I hate when theme authors fixate on certain social network links but not others or add an unneeded sidebar to the theme. But then, I stopped and took a step back.

This planning and plotting I was up to, was it needed? Was it a useful change to my site? Was this theme better than my current theme? I have put many hours into editing my current theme, “Independent Publisher“, to make it look the way I wanted it to look. So should I be putting those same hours again, so soon, into a completely new theme with completely new issues I’d have to fix? I like the challenge, but is the effort valuable? Have I received negative feedback on my theme? Has someone told me that it’s not good the way it looks or maybe it fundamentally conflicts with the content? I write on a variety of subjects – code, fiction, politics, observations about the world, and movie reviews, among others. So it’s been hard to find a theme that fits all that content. Thus, over the years, I’ve experimented with many themes, many plugins and formats to elicit some kind of a reaction from my otherwise passive readership.

I was talking to my brother recently and we were talking about how LinkedIn has the habit of trying new things with their site. I understand the impulse. It’s all about constantly evolving. You have a product, you want to make it better. There’s also the business case for it. For startups and fledgling companies alike, there’s a market to capture and industries to disrupt. Thus, the need for experimentation drives them to keep trying to do new things. If a company working on a professional social network can also act as a Rolodex and be the go-to resource for industry news, that’s better for their business.

But my brother’s point was valid too – you’ve got a product. You’ve released it to the general public. You’re working on minor improvements all the time. Let. it. sit.

There’s oftentimes no need to add that new feature to your current site. If you want to experiment, make a separate platform or a new app to try things. Put it under your label, call it “LinkedIn Connect” or “Facebook Paper”. But don’t try to shove new ideas down the throats of your current users. Let them get used to the current system. Let them complain and argue the merits and demerits of it. Let them give you real feedback and then act on it. At the end of the cycle, if the new idea is that popular, roll it into your current system. Integrate your changes. But don’t start out with the assumption that people will be OK with a constantly changing platform. Most of the time, there’s no need for that.

We talked about all the other companies out there too, including giants such as Google, Cisco and HP. Those who sit on their laurels get surprised by a leaner, smarter company coming along to steal their market share. But those who continually reinvent just to keep the rust off, lose their focus and their customers. If you’ve got a radical improvement to your product, go for it. But make sure you’ve got a second set of eyes telling you that the new is actually better than the old, not just newer than the old.

So, as I looked at Balloons, I silently sighed. There was no need for it. No one is telling me that my tech posts look bad in the new theme. My most popular post ever “Installing Fever on AppFog” still gets visited a few times a week even though it’s years old now. People still read through it on a theme that’s better suited to fiction than tech tutorials and no one seems to mind. Older posts about code are still visited and no one cares if the font is larger than needed.

I bookmarked the theme and closed the tab. One day perhaps, I’ll dust it off and show it to someone and ask if it would make for a better theme for my blog. Until then, my site looks good and I’ve decided what to do with it – Let. It. Sit.


Authors Note – I wrote and edited this post on Hemingwayapp. It’s an amazing editor. It points out sentences that are hard to read, phrases that can be simpler, and the use of adverbs and passive voice. It helped me get rid of all the instances of passive voice in this text. The makers, the Long brothers, have come up with a new Beta version that you should check out. The New Yorker has taken notice of the app too, among other news media. You can read about their coverage here. This article got a grade of 6 on the app, which is not at all bad!

 

A lecture

 
0 Say Thanks!
Don't
move!

Good morning Young Adults,

Welcome to the third lecture on “History of Writing”. I’d like to point out, as mandatory by law, that this history, as with all other histories taught in your school, is disputed in a federal court and may be deemed incorrect at a later date, at which point the syllabus will change. Until then, we are teaching the less popular version of the topic, as dictated by our school’s charter.

Now, to begin the lesson, let’s do a quick recap of what we have learnt in the last two lectures. We began with the idea that writing things down would make them available to more people and preserve knowledge across time. This part, as we discussed, is not disputed.

Then, we saw a presentation on the idea of a universal language. We understood that even though individuality is considered important, the world had settled into the idea that English would be the universal language towards the end of the twenty-first century. We understand that this is disputed by scholars of the day who believe that English was not actually universally accepted but quite simply the de facto language because of the cultural and economic might of a few countries, which, since they no longer exist, can neither prove nor disprove this specific argument in any comprehensive manner.

Today, let’s talk about the Book. Now, please understand that I’m not talking about a specific book, but the Book in general. Since most of you do not know what that is, let me pull up the Wikipedia entry for it.

“A book is a set of written, printed, illustrated, or blank sheets, made of ink, paper, parchment, or other materials, usually fastened together to hinge at one side. A single sheet within a book is called a leaf, and each side of a leaf is called a page.”

Now, who can tell me what they understood from this? Yes, Tomlin, what’s your take on this definition?

Tomlin – I think it’s some kind of a primitive information repository, but I’m not sure how something like that would work.

Very good Tomlin! Yes, books were information repositories that were organized as chapters. Sometimes, there was so much information that it had to be put into a series of books! Yes, Supriya, you have a question?

Supriya – Yes Ma’am, I want to ask how can they put information on something so flat?

A wonderful question Supriya! People nowadays don’t understand how knowledge can be put on something as basic as paper. Let’s do an exercise. If you remember, I showed you all a piece of paper when we went to the museum last month. You’ll recall that there was text on that paper, real words. There was also an image on it.

I want you all to consider information as we know it today and try to think about how that information would be represented on paper during that period of time.

Let’s begin with a simple example. We all know that the best way to capture someone’s speech is to make a 3D replica of it. How do you think humans in the twentieth century captured speech? Yes, Supriya?

Supriya – I have read that they used to takes things called notes, on paper. They would quickly write what people were saying and then save it someplace safe to recall later.

Very good Supriya! But since you’re familiar with the concepts, I don’t think you should be answering any more questions on the topic. No need to frown, you know it’s the best way to involve everybody.

Next question. In our 3D replicas, we often include, in heavy detail, how people emote their speech. We have records of inflection, pronunciation, emotion, body and eye language and many times, environment. How do you think early man did it? Let’s take it to someone other than the front row. Yes, Mary, what’s your take on this?

Mary – Well, they’d have to somehow copy the 3D model and keep changing the emotion.

Close, but not quite. You see, in written language, there is a process to capture emotion. Words expressing emotions such as anger, relief, hatred, and peacefulness were used to write about emotions. Similarly, surroundings were described in great detail. This is one of the reasons why books were actually so big in size. They had to capture a lot more information in order to properly capture the essence of the scene being described.

Let’s change gears and imagine how they would have described something non-human, like a stone or a DNA sequence. Nowadays, if you want to see a stone, the replicator just builds one for you. If you want to study a DNA sequence, you can ask the replicator to create a model for you and you can play with it live, to create your own DNA sequence. How do you think mankind stored that kind of information? Let’s hear from, you, Ching? Ching, I don’t care if you don’t have an answer, I want you to try.

Ching – I dunno. Maybe they tore off some pages and kept a stone there?

No need to laugh class. Ching is not wrong. No one is. But the human race quickly realized that ideas such as putting a piece of stone in books could only work as a novelty and were not practical. Instead, they used to draw elaborate pictures of the item. Sometimes, they’d describe the color, feel and effect of the object in question in great detail and sometimes, they’d just include a picture of the thing and move on. Yes, Maya, do you have a question?

Maya – But Ma’am, how can they draw a rock on a paper and how can they play with a DNA structure on paper? You told us that paper is non-interactive, unlike our 3D models.

You see, Maya, there is an entire art dedicated to drawing 3D shapes on a 2D plane. It may seem ridiculous to you, but some of their greatest achievements were based on that. From the early prototypes of fuel propelled vehicles, which they called ‘space craft’, to large representations of landscapes, everything 3D was represented in 2D using various techniques such as using darker colors for depth perception and light source inference, using layers of color to give the sense of three-dimensional space, using specific colors to represent specific things, such as red for anger, blue for sadness and green for happiness and prosperity. Now, you may all laugh, but this is how it all worked.

Who here remembers that museum exhibit called a painting? It was some lady sitting in an elaborate costume, with a slight smile. Good to see all these hands up! Now, we may not know why she was painted. Was she some politicians wife, or was she the artist’s representation of an angel, but we can study the painting, only half of which remains now, and see that the smile is actually affected by the way it has been painted. We can also see that the background has been shown to be smaller from her own size, so as to represent that it is far from her. These ideas may seem primitive to us all, but these were the best means they had to represent such notions.

Who here has been to MOMA? You realize why the Museum of Modern Art had to be made on half the side of the moon, right? They needed room to let artists display their work. The famous “Events Of Futures Past” by Giraldo Ganeshan is a theater of five thousand replicas of humans at war with five thousand Centauri. Now, we know that we’re at peace with the civilizations of other star systems, but Ganeshan’s representation is an idea, that something like this could happen. Of course, to accurately represent the scene, Ganeshan had to take up half of MOMA’s space in order to present the installation. You can walk through the model and watch as each warrior attacks one from the other side in a battle that would not even be technologically feasible anymore – we no longer have weapons that we can hold in our hands.

Imagine if, instead of taking all that space, Ganeshan would just have written a book, with detailed descriptions of the battle. It would have taken considerably less space. It would have meant that humans would have to use a very infrequently used faculty now, namely, imagination, but it would have done the job.

Now, getting back to the topic at hand. Maya, how do you think people who drew the DNA structure on paper interacted with it?

Maya – With their imagination, I guess.

Bingo

Maya – But how? It’s so hard to imagine this stuff! And reading is so difficult! Why not just call a 3D model of it and let the comp process it for you?

You must realize that there was very little that comps could do back then. In fact, there was a time when comps didn’t even exist! Nowadays, all you have to do to start studying is to press a button on the memlets around your necks. But back then, knowledge was slowly being transformed from paper to comps and comps didn’t have the capabilities needed for this kind of work. In fact, for a long time, people depended on screens to display information to them!

Puline – What? So you’re saying that they moved from 2D paper to 2D screens? Lame!

Decorum Puline. But you’re right. Our ancestors weren’t the brightest bunch. It didn’t occur to them that the upgrade from paper would be something that takes them into a third dimension. Instead, they chose to make it the same 2D structure, but put it on comps and then on what they called the Internet, which is, in a way, the great-great-great grandfather of the psynet.

That brings me to the last question. How do you think peeps took books from one place to another? How about we get an answer out of Anhel. We’ve barely heard a sound out of you today.

Anhel – yeah, I’m a little distracted today.

Is there something you’d like to share with us today?

Anhel – No, no. Anyways, what was the question?

How do you think peeps carried books around?

Anhel – I don’t know. Did they put them on some sort of fuel powered hovers like we use at home?

Haha! No. I like your imagination, but the thing is, hovers came into existence only about five hundred years ago. What they did was that they had things called bags. If you ever observe the food delivery at home, you’ll notice that individual types of foods are packaged in custom-built air silos. Early mankind, however, had to create containers out of paper or cloth or something that they called ‘plastic’ and use those to store and transport goods, including books. Imagine you all, coming to school, with bags hanging over your shoulders, with books inside them!

Well, that’s all for today’s lecture. Have fun at home and don’t forget next week’s timings because they’re a little different. Good day!

Author’s Note – I’d like to thank my friends Ronnie and Rahul for editing this post. They both provided invaluable editorial feedback and are really cool people. I’d recommend you follow them on twitter, because, apparently, that’s a thing to do.

Solving the ten thousand year problem

 
0 Say Thanks!
Don't
move!

While writing a short story today, I started thinking about an issue that I discovered last year. The story is set in the far future, where the dissemination of knowledge has changed so vastly that the idea of a printed page is absurd. I’ll be publishing it in the coming hours. But, as I was writing it, I started thinking about how much our culture and language will change in the next ten thousand or so years, let alone over the next hundred thousand years. That reminded me of an interesting thing I read last year – “Ten Thousand Years”.

Out in the New Mexico desert, stands a government building with a single task – to permanently store nuclear waste from the US’ various nuclear power plants, for at least the next ten thousand years. The date is so chosen because supposedly, thinking beyond that time frame is too mind-boggling to consider. It has nothing to do with Jeff Bezos’ Long Now Foundation, which is building a ten thousand year clock, thought it might as well, because both ideas are equally interesting and convoluted.

Now, one of the issues that the Waste Isolation Pilot Plant (WIPP) in New Mexico is facing is that of language. Since the last ten thousand years, so much has changed in humanity, that the idea that the same language, the same symbols, and the same myths that protect us today will remain even then, is a non-sequitur. English is constantly fighting to be the language of choice while Spanish, French and Chinese are growing their user base. Symbols such as the skull-and-bones are adapted, first by real-life pirates and then by digital pirates to change their meaning completely, transforming something that indicates danger to something indicating excitement and even fun. Even myths change and long-loved black cats are suddenly considered evil and the number 13 bounces around as something lucky, then not.

Thus, assuming that a sign board at the gates of the WIPP, written in English and a battery of other languages, along with ten different types of warning symbols, should be enough to deter people from entering the premises, is foolish. This is one of the smaller issues that the WIPP is facing.

So what’s the solution? While I was pondering on the course of the story, I realized that the answer would have to be a mixture of ingenuity and technology. This is how I believe the problem can be solved –

We need to build a system that’s not just fault-tolerant and self-healing, but also intelligent enough to learn about it’s surroundings. While it may seem enough to place a settlement of scientists nearby who would constantly watch over the plant, recruit future employees and ensure the safety of the rest of the land, humans have a distinct habit of dying, moving away, letting emotions come in the way of logic and duty, and overall being bad protectors of the environment. So, the solution would be to build a system that can be initially supported by humans but must eventually stand on it’s own feet. This Gatekeeper would not just prevent people from walking into the compound, but also learn new languages, understand symbols and changing economics and governmental systems and ensure that no one disturbs the deathly sanctity of the place it protects. It would be able to access the Internet and learn of new technologies to replace it’s old ones. It would learn languages and add them to it’s database, essentially creating a bookmark of human history as it goes about it’s business of preventing nuclear waste from getting out of this burial place. This would have to be a highly fault tolerant system, able to quickly analyse any potential issue such as maintenance, earthquakes, failing parts and changing technologies in order to ensure its continued service. I think only if we are able to build such a powerful system can we promise ourselves that such a dangerous material can be protected over the next ten thousand years.

Or, we could just drop it into a volcano and hope that thing eats it all up.

Word of the Day: illiberalism

 
0 Say Thanks!
Don't
move!

According to the Merriam-Webster dictionary, illiberalism, quite simply, is the lack of liberal values. But that begs the question, what is liberalism? Is it the ability of a community to be forward thinking and self-critical? Is it the incessant forward march of a government without caring for the social, political and emotive values of its peoples? Or is it the protection of the freedom of expression of an author writing about a sensitive topic with the backdrop of a community to which he does not belong? Hindustan Times certainly believes in the last definition.

First, it shows that it is not only the sangh parivar or Islamic organisations that are at the forefront of such illiberalism.

Source: Liberal values are being trampled upon in Tamil Nadu

The issue at hand is that the author Puliyur Murugesan wrote a bookBalachandran Enra Peyarum Enakkundu (I am also known as Balachandran), about the life and troubles of a transgender, who is sexually harassed throughout life and faces an upward battle of identity. The protagonist belongs to the Gounder community and by now, you would have guessed where this is going.

The Gounder community decided to take offence to this ‘insult’ to their people and instead of rationally sitting down with the writer and asking for edits to the story or a total redaction, decided that the better course of action would be to abduct the author and brutally beat him up in the middle of nowhere. To add insult to literal injury, the police has filed a case against the author for provoking a riot, writing and circulating obscene content, selling a book containing defamatory matter, intentional provocation of breach of peace and causing fear or alarm to public. Wonderful, isn’t it?

HT, in their laconic article, asked an interesting question – why is it that only current authors face the brunt of such injustice? Why do authors such as Sarat Chandra Chattopadhyay, the acclaimed author of Devdas, who “had made uncharitable remarks about some non-Bengali Brahmin clans”, not face such public ridicule and outrage? Perhaps, if it were in fashion, political parties and illiberal communities will also start attacking famous people from India’s history books. Oh wait, they already do!

Here’s some love for LinkedIn Users

Just tap that button
 
0 Say Thanks!
Don't
move!

Some time ago, my brother came to me with a problem. He loves LinkedIn. It’s a great service. But as much as he loves connecting with people on that professional network, there are some glaring inefficiencies that he does not appreciate. He wasn’t interested in removing ads or making it look nicer. He just wanted to see the information that people intend on displaying on the site. You see, there’s a plethora of information available on LinkedIn, but it’s mostly hidden.

For some reason, if you’re landing on a user’s profile from LinkedIn’s user search, or from a Google search, you end up seeing this –

But what you should really be seeing is, at least, the user’s name, a little bit about their history and experience. Essentially, you should be seeing something like this –

LinkedIn’s been around since some time now, but they haven’t fixed this weird issue and so, your LinkedIn experience is often curtailed by what can only be called a minor bug.

Not any more. Today, NiKhCo. has launched a new tool, “LinkedIn Reveal”, which will solve this absurdest of LinkedIn woes. It enables you to explore LinkedIn with the depth you never thought possible. We’re not trying to build something that changes the way LinkedIn displays information or makes things look fancy. We’re just building something that lets you see LinkedIn as it truly should be – a beautiful, open, professional network with all the information you need about people, companies, jobs and connections.

LinkedIn Reveal is now available in the Google Chrome Web Store. Do check it out. It’s valuable for everyone who uses LinkedIn. Also, here’s a screenshot, because pictures somethingsomething thousand words somethingsomething. :)

What Facebook needs to do next

messenger
 
0 Say Thanks!
Don't
move!

Update: Facebook has done it! It has finally created a messenger standalone for the web. You can go here to check it out (here if that doesn’t let you login) or here to read about it (but why would you?). Cheers!

Facebook has changed a lot over the past few years. There have been acquisitions, newsfeed, design changes and rollbacks and a whole mess of things. Facebook obviously understands that the future is mobile (hence WhatsApp and Instagram) and that people are moving in all kinds of directions, towards private spaces (hence the separate Messages app) and public posts (hence hashtags and the searchability of your FB post that goes with those).

So what’s next? Well, today, I wanted to send a link to my brother. Since I’m on Windows right now and not on my Mac (I usually just iMessage links to him), I fired up Facebook and sent it to him in a message. Why? Because it’s convenient, because Facebook detects the OpenGraph information about the page and processes it to make a neat link+image combo and because we have many conversations on there anyways.

Why didn’t I send it to him as a directed Facebook post? I could have, but I didn’t care enough to make it public. Our Facebook activity is not our true selves but a reflection of what we want to portray to others, and this link didn’t necessarily fit into any paradigm of my public self. (In other words, it wasn’t epic. Publicly shared links must be epic.)

Why didn’t I send it to him in an email? Hush, don’t ask silly questions.

But then, I wanted to send him another link and another. So what did I do? I clicked on that miserably little link that takes me to Facebook’s dedicated messages page so that I could share links and have conversations in a larger space than that goofy little box that occupies the bottom right corner of my screen.

That’s when it hit me. Facebook has a beautiful Messages app on the iPhone. I’ve sung praises of it before. But there’s a curious lack of a well designed web interface for messages. The old and clunky interface that sits there now has been sitting untouched since a long time.

Now, you’d argue that another private space that I could have used was the Facebook Groups feature. It looks nice, it’s often updated and has the same look-and-feel as the rest of Facebook. But why would I create a group for just myself and my brother? There’s no need for that since /messages exists. The only thing needed is to build a nice looking private messages space that people would use.

I was really tempted, as I started writing this post, to build this webapp myself. Facebook’s API is open and easy to work with. I’m sure I could have found many plugins and libraries to make the task easier for me. But any such project can never be feature complete. I can build it and you can come, but you’ll never stay because of lack of features, because a single guy sitting with a laptop with a limited amount of time can only do so much.

So Facebook, here’s the next thing you need to do – make messenger.com what it really should be – a full-fledged webapp that’s as classy as any other public facing feature of Facebook. I hope to see it soon!

Cities

 
0 Say Thanks!
Don't
move!

Hunkered, cowering, crawling
The masses seething through
An urban jungle sprawling
Sky and stone of same hue

Eyes lowered, heavy
Watching furtively, each stranger
With hopes and dreams, heady
Covered, guarded from each danger

Rain splashing on head and face
As we walk to our destinies
Some to fame’s embrace
Most, to life’s inequities

What if one day, we were to rise
Leave back this dreary, endless ride
Catch the light before it dies
To claim some day that we too tried

But that is for another day
There is work yet to be done
Cash the check, claim some pay
A race yet to be won

And so one sees
The masses seethe
As endless drops in endless seas
With gray up top and gray beneath

A response to NYTimes’ OpEd on Religion in India

 
0 Say Thanks!
Don't
move!

The New York Times has published a Christmas OpEd piece (yesterday online and today in print) that talks about religious intolerance in India and how ‘hindu militants’ are forcing conversions of Muslims and Christians and has blamed our PM, Narendra Modi for not doing enough to stop this and other attacks.

Initially, I wanted to write a long and deeply researched rebuke of this kind of hypocrisy, but I’d rather let a picture say a thousand words –


Continue reading

Fixing Jetpack’s Stats module

 
0 Say Thanks!
Don't
move!

Despite the hate that Jetpack gets for being a bloatware plugin, it is one of my favorite and the first step whenever I setup a new WordPress install. However, Jetpack does have a few irritating habits that I cannot overlook. One of these is the stats module. The module actually does pretty well, posting data to the wordpress.com dashboard and making it easy for me to quickly glance at the number of visitors I’ve had for the day.

However, every so often the module craps out and logs a large number of visits from crawlers, bots and spiders as legitimate hits, since those are not in the official list of crawlers, bot and spiders to look out for. To fix this, I went out to look for the list and to add to it. One quick GitHub code search later, I found that the file class-jetpack-user-agent.php is responsible for hosting the list of non-humans to look out for. What I found inside was actually a pretty comprehensive list of software, but one that definitely needed extending.

If you want to do what I did, find the file in your WP installation at –
/wp-content/plugins/jetpack/class.jetpack-user-agent.php

Inside the file, look for the following array variable –
$bot_agents

You’ll see that the array already contains common bots like alexa, googlebot, baiduspider and so on. However, I deepdived (meaning did a sublime text search) into my access.log files and found some more. To extend the array, simply look for the last element (which should be yammybot) and extend it as follows –
'yammybot', 'ahrefsbot', 'pingdom.com_bot', 'kraken', 'yandexbot', 'twitterbot', 'tweetmemebot', 'openhosebot', 'queryseekerspider', 'linkdexbot', 'grokkit-crawler', 'livelapbot', 'germcrawler', 'domaintunocrawler', 'grapeshotcrawler', 'cloudflare-alwaysonline',

Note that you want to leave in the last comma, and you want all the entries in lower case. This doesn’t actually matter, because the PHP function that does the string compare is case-insensitive, but it just looks neater. You’ll also notice that I’ve added the precise names of the bots, like ‘grokkit-crawler’ and ‘clousflare-alwaysonline’ but you can be less specific and save yourself some pain. This will, however, affect your final stats outcome.

Notes –

  1. Some of the bots are pretty interesting. I saw tweetmemebot, which is from a company called datasift, which seems to be in the business of trawling all social networks for interesting links and providing meaningful insights into them. Another was twitterbot. Why the heck does twitter need to send out a bot? We submit our links to it willingly! Also interesting were livelapbot, germcrawler and kraken. I have no idea why they’re looking at my site.
  2. Although Jetpack does not have a comprehensive list of bots, it still does a pretty good job. I found the main culprit of the stats mess in my case. Turns out, CloudFlare, in an effort to provide their AlwaysOnline service (which is enabled for my site), looks at all our pages frequently and this doesn’t sit well with Jetpack. I hope this tweak will fix this now.
  3. Although this fix is currently in place, every time the Jetpack plugin gets updated, all these entries will disappear. That’s why this blog post is both a tutorial for you all and a reminder and diary entry for me to make this change every time I run a Jetpack update. However, if someone can tell me a way to permanently extend Jetpack, or if someone can reach out to the Jetpack team (hey Nitin, why don’t you file a GitHub issue against this?) it’ll be awesome and I’ll be super thankful!

Update - I was trying to be hip and did a fork of Jetpack and GitHub, made the changes and then tried to make a pull request. Turns out, I don’t know how to do that, so I opened an issue instead. It sits here.