The Algorithm Dilemma

Last weekend, we watched the Netflix documentary, The Social Dilemma, and over the week, I’ve been discussing the content with my wife. We came to several conclusions, including that there are some algorithms and some services we are too dependent on for our entertainment needs. But there are others we can very much get rid of and should, as soon as possible.

The ones we are dependent on are Instagram and YouTube. We’re constantly on Instagram from the moment we wake up to when we go to sleep. It’s unhealthy, and we’re trying to reduce our time on these networks, but it’s a way to cope with all that’s happening out there. We’ve pivoted from just using Insta for getting jealous about travel bloggers to using it for memes, current affairs, and TikTok overflow bloggers. YouTube is our coffee companion. Whenever we sit down after a long day of work, we use it to get the news, weather, movie and show trailers, and catch up on our interests.

In line with that, we’ve noticed that these networks have both gotten better and worse at latching on to our needs. Instagram has gotten frighteningly good with their ad-focus. I’m generally immune to ads – I rarely see them on my computers thanks to uBlock. But the ones I see on Insta are almost always tech focused and I’ve started really salivating on those. On the flip side, Instagram is a well known negative-thought-bringer and I’ve started noticing the general tone of negativity it brings in our lives. YouTube is great at generally recommending time pass videos, but it’s gotten horrible at surfacing new, good content. The same few videos are shoved down our throats every day, all day, until we watch them. Part of the problem is that our main place to watch YouTube is their Apple TV app. This app has terrible UI. There’s no refresh button and the app doesn’t make an API refresh call even if you kill it and start it again. It’s like the algorithm is stuck on these recommendations no matter what you do.

Lately, for my wife, the YouTube app has been recommending a YouTube produced documentary about Paris Hilton’s life. This is despite that she’s never seen any content related to Paris Hilton or her corollaries, has never seen anything related to obscenely rich and spoilt people, and actively avoided this video every day for the past five days. But, like the demon from the movie It Follows, that video recommendation follows her everywhere. Sometimes it’s at position two in the recommended list, sometimes four. It’s present in the Entertainment section of the app, and in the News section, and in Originals. It’s obvious why this is happening – YouTube produced this content and wants to earn it’s money back. It’s like they hired a Netflix PM and he (definitely a HE for ruining a good product) brought the same stupid ideas he implemented there, here. We’ve discussed starting the video and downvoting it. But my wife pointed out that the lesson from The Social Dilemma is that the algorithm doesn’t care about the vote. It just sees engagement as a good sign for their vested interests and will simply count that, discounting everything else. She has actively started skipping over the video, hoping YouTube will finally get the hint one day. Can’t wait.

One of the things our eyes were opened to was how inherently evil this dependence on shady algorithms is. One of the interviewees says, “but it’s easy to forget how much good these technologies have done, how they’ve connected long lost people and found organ donors.” Another says, “when we were building these, we just wanted to build a tool to connect […] but we forgot to look at the flip side of the coin” (quotes fuzzy and from memory, please watch the docu). But every new layer they peeled in the story felt like a revelation that every decision in these companies is made to cater to the bottom line instead of ever bothering to wonder if it’s good for the masses that use the social platforms mentioned. The design ethicist from Google at least mentioned thinking about how their actions affect millions. The folks from Facebook can’t be bothered.

The thing is, none of this is necessary. But it was inevitable. The Internet was always poised to take over the rest of media. A free travel blogger, vlogger, Instagrammer will always throw out the need to subscribe to a travel magazine. A labor of love tech blog will always dismiss the need to pay for PC Magazine. Someone posting news snippets and their commentary in their free time will completely upend the newspaper business. That’s just bound to happen. Video will always kill the radio star.

But this is not just because of the inherent freedom that comes with the Internet. It’s because our society, our norms, and our laws have always operated in whiplash mode, always catching on with something after it has just become passé.

As the documentary moved from the first half to the second, it started focusing on the political ramifications of the freehand these Internet behemoths got and a message came across. It’s not just social. Yes, YouTube is social and Facebook is a place for video. But Google is just as much to blame for inherently bad search algorithms, and Amazon for terrible facial recognition technology as Facebook and twitter are for letting foreign powers turn American politics into a sham, as WhatsApp is for enabling mass state-sponsored violence in parts of the world, and as tech companies are for promulgating the problem of racial and gender inequality while talking about the Internet as an egalitarian utopia.

After the docu, I sat for a long time in conversation with my wife and we discussed ways that we can improve our interactions with the Internet as it is today. We decided to move from Google Search to DuckDuckGo. We decided to uninstall the official twitter client and exclusively use tweetbot and others. We decided, over these past few days as YouTube inundated us with a Million Heiress’ documentary, that we will actively stop using the YouTube recommendations section and start using it’s Search and Channels to find content we want to watch. It’s not like their search is any better, since it shows only a fraction of the content on the service before giving up on you. But at least it’s better than their silly recommendations algorithm, which really needs an overhaul. Lastly, we decided that we’ll police our time on Instagram and tell each other to get off the network as much as possible.

In other news, I was recently reading an article about what Google is doing to keep bad results out of their Search, and here are my notes on the topic –

Google has a new plan to keep junk out of search

Google Search is every bit as important

Yup, we often overlook it, but search is actually way more important in people’s perceptions of the world than we think.

Social media has proved that “people read it and shared it” has no correlation to expertise, relevance or truth.

I would say that there always has been a more discerning, a more learned clientele of knowledge than the common folk. Though it’s not true that common people are in any measure lesser educated, they certainly are less discerning and more prone to peer pressure. If they see something being shared, they are more likely to jump on it as their new belief than some folks who would rather investigate, even though that investigation doesn’t take more than a few minutes in today’s information soaked era. Speed of information veracity has already reached a pretty good point and algorithms and machine learning continue to make it faster. But people’s willingness to ignore all that is also increasing.

So the technological solution is to create better tools to nudge people towards the truth. But the societal solution is what will matter in the end, and one societal solution is to make people less busy in their work lives, giving them more time to look outwards to what’s happening in the world. The current working generation doesn’t have the brain space to deal with everything going on in the world on top all the work they’re expected to do. We’ve all seen the chart where productivity has risen disproportionately to income levels in the last few decades. This has led to a form of inequality where the only people who have the time to ponder over important things are those who are either content with their current means, or have enough means to not worry about money. Now, this has been the case since the time of Socrates, but should not be the case today, should it?

Update: I was thinking about a simpler time when we used to own the knowledge that we bought – whether as newspapers, or books or magazines. Similarly we used to own music and video. But moving online liberated and democratized all these – people who could not afford music players or expensive books could enjoy streaming music, or ad-supported music videos, or read Wikipedia or blogs to gain knowledge. People have built entire careers through learning programming or handiwork on YouTube. We used to own apps on our phones five years ago, and today we’re moving to subscription models and rundles. But this means that if we want to share something, we have to do it on the platform it’s on. If you’re sharing an Instagram post, or a medium blogpost, the receiver is forced to login to see it. If you own Kindle ebooks, you can “lend” it, but only on Kindle. There needs to be a whiplash where we start paying for our knowledge again, for our media again, our ability to share and spread our sources. But that needs a perspective and longer term thinking that’s a longer conversation.

Sourcing information

We all do most of our browsing on our phones. When we come across something we don’t know about, we google it to find out more. More often than not, the link that gives us the most information is either Wikipedia or a news site.

If it’s current affairs, it’s a news site. If it’s general information, Wikipedia. Then why do we still google the thing? Why waste time on the middleman? Is it force of habit? Is it because we believe that google will give us the most comprehensive information and links? Is it just laziness?

Perhaps it’s all of the above. Google is our one stop shop for all information. Whether we’re looking to buy something, looking for a website which we don’t often go to, looking for some news, or solving some mystery on the web, google will give you the knowledge you’re looking for. That’s a great product, regardless of any other implications on privacy, advertising, politics etc.

So why should we opt to change this excellent workflow? (Need information, ask google, get information)

Because it’s worth it to go to the source.

  • Google often scrapes data from Wikipedia, but most of the time, it’s incomplete. It’ll be the first line or paragraph in a topic that’s complex and needs some more study to understand. Or, google will tell you a part of the information, expecting you to select a link to learn more from. So why not go to the source directly?
  • When the topic is a current affair, Google will show you links that it judges to be of your interest, or of value to them (advertising, collaborations with sites like twitter which will be surfaced above others). Instead, if you go to a solution such as Apple News (or Google News perhaps) and search for the topic you’re looking for, you’ll see a more balanced perspective because all Apple News is doing is collecting links from various news sources and presenting those to you. Notice that I didn’t say you should go to a particular news site for this, because if you want real news, you’d better be looking at more than one source.

Now, how do we make this easier? How do we give up our google habit and go to the source? On mobile, the simplest way to do this is to move your apps around. On my phone, the Wikipedia app sits on the main home screen and the Apple News app sits inside a folder on the dock (most of the time, I end up searching for the news app on spotlight search, but I’m trying to get rid that habit too).

This is not ideal. In an ideal world, I would not have to go to each app individually to search for the topic at hand. I would be able to select a word or phrase and use the share sheet in iOS to jump to Wikipedia or Apple News, neither of which seem to support this simple functionality.

But those are the technical details, which may change at any time. What matters is where we source our information from and why. I recommend that you start cutting out the middleman and go directly to the sources, sites, and services that you trust, because those are the same ones your middleman trusts too. As for the why, well, start doing this and you’ll see a change in how you receive information and perceive the news. Search is good, but search algorithms may very well not be.

No Waze

So, I gave Waze a try after being a Google Maps user for a long time. I had enjoyed using Waze way back in 2012, when I drove all over California trusting this app.

But this time, it chose to disappoint. Routes keep changing arbitrarily. When I let Waze decide the route, more often than not, it picks some convoluted route with a lot of loops for no good reason. When I ask it to compute routes again, it straightens up and gives me the right route. Worst of all – when I noticed a mistake on the map, I submitted it. But instead of a streamlined process, I got contacted by another Wazer (probably a map editor) who asked me some more details about the business, and had never heard nor bothered to google for the business. I was told that Google Maps is not a valid source of information because Waze policy says no copyrighted information may be used to correct the map. (That seems like an OK policy.) Eventually, though, the map edit was accepted as is. Perhaps the user trusted me or perhaps they did their due diligence. However, a week later and the map edit has still not appeared on Waze. So much for that bureaucracy.

Waze has also been showing me ads for nearby businesses as soon as I stop at a traffic light or slow down. I don’t actually blame them for this. It could just be a tactic by Google to force people to just skip over to Google Maps.

All in all, I think my little Waze experiment is over. Time to go back to Google Maps, which keeps improving on a daily basis, by hook or by crook.

p.s. The last straw came just yesterday, when I sent my wife a message from the Waze app, to inform her of my ETA. The app asked me to pick out the contact and showed me her name and phone number. Normally, such a notification would be an SMS with a link. Uber does it like that, Lyft does it like that and frankly, that makes the most sense – the other person gets a link, opens it and can track you in their browser. But Waze noticed that my wife had the Waze app installed, so they decided to send her a notification inside the app. She doesn’t have notifications turned on for Waze. Why would she? No one needs random notifications from a low-usage app. So, she never even received the ETA link. This failure from Waze is a UX issue which they should resolve. They don’t need to re-invent the wheel. Just use the same SMS notification services that every other app in the world uses and get it over with. Since they chose to do it in this half-baked, non-thought-out manner, I think it’s high time I part with the service.

Photo by oniitamo

This is why Google+ integration failed

 

google+ spam

I got the above chat request a few days ago. This came in Hangouts, which is tied into my GMail. I opened it today. Amanda wants to chat. OK.

But, who’s Amanda? No where in the above window is there a link that goes out to Amanda’s Google+ profile. I can’t even see Amanda’s gmail ID from this dialog. But that’s supposed to be a moot point if I can get access to her (it’s?) Google+ profile.

But if I can’t see either of those, how do I know it’s spam or a legit chat request. To err on the side of caution, I’ve Ignored, Reported and Blocked Amanda.

This is why the Google+ integration failed from the get go. If you’re going to shove it down our throats, at least be thorough with it.

License, don’t acquire

Silicon Valley has a bad habit – that of buying outright any company that might prove useful to them and the tech community. When Google bought Waze, Facebook bought Spool and Pinterest bought Icebergs, they all did it to bring to their platforms, users and companies, ideas, technologies and features that they believed would be a good fit with their own setup.

But they did it wrong. Waze is a great app and when it finally disappears (as do all Google acquisitions), it will be a great loss for it’s users. Waze has a unique UI, a dedicated user following and features that are not at all present in Google Maps. While the integration went well, Google Maps is an overloaded app with too many features. Eventually, they’ll simplify and drop a few features, getting rid of many core things that Waze is known for. In no circumstance will Waze ever recover from this setback.

Continue reading

Notes for Week 19 of 2014

The last time I did this, it was week 2 of 2014. But here we are again, with a bunch of nice links to share with you nice folks. Enjoy!

 

Internet

Which is the most popular IP among network engineers? It’s 8.8.8.8, which is Google’s DNS. But this wasn’t always the first IP to be pinged. Before this was Level 3’s not-really-public DNS on 4.2.2.2. Here’s an excellent roundup of the story behind the company across the hill.

Critical Thinking

Here’s a very simple, very straightforward approach to critical thinking. Be advised, I love repeating this ‘program’ over and over again. Do bookmark it.

Religion

Here’s an image explaining why religion can be a bad thing sometimes. Enjoy. 🙂

Writing Tools

There are some really interesting writing tools on the Internet. Here are two that blew my mind with their approach – Gingko and Lines. Do tell me what you think about them.

Finally

Speaking of writing tools, here’s one of my favorites. It’s a beautiful idea, embodied by the simple example that the developer created called “I Made Tea”. I’d really like to know what my readers make with something as elegant as Telescopic Text.

 

Notes for Week 2 of 2014

So, it’s been an interesting week. Some observations –

Social

Found this gem of a Difference between Facebook and Twitter –

Facebook – 

“Best Practices

Making API calls directly to Facebook can improve the performance of your app, rather than proxying them through your own server.”

Twitter – 

“Caching

Store API responses in your application or on your site if you expect a lot of use. For example, don’t try to call the Twitter API on every page load of your website landing page. Instead, call the API infrequently and load the response into a local cache. When users hit your website load the cached version of the results.”

< p>Turns out, when not losing market share to a third-party app, Facebook is actually quite nice to developers as compared to Twitter. To be fair, tweets constitute a lot more volume and processing, so it would make sense for Twitter to want the devs to cache their data. Also, even ADN  has rate limits but at least their limits are more generous than Twitter.

Seriously though, twitter has millions of dollars for servers and all I have is a 128MB VPS. What the heck, Twitter?

Google(+)

Google is no longer Google. It’s Google(+). Everything we love about Google and it’s services is being slowly replaced by Google+ and the latest victim is GMail. Now anyone on Google+ can email you without knowing your email ID. As a communication tool, this makes GMail more open. But that’s exactly what people don’t use GMail for. They use it for Email. Big difference there Google. You can opt-out, but what’s the bet that option will be going away soon?

What Google should actually do –

Google understands one thing and one thing alone – Search. Pushing Google+ isn’t going to help them overcome the social networks of the world. But there is one thing I covet – the Search API. Seriously, why don’t we see third-party Search apps that innovate the way we see our Search results. That’s one data stream we’ve not targeted yet. Google needs to let people in, do their thing and pretty soon we’ll see people integrating Search with  social platforms. Oh, you wanna see which of your Facebook friends searched for the latest Tom Hanks movie and then clicked on IMDB? Here’s the data to that. Seriously Google, stop letting one segment of the business take over the other, specially since we know you’ll kill Google+ a couple of years from now.

Advertising

Ah, advertising! The Bane of TV show lovers binge-watchers. Advertising has slowly crept in everywhere on the Internet, from YouTube to Hulu. Towards YouTube, go find YouTube5. It’s an extension that replaces the usual YouTube player with a cool HTML5 one and kills all ads in the process. Enjoy.

To Hulu, I say, well, get rid of the “Brandon Switched to Ford” ad. Seriously. It’s a stupid ad, I’ve seen all too much of it and Brandon looks like a total douche for being the black sheep who abandoned the family tradition and switched from a Honda to a Ford. If ever Hulu fails, it’ll be because they keep repeating the same ads over and over again. I do not want to be bored by ads, I want them to be innovative and interesting. (Coincidentally, Samuel L Jackson staring in my face is not innovative. I’m looking at you, Capital One.)

I finally also saw the KFC ads that look like some woman with a video camera uploaded to YouTube. That’s supposed to be innovative? Nope. She looks drunk/high/both and you’re not fooling anyone with these ads KFC, those are scripted (or worse, they’re not!).

Finally, saw a teeth whitening strips ad on Hulu that said, very specifically, “If your teeth are not getting white, they’re getting yellow”. Ok, first of all, yellow teeth are perfectly normal and more an indication of stomach trouble than a medical emergency. Second, the ad targets people women who drink coffee. First it was guys who smoke who were targeted and now this. Finally, that text up there. That’s a scare tactic. Pretty soon, they’ll come up with a white paper saying that yes, your teeth getting yellow is a medical problem and you need to use teeth whitening strips in conjunction with toothpaste. All of this will be driven by only one thing – Sales telling the Marketing team to get innovative with the ads. There’s no real medical issue that they’ve tried to resolve.

That concludes the rant session on advertising.

Clients from Heaven

I’ve been building a web app for my brother and he mentioned that the text on the screen doesn’t ‘look black’. For a second, I tried hard not to wonder if my brother is a typical MBA Client from Hell but as it turns out, he was right, the text was actually #2C3E50 which is actually a weird dark blue. Thanks Bootstrap for making me look bad in front of my brother!

WordPress

It was an exciting week to be a WordPress user. Snaplive, a front-end text editing solution was showcased to a few who had signed up for updates. It seems to work really well with WordPress, so expecting some really good things in the future.

Ghost had promised to revolutionize WordPress, but instead it went and setup shop elsewhere. That’s ok, since we have Gust, which is a plugin that ports the awesome Ghost Admin panel functionality to WordPress. Mind you, this just released, so if you’re not ready for bugs (which software doesn’t have bugs?), don’t install this yet.

Finally, a shout out to whatweekisit.com, which I used to, umm, calculate which week of 2014 we’re in. Yeah, I should have just looked at a calendar.

How the Common Man will pay for the Internet soon

If there’s one buzz word that’s promised to solve all the monetary problems of every Internet-based startup in the past two years, its “Big Data”. Everyone’s collecting it, everyone’s recording it and everyone’s saving it for tomorrow. From Twitter’s Billions(of tweets) to your Netflix queue and even your Google Search history, everyone’s looking to figure out how to sell you more stuff based on your habits.

But no one’s actually selling the data. The data in itself is useless, no matter how much of it you have. It’s the connections that are formed from it that are important and that’s what everyone is hankering to sell – information. However, all that these companies are trying to do is sell information to advertising sources and point of sales organizations like Amazon because that’s where they can easily get a large paycheck in exchange for a much larger database of customer information. Continue reading