The Algorithm Dilemma

Last weekend, we watched the Netflix documentary, The Social Dilemma, and over the week, I’ve been discussing the content with my wife. We came to several conclusions, including that there are some algorithms and some services we are too dependent on for our entertainment needs. But there are others we can very much get rid of and should, as soon as possible.

The ones we are dependent on are Instagram and YouTube. We’re constantly on Instagram from the moment we wake up to when we go to sleep. It’s unhealthy, and we’re trying to reduce our time on these networks, but it’s a way to cope with all that’s happening out there. We’ve pivoted from just using Insta for getting jealous about travel bloggers to using it for memes, current affairs, and TikTok overflow bloggers. YouTube is our coffee companion. Whenever we sit down after a long day of work, we use it to get the news, weather, movie and show trailers, and catch up on our interests.

In line with that, we’ve noticed that these networks have both gotten better and worse at latching on to our needs. Instagram has gotten frighteningly good with their ad-focus. I’m generally immune to ads – I rarely see them on my computers thanks to uBlock. But the ones I see on Insta are almost always tech focused and I’ve started really salivating on those. On the flip side, Instagram is a well known negative-thought-bringer and I’ve started noticing the general tone of negativity it brings in our lives. YouTube is great at generally recommending time pass videos, but it’s gotten horrible at surfacing new, good content. The same few videos are shoved down our throats every day, all day, until we watch them. Part of the problem is that our main place to watch YouTube is their Apple TV app. This app has terrible UI. There’s no refresh button and the app doesn’t make an API refresh call even if you kill it and start it again. It’s like the algorithm is stuck on these recommendations no matter what you do.

Lately, for my wife, the YouTube app has been recommending a YouTube produced documentary about Paris Hilton’s life. This is despite that she’s never seen any content related to Paris Hilton or her corollaries, has never seen anything related to obscenely rich and spoilt people, and actively avoided this video every day for the past five days. But, like the demon from the movie It Follows, that video recommendation follows her everywhere. Sometimes it’s at position two in the recommended list, sometimes four. It’s present in the Entertainment section of the app, and in the News section, and in Originals. It’s obvious why this is happening – YouTube produced this content and wants to earn it’s money back. It’s like they hired a Netflix PM and he (definitely a HE for ruining a good product) brought the same stupid ideas he implemented there, here. We’ve discussed starting the video and downvoting it. But my wife pointed out that the lesson from The Social Dilemma is that the algorithm doesn’t care about the vote. It just sees engagement as a good sign for their vested interests and will simply count that, discounting everything else. She has actively started skipping over the video, hoping YouTube will finally get the hint one day. Can’t wait.

One of the things our eyes were opened to was how inherently evil this dependence on shady algorithms is. One of the interviewees says, “but it’s easy to forget how much good these technologies have done, how they’ve connected long lost people and found organ donors.” Another says, “when we were building these, we just wanted to build a tool to connect […] but we forgot to look at the flip side of the coin” (quotes fuzzy and from memory, please watch the docu). But every new layer they peeled in the story felt like a revelation that every decision in these companies is made to cater to the bottom line instead of ever bothering to wonder if it’s good for the masses that use the social platforms mentioned. The design ethicist from Google at least mentioned thinking about how their actions affect millions. The folks from Facebook can’t be bothered.

The thing is, none of this is necessary. But it was inevitable. The Internet was always poised to take over the rest of media. A free travel blogger, vlogger, Instagrammer will always throw out the need to subscribe to a travel magazine. A labor of love tech blog will always dismiss the need to pay for PC Magazine. Someone posting news snippets and their commentary in their free time will completely upend the newspaper business. That’s just bound to happen. Video will always kill the radio star.

But this is not just because of the inherent freedom that comes with the Internet. It’s because our society, our norms, and our laws have always operated in whiplash mode, always catching on with something after it has just become passé.

As the documentary moved from the first half to the second, it started focusing on the political ramifications of the freehand these Internet behemoths got and a message came across. It’s not just social. Yes, YouTube is social and Facebook is a place for video. But Google is just as much to blame for inherently bad search algorithms, and Amazon for terrible facial recognition technology as Facebook and twitter are for letting foreign powers turn American politics into a sham, as WhatsApp is for enabling mass state-sponsored violence in parts of the world, and as tech companies are for promulgating the problem of racial and gender inequality while talking about the Internet as an egalitarian utopia.

After the docu, I sat for a long time in conversation with my wife and we discussed ways that we can improve our interactions with the Internet as it is today. We decided to move from Google Search to DuckDuckGo. We decided to uninstall the official twitter client and exclusively use tweetbot and others. We decided, over these past few days as YouTube inundated us with a Million Heiress’ documentary, that we will actively stop using the YouTube recommendations section and start using it’s Search and Channels to find content we want to watch. It’s not like their search is any better, since it shows only a fraction of the content on the service before giving up on you. But at least it’s better than their silly recommendations algorithm, which really needs an overhaul. Lastly, we decided that we’ll police our time on Instagram and tell each other to get off the network as much as possible.

In other news, I was recently reading an article about what Google is doing to keep bad results out of their Search, and here are my notes on the topic –

Google has a new plan to keep junk out of search

Google Search is every bit as important

Yup, we often overlook it, but search is actually way more important in people’s perceptions of the world than we think.

Social media has proved that “people read it and shared it” has no correlation to expertise, relevance or truth.

I would say that there always has been a more discerning, a more learned clientele of knowledge than the common folk. Though it’s not true that common people are in any measure lesser educated, they certainly are less discerning and more prone to peer pressure. If they see something being shared, they are more likely to jump on it as their new belief than some folks who would rather investigate, even though that investigation doesn’t take more than a few minutes in today’s information soaked era. Speed of information veracity has already reached a pretty good point and algorithms and machine learning continue to make it faster. But people’s willingness to ignore all that is also increasing.

So the technological solution is to create better tools to nudge people towards the truth. But the societal solution is what will matter in the end, and one societal solution is to make people less busy in their work lives, giving them more time to look outwards to what’s happening in the world. The current working generation doesn’t have the brain space to deal with everything going on in the world on top all the work they’re expected to do. We’ve all seen the chart where productivity has risen disproportionately to income levels in the last few decades. This has led to a form of inequality where the only people who have the time to ponder over important things are those who are either content with their current means, or have enough means to not worry about money. Now, this has been the case since the time of Socrates, but should not be the case today, should it?

Update: I was thinking about a simpler time when we used to own the knowledge that we bought – whether as newspapers, or books or magazines. Similarly we used to own music and video. But moving online liberated and democratized all these – people who could not afford music players or expensive books could enjoy streaming music, or ad-supported music videos, or read Wikipedia or blogs to gain knowledge. People have built entire careers through learning programming or handiwork on YouTube. We used to own apps on our phones five years ago, and today we’re moving to subscription models and rundles. But this means that if we want to share something, we have to do it on the platform it’s on. If you’re sharing an Instagram post, or a medium blogpost, the receiver is forced to login to see it. If you own Kindle ebooks, you can “lend” it, but only on Kindle. There needs to be a whiplash where we start paying for our knowledge again, for our media again, our ability to share and spread our sources. But that needs a perspective and longer term thinking that’s a longer conversation.

Things the YouTube Apple TV app needs to improve

black remote control

Just a short list of things that’s wrong with the Apple TV YouTube app –

1. Every time you pause and play a video the entire video interface comes up and doesn’t go away for a good ten seconds. Users have to hit the back button to get rid of that shitty interface. Learning this behavior is bad, because NO OTHER app does this and then the reinforced behavior causes a lot of headache when using other, better apps.

2. There’s no fast switching between user profiles. This is true for other apps as well, but particularly needed for YouTube, where since a very long time we’ll all had separate profiles and thus the algorithm has learnt to show us different things. Going from one profile to another is just irritatingly long a process.

3. There’s no way to quickly refresh the recommendations the algorithm throws at us. There are some recommendations that we get shown for days at end, and we have to just watch the video to get rid of it. There needs to be a clear refresh button that just washes the slate clean.

4. There’s no “Live” section. In today’s day and age, there are a LOT of livestreams going on and we’d like to see them. Right now, the only way you can see them is if you search for them with the keyword “live” and hope to see it in the results, or if the algorithm shows them to you. There’s a section for “gaming”, which we NEVER see, but clearly they’ve not made the sections “learnable”.

5. When you’re watching a video and you want to interact with the “interface”, you can’t pause the video and interact with it. As soon as you move your cursor, the video starts playing. This is highly unwieldy.

6. Their end-of-video interface is shit. SOOOO many videos have content all the way till the end, but they wash over the last ten or fifteen seconds with the “in-video links”, thereby destroying my ability to see the content all the way to the end. This also creates a learning for channel owners to add a goodbye template at the end, which major news channels do, but this is not a good learning for when you’re watching a video on mobile or desktop. So for YouTube to do this, just focusing on Apple TV, is terrible!

There are some good things too –

1. The keyword search is epic. Much better than other apps and well integrated into the Apple TV search by voice model (here’s looking at your shitty search HBO). Though I’d like for the search to be deeper, with more sections to separate the searched content as “Live”, “popular”, “recent”, etc.

2. They’ve finally added a quick button to access the Channel from the video interface, but they’ve hidden the like/dislike buttons behind the “additional options” section in the video interface. This is, I guess, a move to trust the algorithm more, but it’s terrible, because I’d LOVE to downvote some videos after watching them.

3. The algorithm is actually doing well in recommending similar content over time. Not in the immediate instance, but over time.

That’s all for now. If you’ve been watching the news, you know that the police in the US has been attacking and arresting peaceful protestors all over the country, so please consider donating to the National Bail Fund Network here.

A comment on The YouTube Conundrum

The following is a comment I was writing on the above post. It became long enough that I’d rather just throw it on here for posterity. A couple of loosely knit thoughts on YouTube –

YouTube seems different.

Source: Is YouTube Fundamental or Trivial? – Study Hacks – Cal Newport

YouTube is different.

Instagram, twitter etc have a feed. YouTube doesn’t have a feed like that. YouTube does have an autoplay option, but in my experience most people prefer to keep it turned off. It’s a fundamentally different browsing model than these other social networks.

The author wishes people use YouTube as a sort of backend to embed videos into their websites. I’d say that a lot of people did initially experiment with video embeds as a means of ‘indiewebifying’ YouTube and Vimeo. Many still continue to do so. So many methods of embeds exist, from WordPress shortcodes to YouTube itself giving you easy to copy iframe and html5 snippets. But that’s not how YouTube is truly consumed. Just like those other social networks, YouTube is consumed mainly within their app. There’s true continuity there, even though most people don’t actually use it.

The other point is that content is king. When you’re chasing silly cat videos, whatever YouTube suggests seems fine. Similarly, when my wife has to do some housework, she puts on one fashion blogger or the other and the algorithm takes her on a journey of background noise that’s more than adequate.

However, when we get home and want to either watch some news/latenight commentary/random funny videos from specific content creators, we specifically select a video, play it, and exit after it’s done.

My main method of consuming YouTube is on the Apple TV. With the new version of their app, YouTube has effectively shot themselves in the foot. The app doesn’t do a very good job of good, engaging, never-ending recommendations. We’re a little more discerning than letting complete random videos play when we’re actively looking at the screen, so after a few refreshes, the content of the day dries up and we can actually get out and watch something else we’ve been paying for – Netflix or Amazon.

So what’s the right way to think about YouTube: is it fundamental to the internet revolution, or just another source of social media distraction?

YouTube is both, true. But it’s both because people have recognized the value of uploading serious content on there. Now, serious content isn’t only suited to video format. It can be made in photos (see brainpickings Instagram) and in tweets (the reuters twitter feed). But can it be consumed in those formats easily? No, and that’s why YouTube stands out.

YouTube is a conundrum because people actively upload cat videos on it.