Notes on setting up Freedbin

Here are some notes on how to setup Rachel Sharp‘s Freedbin, which is a docker version of the popular Feedbin RSS feed reader.

I had some trouble setting this up on my Windows 10 machine. A lot of issues I faced had to do with setup and environmental variables. I don’t think I faced any real issues due to my host being Windows, other than the terrible thing that Windows 10 itself is. Anyways.

First of all, I had an already running version of postgres for other docker images, so there was a conflict I was not able to resolve, since Rachel’s docker compose file calls its images directly from Docker Hub which are not easily configurable. If someone can guide me to using the same postgres instance for two docker projects, that would be great! Right now, I have two docker containers running postgres.

So, (real) first of all, I downloaded the repo to my own machine to make modifications.

To begin, in the docker-compose.yml, I changed the name of the service from postgres to postgresfeedbin and changed the port to 5433 instead of the 5432 which was already in use.

I also changed the app image from rachsharp/feedbin to a local name freedbin_app and added the build line, so I could build the changes I’m putting in.

I added the restart unless-stopped line to ensure my containers never stop! 🙂

There’s a discussion on the github repo about replacing Postlight’s Mercury service with our own open source version of the same. Postlight has sunset their own servers, so it makes sense to use our own. One alternative is to use Feedbin’s own extract service, but that is available only in the newer version of Feedbin, which Rachel’s docker container doesn’t use. Instead, I already had a docker image of Mercury from the docker hub that I’ve setup for tt-rss and other projects, which I just connected to, using the MERCURY_HOST environment variable. In this setup, the MERCURY_API_KEY doesn’t do anything. Mercury just ignores it and it seems that so does Feedbin.

All of the above are summarized here, as part of the docker-compose.yml file –

app:
    # image: rachsharp/feedbin
    image: freedbin_app
    build: .
    restart: unless-stopped
    environment:
      - MERCURY_HOST=http://192.168.99.100:3000
      - MERCURY_API_KEY=abcd
      - SECRET_KEY_BASE=abcd
      - POSTGRES=postgresfeedbin
      - POSTGRES_USERNAME=feedbiner
      - POSTGRES_PASSWORD=feedbiner
      - PGPASSWORD=feedbin
      - DATABASE_URL=postgres://feedbiner:[email protected]:5433/feedbin_production
[...]
  postgresfeedbin:
    image: postgres
    restart: unless-stopped
    command: -p 5433
    environment:
      - POSTGRES_USER=feedbiner
      - POSTGRES_PASSWORD=feedbiner
    ports:
      - 5433:5433
    expose:
      - 5433
    volumes:
      - postgres_data_feedbin:/var/lib/postgresql/data
volumes:
  redis_data:
  postgres_data_feedbin:

I further had to make changes to the startup_script.sh file as here –

if psql -h postgresfeedbin -p 5433 -U feedbin -lqt | cut -d \| -f 1 | grep -qw feedbin_production; then

As seen, I’ve just pointed it to the new service name and port.

At this point, the service was able to start. I was able to create an account and get in and add feeds. However, I follow a lot of feeds and importing an OPML file makes good sense for me. But, the import settings page was failing to import due to a failed AWS config. I looked up solutions and one way around is just to disable a connector called CarrierWave, which connects to AWS. Guess what gets disabled if you disable CarrierWave? The import/export page.

So, I went about creating an S3 bucket on AWS, getting credentials, and making the S3 bucket publicly accessible. I don’t know why this is the case. Perhaps if we use a newer version of Feedbin, these issues will not pop up, but in Rachel’s version, this is the case, so I went with it.

After I made my S3 bucket and got the AWS credentials, I added them to the Dockerfile as here. The variables are already there, just need to be filled up –

ENV FONT_STYLESHEET=https://fonts.googleapis.com/css?family=Crimson+Text|Domine|Fanwood+Text|Lora|Open+Sans RAILS_ENV=production RACK_ENV=production AWS_ACCESS_KEY_ID='my_key_id' AWS_S3_BUCKET='my_bucket_name' AWS_SECRET_ACCESS_KEY='sooooo_secret!' DEFAULT_URL_OPTIONS_HOST=http://localhost FEEDBIN_URL=http://localhost PUSH_URL=http://example.com RAILS_SERVE_STATIC_FILES=true

There’s one more catch. The Feedbin code uses its own version of CarrierWave called CarrierWave Direct, which defaults to try to use the ‘us-east-1’ region for AWS. If your bucket is there, you’re fine. Mine is in ‘us-west-1’, so I had to go into the /config/initializers/carrierwave.rb file and change the following to add my region –

config.fog_credentials = {
      provider: "AWS",
      aws_access_key_id: ENV["AWS_ACCESS_KEY_ID"],
      aws_secret_access_key: ENV["AWS_SECRET_ACCESS_KEY"],
      region: 'us-west-1',
}

Finally, I am ready to build and deploy. Running the following command –

docker-compose build

You’ll notice a new image in your docker images list –

$ docker images
REPOSITORY                    TAG                 IMAGE ID            CREATED             SIZE
freedbin_app                  latest              20a0334cd11c        30 minutes ago      1.27GB

and now you can deploy –

docker-compose up

It takes a while, as Rachel mentions somewhere, but all services come up perfectly, and I was able to import my OPML file. I noticed that the S3 bucket holds the lone OPML file, so perhaps it won’t cost me any money? Eventually, once I know that the import is done, I’ll go in and delete the bucket.

Big, big thanks to Rachel Sharp for creating Freedbin. It’s a great way to get started on Feedbin and while I was working on setting this up, I learnt how to use docker, created my first Docker container and uploaded my first project to Docker Hub. Hopefully, I’ll be able to build Freedbin from scratch using the latest Feedbin code and Feedbin’s extract service, and using the principles set down by Rachel.

Reeder 4 is here?!

I’ve been experimenting with Tiny Tiny RSS for a couple of weeks now, to figure out if I’m done with Fever RSS. Part of it is that Fever runs on a Digital Ocean VPS and I’d like to reduce the load on that server as much as possible.

I run TT-RSS on my home PC and get access to it using ZeroTier wherever I am. My app of choice for it on the move is Fiery Feeds. It’s a beautiful app with an amazing, rich feature set. But the main reason I use it is because Reeder doesn’t natively support TT-RSS.

But, I’m learning that I’m set in my ways. the TT-RSS web app feels just as dated as Fever, and there are some features in Fever that I really enjoyed, which I miss in TT-RSS. Since I’m looking at two dated, self-hosted RSS readers, why not go with the one I’ve used more?

Also, Fiery Feeds is gorgeous, but it’s not Reeder. Reeder is familiar and cozy. It’s got matured syncing and great UX. Fiery Feeds comes close, but there are some things which are just irksome, such as the way it opens up articles as a pop up. The cadence of Reeder’s panes is just beautiful to work with.

Today I learnt that Reeder 4 has been out since April 25th. At an absurd cross-device cost of $5, it’s a steal for all the features Reeder gives you.

But here’s the flip side – I only use Fever (and now TT-RSS) and while TT-RSS has a plug-in for Fever, I’m more inclined to just use Fever. Which means I have no space for using any other paid or free RSS reader service. So I’m really restricted in what I’m doing with Reeder. Further, though I’ve started using Instapaper as my read later solution, I’ve never really understood the point of using Reeder for Instapaper. It’s fine to skim through, but wouldn’t you rather use the Instapaper apps to read the content? For me, the highlighting and notes functionality of Instapaper is essential. How does one ignore that to read content on Reeder?

Lastly, there’s the demise of Mercury. Mercury has been my savior in this mess of RSS feeds. My TT-RSS installation has a Mercury plugin that cleans up and extracts content from the feeds. It really just works, and since they’ve made it open source and unavailable as an API, Reeder is bound to suffer. Reeder 4 is now experimenting with something called Bionic Reading and it’s a hit or miss. We’ll see.

So here it is –

  1. I’m tired of TT-RSS and untired of Fever (for now).
  2. I love Reeder 3 and Reeder 4 is ridiculously cheap for an upgrade. I have an older iPad Pro, so Reeder 3 works really well for me. But buying 4 means supporting Rizzi in the amazing work he’s doing.

  3. Fiery Feeds is awesome and pretty and useful, but I don’t pay for the subscription, so I don’t get to use the fancy automatic folders that it creates.

  4. RIP Mercury

  5. I really want to get my RSS feed reader off the VPS and onto my local setup, secured and made accessible through ZeroTier.

How many people reading this have already bought Reeder 4?

How is it?

Have you used Bionic Reading?

Which RSS reader service do you use it with?

Do you read Instapaper articles on Reeder 4? How is that experience?

Squarespace is the best and the worst at RSS

Within the last 12 hours, I’ve come across two websites hosted on Squarespace that portray how one mustn’t do RSS. Sadly, at some level, it’s not necessary that the owners of these websites even know what I’m talking about.

I’d like to name these sites –

Soup

and

StephenMarche

These are nice sites – well designed, purposeful, vibrant. But their content is so pitifully inaccessible through RSS. Here’s why –

With Soup, I really wanted to get RSS access to all of the topics they cover. These are Culture, Food, Interviews, Features, to name a few. Usually, when I’m on a site that has RSS feeds, the SubToMe extension tells me how to get to it. In the case of Soup, it failed. The content is visible on the homepage, but the RSS feed that it picked up was blank –

http://thesoup.website/index-rally?format=RSS

‘rally’ is what piqued my interest. What is this CMS?

WhatCMS says that it’s Squarespace.

Well, what do Squarespace’s docs say about RSS feeds? Do they even support them?

As it turns out, they do, and quite well! (or so say their docs)

So I opened each of the ‘topics’ I wanted to subscribe to on Soup’s site and found their RSS feeds using SubToMe. One example –

http://thesoup.website/culturesoup/ -> http://thesoup.website/culturesoup?format=RSS

I immediately noticed that the content is there in it’s entirety! That’s amazing. It almost never happens on commercial sites that the RSS feed carries the entire content.

Good – RSS feeds contain entire content

Bad – I had to subscribe to eight different feeds. There’s no parent or ‘all’ feed

 

Later, I came across Stephen Marche’s writing in NYT and that led me to his site. Again, beautiful site, really modern, really functional and pleasing. I jumped to the Essays -> Recent Work section but alas, SubToMe didn’t find any RSS feed!

By now, I’d wizened up. I know that on most pages, just adding ‘?format=rss’ at the end will get me the RSS feed. So I did that. Nothing. Why is that? Perhaps because the recent work page isn’t really a traditional list of items that Squarespace converts into RSS. It’s a static page which the Admin just adds URLs to the top of. But how would I know the difference? There’s no way. So as of right now, I’m subscribed to Soup’s RSS but not to Stephen Marche’s. I followed him on Medium, but ugh.

Pro – ???

Con – Maybe the admin turned off RSS on purpose? Maybe the page I’m looking at cannot support RSS?

Now, I can reach out to the owners of these sites to figure things out. Maybe I’ll end up educating them on the importance of RSS and maybe I’ll learn something new about Squarespace (do they even support an ‘all’ RSS feed? I don’t know, I’ve never used the platform). Maybe all they need is a slight push in the right direction, or maybe it’s a long project that’ll require a reworking of their workflow (which, tbh, why would they do that for me?)

But I don’t want to do any of this. RSS is the perfect stalker medium on the Internet. Facebook and WhatsApp show you read notifications. On twitter and Instagram you’d end up hitting ‘like’ by mistake. But RSS is one-way (depending on which RSS reader you use) and so it’s perfect for people like me who just want to cultivate their little corner of the Internet.

There’s a post out today by Brent Simmons talking about an article that’s talking about the demise of RSS. Brent points out that RSS doesn’t need to be the ‘default’ for everyone and RSS readers don’t need to be installed on every device on Earth for this to be a successful technology. It already is.

This is most visible with beautiful walled gardens such as Squarespace. Most people who host with Squarespace do it because it’s commercial and aligns with their interests. The primary method of communication for consumers is the newsletter. There are options for eCommerce shops, podcasting, and email campaigns. Much off this could happen without RSS. But Squarespace took the basic RSS technology and chose to use it as the back-end for most of these things. Podcasting is basically an RSS feed with audio attached, so there really was no choice but to use this open standard. Wherever RSS feeds are available, they’re full length and rather useful. So could RSS have a place on the Internet? It already does.

Finding my space

A large portion of the Internet is just about discovering interesting things. A part of that is just generally interesting things. But the other part is things that interest us. These two are different.

For most of my lifetime on the Internet, I’ve sought, and found, interesting things. My media diet has varied a lot over the years, flicking from one service and form of information to another. I’ve frequented twitter, Facebook, reddit, news sites, Instagram, blogspot, imgur, tumblr, self hosted blogs, forums, and a whole lot of the Internet I’d rather not talk about. I’ve seen memes (I hate memes), I’ve been caustic (I’ve learnt that’s just not useful to anyone), I’ve read entire books on Gutenberg.

But of late, I’ve noticed that I’ve finally found my space. Some people find it on tumblr or twitter because that’s where the people are. I’ve found it on RSS. I follow, unfollow, cull, clean, unsubscribe and resubscribe to blogs a lot. Whenever I think about moving away from my current self-hosted RSS feed solution, I look at the 700 odd blogs I follow and think that I’ve got better things to do than to reduce this list to an acceptable-by-the-service-I-want-to-move-to number. I used to follow well over twelve hundred sites,  but I realized that I don’t follow the news the way I used to (now I seek it out myself, when I want to, via Reuters or Apple News), so I unsubscribed every single news-site RSS feed and this is where I am today.

For a short, shining time, I was a part of the App.net story. I wasn’t particularly involved, but I did pay for the API and I did learn a few things along the way. I also made some friends and found more people to follow (overwhelmingly, these are old white guys. Just the demographic frequenting that service, I guess). When ADN went away, I still followed these people’s stories, through other social networks that sprung up (pnut, 10C, micro.blog) but also partly, through their blogs. On these social networks, I found more people to follow their blogs of.

What prompted me to write all of the above? I saw the following post by Colin Walker on his ridiculously well-built blog today –

“It’s not about being perfect, just about being.”

He’d written it in his notebook at some point and took the time to remind readers like me of it.

This idea resounds with me. This is something I’ve struggled a lot with. I’ve tried daily blogging, daily journaling, daily private blogging, scribbling notes on a throwaway page on the net, all in an effort to just put words on the screen, to just ‘be’. It doesn’t matter that those words are perfect. Or, well, it shouldn’t. I still fret over it. I still write something, save the draft, and push it out of my memory, because I worry that it’s not up to the mark. I still feel that a lot of my writing is either too laborious, or too much of a rant, or that I drone on.

Meanwhile, there are people like Colin out there, reassuring us that no one is perfect, that there is nothing more important than putting those words, and oneself, out there. I’m glad I follow his blog, and so, follow him.

I’ve found my space in this one field of interest – writing. There are others I’d like to sate, but I believe I can find blogs for those too. If not, I’ll write about that too, right here, asking for your help, dear reader.

Photo by Blue Trail Photography

Deleting Duplicate items in Fever RSS

My Fever RSS setup has a lot of feeds that often duplicate items. There are feeds from news sites such a The Times of India corresponding to National, International and Governmental news as well as feeds from tech sites that often repeat things. The end result is that I often see the same title, the same post and thus repetitive news many times during the day. I found the following script to be an excellent way to remove duplicate items from Fever. This works on the MySQL level and so you should be careful when using it, lest you delete everything because of some coding error on my part (though I’ve checked and this works). Continue reading

How I Follow Blogs on the Open Internet

Colin Devroe’s post about Fred Wilson’s post about how hard it is to follow blogs on the Open Internet is interesting to me.

Ok, before we go any further, yes, this is very meta. Yes, I could have written this entire thing as comments on Colin’s blog (no, it doesn’t support comments) or Fred’s blog (has nice disqus comments) but I didn’t because that’s the point of blogging. I can write this ‘commentary’ on my blog. Sort of like Greek philosophers writing entire books just discussing each other’s books. Very meta indeed. Continue reading

A brief update on Fever/AppFog

Recently, AppFog sent out an email telling us that free accounts will be further restricted in what features and resources they are will receive. This felt like a major issue for me at that time, since I have Fever running on the service and I’ve dedicated close to 1 GB of RAM to the app.

When the changes finally made through, I realized that I was wrong. Upon monitoring my Fever installation during updates, I realized that it doesn’t use more than ~150 MB of RAM at a time. The only other thing is the database size, which is more than 300 MB for me, something which cannot be easily hosted anywhere else.

I ran some numbers and have found that the most basic paid plan from AppFog can allow for 8 Fever installs with 256 MB each but with the restriction of 200 MB of database storage per install for $2.5/mo. So, if you can find 7 other people who don’t have more than, say, 300 feeds in their Fever installation, AppFog would be the perfect place for you. It would also be a good way of giving back to the service that has supported free Fever installs for so long.

To everyone else, I must ask this – tell me about your Fever installs. How much are you paying? How much RAM and db are you using? Would you be open to sharing space with me (and possibly others) to reduce hosting costs?

I love Fever. It’s one of those services that are just the perfect fit, in this case for reading RSS feeds. I’m grateful to Shaun Inman for continuously working on this application, even though it seems that new installs are at an all time low and he’s busy with personal stuff. I am ready to pay for the hosting, but I figure that if we work together, we can reduce our costs greatly.

On Google Reader

Ever since the Google Reader news of day before, I’ve noticed a marked increase in traffic to my blog posts regarding Fever as an alternative RSS reader. In fact, I’ve seen my previous record for views in a day of 495 was broken yesterday with 539 views. I’d like to comment on a few things while I have your attention.

First of all, yes, Google Reader was a free service and yes Fever is not. You pay $30 upfront and if you’re not able to get free Appfog hosting, you end up paying about $4-6 per month for hosting it on a fairly cheap host like NearlyFreeSpeech. Why is it variable? Because Fever’s hosting costs you based on how many feeds you want to add to it (my Fever MySQL database is 200 MB at the moment). But even though Fever is a paid solution, I’d still make the case for it. Continue reading

DotIt!

A bookmarklet for importing to DotDotDot –

A few days ago I posted about an up and coming service called DotDotDot that is a great replacement for Instapaper. That day someone posted a link to my blog on HackerNews and my site got 280 hits in a day, a record for my blog… 🙂 Continue reading

Back to Instapaper

I’ve recently returned to Instapaper.

Why? Because it’s neat. I use Fever exclusively for my RSS consumption but the feed view in Fever is pretty bad. So, almost always, I found myself looking for a way to read interesting articles without visiting their ad-filled websites. Instapaper was embedded in Fever, but I discovered that if I don’t just want to save the article to Instapaper, I want to read it right then, I could easily integrate the “Instapaper Text” bookmarklet into Fever and go from there. Further, Instapaper’s code is smart enough to parse 99% of the sites I want to read cleanly (including, amusingly, Google.com :D)

Continue reading