Automatically post from your RSS feed to Mastodon using Netlify

Some time ago, I experimented with ActivityPub, delving into the spec trying to implement (a very simple limited subset of a) a server for this website. This has left me with opinions. If you're just here for the title, feel free to skip to the second section.

The protocol that wasn't

While ActivityPub goes into a lot of detail how the JSON objects should look like, it leaves a lot of things unspecified or completely ignores them. This is why the Mastodon docs don't just cover ActivityPub, but also include Webfinger (a protocol for translating user handles to ActivityPub endpoints), Microformats (a class-based semantic markup), and several others. Despite this, numerous features are not officially specified or are Mastodon-specific extensions: Many admin tasks (like blocks or announcements), custom emojis, how polls work, hashtags, pinned toots, profile discovery, and much more are all non-standard extensions.

Surprisingly, ActivityPub actually has more features in one area not commonly implemented by most Fediverse servers: Client-to-server interactions.

In theory, ActivityPub isn't limited to just the servers, clients may use it too! This is possible because ActivityPub actually has 2 modes of operation:

This would theoretically not only allow for clients to work with ActivityPub directly, without relying on a Mastodon-specific API (or wrappers like masto or megalodon), but also crucially permits fully static-file fedi servers! Instead of a backend with a full database, a queue with automatic fallover (to push statuses), and probably a bunch more stuff that I forget, an ActivityPub server could just be a collection of a few files:

Unfortunately, none of this works. It's a shame, I really tried - but all ActivityPub implementations cite some privacy concerns when you search their forums for reasons why they don't implement this style as well.

Being able to GET your outbox would also trivially mean that you could move your posts with your account if you need to switch servers, etc. Bummer!

I still think there is value in getting rid of implementation-specific APIs and having a uniform client API as well - maybe I will get around and build a "translator" middleware server at some point.

A different approach

All I wanted to do is for my website to be part of the fediverse - but that doesn't seem to work without having a complex server setup, and I really like just having a bunch of static files on Netlify! But then I discovered that Netlify can trigger a function whenever you push a new version, and that Netlify deployments all have a unique, static link that even works after that new version is deployed. Since I use a static site generator anyways, I deploy a new version every time there is something new in the RSS feed, so...

Here is the new plan:

  1. Write a function that gets triggered after each new deployment
  2. Figure out what the new and previous deploy permalinks are
  3. Load the RSS feed for both of them!
  4. Post new items using Mastodons API for all new items you discover

Preparation

Since we want to talk to the Netlify and Mastodon APIs, we will need access tokens for both of them.

On Netlify, you can create a personal access token here. Just make sure that the token doesn't expire and that you copy the token before leaving the page.

For Mastodon, the process is similar, except that I can't just link the page: You'll want to navigate to Preferences -> Development -> New application. Make sure you enter proper application names and urls, since some clients display those. The redirect URI doesn't matter, since we are not actually doing any OAuth. Regarding permissions, only write:statuses is strictly required, but I suspected I might want write:media in the future as well.

Now that you have both access tokens, you can add them to the environment of your Netlify site. I called them NETLIFY_ACCESS_TOKEN and MASTODON_ACCESS_TOKEN (make sure you get the order right!). I also only have them set in the production context just to make sure it doesn't accidentily post new statuses while I work on a post locally.

Let's write some code

Netlify automatically triggers functions after certain events, if that function just happens to be called the right name - in our case, we need to call the function deploy-succeeded, so with the default configuration, we would need to create a file called netlify/functions/deploy-succeeded.mjs. I will use masto to publish new statuses and htmlparser2 to parse our RSS feed, as it provides a really convenient method for this purpose:

npm i --save masto htmlparser2

If you are wondering: Yes, Netlify provides an API wrapper as well, but that one doesn't have proper types, and also just doesn't work in functions! It has probably something to do with the way esbuild bundles stuff, and why there is a external_node_modules option, but I still couldn't get it to work. Since all we need to do is one simple GET request, I just did it manually.

For the curious: It always complained that @netlify/open-api could not be found, which is imported using createRequire, which probably causes everything to break...

...but other than that that is really it. The actual code is embarrassingly straightforward in its stripped-down version. We literally only need to follow our four steps, so here is the entire thing:

import { parseFeed } from 'htmlparser2'
import { createRestAPIClient } from 'masto'

const RSS_URL = '/rss.xml'
const MASTODON_INSTANCE = 'https://mastodon.social'

export default async function handler(req, context) {
    const NETLIFY_ACCESS_TOKEN = Netlify.env.get('NETLIFY_ACCESS_TOKEN')
    const MASTODON_ACCESS_TOKEN = Netlify.env.get('MASTODON_ACCESS_TOKEN')

    const mastodon = createRestAPIClient({
        url: MASTODON_INSTANCE,
        accessToken: MASTODON_ACCESS_TOKEN
    })

    // step 1: get the deployment permalinks from netlify
    const siteId = context.site.id
    const url = new URL(`https://api.netlify.com/api/v1/sites/${siteId}/deploys`)
    url.searchParams.append('state', 'ready')
    url.searchParams.append('production', 'true')
    url.searchParams.append('per_page', '2')

    const [newDeploy, oldDeploy] = await fetch(url, {
        headers: {
            Authorization: 'Bearer ' + NETLIFY_ACCESS_TOKEN
        }
    }).then(res => res.json())

    // step 2: load the rss file for both of them
    const newFeedUrl = new URL(RSS_URL, newDeploy.links.permalink)
    const oldFeedUrl = new URL(RSS_URL, oldDeploy.links.permalink)

    const [newFeed, oldFeed] = await Promise.all([
        fetch(newFeedUrl).then(res => res.text()).then(parseFeed),
        fetch(oldFeedUrl).then(res => res.text()).then(parseFeed)
    ])

    // step 3: figure out which items are new
    const itemsToPublish = newFeed.items
        .filter(a => oldFeed.items.findIndex(b => a.id === b.id) < 0)
        .sort((a, b) => a.pubDate - b.pubDate)

    // step 4: publish them using mastodons API
    for (const item of itemsToPublish) {
        const status = await mastodon.v1.statuses.create({
            visibility: 'public',
            status: `${item.description || item.title}\n\n${item.link}`,
        })
    }
}

Of course, in a real-world scenario, you'd include some error checks along the way. And you probably don't want these 2 nested loops just to find the right items to publish, especially if your RSS file gets bigger. And maybe you get the idea that you want to abuse the <category> tag in the RSS feed for hashtags in your statuses, but then parseFeed no longer works, so you have to make your own implementation?

You get the idea. If you want to know what I actually do in production, you can always look at the real source instead.