Marty McGuire

Posts Tagged webmention

2020
Wed Jul 15

What we talk about when we're talking about "Webmentions"

I have a great fondness for IndieWeb building blocks, and Webmention is a wonderful meta-building-block that connects so many individual websites together.

Obligatory "what is Webmention?": it's a specification that describes a way to "tell" a website that some document out on the web links to one of the pages on that site.

Sound simple? It is! Perhaps even suspiciously simple. Webmention enables whole new kinds of interactions between sites (there are some great examples in this A List Apart piece). Unfortunately, almost all of the coordination to support these interactions happens outside of the "Webmention" spec itself.

So when I see blog post titles like these I am not sure exactly what to expect:

There are many more like this! I just grabbed a handful off the last few pages of news.indieweb.org.

These examples and many more are referring to the most talked-about use of Webmention: enabling site-to-site comments and responses, like a souped-up independent alternative to terrible embeds like Disqus.

This may sound like a simple feature! We might expect it to look like this:

  • You see a post on the web that you like. Let's call that "their post".
  • On your own site, you make a post that links to theirs with some comment like "Nice post!". We'll call that "your post".
  • Assuming that you both "have Webmention support", you might check their post a little later and see a nice summary of your post as a comment below their content.

However, for a webmention to "succeed", a lot of coordination needs to happen.

On your side:

  • You publish "your post" which links to "their post". So far, so good. This is the web! You probably publish links on your site all the time.
  • When your post is live, you can try to send a webmention. How do you do that? It depends.

From here, it's pretty much out of your hands. On their side:

  • Their post must advertise the URL of a service that will accept webmentions for that post.
  • Their service must check that your post is a real post on the web, and that it contains a link to their post. Then it ... stores it somewhere? Maybe it goes into a moderation queue?

So then they have the webmention, but to actually display it, their site must:

  • Pull your post out of wherever their webmentions are stored.
  • Somehow understand what your post "is".
  • Render that into their post. If they want to.

When I see folks posting "I added Webmentions to my site" I want to believe that they have some version of all of the bullet points above.

Unfortunately, there are lots of incompletes.

A list, without references, of partial Webmention support I have seen

The Junk Drawer

Folks in this category are collecting webmentions, probably by signing up for a receiving service like webmention.io. Their posts advertise webmention.io as the place to send mentions, webmention.io dutifully checks and stores them, and ... that's it.

This kind of "Webmention support" is often announced alongside a sentence like "Next up I'll figure out how to display them!" For me, this conjures up images of the warehouse at the end of Raiders of the Lost Ark, or a house filled to the ceiling with stacks of moldy newspapers.

There are, I recognize, lots of good reasons not to display webmentions, beyond some of the technical speedbumps and pitfalls I talk about below. For example there are a lot of unanswered questions and not-yet-built tools and services for dealing with moderation and abuse.

"Why didn't my reply show up on your site?"

Static sites are back and I love it. But if there's one thing that static sites do extremely poorly it is responding dynamically to outside events. Some static sites (including my own!) will save webmentions as they come in, but won't display them until the next time a post is added or modified on the site.

"Why does my reply look weird on your site?"

From services to command line tools to fancy build hooks on your fancy hosting service, there are many ways to automate sending webmentions. But unless the receiving end can understand the content of your post, it might not show up as you intended (or at all) on the receiving end. Current best practices would have you add some extra markup to your post so that the receiver can know that your post is a reply, or a like, or an RSVP, or something else. This is a hard coordination problem between your site and theirs! In fact, there is a whole community and standards process for figuring out how to do these things!

Making sure your posts have the "correct" markup to look like you want can be difficult even for developers writing their own HTML. Tools like indiewebify.me, Monocle's preview, and microformats.io can help if you are getting your hands dirty. It's much harder for folks that just redesigned their site with a new WordPress theme.

Bridgy Over Troubled Waters

Bridgy is an absolutely incredible suite of services provided by Ryan, also for free, for the community.

With the power of Bridgy Backfeed you can use Webmention to feed replies, likes, and reposts from your Twitter tweets to their corresponding post on your own site! This works despite the fact that twitter.com does not link to your website because Bridgy generates little "bridge" pages for which to send webmentions. And it's just a little bit of tweaking to have your Webmention display code handle the quirks.

With the power of Bridgy Publish you can use Webmention to automatically copy posts from your website directly to social media silos like Twitter! You do this by hiding a link to Bridgy in your post, which sends a Webmention to Bridgy, and then Bridgy parses your post to understand it and figure out which bits to tweet. And then Bridgy responds with info about your new tweet. And it's just a little bit of tweaking to have your Webmention sender handle those quirks and update your post with that link.

With the power of Bridgy Fed you can use Webmention to automatically copy posts from your website directly into The Fediverse where yadda yadda yadda. And it's just a bit of tweaking to have your Webmention yadda yadda yadda yadda. 😩 I've tired myself out.

These are all fantastic things that are built on top of Webmention but that I often feel are conflated with Webmention.

"Just let JavaScript do it!"

This one is a bit... unfair on my part. In fact, I think this setup is the best you can get for the least effort, and I encourage folks to go for it. It looks like this:

  • Register with webmention.io to receive, verify, and store your webmentions.
  • Put webmention.js in the template for your posts.

I love webmention.io and use it myself. It is an amazing community resource run by Aaron at no charge! Kevin's mention.tech is another great tool, as is VoxPelli's webmention.herokuapp.com. By configuring one of them to accept webmentions on your behalf, you save yourself a lot of self-hosting trouble. Each of these services provides APIs that let you pull out the mentions for pages across your site.

Similarly, webmention.js is a really great tool developed by fluffy (at no charge!) that hides a lot of complexity and forethought about how to display webmentions with a single JavaScript include.

All that said, I have some issues with this particular combo long-term because all the fetching and display of webmentions happens in the browser of the person viewing your post.

If 1,000 people visit your post, that's 1,000 requests to webmention.io, putting load on a service being run by one individual for free.

This setup also means that the webmentions for a post aren't included in the original HTML. So, if your site sends a webmention and wants to check back automatically to see if it's shown up, but their site only displays webmentions via JavaScript, then your site will never see it. Likewise, it becomes much harder to keep track of reply chains, for example.

Why are you being such a downer about this?

Despite, apparently, being a bit salty today, I really do get excited about Webmention, how it's being used in so many ways to connect independent sites, and new ways it can be used in the future.

I'm worried, a bit, that "Webmention" is starting to lose its meaning in conversation. It's starting to feel like a shorthand that hides important details.

Maybe Webmention can be thought of as less of a "building block" and more like a glue. You can do so many things with glue, like combining a bunch of planks into a table, or building a parade float sculpture with papier-mâchÊ, or doctoring the photo in a passport!

You wouldn't call them all "glue".

2018
Sat Apr 14

Site Updates: automatic webmentions, person tags, better reply image handling

I had a bit of time this weekend to tinker with my site.

The biggest change is one that happens totally behind the scenes - I've added a step to the service that compiles and deploys my site to automatically send Webmentions from my posts to posts that I am responding to or mentioning. There are many folks with projects around this, but they tend to be a little more aggressive than I like in finding URLs to send mentions to. For example, they often want to send mentions to my homepage and avatar image for my own posts!

So, I have added a Webmention sender to my site that is triggered by new and updated pages and sends mentions to any URLs that I am specifically responding to or person tagging, along with any URLs mentioned in the actual content of the post.

Along with this, I now support person tags on regular entries like this one. I can add a tag like http://tantek.com to a post and it will tag them with proper markup and, if possible, show the name and photo from their homepage h-card.

Finally, I did a little cleanup in how I display reply contexts for other people's posts that have multiple images in them, like this one from Aaron Parecki.

It's nice to be able to sit down and knock off the little nagging TODOs here on the site. That said, I'm hoping to be able to make some big changes, soon! We'll see what my schedule allows.

2017
Sat Dec 30
↩ Replied to https://strugee.net/blog/2017/12/announcing-lazymention-elegant-outbound-webmention-for-static-sites
post Announcing lazymention: elegant outbound Webmention for static sites
This post also appeared on IndieNews. Last night I hit publish on version 1.0.0 of a new project, lazymention! Whoohoo!

Congrats! Lazymention looks great! I recently fixed up my site to properly send WebSub notifications to begin building exactly something like lazymention, so I look forward to WebSub support!

Fri May 26

Site Updates: Webmention Notifications in Matrix with Hubot

Jonathan Prozzi and I have challenged one another to make a post about improving our websites once a week. This is me getting back on the train!

In a previous site update I wrote about setting up a system to notify me whenever my site received webmentions. Essentially, this meant that I could now get notifications on my phone and desktop whenever somebody interacted with my site, such as: replying to one of my posts on their own site, retweeting or favoriting one of my posts, or even RSVPs to my Facebook events.

One thing I didn't super like about this system is that it used the Pushbullet service which, while great, is not under my control.

I've been running a Matrix chat server at home for a while now. I primarily use it to chat with people in my household in IRC channels. I use a really nice client for Matrix called Riot, which runs in the browser, but is also available on Android and iOS, and is capable of sending notifications about chat events, which I have found really handy.

Recently, I've added a chatbot to my Matrix server named Hubot, thanks to the Hubot-Matrix adapter. Hubot is super neat because it is fairly easy to script up new behaviors, and it has nice built-in support for the web - both for making web requests, but Hubot also runs a server for accepting web requests. Once I realized this, it occurred to me that I could replace my previous notification system that uses Pushbullet with one that goes through Hubot.

First, a note on security. Exposing a chatbot's HTTP listener interface to the great wide internet comes at some risk! I made sure to the following:

  • I run Hubot behind a firewall, so no plain HTTP traffic can come directly across the internet.
  • Using another home server, I set up nginx to act as a secure HTTPS proxy, using a certificate from Let's Encrypt to encrypt all traffic that goes over the internet.
  • I decided that any behaviors I write for Hubot that use the HTTP listener will use some kind of secret token to ensure that the request is valid. I don't want spammers blowing up my chatrooms!

I decided that the bot should:

  • Allow a user to request webmention.io notifications for a given site into any room.
  • Generate and store a "callback secret" to work with webmention.io's Web Hook system and tell the user the URL and callback secret to configure over on the Webmention.io Dashboard.
  • Accept HTTP requests from webmention.io at something like <HUBOT_HOST>/hubot/wmio/notify
  • Verify that the request contains the callback secret
  • Generate a nice text summary of the notification based on its contents
  • Send the notification to the room that the user was in when they made the follow request.

With that in mind, I began learning lots about testing Hubot scripts, refreshing myself on Coffeescript, and so on.

I am now happy to introduce this first (janky) release of my Hubot Script, hubot-webmentionio-notify!

Once installed, you can start a conversation with your hubot and ask it to follow a site:

  you> hubot wmio follow mycoolsite.biz
  
hubot> @you OK! Use this as your Web Hook: <HUBOT_URL>/hubot/wmio/notify And use this as your callback secret: 1a2b3c4d5e6f7890000

The string "mycoolsite.biz" can actually be anything and should be something easy to remember in case you want to unfollow notifications later. Hubot doesn't check incoming mentions against it at the moment.

You can enter the URL and callback secret in the Webmention.io dashboard, and future webmentions will be sent to your Hubot and output into the room of your choice.

Notification example - a user on Twitter mentioned my Twitter handle in a post there.

I don't know how useful hubot-webmentionio-notify will be for other folks at the moment, but I am excited be getting these notifications via services that I control. I look forward to building more fun things with Hubot!

Wed Apr 5

Site updates: showing emoji reactions

Inspired by Eddie Hinkle's recent post about viewing webmentions, I decided to improve the way I display webmentions on my site.

TL;DR, my site now pulls attempts to recognize single-emoji comments and display them as a "Reaction".

Slightly longer version - my site uses webmention.io for handling webmentions, and I use brid.gy to backfeed interactions from Facebook to my own site. The way brid.gy handles Facebook reactions other than the standard "like" is a little quirky - they show up in webmention.io as a "reply" with a single emoji as the "content".

Using the Ruby twemoji library, my site checks the "content" of a reply against the emoji index and, if the content is a single emoji, pulls it out of the usual "reply" display and puts it in a facepile. The emoji itself is shown as an icon in the corner of the little face image.

Example of some ❤️ reactions from Facebook

While I was at it, I cleaned up a lot of my webmention-handling template to make things much clearer. This will make things easier for folks that want to re-use this code when I (eventually) release this as a Jekyll plugin.

Tonight is a Homebrew Website Club night, but Baltimore is not having another official meetup until April 19th. Still, I wanted to get something done to continue my deal with Jonathan to post something IndieWeb related at least once per week.

Wed Mar 29

Site updates: /mentions page and notifications

Jonathan Prozzi and I have challenged one another to make a post about improving our websites once a week. I'm a little late with this one!

I recently added support for displaying mentions, such as likes, reposts, comments, etc. from around the web that refer to the posts on my site. One thing the update didn't do is catch another type of mention, such as when someone mentions me in a tweet (example). These get fed to my website by brid.gy, but weren't displayed anywhere.

So, I created a /mentions page for displaying these mentions. In the future, when a post mentions my homepage, the result will show up on the mentions page.

Screenshot of the Recent Mentions page, including some tweets that mention @schmarty

My mentions still don't yet update in real time - they are compiled into my site whenever I make a new post. That's coming up in the future, but I have taken one more step towards real-time interactions with notifications!

Webmention.io, the service that I use for accepting and storing webmentions, has a WebHook option that can notify your site whenever a new webmention has been received. I wrote up a simple Python service using Flask that will listen for these messages from webmention.io and send them to me via PushBullet, a notification service that I've been using for a while for other projects.

Webmention.io WebHook configuration

Now, I'll see a notification on my phone and laptop when another site sends me a webmention!

Update: Here's an example notification!

Screenshot of a mobile notification from Pushbullet
PushBullet notification that I have received a new mention.
Thu Mar 23
↩ Replied to http://eddiehinkle.com/article/2017/03/receiving-webmentions.html

Congrats and have a Webmention! I love that you have a roadmap.

Something like WebSub (formerly PubSub) would be great for triggering a notification when a new Webmention arrives, without having to poll!

Wed Mar 22

Site updates: Displaying Webmentions!

Webmentions are one of the most interesting and powerful technologies floating around the IndieWeb. At their most basic, they sites on the web to interact by sending a notification when a page on one site links to a page on another. When combined with machine-readable metadata like microformats2, they enable really neat social interactions between sites, feeding back likes, comments, bookmarks, shares, event RSVPs, and plenty more.

Receiving Webmentions

A site doesn't have to do all its own Webmention handling, and there are a few services that will handle them for you. I set up my website with the Webmention.io service back in August 2016 (so long ago!) and it's been accepting mentions from other sites since then. And, while there aren't a lot of websites that send Webmentions natively, there are services like Bridgy which uses Webmentions to backfeed social interactions to my site from sites like Facebook and Twitter. Pretty neat!

Sending Webmentions

When I publish a post with a link to a site that support Webmentions, I still need to actually send that notification. I haven't yet built a tool that does that for my own website, but I have been able to make use of Aaron Parecki's Telegraph, which will take in a link to one of my posts and parse it for outgoing links, find out of the targets of those links support Webmentions, and allow me to send them with the press of a button. It's ridiculously easy to use and has the added benefit of letting me pick-and-choose which links go out as Webmentions.

Displaying Webmentions

Webmention.io has been collecting mentions for my site for something like 6 months, but they don't just magically show up on my site! Webmention.io provides an API for fetching the mention data for individual pages, or all mentions for my domain.

My site is built on Jekyll, a static site generator, and I like that so far it doesn't rely on JavaScript for folks to read it. I didn't want to require JavaScript for displaying mentions, so I needed a way to "bake in" my mentions for each post. I was inspired by Aaron Gustafson's jekyll-webmention_io, but found that I didn't like some of the choices in markup or the way that it stored the mention data, so I went ahead and wrote my own. It's still heavily a work-in-progress, but I do hope to release it for other folks to use once it's more stable.

What works? Let's see!

Here's an example post with some Likes and RSVPs (both "yes"es and "maybe"s):

And an example post with some replies backfed from Facebook:

All of these are being displayed with the data that Webmention.io provides with its API, and there are some types of post that I would like to handle differently such as the ❤️ above (which was a Facebook "heart" reaction), and I'd like to include a JavaScript enhancement that will show any new mentions, so they aren't sitting in "limbo" until I make a new post.

Overall, I'm really excited to finally be showing these on my site! I think Webmention is a pretty critical part of bringing the "social web" into the IndieWeb and back out of the silos. I am grateful to all the folks that have made this possible with their work on standards and tools!