Photo for tonight’s HWC Baltimore

Jonathan Prozzi and I have challenged one another to make a post about improving our websites once a week. This one should have gone up last week!

A few weeks ago I posted some thoughts about my IndieWeb setup called "Easier POSSE with Micropub Edits?" in which I wished for a tool that would let me take a given post from my site, syndicate it to silos like Twitter and Facebook (tweaking the content if I want), and updating the post on my site to show the links to those syndicated copies.

Why?

I failed to make at least one important thing clear in my original post – why do I care about syndication links? There are many reasons.

• I have set up a system to notify me when my posts get webmentions from other sites.
• Brid.gy will backfeed comments and reactions from sites like Twitter and Facebook to my site via webmention, but only if brid.gy knows which posts on those silos correspond to a post on my site.
  

If I decide that a post should be syndicated to a silo, it's because I want it to reach the people who follow me there and, if that is true, I also want their interactions to come back to my site. So, in these ways, a post isn't "done" unless it is on my site, with syndicated copies on the silos I care about, and with syndication links for brid.gy to feed the interactions back.

Starting at the End

I decided to start by making my site's Micropub server support Micropub Source Content Queries and Micropub Updates. Any tool that helped automate syndication would need this plumbing to operate.

When implementing a new feature, it always helps to have something to test against. So, I went looking for a Micropub client which supported queries and edits. The test suite for Micropub at micropub.rocks includes a lovely implementation report grid, showing which Micropub clients support what features of the spec.

Of the clients listed, two of them were web-based and Open Source. I had played with and liked Inkstone in the past, but its edit features are currently considered a work-in-progress. So, I tried out Micropublish.net, and it was exactly what I was looking for.

Micropublish has a feature to let you enter a URL for a post on your site to edit. It will use Micropub source content queries to get the source data for that post and let you edit the content and other properties of the post. It can then send a Micropub update to save the updated version of the post back to your site, if your server supports updates. It even has a great feature for developers - a "Preview" button will show you exactly what request will be sent to your server for the update.

Micropublish.net is a great tool for testing out Micropub query and update support, but my Micropub server is bespoke, hastily-written, hand-rolled Python. So, while it was easy enough to add query support, it took me a while to get my code structure cleaned up, write some tests, and actually implement updates.

A New Workflow

I am pleased to say that it works and, with the help of Micropublish.net, I now have a functioning workflow for publishing to my site and syndicating to silos like Twitter and Facebook, even from my phone, without having to open my laptop, edit YAML data, and push git repositories around. It looks like this.

• Make a new post to my site with a micropub client like Quill.
  
• Open the post for editing in micropublish.net (I use Url Forwarder for Android to make this super easy on my phone, a bookmarklet makes it easy on my laptop).
• In a new tab, log in to Twitter and make a similar post, copy the URL to the new tweet into the Syndication field on my post.
  
• Repeat the steps to make posts on Facebook, Mastodon, etc., copying their URLs into the Syndication field.
• Finally, hit "Update" in micropublish.net to update my post with the syndication links.
  

This is still a very manual process, but it now makes it possible to finish a post in a way that I couldn't before. In the spirit of manual until it hurts, I will use this for a while and see what existing pain points remain, and what new ones appear, to help decide what comes next.

Thanks to Barry Frost for micropublish.net and to Tantek for the nudge to write an update!

Audio edition for This Week in the IndieWeb for April 29th - May 5th, 2017.

You can find all of my audio editions here.

You can subscribe with your favorite podcast app on huffduffer.

Music from Aaron Parecki’s 100DaysOfMusic project: Day 48 - Glitch, Day 49 - Floating, Day 9, and Day 11

Thanks to everyone in the IndieWeb chat for their feedback and suggestions. Please drop me a note if there are any changes you’d like to see for this audio edition!

Audio edition for This Week in the IndieWeb for April 15th - 21st, 2017.

You can find all of my audio editions here.

You can subscribe with your favorite podcast app on huffduffer.

Music from Aaron Parecki’s 100DaysOfMusic project: Day 48 - Glitch, Day 49 - Floating, Day 9, and Day 11

Thanks to everyone in the IndieWeb chat for their feedback and suggestions. Please drop me a note if there are any changes you’d like to see for this audio edition!

Baltimore's April 2017 meetup for Homebrew Website Club met at the Digital Harbor Foundation Tech Center on April 19th.

Notes from the "broadcast" portion of the meetup.

brianey.com – Been working on some content based on personal notes, which he takes all the time, about things like stuff he's read, etc. Thought about turning ~5 of his notes at a time into monthly lists, but has been writing a lot. Might be more like each note becomes a paragraph-long post. He also manages a newsletter that usually covers one topic, but might automate these new "listy" posts into a collection for the newsletter.

jonathanprozzi.net – Been working on a longer content post, part two of a series that started in March about creativity and code and his personal learning journey. Wants to keep up with post-a-week challenge. Started capturing ideas because he tends to forget topics if he doesn't get started on them. Doesn't want to have long spans of time between posts, so needs a system to keep track of posting as he gets too busy to take big chunks of time.

martymcgui.re – Been working on cleaning up cruft in his site implementation, not a lot of publicly visible stuff. Also been thinking a lot about things that stop him from posting, like not being able to easily syndicate posts while on mobile, and thinking of plans to make it easier. Tonight worked on the first part of supporting micropub edits for his site by working on source queries.

We also had a good discussion about how folks track their projects, keep ongoing notes, nudge themselves to make progress, and more. We talked about tools like Google Calendar and Tasks, laverna.cc, "GTD", and more that I forgot to write down.

We hope that you'll join us for the next HWC Baltimore on May 10th back here at the Digital Harbor Foundation Tech Center! It's an "Off-Week" for the usual HWC schedule. After that we're back on track with another "On-Week" HWC on May 31st!

Update: thanks to Ryan Barrett for pointing me to his Keep Bridgy Publish dumb post, which explains why Bridgy doesn't include the features mentioned below!

Nerd alert: This post is me geeking out and will involve talk of protocols.

In keeping with the IndieWeb concept of POSSE (Publishing on my Own Site, Syndicating Elsewhere), I try to make social media posts on my own site first and then make similar (not always identical!) posts to my accounts on silos like Twitter and Facebook. I then add links to the posts on my site indicating that you can find the "syndicated copies" of that post on those silos.

My process for doing this is something like:

• Make a new post, likely with a micropub client like Quill.
  
• Log in to Twitter and make a similar post, making note of the URL to the new tweet.
  
• Log in to Facebook and make a similar post, making note of the URL to the new FB post.
• Edit the metadata to my post to indicate the new syndication links.
• Re-publish the post on my site.

Because of the way my site is set up, this manual process requires the use of my laptop, so I can't do it on the go.

When thinking of ways to automate this process, I found myself drawn to another nice IndieWeb tool called Telegraph. Telegraph takes the URL for a post, finds links inside that post, and (if they support webmentions) allows me to notify those sites about that my post links to theirs with a single click.

I like the way Telegraph works for several reasons:

• It's not purpose-built for a single website - it can be used for sending webmentions from any site that publishes their content with the right markup to any site that can receive them.
  
• It puts the final decision to send a mention in my hands - I can choose to send mentions to any particular link mentioned in my post, or not.
• With a bookmarklet, the process of sending webmentions becomes very simple. I visit the page for my post, click my Telegraph bookmark in my browser, and Telegraph shows me the links and send buttons for my post.

There already exists a great tool for copying content from my site to certain silos: brid.gy. While brid.gy's primary use case is to use webmentions to syndicate comments and other activity from silos onto your own site, brid.gy also has a Publish feature which accepts a URL from your site and attempts to create a similar post on the silo of your choice. Brid.gy Publish is a great feature, and I make good use of it. However, there are still a few pain points that I feel when using it:

• While Brid.gy Publish gives a nice preview of what it will do, I can't tweak the content before publishing without editing my own post.
  
• Brid.gy does not, as far as I can tell, support bookmarklet functionality. So, publishing takes multiple steps:
  1. Visit my brid.gy profile page for a particular silo account.
  2. Enter the URL for the post on my site.
  3. Approve the post.
  4. Copy the URL for the new silo post from brid.gy.
     • Note: These steps can be automated via hidden links and webmentions, and it even returns the URL for the new post. However, this comes at the cost of the ability to preview what is to be posted.
• Finally, while Brid.gy lets me know in its UI that my post succeeded, Brid.gy has no mechanism for informing my site that the new syndicated post is available. I still need to enter the syndication URLs into my post manually and re-publish my post.

With that groundwork of existing tools, here is what (I think) my ideal workflow would look like:

1. Make a new post to my site, likely with a micropub client like Quill.
2. While looking at my post, click a bookmarklet that takes me to a syndication tool.
3. The syndication tool shows me previews of what my post would look like on each silo where I'd like to publish. I can tweak the content, if needed.
4. A single "Publish" click for each silo would create the post on that silo, but would also update my website with the new syndication link.

This tool seems non-trivial to implement, but I think there are several building blocks which could be quite useful:

• silo.pub is a micropub-to-multiple-silos service that can simplify the process of getting content into silos. Brid.gy Publish also does much of this.
  
• Micropub supports querying source properties of a given post. (it also supports querying for a list of syndication targets, but this seems primarily intended for setups where the micropub server itself will handle the syndication).
• Micropub supports updates. this could be used to add the syndication targets back to the original post. In fact, this use case is in the spec.

I am not currently aware of anyone who does POSSE with this particular flow. I would be interested to know other folks' thoughts on this! Feel free to let me know with a post on your own site that mentions this one, or hit me up in the #indieweb-dev IRC chat!

Further Reading:

• https://indieweb.org/syndication-brainstorming
• https://indieweb.org/micropub_chaining

Audio edition for This Week in the IndieWeb for April 8th - 14th, 2017.

You can find all of my audio editions here.

You can subscribe with your favorite podcast app on huffduffer.

Music from Aaron Parecki’s 100DaysOfMusic project: Day 48 - Glitch, Day 49 - Floating, Day 9, and Day 11

Thanks to everyone in the IndieWeb chat for their feedback and suggestions. Please drop me a note if there are any changes you’d like to see for this audio edition!

Jonathan Prozzi and I have challenged one another to make a post about improving our websites once a week. I'm late with this one!

Most of the features on my website are experiments in learning new things. Sometimes I learn a better way of doing something that I've already built into the site and it's time to migrate!

Moving Media files from Git LFS to a Media Endpoint

I build my site with Jekyll, and I store my site's configuration and text content via Git. One of the things that most folks avoid with Git is storing text content (which fits into Git's model of efficiently storing differences over time) with large binary files like images, etc. (which Git cannot manage as efficiently).

When I first set up my site, I made use of Git LFS ("Large File Storage") for managing anything that wasn't text. Any images, video, or audio that I added to my site was stored in an _assets/ folder in a way that matched uploaded files to the posts they were a part of. Git LFS would transparently ship those files off to a secondary server rather than include their content in the Git repository itself. I had to go through some hoops to set up my local GitLab server to support Git LFS and to set up Git LFS with the server that handles receiving new posts via Micropub, compiling and deploying the site.

It turns out that there are many reasons that a site would want to handle media files separately from the text content that refers to them. In fact, it is a common enough pattern that the Micropub standard includes a definition for a separate "media endpoint" to handle file uploads. I shared a Micropub media endpoint implementation that I built called Spano a while back, and it has been working well with support from tools like Quill. So the text content of my site is served from https://martymcgui.re/, and my media files from https://media.martymcgui.re/. With a couple of changes in my code and my workflows, this has become the way I handle all media files for my site.

However, I still had a bunch of files in site being handled by Git LFS, and some of my Jekyll code (plugins and templates) for showing embeds expected files to be on the local filesystem. This past week I took some time to write some scripts to find all references to those local files, migrate them to my media server, and update the outgoing links. I also updated my embed handling so it didn't rely on local files. This let me delete a lot of local metadata I was keeping but not using, like all the EXIF tags in uploaded photos. I am now Git LFS free and it feels like one less thing to worry about.

Better Caching for Mentions from Webmention.io

When I finally started displaying webmentions, I had a very simple model for how to cache all the info from webmention.io. Basically: I stored all mentions in a big array and, when my site went to fetch new mentions, it would keep fetching until it saw the "last" mention again. This led to a bit of a bug where someone might send me a mention, update their page, and send the mention again. My site would not be able to recognize the "last" mention, so it would fetch all my mentions again, leading to everything appearing twice.

This past week I rewrote my mention handling to avoid this problem by replacing this array and storing mentions in a hash based on the source and target. The new code also checks to see if the verification date of the mention has changed (giving me a way to detect and notify about changed mentions in the future). I also reorganized my mention cache to include an index by the target URL on my site. This makes it a bit quicker to find mentions for a given page when rendering out the site.

Neither of these changes are really visible to readers of my site, but they have been useful for cleaning things up. The webmention.io handling in particular has brought my plugin a lot closer to being something I could release for other people to use!