Posts Tagged indieweb

Mon Feb 20

Site updates: Tags and Media Fragments

Over the weekend I created an English audio version of the most recent This Week in the IndieWeb newsletter. This led to some great discussion in the #indieweb chat about improvements and next-steps in creating a podcast from audio posts on one's own website. Today I added a couple of features to my site towards that end.

First up, I added support for "tag aggregations" - essentially, pages that list all posts with a certain tag. So, any future editions of this audio newsletter that I post can be tagged with "this-week-indieweb-podcast" and will then show up on the "This Week in the IndieWeb Podcast" page. It should soon be possible to feed that page to a tool like Granary to convert the feed on that page, with its audio entries, into an RSS feed suitable for subscribing with a podcast app.

Next up, I added support for "Media Fragments", a W3C recommendation that allows linking to a specific timestamp to start (and even stop!) playback of video and audio. Aaron Parecki's recently implemented this on his own site and was kind enough to share the implementation! Now, you can create links that jump to a specific time of any audio or video post on my site.

For example, if you want to quickly jump to the part of the This Week in the IndieWeb audio edition that contains info about the next upcoming Homebrew Website Club meetings, it looks like this: https://martymcgui.re/2017/02/18/151503/#t=54

Media fragments could enable some fun things, such as a list of links that index directly to particular sections of a long recording.

Aaron also documented a fun way to use media fragments for attribution of other people's audio or video posts. For example, my audio newsletter made use of several of Aaron Parecki's pieces from his 100DaysOfMusic project. I gave attribution by linking to Aaron's posts from my post, and because Aaron's site supports Webmentions, you can see that my post shows up in the "mentions" list for one of the clips I used. With media fragment support, it should be possible to have the mentions on Aaron's post link directly to the exact portion of my audio post where it appears!

Features like this give me hope that it could be possible to make an IndieWeb podcasting experience that is richer and more interactive than the current directory model.

Sat Feb 18

This Week in the IndieWeb February 10 - 17, 2017

Audio edition for This Week in the IndieWeb for February 10th - 17th, 2017

Thinking about doing this as a regular thing, if I can get the production time down. Feedback welcome!

Music from Aaron Parecki’s 100DaysOfMusic project:

Wed Feb 8

HWC Baltimore 2017-02-08 Wrap-Up

Baltimore's February 2017 meetup for Homebrew Website Club met at the Digital Harbor Foundation Tech Center on Feb 8th.

Notes from the "broadcast" portion of the meetup:

martymcgui.re - reworked his micropub server to support the media endpoint he was working on last time. also added venue pages to his site for event locations. made progress on event-posting micropub client (eventually will be released publicly). also added support for deleted posts with dt-deleted and meta http-equiv status 410 gone.

jonathanprozzi.net - refactoring site templates. spent a lot of time debugging some simple problems, h-cards. wants to figure out a way to track incremental work on website when other work things are crazy so threads don't get lost. wants to keep segmenting his Hugo template while maintaining mf2 stuff. goal is to get to working on appearance of his site theme.

jjuran.org - currently working on a CV on his site. first use of img srcset to feed high-res images to clients that want them. uses his own Perl-based static site generator that is "showing its age". working on his own programming language so he can spend most of his time writing code in a high-level language.

We discussed the challenges of building your own tools for fun vs. starting from the goal of posting more (e.g. via Wordpress).

Left-to-right: martymcgui.re, jonathanprozzi.net, jjuran.org

We hope that you'll join us for the next HWC Baltimore on March 22nd back here at the Digital Harbor Foundation Tech Center.

Photo for HWC Baltimore 2017-02-08!

Thu Jan 26

Spano - a minimum-viable Micropub Media Endpoint

Micropub is an open API standard to create posts on one's own domain using third-party clients  and currently a W3C Candidate Recommendation. One of the (semi-) recent additions is the idea of a Micropub Media Endpoint. The Media Endpoint provides a way for Micropub clients to upload media files to a Micropub service, receiving a URL that is sent along in place of the file contents when the post is published.

Some of the things I like about Micropub media endpoints include:

  • The spec allows the media endpoint to be on a completely separate domain from the "full" micropub endpoint.
  • The spec doesn't specify anything about how the files are stored or their final URLs or filenames.
  • They make it easy to separate the handling of (large) media files from the (presumably much smaller) content and metadata of a post.
  • They enable Micropub clients to upload multiple files without creating multiple posts. This makes it simpler to create posts that contain multiple images, like a gallery.

Personally, I wanted a Micropub media endpoint server with a few extra properties:

  • It should be able to run completely separately from, and therefore work in conjunction with, any other micropub server implementation.
  • It should not store duplicate files. If the same file is uploaded twice, the same URL should be returned both times.
  • It should not allow overwriting files. If two images of the same name are uploaded, both are kept and receive different URLs.

Enter HashFS

My extra features above essentially describe a content-addressable storage storage system. CAS is a way of storing and accessing data based on some property of the actual content, rather than (potentially arbitrary) files and folders.

HashFS is a Python implementation of a content-addressable file management system. You give it files, it will put them in a directory structure based on a cryptographic hash function of the contents of that file. In other words - HashFS can take any file and give back a unique path to that file which will never change (if you later upload a new version of the file, it gets a different path).

To add the the fun of HashFS, there is a Flask extension called Flask-HashFS which makes it easy to expose a HashFS file store on the web via the Python Flask framework.

Introducing Spano

Spano is a Micropub Media Endpoint server written in Python via the Flask framework which combines Flask-HashFS for file storage with Flask-IndieAuth (introduced earlier) to handle authentication and authorization.

Spano is a server-side web app that basically does one thing: it accepts HTTP POST requests with a valid IndieAuth token and a file named "file", stores that file, and returns a URL to that file. The task of serving uploaded files is left to a dedicated web server like nginx or Apache.

Using Spano

Once Spano has been set up and configured for your domain, uploading is a matter of getting a valid IndieAuth token. IndieAuth-enabled Micropub clients will do this automatically. For testing by hand I like to log in to Quill and copy the access token from the Quill settings page. With token in hand, uploads are as easy as:

curl -D - -F "file=@myfile.jpg" \
  -H"Authorization: Bearer xxxx..." \
  https://media.example.com/micropub/

Which should output a response like:

HTTP/1.1 100 Continue

HTTP/1.0 201 CREATED
Content-Type: text/html; charset=utf-8
Content-Length: 108
Location: https://media.example.com/cc/a5/97/7c/2004..2cb.jpg
Server: Werkzeug/0.11.4 Python/2.7.11
Date: Thu, 26 Jan 2017 02:40:05 GMT

File created: https://media.example.com/cc/a5/97/7c/2004..2cb.jpg

Integrating Spano with your Micropub Endpoint

If you want Micropub clients to use Spano as your Media Endpoint, you need to advertise it. This is handled by your "main" Micropub server using discovery. Essentially, a client will make a configuration request to your server like so:

https://example.com/micropub?q=config

And your server's response should be a JSON-formatted object specifying the "media-endpoint". A bare minimum example:

{
  "media-endpoint": "https://media.example.com/micropub/"
}

In addition to advertising the media-endpoint, your Micropub server must be able to handle lists of URLs in places where it would normally expect a file.

For example, when posting a photo from Quill without a media endpoint, your Micropub server will receive a multipart/form-data encoded file named "photo". When posting from Quill with a media endpoint, your Micropub server will instead receive a list of URLs represented as "photo[]=https://media.example.com/cc/...2cb.jpg". Presumably this pattern would hold for other media types such as video and audio, if you are using Micropub clients that support them.

This particular step has been an interesting challenge for my site, which is a static site generated by Jekyll. My previous Micropub file-handling implementation expected all uploaded assets to live on disk next to the post files, and updating my Jekyll theme and plugins to handle the change is a work in progress. I eventually plan to move all my uploads out of the source for my project in favor of storing them with Spano.

Feedback Welcome!

Spano is probably my second public Python project, so I'd love feedback! If you try it out and run into issues, please drop me a line on GitHub. Or you can find me in the #indieweb chat on freenode IRC.

I'd also like to thank Kyle Mahan for his Woodwind Flask server application, which inspired the structure of Spano.

Flask-IndieAuth - A Python Library for Micropub Servers

One of the things I like about the IndieWeb community is that while they are building tools for themselves, they also tend to release useful parts under Free Software licenses. This helps other developers join the community more quickly, but it also tends to help improve the quality and feature sets of these projects as others use and add to the source.

One of my favorite things to come from the IndieWeb folks is the Micropub API standard, which defines some simple protocols for clients to send post data (the kinds of things you'd share on a blog or social media: images, short plain text, long articles, tags, and more) to servers for posting. One upshot is that if your server accepts Micropub, you can use one of many clients to put content on your site. I'm using a dedicated editor from Aaron Parecki's Quill to write this post, but there are lots of alternatives that are aimed at special use cases. For example, Kyle Mahan's Woodwind is an IndieWeb reader app that happens to include functionality for posting replies, favorites, reposts, and even RSVPs directly to my site via Micropub.

Another favorite is the idea of IndieAuth for web sign-in. At a high level, the idea is that you create two-way links between your website and your user profile on some other silo. For example, on your homepage you add a link to your Twitter profile and on your Twitter profile you link back to your homepage. For a client that supports IndieAuth, I can log in using my homepage URL by verifying that I can log in to my Twitter account.

My own personal Micropub implementation is a little pile of spaghetti Python code making use of the Flask framework. I use IndieAuth to handle authentication (i.e. - proving that a post comes from an app that I've logged into) and authorization (i.e. - proving that I gave that app permission to post to my site). As I've started improving my Micropub implementation, I found it useful to extract that portion of my code into a library that can be used with other Flask applications.

Introducing Flask-IndieAuth

Flask-IndieAuth is a Flask extension that adds the ability to require a client to send a valid IndieAuth token when making requests to any route. For example:

from flask_indieauth import requires_indieauth

@app.route('/micropub', methods=['GET','POST'])
@requires_indieauth
def handle_micropub():
    # ... handle the request

The @requires_indieauth decorator runs before the code for the route. It currently looks for an IndieAuth token in one of three places, in order:

  • HTTP Header (e.g. "Authorization: Bearer xxxx...xx")
  • HTTP form data or query string (e.g. "?access_token=xxxx...xx")
  • The body of a JSON-encoded POST (e.g. {"access_token": "xxxx...xx"})

If a token is found, it will be verified against the configured Token Endpoint to confirm that it is a valid token issued for your server's configured homepage with a sufficient scope.

For more information on how to install, configure, and use Flask-IndieAuth, please check out the README on GitHub.

Next Steps

I'll be using this extension to build my Micropub media endpoint (coming up in a future post) and so far it is working just fine. That said, I know there is a lot of room for improvement. Some things on my list:

  • "scope" can have many values, but only "post" is supported for now. It should probably be passed as an argument to @requires_indieauth so different routes can have different requirements.
  • The configured homepage ("ME") is currently expected in the Flask app's config. I'm not sure if that's "standard".
  • "TOKEN_ENDPOINT" is currently expected in the Flask app's configuration, but since it is required to be specified in the HTTP headers for or as a <link> in the content for the homepage, this could be fetched by the server.
  • Error handling isn't great - all failure conditions currently return HTTP 400 (Bad Request) but should probably be diversified a bit.

Feedback Welcome!

This is my first published Flask extension (heck, it's my first public Python package on PyPI), and I'd really appreciate comments, questions, pull requests, etc. Feel free to reach out on GitHub, or you can find me in the #indieweb chat on freenode IRC.

Wed Jan 25

HWC Baltimore 1/25 Wrap-Up

Baltimore's first Homebrew Website Club of 2017 met at the Digital Harbor Foundation Tech Center on 2017-01-25.

Notes from the "broadcast" portion of the meetup:

martymcgui.re - mf2 for deleted posts. Packaged and released his first Python package, a Flask IndieAuth lib for micropub servers, has a proof of concept micropub media server built on Flask-HashFS.

brianey.com - lots of WordPress IndieWeb plugins setup on his site. Has IndieAuth working. Thinks micropub is working. Also integrating these into humor/fake-advice site imnotwrong.com, POSSE to Medium. Discovered that people still use StumbleUpon and got some good traffic from there.

jeancedre.com - wrote down list of things to do to refresh portfolio site. Chose new color scheme and brought it into Sketch to start designing. New website will mean new logo, layout, etc., and eventually business cards based on that. Not sure if wants to use WordPress "because it's overkill". Discussed how he wants to "use" the site, as the author, since WP provides a nice CMS that supports lots of workflows.

jonathanprozzi.net - did lots of refactoring of his Hugo theme into logical partials. Goal is to make a Hugo theme that supports IndieWeb via mf2 out of the box. Has also started making an ongoing list of all the changes he has been making, or wants to make, on his site.

We discussed the renewed IndieWeb interest in automatically archiving our posts and links to the Internet Archive, prompted by the content deletions and Twitter censoring of the new U.S. gov't administration. We also discussed the upcoming CryptoParty Baltimore happening in a little over a week on Feb 4th.

Left-to-right: brianey.com, jonathanprozzi.net, jeancedre.com, martymcgui.re

We hope you'll join us for the next HWC Baltimore meetups: Feb 8th and March 22nd!

Photo for tonight’s Homebrew Website Club Baltimore.

Wed Dec 21
↩ Replied to https://aaronparecki.com/2016/12/21/7/100days
post from
Currently debating between #100DaysOfMusic or #100DaysOfIndieWeb for my #100Days project.

This sounds like a really great challenge either way! I am rooting for #100DaysOfIndieWeb because I learn so much about #IndieWeb from your stuff.

Tue Dec 13

Great post from @aaronpk about his #IndieWeb setup. I should make one of these!

https://aaronparecki.com/2016/12/12/9/my-website

post from A Brief Intro to My Website Architecture
This is a description of all the pieces and tools that I use to post to my website and handle comments and responses. Many of the pieces are open source and/or based on open protocols that you can implement yourself.WritingI write posts using an app called Quill. It's an open …