ox-activity-streams

ox-activity-streams is a publish backend for Org-mode, which allows to publish org files as fediverse posts by serving only static files. It assumes that the org files are already published on the Web as html files. ox-activity-streams creates additional JSON-LD files, which can be alternatively served by your server as activity streams objects.

Below, the video shows this article display in Mastodon, when we look for its URL: https://buron.coffee/entity/ox_activity_streams.

I want to acknowledge David Larlet for our discussion about this, which gives me the motivation to continue.

Requirements

First, we have to create a file describing the activitypub actor (or the user in common language) to which all the publication will be attributed. Second, we set up a webfinger route such that the other fediverse applications are able to resolve the actor. Finally, in order to correctly serve activity-streams objects, we can configure a content negotiation which allows to gracefully serve a HTML or JSON file using the same IRI (or URL) following the requested content type.

In the following, we assume that the web site files are stored in a directory accessible at the URL https://buron.coffee/ (in the following the https is required by activitypub).

Activitypub actor

We create a JSON file becasse.jsonac for the actor becasse@buron.coffee: 

{
  "@context": [
    "https://www.w3.org/ns/activitystreams",
    "https://w3id.org/security/v1"
  ],
  "type": "Person",
  "id": "https://buron.coffee/becasse",
  "inbox": "https://buron.coffee/becasse-inbox",
  "outbox": "https://buron.coffee/becasse-outbox",
  "preferredUsername": "becasse",
  "url": "https://buron.coffee",
  "name": "La Bécasse (website)",
  "icon": {
    "type": "Image",
    "mediaType": "image/png",
    "url": "https://buron.coffee/img/logo.png"
  },
  "following": "https://buron.coffee/becasse-following",
  "followers": "https://buron.coffee/becasse-followers",
  "summary": "<p>Welcome to my website</p>",
  "publicKey": {
    "id": "https://buron.coffee/becasse#main-key",
    "owner": "https://buron.coffee/becasse",
    "publicKeyPem": "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAt0YbiewxvvDjoV5yiwya\nBFZmOOBLPZJAkp3ounbvkYlinslpBXCA8qQWER6XrMemaEBZxfXPvuKNCJEnGZZJ\nI17nbcgcMoqfw7l8qJBEx06MSOxKkcdVAcHLkteHMw4K3xw4M2t+PoA3XZt2AWts\nHPBgaosGCRVAR89KUr0jehOBY2KrQWlWIWGnomKsfUNZ3WcJ7NsocWxeycKU/U3+\nH4g6SzjkjhJgHX2D+oO7APiWSp5F107Imo5aV5elfie7wAKsDiQYVv/URGpYQyjl\nkkW/6dd78ZskV4Yof6LS2Ie4v/JI9EVSf4hgiWGrGSUbmHUEij5ZV9Xz2xKJVMXf\nZwIDAQAB\n-----END PUBLIC KEY-----\n"
  },
  "endpoints": {}
}

I will explain why we use the extension .jsonac below. The reason why the actor's IRI (the id value above) does not contain this extension is explained with content negiation, if you choose to skip it you have to add to extension to your links. Up to now, I don't know if the public key is necessary for the rest.

Similarly, we create four files for the inbox, outbox, following, follower, for example named respectively becasse-inbox.jsonac, becasse-outbox.jsonac, becasse-following.jsonac, becasse-follower.jsonac, which contains the same JSON document, which the id's value is adapted: 

{
  "@context": [
    "https://www.w3.org/ns/activitystreams"
  ],
  "type": "OrderedCollection",
  "id": "https://buron.coffee/becasse-outbox",
  "totalItems": 0,
  "orderedItems": []
}

Finally, we have to ensure that the previous files about actor and the file about the publication are served with content type application/activity+json. To implement it using Nginx, I add a mime type to /etc/nginx/mime.types. The following line tells Nginx that the mime type of files with the extension .jsonac is application/activity+json: 

application/activity+json             jsonac;

Webfinger

It also is required that the actor file can be discovered using webfinger. So, we create another JSON file (e.g. becasse-webfinger.json) containing information related to the actor becasse@buron.coffee: 

{
  "subject": "acct:becasse@buron.coffee",
  "aliases": [
    "https://buron.coffee/becasse"
  ],
  "links": [
    {
      "rel": "http://webfinger.net/rel/profile-page",
      "type": "text/html",
      "href": "https://buron.coffee"
    },
    {
      "rel": "self",
      "type": "application/activity+json",
      "href": "https://buron.coffee/becasse"
    }
  ]
}

This file should be accessible with an URL of the form http://buron.coffee/.well-known/webfinger?resource=acct:becasse@buron.coffee. For example with Nginx we can use the following an route of the server for buron.coffee: 

location = /.well-known/webfinger {
    add_header 'Access-Control-Allow-Origin' '*';      
    root /var/www/buron.coffee/notes/;
    try_files /becasse-webfinger.json =404;
}

This instruction can be extended if you use several actors, see: https://willnorris.com/2014/07/webfinger-with-static-files-nginx

Content negotiation

The content negotiation is an optional configuration, which allows to mimic the behaviour of several fediverse application concerning their links: they are both used for serving HTML page and content of type application/activity+json. For example, the link https://mastodon.social/@Gargron corresponds to a HTML page and an actor JSON document, which is accessible using the following command line: 

curl -H "Accept: application/activity+json" https://mastodon.social/@Gargron

So, I aim to have an IRI for each post (or actor) that will be the URL of the HTML file without its extension and requesting this IRI should respond the JSON activity streams object (or JSON actor) if the Accept HTTP header is application/activity+json and otherwise the HTML page.

In Nginx, we first need to define a map from the accept value to the request file's extension (it can be added in /etc/nginx/nginx.conf): 

  ## activitypub extention mapping

 map $http_accept $ac_ext{
     "~application/activity\+json" ".jsonac";
     default ".html";
}

Finally, we use the $ac_ext value in an route of the server for buron.coffee: 

location / {
    root /var/www/buron.coffee/notes/;
    try_files $uri $uri$ac_ext $uri.html $uri/index.html =404;
}

Source code and configuration

You can find below the code of ox-activity-streams.el: 

(require 'ox-html)

(defgroup org-export-activity-streams nil
  "Options specific to activity streams export back-end."
  :tag "Org activity streams"
  :group 'org-export
  :version "24.4"
  :package-version '(Org . "8.0"))

(defcustom org-activity-streams-actor-url "https://mastodon.social/@Mastodon"
  "The URL of the actor."
  :group 'org-export-activity-streams
  :type 'string)

(defcustom org-activity-streams-tag-name "@Mastodon@mastodon.social"
  "The name of the actor."
  :group 'org-export-activity-streams
  :type 'string)

(defcustom org-activity-streams-tag-url "https://mastodon.social/@Mastodon"
  "The url of the actor."
  :group 'org-export-activity-streams
  :type 'string)

(defcustom org-activity-streams-ext "as.json"
  "The extension of the exported file."
  :group 'org-export-activity-streams
  :type 'string)

;;; Define backend



(org-export-define-derived-backend 'activity 'html
  :menu-entry
  '(?a "Export to Activity Streams"
       ((?A "As JSON buffer"
            (lambda (a s v b) (org-activity-streams-export-in-buffer a s v)))
        (?r "As JSON file" (lambda (a s v b) (org-activity-streams-export-to-json a s v)))
        (?o "As JSON file and open"
            (lambda (a s v b)
              (if a (org-activity-streams-export-to-json t s v)
                (org-open-file (org-activity-streams-export-to-json nil s v)))))))
  :options-alist
  '((:activity-streams-tag-name nil nil org-activity-streams-tag-name)
    (:activity-streams-tag-name nil nil org-activity-streams-tag-url)
    (:activity-streams-actor-url nil nil org-activity-streams-actor-url)
    (:activity-streams-extension nil nil org-activity-streams-ext))
  :translate-alist '((comment . (lambda (&rest args) ""))
                     (template . org-activity-streams-template)))

(defun org-activity-streams-template (contents info)
  "Return complete document string after RSS conversion.
CONTENTS is the transcoded contents string.  INFO is a plist used
as a communication channel."
  (let* ((title (org-html-plain-text
                 (org-element-interpret-data (plist-get info :title)) info))
         (hl-pdir (plist-get info :publishing-directory))
         (actor-url (url-encode-url (plist-get info :activity-streams-actor-url)))
         (tag-url (url-encode-url (plist-get info :activity-streams-tag-url)))
         (tag-name (plist-get info :activity-streams-tag-name))
         (hl-home (plist-get info :html-link-home))
         (as-ext (plist-get info :activity-streams-extension))
         (input-file (plist-get info :input-file))
         (publink
          (concat
           (or hl-home hl-pdir)
           (file-name-sans-extension
            (file-relative-name input-file (plist-get info :root-directory)))))
         (pubdate (format-time-string "%Y-%m-%dT%T%z" 
                                      (and input-file (file-attribute-modification-time
                                                       (file-attributes input-file)))))
         ;; in the following, we could introduce a function for computing a description from the html content following this example:
         (description (or (plist-get info :description)
                          (and (string-match "<p>\n*\\(.*?\\)\n*</p>" contents)
                               (replace-regexp-in-string "</?[^>]*>" ""
                                                         (match-string 1 contents))))))


    (s-replace "[null," "["
               (json-encode
                (plist-put
                 (plist-put
                  (plist-put
                   (plist-put
                    (plist-put
                     (plist-put
                      (plist-put
                       (plist-put
                        (plist-put
                         (plist-put
                          (plist-put nil :@context "https://www.w3.org/ns/activitystreams")
                          :type "Article")
                         :id publink)
                        :published pubdate)
                       :url publink)
                      :attributedTo actor-url)
                     :to "https://www.w3.org/ns/activitystreams#Public")
                    :name title)
                   :summary description)
                  :content contents)
                 ;; insert null in the array because json-encode have a bug ? 
                 :tag (list nil
                            (plist-put
                             (plist-put
                              (plist-put nil :type "Mention")
                              :href tag-url)
                             :name tag-name)))))))

;;;###autoload
(defun org-activity-streams-export-in-buffer (&optional async subtreep visible-only)
  "Export current buffer to a json buffer.

If narrowing is active in the current buffer, only export its
narrowed part.

If a region is active, export that region.

A non-nil optional argument ASYNC means the process should happen
asynchronously.  The resulting buffer should be accessible
through the `org-export-stack' interface.

When optional argument SUBTREEP is non-nil, export the sub-tree
at point, extracting information from the headline properties
first.

When optional argument VISIBLE-ONLY is non-nil, don't export
contents of hidden elements.

Export is done in a buffer named \"*Org  Activity Streams Export*\", which will
be displayed when `org-export-show-temporary-export-buffer' is
non-nil."
  (interactive)
  (org-export-to-buffer 'activity "*Org Activity Streams Export*"
    async subtreep visible-only nil nil (lambda () (json-mode))))

;;;###autoload
(defun org-activity-streams-export-to-json (&optional async subtreep visible-only)
  "Export current buffer to a JSON file.

If narrowing is active in the current buffer, only export its
narrowed part.

If a region is active, export that region.

A non-nil optional argument ASYNC means the process should happen
asynchronously.  The resulting file should be accessible through
the `org-export-stack' interface.

When optional argument SUBTREEP is non-nil, export the sub-tree
at point, extracting information from the headline properties
first.

When optional argument VISIBLE-ONLY is non-nil, don't export
contents of hidden elements.

Return output file's name."
  (interactive)
  (let ((outfile (org-export-output-file-name
                  (concat "." org-activity-streams-ext) subtreep)))
    (org-export-to-file 'activity outfile async subtreep visible-only)))

;;;###autoload
(defun org-activity-streams-publish-to-json (plist filename pub-dir)
  "Publish an org file to a JSON file.

FILENAME is the filename of the Org file to be published.  PLIST
is the property list for the given project.  PUB-DIR is the
publishing directory.

Return output file name."
  (let ((bf (get-file-buffer filename)))
    (if bf
        (with-current-buffer bf
          (write-file filename))
      (find-file filename)
      (write-file filename) (kill-buffer)))
  (let ((as-ext (plist-get plist :activity-streams-extension)))
    (org-publish-org-to
     'activity filename (concat "." as-ext) plist pub-dir)))


(provide 'ox-activity)

Here an example of configuration: 

(require 'ox-activity-streams)

(setq org-publish-project-alist
      '(("notes-activity-streams"
         :base-directory "~/documents/notes/"
         :root-directory "~/documents/notes/"
         :html-link-home "https://buron.coffee/"
         :activity-streams-extension "jsonac"
         :activity-streams-tag-name "@LaBecasse@mastodon.social"
         :activity-streams-tag-url "https://mastodon.social/users/LaBecasse"
         :activity-streams-actor-url "https://buron.coffee/becasse"
         :publishing-directory "/var/www/buron.coffee/book/"
         :publishing-function org-activity-streams-publish-to-json)))

ox-activity-streams is based on the HTML publish backend of org-mode, so it supports some of the HTML properties. Its specific properties are:

  • activity-streams-extension the extension used for the outputted files,
  • activity-streams-tag-name and activity-streams-tag-url specifies the name and the URL of the actor to add in copy when some reply to the publication,

  • activity-streams-actor-url is the URL of the actor as configured above.

    The result for the current page have the following form and is accessible at https://buron.coffee/entity/ox_activity_streams.jsonac: 

    {
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Article",
  "id": "https://buron.coffee/entity/ox_activity_streams",
  "published": "2021-08-25T00:13:39+0100",
  "url": "https://buron.coffee/entity/ox_activity_streams",
  "attributedTo": "https://buron.coffee/becasse",
  "to": "https://www.w3.org/ns/activitystreams#Public",
  "name": "ox-activity-streams",
  "content": "--- published HTML ---",
  "tag": [
    {
      "type": "Mention",
      "href": "https://mastodon.social/users/LaBecasse",
      "name": "@LaBecasse@mastodon.social"
    }
  ]
}

    As you can see it defines an "Article", which is not the object type used by default by Mastodon being "Note". For this reason, only the title and the URL will be visible from Mastodon, the other fediverse applications may display it differently. An additional "summary" properties could have defined to summarize the content of the Article, for example by exporting in plain text the first line of the org file, but I don't know how to do this.

Extensions

Fediverse forms from the Web page

Here, I propose to add a form to the exported HTML pages, in order that people can interact (reply, boost, favorite) with the post from the fediverse. You can test it by using the button "answer from fediverse" at the end of this page. The needed HTML code is: 

<p>Interact from the fediverse with your username:</p>
<form onsubmit="event.preventDefault();subcribeComponent(event)">
  <input name="text" type="email" placeholder="user@instance">
  <input value="answer from fediverse" type="submit">
</form>

It requires the following JS function to get the link of interaction from the username using webfinger: 

function subcribeComponent(e) {
  // this is my way to IRI from page content, you have to adapt it to your case
  const uri = document.querySelector('.h-entry .u-url').href;

  const address = e.target.querySelector('input[name=text]').value;
  const [ username, hostname ] = address.split('@')

  const protocol = window.location.protocol

  // Should not have CORS error because https://tools.ietf.org/html/rfc7033#section-5
  fetch(`${protocol}//${hostname}/.well-known/webfinger?resource=acct:${username}@${hostname}`)
    .then(response => response.json())
    .then(data => {
      if (!data || Array.isArray(data.links) === false) {
        throw new Error('Not links in webfinger response')
      }

      const link = data.links.find(link => {
        return link && typeof link.template === 'string' && link.rel === 'http://ostatus.org/schema/1.0/subscribe'
      })

      if (link !== null && link.template.includes('{uri}')) {
        return link.template.replace('{uri}', encodeURIComponent(uri))
      }

      throw new Error('No subscribe template in webfinger response')
    })
    .then(window.open)
    .catch(err => {
      alert('Cannot fetch information of this remote account:\n ' +err)
    })
}

Storing the received activities

We can also use Nginx to store the activities received on actor inbox: 

location = /becasse-inbox {
   access_log /var/log/nginx/buron.coffee/becasse-activities.log postdata;
   echo_read_request_body;

   root /var/www/buron.coffee/notes/;
   try_files /becasse-inbox.jsonac =404;
}

Related project

Bopwiki seems to be a promising alternative to org-roam and ox-activity-streams as a simpler way to create knowledge base following the slip box principle with links and backlinks. I discussed with its creator about the differences in this thread.

This post accepts webmentions. Do you have the URL to your post?

Otherwise, send your comment on my service.

Or interact from the fediverse with your username:

fediverse logo Share on the Fediverse