Issue list for mrblog.nl

Because everything is orgmode now, I might as well keep an issue list for the site and publish it. Managing todo items is pretty well covered in org-mode ;-)

Current issues

TODO CSS changes to do   build
  • titles blue
  • tags smaller
  • drop shadow for the images, taking the auto resize of the theme into account
  • make use of the icon font (calendar and home icon)
  • restyle main titles, should be more condensed, see old site
  • mail icon for about is not very fitting
TODO Configure multiple output formats for mrblog.nl blog postings   build

Now all source is valid org-mode we have a plethora of nice options:

  • deliver original source format (org-mode)
  • pdf format (for printing)
  • openoffice format (for additional editing)
  • plain text format (for email)
  • activitystreams for listings (connect to mastodon)

In the jekyll spirit, this should really all be generated statically and published in the same way as org->html For the latter we use org-ruby. Using hugo now

See also: https://github.com/fauno/jekyll-pandoc-multiple-formats

TODO This would be useful to have when writing longer documents   check

https: martignoni/hugo-notice: A Hugo theme component to display nice notices

TODO Do not render /tags or make it the tag cloud   fix

It is the normal list page now

TODO Scrolling on the map component starts zoom when scrolling page   fix

Not sure if I want to change it, but it is annoying.

Remnants from earlier iterations
TODO Create a better way for handling remote images   build

Similar to the gmg plugin, but more generic. Goal is to have a quick way to insert a properly styled image, based on a class or keyword.

TODO Make pages searchable   fix
TODO Pullquote needs to be reimplemented   build

Perhaps use a functionField for this, this allows for the syntax

1$somefunction("String argument")$

in templates. This function is defined in a context like

1postCtx :: Context String
2postCtx = functionField "somefunction" (\args _ -> error $ show args) <> defaultContext

which will just print the arguments while compiling the templates.

An example for pullquote might be:

1$pullquote("right", "Paragraph with {marker that signals begin and} marker that signals end")$
2
3Resulting rendering mockup:
4"Paragraph with marker that signals beging and maker that signals end"
5                                 "...marker that signals begin and..."

In html this needs to happen:

1  <span class="pullquote-right" data-pullquote="...marker that signals begin and...">
2
3    Paragraph with marker that signals begin and marker that signals end
4  </span>

And the CSS can take care of the actual rendering.

TODO Show post titles in prev/next links   build
TODO Gitactivity needs to be reimplemented   build
TODO Have /tag hold a tag cloud   build
TODO Generate multiple versions for each post   build

This is now relatively easy. Minimal set: source+html

  • source
  • html
  • pdf
  • epub

And provide some icon after the title or on the same line as the

TODO Replace feed with my own template   fix

Problem:

  • the updated field is a bit special, but i can copy code from
TODO Implement the projects or subblog functionality   build
TODO Implement automatic linkchecker for mrblog.nl   build

Perhaps a cron job which does this.

  • only report what can be fixed by me;
  • report only if there is something to be fixed;
  • check daily

There is a check command for hakyll which does link checking. The question is if we can react to this while building the site, for example by striking through the text and altering the link to something else.

TODO Implement subsites with subdomains while keeping the main site the same   build

Usecase: cobra.mrblog.nl content is now also on mrblog.nl tagged with 'cobra'

TODO Expose subdomain structure on front page   build
TODO Create an anonymous blog, as some sort of game. "Find out who I am"   build

Needs:

  • reliable TOR network access
  • anonymous email
  • encrypted connections to everywhere
  • damage controle system (assume new identity quickly when exposure signs are there)
  • think about how
  • http://ur1.ca/ecl5w has some good information
  • the idea of findingout who I am is for play, it is not enough that people "know" who I am, but they need a record of how they find out. (I even consider spilling my identity upfront but that would make it much less interesting)
  • researching the HOWTO is already leaking information, so the first and foremost action is to get access to the TOR network completely.

Recently acquired book from Mitnick may help with this.

Thoughts on having a commenting system

The site is using disqus (not anymore) now, which is comfortable, but a proprietary solution.

Requirements
  • based on open source solutions only
  • multiple interfaces (web, mail, xmpp etc.)
  • static rendering of pages must be preserved (TODO: clarify this in relation to on-site rendering with javascript)
  • authentication not needed, anyone may comment –> thus moderation
  • plain text storage (git based would be nice)

Note: staticman.net more or less satisfies all criteria.

Workflow

Someone visits a page, wants to make a comment. There is one thing that user wants to do and that is fill a field with their comments. Extensive login and identification procedures are just getting in the way. What they will be prepared to do is easy identification, so they are credited with the comment or are able to follow the thread of discussion.

Viable options
  • use gnu-social (non-static)
  • use pump.io (non-static)
  • homebrew js only solution
  • staticman.net :: automated solution to post files to git repository, which then can be picked up by the site compiler
Scenario: using pump.io

The first assumption I am going to make is that there is a pump.io note somewhere, i will be using:

http://qua.name/mrb/note/gtOv6E2aSle9P2jMOGlUKA

as the example. The comments to that note are considered comments to the posting on the website. Using the assumption the task can be split into the following sub-parts:

  1. Displaying the comments in the relevant page

(the next section contains api keys, so is encrypted by default, in case this gets published somewhere)

Ad 1. Displaying comments for a specific note

—–BEGIN PGP MESSAGE—–

hQEMA79Ps5L+qnrdAQgAyjV6+jk8SoGTKzf8+SuTUIN9M1Z4lvUWvkyF2SU+W8iR OH62xh0DldMlluvb191fsA8S81Qkrl6FecvSamnJNGOpXL9vsXdkKzXpn109zkoX +IjskILrOzcmt/Vjt6bMrPj3HE3TaGv1FY71sGF/n4XIlxSngbZvx3b0z8RPmGP6 1mC08vGIIzXbrKJGQbKGA+f4vUUAg0GsUy+QZ03N722G9Se+QPhyQ87GIFzuCm78 Jn0r2FxHgqQgaP1Q3P5rWDnPy8oT1cshZccw9NYWYj7TDpXVdmSMPu2VZ46fXY3q ygsIVyHPuL9fJWv2tz8u3fbcDgJZ5nrhJoDGZt7cLdLpAZYAtcELZJAPdC4oXhWR HVWjC/G/+KjdhS4Xv2vzUcTZZwnVSJQmvl2/WJdjyW6Poa5z9onGztMLTORlgKr8 CFQlw1TtDTwBmC8kGoVpGIAyazHlg55/Aj3eNODPStQJBOz7vsOp8fCpGa8TreL1 5KdEUt9DHVL3EenedQXKY1Sp1hAJFH0jg/XqZJbG+MJjz/18yXs6kVAxgXZiJo1Z AVlS6Aa+UcHju3fO9mBHwUFNaQ/oPFQLmrOV2AH5UpBfNu+k3sHyaM/5Ntmo7Sh0 /1nebKsMbJpJsMXZTNAPpIYusqOZJ2e2lgehMoq4po4+2uzRaqwloImmqOurmezt jQccNECGEDwi2Al14w1V3sm6xoa8f1pbkVxW7/GV/ayIBL2IDr6mS2zoEidDyelx GPj66PNw5P9JKF2zHP/FZtWzfyi0bzwOoNb3A4fIEb9EgLTi1eCqePML4TgFwhAn 9rvk3fzE3i5pUkfMj0FOLMDk2igK918SATH6byfgAVGBgNPerLfHCGZotYiULUqD ShZUz/tKa3GVtniluts+xJAObtl7xKjwtiVMbU0N4tfsqOxjgmJYCVk9qoK2a/XU BSpfCVzXkxyH6vJzD3wSXWWX2HEtAGBPGUdapH2Zx6I0Y0up7VEKmmGEn0WJIu8x AoFGEDrD+Ng0r0HK2ZjBcVzAjnysFBT4j9DJKTDxjp5UjKiPfFnmAP8U9XBolYT3 aPrHhSNCCguLDqQ1KrLxx6Lk2WjcommfGOhZ8/it1LzJE9MomNhY8FHn3YZB5cbe zWRYyiDahxseRli4sHaGGTkWX/DWfDV3qI61BT6sKgNJ1Q+838PhZlHIuKqoZeia c5w9EBd3nSdC8K9aF5bE/eQKG0ZK77ThYw2G+rIWAVyRSWwmMR3sUoA+YPl1DPIL vCMzx/FYiti8w1qihrI6xJD360V+TO9A3xIdeEBSiQwhgyaW+Hf8a/GcqftMj8iQ ihtLcIMD8hkPaIJDjDttk4+7DY7O1uCNWWvM5/WdurFOH+iheVeNp4EqqJvic/wU 6Y7sAjhx8IQ8gNBsgiJXhEVUcUxSsZ8cRh4B+6KSmvWWtoX5YFXjOAeqB7opkISI VCBYPrar7YqrDuZnNJ5rUn8= =5atE —–END PGP MESSAGE—–

Scenario: using staticman.net

I've begun implementing staticman.net usage. This is a log of that effort

First step: getting the data

The current commenting system is disquss and they offer an export which dumps all comments into a proprietary xml file.

I converted this xml file to a representation in json with xml2json to get the data in json format. Reason is that the staticman implementation directly produces json too, so I'm trying to mimick that. The reason I chose for staticman.net to produce json over yaml is that I already use json data for the site search engine on the blog.

The first idea was to extract a json file for every comment from the xml file and use that directly on site with a javascript piece to bridge it to rendering.

Extracting the json file per comment is done with jq in a number of distinct steps (basically for loops). While tedious to do, it's scripted and now a reproduceable step.

TODO The messages sometimes hold html, do we want to keep this?   check

It seems to be only <p> tags mostly, so I guess we can clean it a bit.

END

Having separate files for each comment is less suitable for web exposure, because they can only be accessed by their URI. This means that their filenames must be known for them to be rendered by the browser. Gathering these filenames by hakyll or combining them into one 'well-known' file is then the next step.

Choosing them to be gathered into one well-known filename is attractive because it fits nicely with the techniques already in use on my blog. The output of hakyll is a directory per post/page which contains, so far, 1 file per format of the content. (.html and .org so far) With that one file in place, a piece of client side code can then be written to render the page based on that file.

So, first goal is to have in the directory:

blog-slug.html
the html page that is rendered
blog-slug.org
the original content
blog-slug.json
data related to the page (comments, other metadata?)

It's fairly trivial to also include the blog contents into the .json file also. In fact this might be a good starting point.

Multisites

I want to host multiple blogs below mrblog.nl, like:

(me.)mrblog.nl
main blog, having all content
cobra.mrblog.nl
the cobra build
hbx360.mrblog.nl
cazeneuve hbx360 related stuff
sacia.mrblog.nl
sacia shaper related stuff
photo.mrblog.nl
photoblog (sparse subset of media.mrblog.nl)
tools.mrblog.nl
tools in general
media.mrblog.nl
mediastorage for all mrblog.nl sites (mediagoblin)

The hosting of sub-blogs is probably quite easy to do by defining to publish them to subdirectories and make sure they are self-sufficient, meaning they have an index.html file there.

One way to help with this is to use categories which are a bit special:

  • /work/code/_posts/blahblah.md -> post is in cats work and code and if the permalink variable contains :categories the live site will contain folders for the categories.
  • categories variable in front matter: YAML list or space separated string; this just registers the category and makes the variables work.

I like the specification of categories and blogs to be separate really, but file organisation above _posts is unavoidable to get them into categories so might as well use that to my advantage

Another option could be to have a data file below _data which contains information on the sub-blogs.

In URI terms this would be:

http://mrblog.nl
main blog, containing everything
http://mrblog.nl/cobra
everything in http://cobra.mrblog.nl
http://mrblog.nl/hbx360
everything in http://hbx360.mrblog.nl
?

These should all be the same:

  1. http://cobra.mrblog.nl/2013/11/15/the-title-of-the-post.html
  2. http://mrblog.nl/cobra/2013/11/15/the-title-of-the-post.html
  3. http://mrblog.nl/2013/11/15/the-title-of-the-post.html

Item 4.3 is now in place. 4.2 is not really necessary

Items 2. and 3. more or less dicate there should be folders in the sources named cobra and hbx360 which have their own index.html

Using a make/rake file different config files per site in the same directory (basically making multiple jekyll runs) is possible but seems a bit sensitive for conflicts. So, trying to avoid that is worth something.

Some things (like having cobra.mrblog.nl serve up from a cobra (virtual) subdirectory can be arranged from nginx.

Jekyll 2.0 adds a notion of collections which are like pages (not time based content like posts) gathered in a _collection directory, exposed in a /collection url part. They can be made to render (treated as an input format to be transformed) or just plain copied.

[2017-01-14 za] As I'm progressing I'm more and more leaning towards one base domain (https://mrblog.nl in this case) and have features in the blog system to define my tags categories and what have we. This makes everything a lot simpler and probably more flexible.

Requirements
  • each subsite has its own feeds under its own domain
  • links in the feed point to the proper subdomain
  • main site has all content of all subsites, properly tagged and least, preferably categorized too
  • everything can be generated and previewed as if it was one site
  • if resources are duplicated, the process to manage them should be automated.
  • if not a separate theme, show the use that he is on a sub-blog.
  • http://sub.mrblog.nl/YYYY/MM/DD/title.html point to proper contents
  • http://mrblog.nl/sub/YYYY/MM/DD/title.thml point to proper contents
  • http://mrblog.nl/YYY/MM/DD/title.html points to proper contents
  • a simple mechanism is in place for local testing.
Implementation

Taking photo.mrblog.nl as example, the steps to implement it are as follows.

  • The source documents are in folders sites/blog1,

sites/blog2… etc. This makes sure the documents are in categories automatically which we can filter on. Example: ./sites/photo/*.org contains documents for http://photo.mrblog.nl The index.html loops over all content which is in category 'photo'. In addition, cross-blog posting can be done by setting the category in the frontmatter manually.

  • nginx points photo.mrblog.nl to ./photo as root dir

Given the amount of time I already spent on this, perhaps a separate subblog with syndication approach is simpler (it is).

Blog article ideas
TODO Consider a blog series on yubikey applications   write

Series: Authenticating:

  • OTP usage (with and without running own validation server)
  • FIDO U2F (only google uses this?)
  • OATH (use android as example, compare with google auth app)
  • OpenPGP usage (gnupg, ssh, firefox, openkeychain, pass, password store)
  • PIV (no usecase found yet)

Encrypting

  • OpenPGP
  • PIV
TODO the conflict of being open source and having 'next new features'   write
TODO Project: cobra   write
TODO Project: Cazeneuve HBX   write
TODO Project: Sacia Shaper   write
TODO Project: Power9 machine   write
TODO Project: Cars (for each one?)   write

TODO Perhaps put mastodon shortcode into my blog to use   check

https: Mastodon Embed Shortcode for hugo — KevOps — DevOps, Cooking, and everything else I want to write about

TODO consider a 'short note' like publication type   build

This would not be posted in the main line, but appear in a footer, sidebar or something else. It's a microblog like thing.

TODO Look into having an automated list of updates on my blog   check

https: Maintenance | Hugo

After first deployment

TODO add search functionality   build

https://gohugo.io/tools/search/

https://gist.github.com/eddiewebb/735feb48f50f0ddd65ae5606a1cb41ae looks ok-ish

The basic method is the same, generate some data file to query with client side code. Typically a json file. While this is fine for small sites, this doesn't scale very well, the json file gets loaded completely for every search. At some point a smarter solution is needed.

In terms of implementation, typically some template is constructed, which is then generated alongside the pages, uploaded so the client side code can access it.

fuse.js gets the most votes

The theme I am using is planning an implementation for search in their 2.0 version which may be fine to wait for.

TODO integrate existing comments   build

They are not on site right now. But I have json exports from the disqus days.

No need to do this directly, but I don want to save the data.

I like the idea of staticman although i do not want to rely on github for the comments, I think it is possible to self host, but this may be quite a bit of work. Probably postponing.

I also like the idea of having the comments on a social site, and not having them directly on the blog itself, but that has a few nicknacks as well. I haven't seen an implementation yet which I liked.

TODO When viewing a post, show related content in the sidebar   build
TODO look at automatically minifying   check

I think there is a section in the documentation about the hugo pipe which may be related to this? Ideally this should be an option in the staging/production environment to set

TODO After deploy: reorganize content files   build

There are a couple of folders below content which shouldn't really be there. The files folder should be examined and most of the files should probably move to its relevant document / posting where the post is converted to a page bundle. The downside is that the filesystem will then be littered with all these files and it will be hard to optimize for them or move them to a CDN if needed.

TODO Extend the deployment script   write

hugo deploy does only supports some cloud services, so i just need some sort of script to rsync the whole shebang.

Options:

  1. rsync script to sync .site dir This is basically what the old site did, manual action, no frills. Good way to start I guess. I did this now using a local git alias git deploy which runs an rsync script from the root of the repository. I would have liked a post-push-hook, but apparently that is not the proper way to use git according to the bosses.
  2. Use ansible for deployment Another option is to use an ansible script which can do the rsync, but also help configure the webserver and other things that may help manage the deployment. The latter part is unquestioned, but I'd hesitatie to use ansible to publish content, if only for speed reasons.
  3. Post receive hook on server An alternative deploy option could be a post-receive hook on the server side, this means only git and editor is enough (locally) to publish content, instead of hugo/git/rsync. This requires hugo on the server though, so it's not much better in general. The site would then run a script after each push, taking care of updating the site on the right location on the server. An extra pro is that the amount of content that needs to go over the connection is probably a lot less (although rsync should be fairly efficient in just transferring the differences to what has already been published)
TODO Make the gpx shortcode into a hugo module   write

Or even better: use https://github.com/altrdev/hugo-leaflet

This requires fontawesome which is also helpful for some other items such as using a calendar icon and share icons.

TODO Look at https://fed.brid.gy   check

It's a procedure to bring normal websites into the fediverse without having a framework, it's a modular setup which can be implemented in different levels, depending on the need.

Not sure where I would use this though.