In Working with Jekyll and Org-Mode a simple solution was given to use org-mode format files as posts.
That solution had quite a few limitations:
- the files still needed the ‘
---’ yaml header to be present on the first lines, making it invalid org-mode documents - only posts were supported, not documents, collections and pages
- the
#+key: valuesyntax of org-mode was not used for settings, or at least only partially - working with liquid tags was cumbersome.
What follows are some notes on how I solved the issues.
Getting rid of the ‘---’ requirement
Jekyll uses the ‘---’ header to determine which files need to be processed. If such a header is not present, the file is either copied verbatim or ignored, depending on the settings. Having this header in an org-mode file makes it somewhat invalid. Not a bit deal, but luckily it’s rather easy to get rid of.
The function in standard Jekyll that determines this is has_yaml_header so by extending that function we can make sure org-mode files are treated to be processed like there was a header.
module Utils
def has_yaml_header?(file)
!!((File.open(file, 'rb') { |f| f.read(2) } =~ /^#\+/) or
(File.open(file, 'rb') { |f| f.read(5) } =~ /\A---\r?\n/))
end
endThe check is just for the file to start with #+ which is enough for me, for now.
Dealing with the yaml header settings
Getting rid of the ‘---’–marker is one thing, but between those markers are settings which are relevant for the document. At least title, tags and layout are usually present in the header. With the yaml-header gone we need a way to register those variables in some org-mode syntax.
The standard org-mode key/value pairs as mentioned above are suitable for that. The method is implemented statically in the OrgConverter class:
def self.process_options(content, data)
org_text = Orgmode::Parser.new(content, {markup_file: "html.tags.yml" })
org_text.in_buffer_settings.each_pair do |key, value|
# We need true/false as booleans, not string.
if key.downcase == 'published' #(any others?)
value = value.to_b
end
data[key.downcase] = value
end
data
endThe implementation is not very elegant, but it registers all org-mode buffer settings as values. An exception must be coded for values which must be boolean. (I use only the published property, but there may be more)
Handling liquid tags
Here’s where things get a bit hairy. Formally, when writing content which requires a liquid construct, it should be enclosed in the proper org-mode block delimiters, something like:
#+BEGIN_HTML
{%raw%}{% liquid construct here %}{%endraw%}
#+END_HTML
such that the whole document is a normal org-mode document. As my current documents do not have the ‘BEGIN_HTML/END_HTML’ delimiters this would be annoying to change (again). So, the first thing I tried is to implement it to avoid all that changing again. That worked, a fully working implementation is at commit cc2081
However, there are significant issues with this:
- documents are still odd org-mode documents; no real improvement from the originals (just a bit less non-org-mode constructs)
- build performance of jekyll is unacceptable in the implementation (roughly 3 times slower!!)
- to solve the problems plugins need to be adapted, so it’ll quickly become a nightmare.
- the code already is more complex than it needs to be (include tags, quote replacement)
After realizing this, I started over and just coded the convert method as:
def convert(content)
org_text = Orgmode::Parser.new(content, {markup_file: "html.tags.yml" })
org_text.in_buffer_settings.delete("TITLE") # We already captured the title on processing the options
org_text.to_html
endGiven I made the change to all my orgmode files as mentioned above, which is not that much work with emacs’ dired-do-query-replace-regexp command, the conversion is complete. The only thing to make sure is that for each type of source file (post, page, document) the header options are processed with the process_options function above.
You can view the end result in commit 40dad2
Many people in the Emacs community use orgmode. Quite a few of them use the org-babel system to write and maintain their emacs configuration, and so am I. I find the biggest advantage to be able to document my thought process as well as the configuration details. Emacs configuration can get pretty big and maintaining it becomes a lot easier for me if I can read back what I was thinking 6 months ago.
The system borrows from literate programming concepts where you write code and documentation in one document and let tools tangle the code and documentation into separate documents.
{%pullquote right %} The default way to publish an Emacs configuration with that system would be to let org-mode export the configuration document to html and publish that somewhere. What I wanted to do was to use the in-place org-mode converter I am using with Jekyll. In short, {"the ideal would be that jekyll fetches the authoritative version of my emacs configuration in org-mode syntax"} and treats that as /normal/ content and publish it. {% endpullquote %}I started off by defining a collection, a new feature of jekyll, which gave me a chance to use it. The collection is defined in the _config.yml file:
# Emacs collection contains emacs 'generated' documents
collections:
emacs:
# Render them, i.e. convert them to html.
output: trueThis will render source files in the _emacs folder as if they were pages. The url generated for them will be prefixed by /emacs/.
In this folder I placed a file config.org with the following contents
---
layout: orgmode
---
[... text left out ..]
{% raw %}
{% include extern/mrb.org %}
{% endraw %}That includes the file <root>/_includes/extern/mrb.org from the jekyll installation and renders its content. The only correction I had to make was to insert a raw...endraw block around some org-mode settings.
You can view the result of the rendering in /emacs/config.html
Most of my notetaking and task management is in org-mode so it makes sense to use that as the basic format of my postings too. This is usually done by publishing a project from org-mode into the location where jekyll keeps its files and let jekyll convert that into something publishable. I’m using a slightly different setup which is more effective for blog posting.
Publishing with org-mode turned out harder than I thought. The publish process is pretty demanding in org-mode and you only end up with raw documents that still need to be processed by jekyll. It occurred to me that http://github.com has the ability to render org-mode documents directly and perhaps that library could be used to turn it into a jekyll plugin so I could use the org-mode format directly.
{% pullquote left %} Turns out there was a recent commit which mentioned an org-mode converter plugin; long story short: installed it and never looked back. {" Having jekyll convert org-mode files directly saves the whole publishing configuration step in Emacs "} which would otherwise be needed from within org-mode. {% endpullquote %}The converter is simple; it uses an org-ruby call to convert the org-file to html and that’s it really:
module Jekyll
# Convert org-mode files.
require 'org-ruby'
class OrgConverter < Converter
safe true
def setup
# No-op
end
def matches(ext)
ext =~ /org$/i
end
def output_ext(ext)
".html"
end
def convert(content)
setup
Orgmode::Parser.new(content).to_html
end
end
endI can now just write org-mode files with a frontmatter and they’ll end up automatically as blog postings. As yaml frontmatter needs to come first in the file to make jekyll happy this can’t be hidden in an org-mode construct like a comment block or something else that org-mode itself ignores. This makes it harder to use the blog postings for anything else than jekyll because the frontmatter will get in the way; exporting the file to PDF for example. There is obviously room for improvement, but this simple plugin directly gives a workable system.
To have a reference document for writing I created a test org-mode file, with the rendered result here. This file helps to check what org-mode constructs render into something useful and verifying visual layout of them. Not everything worked as I had hoped, but given the amount of complexity that got eliminated I’m quite happy with it.
Issues that I found in the rendering:
- the headers start at level 1 which is probably 1 or 2 levels too high for my purpose; I haven’t found a way to correct this yet. I probably should file a feature request for this;
- footnotes do not work, which I would use to keep links nicely at the bottom of an article.
- some rendering is ugly (blockquotes for example), but that’s probably not a direct consequence of the org-mode converter
- there are only a couple of org-mode environments supported;
- the use of liquid tags that jekyll uses is somewhat cumbersome.
I was pleasantly surprised by the code highlighting though, which worked out of the box for me.
The next step is finding or making some helper functions in emacs lisp to support working with drafts and publishing.
Many people got attracted to bitcoin the last couple of months. This explosion of interest probably caused the monetary value of bitcoins to show the same shape as the google trend-line chart pictured below.
In this article I will ignore the exponential increase in monetary value since its inception and focus on some intrinsic properties of bitcoin and why those are valuable as such.
The sudden interest in bitcoin is not coincidental in my opinion. The bitcoin ecosystem tries to solve a number of problems with the current systems of trade, currencies and the economical structures in general we have in place for them, like our national banks, the currency and stock exchanges and the IMF. The last couple of years we have been confronted, sometimes in very painful ways, with the shortcomings of those systems and, not in the last place, the shortcomings of the people involved; all of us.
The financial crisis is obviously a complex issue and no single cause can be determined. Whatever these causes are however, they have provided a trigger for many to look for alternatives to create wealth or, more important, create ways to hang on to wealth, especially if there is not a lot of it to hang on to.
For the rest of this article I’m going to assume you know how bitcoins work. You will not have to be an expert in economics, cryptography or computer programming, but a basic understanding of the workings of bitcoin will be necessary. The websites http://bitcoin.org en http://weusecoins.com are good starting points. Some of the points below won’t make sense if you do not have at least a rudimentary understanding of how the system operates.
The trust issue
Most people will worry (a lot) if they have money issues. This alone dictates that you absolutely want to trust all parties involved with handling your money.
These parties include the people you receive money from, like your employer, the local banks you trust to keep your money safe for you, the government which monitors the usage of it, but also the ‘coins’ that represent the money. All of these need your continued trust, and that is a lot to ask. Especially if things do go wrong at times.
Employers need your trust that they possess enough of the ‘stuff’ to be able pay you for work, local banks need your trust so you let them keep your savings, governments and central banks need your trust that they won’t screw up (by printing too much money for example) and the coins also need your trust that they are the real deal. All of the examples in some way betrayed our trust in the past. That has an effect on people. It may not be visible for a while, but broken trust sticks.
Bitcoin tries to address some of the trust issues people have; I think it was one of the main motivations for creating a system like bitcoin. It does this, amongst other things, by shifting a few of the trust items from people/institutions to verifiable technology.
For example, instead of trusting a bank to verify transactions to be valid, because they are the only party who can oversee all transactions, trust is placed in hashing techniques to demonstrate that, for example, double spending is very, very unlikely. These techniques are easier verified and proven to be right than the bank which is now responsible for it, if only because we don’t have access to these verifications. The major goal of both verifications is to prevent the same coin to be spent multiple times. (fraud)
Another trust shift is the ‘keep-save’ mechanism. If you keep your savings on your savings-account at your bank, the combination of the banks trustworthiness and, should that fail, the (limited) guarantee the government gives on your savings makes that you can feel comfortable on parking your money there. With bitcoin, your trust will be in cryptographic tools and the network so you keep all your savings in a computer file. The mechanism you could use to keep it safe is to encrypt that file and spread it all over the network to many places to minimise the chances of losing all copies of it. There is no-one to trust but yourself, but there’s also no-one which can protect you in case you screw up yourself.
Distance is not important, value is
Another property of the bitcoin system, not unique to it but especially well implemented I think, is the way it makes the distance to receivers irrelevant and allows value to be put to use effectively. I’ll give an example below.
Say I want to transfer 2 euros to someone which is in a country far away from mine. The amount of time and money it takes to get this modest amount into the hands of that someone distant is ridiculous in the current financial system. My bank does provide a service but it will cost me at least 10 euros, double that amount if I want to get it done ‘fast’. Fast, in this case meaning within 24 hours! For larger sums, the cost may be acceptable, but for small amounts both time and cost are ridiculous.
There are many services which try to solve at least part of the problem outlined above. Services like paypal with on-line accounts to make things go faster, or proxy companies which gather up all the small amounts and transfer to the real supplier when things have piled up. Up until bitcoin I did not encounter a service which chose the simplest concept for this problem: “Set up a secure, verifiable, immediate non-refundable transaction between the involved parties.”
I do not believe the technology to do this has not been available to banks and/or credit card companies, so that can’t be the reason they have not implemented a cheaper and more efficient system. It’s not very hard to imagine what their reason is though. Distance used to be a major hurdle, it is not anymore.
The key differences bitcoin provides here are:
- the receiver and sender communicate directly, trust is a lot easier to maintain if there are less parties involved. “No middle man needed, nor wanted”
- the ‘act’ of payment is almost immediate, the receiver can check almost immediately that a transaction has been made. (Verification for validity by the network can take a while though) In relation to the 24 hours described in the first paragraph this can certainly be considered very fast, near real-time
- a transaction fee is optional. If you specify one, you make it more attractive for others in the network to check your transaction and have a go at collecting that fee. If swift transaction handling is not important, but transferring, say 0.05 euro, to a certain person is important, bitcoin is about the only way I know to do that effectively.
Remember, the amount of 0.05 euro may not be much to you and me, but there are parts in the world where it can buy you a meal or a bottle of water. The fact alone that bitcoin makes these kinds of transactions possible is enough reason to give it more than a casual look.
Bitcoin increases the value of my € 0.05 by allowing effective use.
No unreasonable control
It’s probably true that bitcoin, or systems like it, scares financial companies and governments and therefore will have a rough time ahead. This scare is in part caused by a fear of decreasing control over the system compared to the classic system. Almost all economic commentators or government representatives will argue that ‘some form of control’ is needed to correct and stabilise the system. I’m not very convinced of that being effective or wanted anymore.
Recently, the unreasonable control over money flow in the wikileaks dry-out attempt confirmed this once again for me. It doesn’t really matter if companies like mastercard and paypal decide not to handle transactions for wikileaks themselves or that they have been put under pressure to do so. The fact that it happens shows they have control over where I spent my money. I don’t want that. Bitcoin offers a system where this type of control is impossible by means of the system itself; personal threats will be effective I’m afraid with any system.
Next to the self-control over spending purposes, anonymity is also important for some people. The example often used, mostly in critical pieces on bitcoin, are criminals. Bitcoin makes it possible, when used in certain ways, to bring money from A to B without exposing identities to each-other and to third parties. This is obviously attractive for criminals, including people who want to evade taxes. This is a valid concern and should be addressed properly, but I don’t think it has anything to do with bitcoin as such. With regard to this aspect, bitcoin has no other properties than cash, it’s just more effective and easier to use than exchanging bits of paper money. The real use-case here is the non-criminal people who want to perform semi-anonymous transactions for valid reasons.
So, what’s the verdict?
Bitcoin is a good idea, generally speaking. From a technological viewpoint it’s excellent. It’s trivial that libertarians and anarchists will be attracted by bitcoins, we don’t need to argue the case for them. The challenge is to present the extra-, not the replacement-, values of bitcoin for all the other people out there. I have touched on three of the most important ones to me. There are more properties which make it very attractive as an alternate choice for exchanging value.
Many ‘digital cash’ systems have been presented before bitcoin, but for all of them I could point out critical weaknesses within a very short time. For many of them this was not even a technical weakness, but an organisational (like a paranoid initiator, looking for patent protection) or an economical issue (creating a metal backed currency in the hands of a private company). With bitcoin there are certainly weaknesses in the system, but I have not been able to find a critical one upfront.
The goal of bitcoin is not necessarily to take over existing currencies or existing financial systems, although I would love to see that play out. I would like it to augment the current systems with new ways to trade, more effective ways to put wealth to use, more transparent ways to work together. It needs to put banks and governments on the edge of their seats and keep them a lot more aware of their obligation to reasonably deal with their control over over money.
Having a transparent, technologically sound system for exchanging value is in the interest of many. I’m sure bitcoin has many things that can be improved. Its its complexity of use and the rather clumsy exposure of meaningless addresses come to mind, but the foundation is solid and the issues I found are by no means critical or unsolvable. The fact that bitcoin, the program, is open source does help to understand and validate the system and thus gain my trust. This contrasts on many levels with the services offered to me through financial companies.
When was the last time you validated your bank’s software?