I happen to have a repository of xaraya going back to 2002 in git, so I thought I’d give it a shot.

…more time passes than originally planned…

Result

8 years of commit history in 10 minutes of video. Produced with gource from a git repository of Xaraya. The video traces the 2.x main branch back to its origin, repository wise. The start is the import of the CVS postnuke repository into Bitkeeper. Since then xaraya has switched to monotone.

Gource settings used:

gource --stop-position 1.0 \
       --camera-mode overview \
       --bloom-intensity 0.18 \
       --user-image-dir .git/avatar \
       --highlight-all-users \
       --output-framerate 60 \
       -s 0.5 \
       -720x576 \
       -a 0.3 \
       --hide filenames \
       --user-scale 1.4 \
       --date-format %Y-%m-%d \
       --disable-progress \
       --output-ppm-stream - \
| ffmpeg -y -b 3000K -r 60 -f image2pipe -vcodec ppm -i - -vcodec libx264 -vpre default gource.mp4

This produced a video of 11 min. 35 seconds. To bring it back to 10 minutes the framerate was increased until the total time fell just below 10 minutes.

I specifically wanted the whole history to be in one video of 10 minutes (the Youtube maximum) which means compromising a bit on the quality. If there are things I can do within these assumptions to improve the video, I’d like to hear them.

I don’t think I used MT for a long time. At the time I was maintaining/rewriting all the xmlrpc/blogging modules for Xaraya and it made sense to use my own blog as a testenvironment for it. This weekend I installed WordPress on our servers. Both the “normal” version and the WordPress-µ (Multi-User) were installed. (I had already started with the 2.7 beta, but along the way I sort of realised that the multi user variety is a better choice for me).

So, all new software again for the measly 10 posts I do a year.

The plan is to have mrblog.nl be a ‘top-level’kinda thing, perhaps multi-domain to allow quick creation of blogs and use subdomains for specific purposes (like the cobra blog and I want to experiment with a photo-blog at some point too).

We’ll see.

The monotone setup we use allows us to test all this without disturbing main line development. Organizing these bigger changes into scenario’s allows us to isolate these changes and see where they lead to. The decision to use or to throw away can thus be postponed to a time when we’re ready to do so.

Since replacing the middleware is potentially a very big step, I noticed we (the Xaraya group) have some trouble making a decision on what to do with this. Therefore, the creole scenario has been merged into this site to show that a real life site (albeit a small one) can be migrated to creole without any effect. (Note: I’ve been running creole on other installs for a while now, so I’ve known this for a while)

Verdict: one change was necessary to wrap the ‘GetRowAssoc’method present in adodb but not in creole. Everything else worked directly after the merge, so my preliminary conclusion if the decision hangs on ‘what impact does this have?’is that the impact is very minimal. On top of that I think that the issues we might run into are not bad things as such, but chances to improve Xaraya.

Creating content with Blocklayout in Xaraya is flexible and every once in a while i want to take it for a spin. In the past i’ve played with creating a XUL theme where Xaraya served it directly as the XUL mimetype, this time i’ve been playing with creating a calendar feed of any Xaraya content which has some sort of a date attached to it.

Assuming it’s common knowledge that if you want to run both php versions on the same server you’ll have to run two different instances of your webserver, usually on two different ports. In our case, when you access a site you get apache 1.3 with php4 by default. If you access the same site at port 81 (like site.com:81) you get apache 2 with php5.

When you want to make sure that a particular site runs php5 even if accessed through the default port (80) you can include the following in your htaccess file:

# Force this site to use port 81 on which php5 is served
RewriteCond %{SERVER_PORT} !81
RewriteRule ^.* http://marcel.hsdev.com:81%{REQUEST_URI} [L,R=301]

The first line tests whether the port accessed is NOT port 81 and, if the case, redirects the browser to the same request uri, using the 301 status code (“moved permanently”) and makes sure it is the last rewrite rule

This setup allows each individual site to decide whether to run on php5, optionally using bleeding edge Xaraya features like the creole scenario (which requires php5 ) or to server on both php4 and php5. Likewise if you want to prevent using php5 it’s easy to see how the htaccess could be adapted. This situation is however far less common.

I tend to view at an application development process like this:

1. make it work
2. make it right
3. make it nice
4. make if fast

For Xaraya, the 1.0 release signals to me we’ve been able to make it work. Now, let’s try to make it right. Some ideas on how follow.

Worldkit uses an RSS feed with location information to construct an image where the data points are shown on a map. In doing so, of course the RSS feed can be subscribed to in the ‘normal’way too, giving you an addressbook directly in your aggregator of all people who have provided their longitude and latitude information on the site.

Chris compares the two different mechanisms used for a concept introduced in Team Foundation (TFS) known as Shelving. It is a concept where you put a certain state of your development environment ( workspace, repository, whatever the local term is you are used to) aside (on a shelf), work on some other stuff and take the shelved state back into your workspace (UnShelve) when ready.

Let me add a few notes on how we do this in Xaraya with BitKeeper.

If you view this article by itself (what we call the display view in Xaraya), you’ll notice an extra part on the right side “Related entries” which lists other entries which have the same keywords attached to them. This allows a reader to quickly read all entries which have the same or similar taxonomy. The block isnt fully functional yet, because it reacts to some keywords while it doesn’t for others, so there’s still some digging to do there. (FIXED)

Outstanding task list:

• making Xaraya react on the mt_tb_ping_urls member
• deal with the allow/disallow flags for trackback and comments
• find a home for mt_excerpt
• implement the mt.supportedTextFilters method to allow for markdown or textile input

This morning, I fired up Ecto, and the update which contained it was already there! Man, this guy is quick.

Awesome!

The more advanced features of the MT API I havent implemented (yet), but one thing is supported from the beginning, which is the Extended Entry.

This allows (from capable clients) to upload files to Xaraya directly from the blogging client program.

Like this:

The metaweblog api in Xaraya is incomplete, but the first result is there: multiple category support, directly from the blog client. (This post is in two for example)

I tried to mimick the old look as much as possible. While Xaraya is certainly more flexible and some things were much easier to do than in MT, I also noticed that the finesses of MT will take a while to move over.
Some things which will take some time to come back the full MT way:

• Trackback functionaliy; while Xaraya supports this, it will take some manual effort to get it to be a friendly player;
• Inline RDF Moveable type supports by default for autodiscovery;
• Some templating work is still left (like directly showing the older entries).
• The support for uploading images directly with blog postings
• The extended API support

As you can see in the comments, it’s not a good one, basically cleaning up some mess i made earlier that day.

Nevertheless, it’s there

When it finds those fragments, the following happens:

• the change committed is attached as a patch to the bug; (this patch can in most cases be applied to source trees out there, while maintaining upgradeability)
• a comment is entered for the bug, summarizing what happened;
• links are created in the comment to view the path in the bitkeeper web interface;
• bugzilla mails the relevant parties as usual.

This script is smart enough, not to enter the same information on the same bug and changeset twice which, if you know how bitkeeper works, is not uncommon if you have multiple repositories. As always, on deploying something new, you always start to think immediately about features which you could add (example: scanning for fixed  or fix  and setting the state of the bug accordingly).

During the period of  little over a year, we’ve built ourselves an infrastructure which is still growing and is literally improved upon everyday. Some elements of it:

• automatic updates of mirror repositories;
• fully integrated patch system;
• custom command system, defining new commands in the repository which propagate to all developer repositories;
• funny stats output;
• getting information on RFC status;
• integrated build system based on phing;
• integrated source documentation generator based on phpdoc;
• repository web interface as module of xaraya;
• adding comments to files which propagate across repositories;
• integrated unit test system;
• semi-automatic archiving of old files in the repositories;
• possibility to test for compliance with coding standards on every commit automatically;

With hindsight, a rather long list of tools, mainly created for fun. Some are used extensively, others are only used by myself.

As of now it allows you to register testsuites and testcases which can be run once or repeatedly with

bk tests

I expected this to be an investment for the long run. The longer we used it, sometimes the tests would fail and we could fix things
more quickly. To my surprise the tests started failing basically right from the beginning, even when very premature code was available.
Both bugs in the testframework as in the codebase itself showed up immediately.

I found that the crux to this is that having a test framework puts your mind in a different state. While developing you’re trying to make things work, while writing tests you’re trying to make things break. This combination has worked already in the early stages, i can hardly wait to see how this is going to work when the number of tests starts to grow and we actually use that information in our process.

The scepsis has gone, unit testing rocks!

The IETF (Internet Engineering Task Force) uses a system for specifying internet protocols. The documents in which these protocols are described are called RFC-s (Request For Comments) and they are subject to a complicated process of drafting, reviewing becoming a proposal and utlitmately hoping to be a standard.

Earlier tonight there was a discussion about RFC-s in the Xaraya group. We use a RFC system in the group as well. Specifying the way we want to code up cool modules in the system are documented in a rigorous XML format which doesn’t leave much room for freedom. No layout, no design, just content. The funny thing is we have a simple process of writing the RFC, discussing it on our mailing lists, adapting the RFCs, the PMC approves them and we use the RFC as a reference to determine whether an implementation is complete. Simple yet effective it would seem.

Nope.

In 5 months we have written just over 35 RFCs, that is, reserved the numbers. The content is mostly lacking and comments on the RFC must be begged for almost. Is the solution to make the process more complicated? Better tools? Allow RFCs to be “designed” so they are more attractive for people to read?

My own belief, was just the opposite, based on the assumption that we wanted to create code, not RFCs. So, I implemented a very simple xml tool to write up RFCs which forced people to focus on the content, not on the layout. It seems I need to rethink a bit.

Well, this would be perfect for Xaraya and DDF operations. The irc voting we sometimes do with the notorious +1 and -1 lines works only so far. If someone doesn’t sum up the votes and puts it on a mailinglist, it’s quickly forgotten what the vote was about and the decision gets lost in the day-to-day work.

With eVote connecting to mailman, which we use in the Xaraya group, this would theoretically mean, as we have connected all mailing lists to NNTP that we can have votes and petitions on all mailing lists and newsgroups.

Especially for the DDF, which can possibly hold a greater number of projects this can highly simplify the decision making process. Definitely worth looking into.

In BL there is some PHP syntax hidden in the tags (for example the evaluation of conditions in if and while constructs).

Here’s a good example why the domain of the blocklayout templating system and the programming language in which it was implemented should be separated.

Because the domain of blocklayout and PHP are so tight together now, this is just one example of what can happen.
Bug 107 – <xar:set> allows PHP to be executed

This has the following advantages:

• guaranteed loading of API
• smaller code size
• simplification for module developers

This changeset makes it a lot easier for module developers. They can just use xarModAPIFunc in their modules, without calling xarModAPILoad each time. Basically this lil change does the same as the developer did. Check whether the module API was already loaded, if not load the module API.

All diffs for ChangeSet 1.498