revision control Archive

Played a little with the gource log visualizer today. It is typically used to visualize committed revisions in a revision control system, although any system that logs events of some kind could be made to work with it I suppose.

I happen to have a repository of xaraya going back to 2002 in git, so I thought I’d give it a shot.

…more time passes than originally planned…

Result

8 years of commit history in 10 minutes of video. Produced with gource from a git repository of Xaraya. The video traces the 2.x main branch back to its origin, repository wise. The start is the import of the CVS postnuke repository into Bitkeeper. Since then xaraya has switched to monotone.

Gource settings used:

gource --stop-position 1.0 \
       --camera-mode overview \
       --bloom-intensity 0.18 \
       --user-image-dir .git/avatar \
       --highlight-all-users \
       --output-framerate 60 \
       -s 0.5 \
       -720x576 \
       -a 0.3 \
       --hide filenames \
       --user-scale 1.4 \
       --date-format %Y-%m-%d \
       --disable-progress \
       --output-ppm-stream - \
| ffmpeg -y -b 3000K -r 60 -f image2pipe -vcodec ppm -i - -vcodec libx264 -vpre default gource.mp4

This produced a video of 11 min. 35 seconds. To bring it back to 10 minutes the framerate was increased until the total time fell just below 10 minutes.

I specifically wanted the whole history to be in one video of 10 minutes (the Youtube maximum) which means compromising a bit on the quality. If there are things I can do within these assumptions to improve the video, I’d like to hear them.

We have been considering migrating Xaraya to the creole abstraction layer in the future. Creole is similar to the adodb library we use now. Originally the support for Creole was created to be able to implement a MS-SQL proof of concept quickly. After using the library for a bit I found it much easier to work with than adodb. Creole’s API is much cleaner and it is PHP5 only, which gives us a head start for future Xaraya versions.

The monotone setup we use allows us to test all this without disturbing main line development. Organizing these bigger changes into scenario’s allows us to isolate these changes and see where they lead to. The decision to use or to throw away can thus be postponed to a time when we’re ready to do so.

Since replacing the middleware is potentially a very big step, I noticed we (the Xaraya group) have some trouble making a decision on what to do with this. Therefore, the creole scenario has been merged into this site to show that a real life site (albeit a small one) can be migrated to creole without any effect. (Note: I’ve been running creole on other installs for a while now, so I’ve known this for a while)

Verdict: one change was necessary to wrap the ‘GetRowAssoc’ method present in adodb but not in creole. Everything else worked directly after the merge, so my preliminary conclusion if the decision hangs on ‘what impact does this have?’ is that the impact is very minimal. On top of that I think that the issues we might run into are not bad things as such, but chances to improve Xaraya.

Chris Rathjen : Team Foundation vs. Subversion: Shelving:

Chris compares the two different mechanisms used for a concept introduced in Team Foundation (TFS) known as Shelving. It is a concept where you put a certain state of your development environment ( workspace, repository, whatever the local term is you are used to) aside (on a shelf), work on some other stuff and take the shelved state back into your workspace (UnShelve) when ready.

Let me add a few notes on how we do this in Xaraya with BitKeeper.

Feature: No More Free BitKeeper:
Linus, not interested in a one sided deal even when he’s on the winning side, decided for this and other reasons to migrate away from BitKeeper and to begin looking for an alternative.

It will be interesting to see how this works out.

Some hours ago my changeset number 1000 went into the xaraya repository.

As you can see in the comments, it’s not a good one, basically cleaning up some mess i made earlier that day.

Nevertheless, it’s there

Using a script from Bryan O’Sullivans bitkeeper repository we integrated the bitkeeper repositories for Xaraya with our bugzilla installation. The script is used as a trigger everytime changes are committed to the main repositories on the server and scans the changeset comments for text fragments mentioning bugs.

When it finds those fragments, the following happens:

• the change committed is attached as a patch to the bug; (this patch can in most cases be applied to source trees out there, while maintaining upgradeability)
• a comment is entered for the bug, summarizing what happened;
• links are created in the comment to view the path in the bitkeeper web interface;
• bugzilla mails the relevant parties as usual.

This script is smart enough, not to enter the same information on the same bug and changeset twice which, if you know how bitkeeper works, is not uncommon if you have multiple repositories. As always, on deploying something new, you always start to think immediately about features which you could add (example: scanning for fixed  or fix  and setting the state of the bug accordingly).

During the period of  little over a year, we’ve built ourselves an infrastructure which is still growing and is literally improved upon everyday. Some elements of it:

• automatic updates of mirror repositories;
• fully integrated patch system;
• custom command system, defining new commands in the repository which propagate to all developer repositories;
• funny stats output;
• getting information on RFC status;
• integrated build system based on phing;
• integrated source documentation generator based on phpdoc;
• repository web interface as module of xaraya;
• adding comments to files which propagate across repositories;
• integrated unit test system;
• semi-automatic archiving of old files in the repositories;
• possibility to test for compliance with coding standards on every commit automatically;

With hindsight, a rather long list of tools, mainly created for fun. Some are used extensively, others are only used by myself.

Last week we started implementing a unit testing framework for Xaraya, which is highly integrated with the bitkeeper repository.

As of now it allows you to register testsuites and testcases which can be run once or repeatedly with

bk tests

I expected this to be an investment for the long run. The longer we used it, sometimes the tests would fail and we could fix things
more quickly. To my surprise the tests started failing basically right from the beginning, even when very premature code was available.
Both bugs in the testframework as in the codebase itself showed up immediately.

I found that the crux to this is that having a test framework puts your mind in a different state. While developing you’re trying to make things work, while writing tests you’re trying to make things break. This combination has worked already in the early stages, i can hardly wait to see how this is going to work when the number of tests starts to grow and we actually use that information in our process.

The scepsis has gone, unit testing rocks!

Today the BK-view module I wrote for Xaraya went into a live site which can actually be used for something usefull.

The site http://bk.xaraya.com is the BitKeeper central for the Xaraya project. The BK-view module is used to browse through all of the clones used in the project, which is in the mean time a rather large number, and I’ve only included the top level clones.

The clones from the clones are not even in there.