Command of the day: git svn (Using git over a subversion repository)

This may be the end of a long quest for a tool to do 'offline commits on subversion'. That is, being able to work with small commits when you are not able to commit because you are off-line (on-flight or on-train hacking) or you are not a commiter yet (menthorship).

The fact is that, we can benefit of some nice features of git without having to migrate from subversion. I found a very interesting blog entry explaining that. I compiled my own notes for CLAM:

Install the required packages:
sudo apt-get install git-svn

Configure your user:
git config --global user.name 'David García Garzón'
git config --global user.email 'yo@alli.com'

Create a git clone of the svn:
git svn clone http://dgarcia@clam-project.org/clam --stdlayout -r HEAD
Notice that i skiped the 'trunk' part of the url we normally use. '--stdlayout' is to interpret trunk/ branches/ tags/ as such. '-rHEAD' avoids cloning the whole history which is faster.

Work with the repository as with a normal git one, avoiding remote operations (push/pull). Git can fustrate SVN users with a lack of shortcuts (ci, di, up...). You can define them at will:
git config --global alias.ci "commit -a"
git config --global alias.di diff
git config --global alias.stat status
git config --global alias.up "svn rebase"
git config --global alias.rci "svn dcommit"

To update your git repository to server changes:
git svn rebase
If you have uncommited local changes you have to 'stash' them first.

To apply all the local commits to subversion, once rebased:
git svn dcommit


To generate, instead, an incremental set of patches:
git format-patch -M -C -s --inline --stdout remotes/trunk > mypatch.mbox

Note that this is not a regular patch file but an mbox file containing several mails, one per commit, each with the patch attached and some aditional information such as the commit message.

Those commits can be sent to the menthor and applied to git sandbox with svn write access.
git am mypatch.mbox

An then the menthor can commit using previous command 'dcommit'. That easy... or not so.

Some other highlights i would do on git: Nice programmer oriented subcomands such as git bisect, to help bug hunting by doing a dicotomic search on revisions, or git stash, to discard temporarily uncommited changes and recovering them later, or 'git grep', which executes grep ignoring git control files and generated files. Tools like StGit and guilt that allows to edit the order of the local commits considering them a serie of patches similarly to quilt.

Command of the day: interdiff

Often, when working in code projects, I have to keep some local changes uncommited because I am offline or because i am doing some experiment and i am not sure that it will be successful. In those cases you end up doing a big commit with has very low grain to rollback any regressive change. Sure new distributed VCS allow offline commits but we are working with subversion. So I was glad when i recently discovered how to use interdiff command.

If you plan to make a set of offline changes, on each point you would commit, just take an 'svn diff' on a file. Given those accomulative diffs respect the BASE revision, interdiff knows how to extract the incremental ones from one state to the next one so. When you are back online, you revert all changes, apply and commit each patch separately.

Other interesting commands to manage patches (available in the patchutils package):

• combinediff: the reverse, joins two patches together
• recountdiff: very useful when you want to reapply a patch but the state of the code has changed.
• flipdiff: exchanges the order of two pages
• filterdiff: remove the changes affecting a set of files from an existing patch

No excuses for coarse commits after on-flight coding to Parma!

Command of the day: svk ls

Why not calling them command-of-the-year as the one and only command-of-the-day i posted until now was posted one year ago? Well, they are intended to be commands that saved me the day like the one i explain today.

On my previous entry, I explained that Pau and me were publishing on sourceforge a python script, WiKo, that we used to edit our latest articles and thesis. Because we used it in such context, latest version have been commited in a private repository we have for articles and the like.

I wanted to move the files to a public repository at sourceforge. Sadly, svnadmin dump just works when you are in the same host as the repository is and we have no regular ssh access to the subversion. We could bother our admin, Jordi Funollet, but he is always very busy and we are always requesting him. (Thanks, Jordi for your support!!)

Luckily i found a nice entry blog on how to use svk to remotely dump a svn repository.

You need to install svk, in debian based:

 sudo apt-get install svk 

Then we should create a mirror repository with svk just containing the revisions concerning the selected path

 svk ls svn+ssh://username@svn.myserver.com/path/to/repo/sub/path/to/wiko 

svk will ask you several questions:

• Choose a base URI to mirror from (press enter to use the full URI): To filter by path, i recommend to do such filtering on the command line by specifying the full path, so return.
• Depot path: [//mirror/wiko] Is very important here to change the depot name so you can skip that '//mirror' part, so answer //yourproject beginning with a double slash!!
• To end up, it asks which revisions do you want, normally the answer is 'a' (all)

Then dump the local svk repository:

 svnadmin dump ~/.svk/local > my-repository-dump 

And now you can import it into sourceforge as normal:

 svnadmin load [the-new-repository]  < my-reposotory-dump 

With sourceforge the process is more complex. You should upload the dumpfile to the shell server. For example, for wiko it was at: sftp://wiko.sourceforge.net/home/groups/w/wi/wiko/

Then you should specify the uploaded dump file as it was a cvs to svn migration. It last a while, after that you can checkout wiko code!

 svn co https://wiko.svn.sourceforge.net/svnroot/wiko/wiko 

I lost the history of the file before a move operation, but at least we recovered some of the history.

Taking a look at the man page you can see that svk is really helpfull for other purposes, but definitely, its ability of dumping a remote repository saved my day :-)

WiKo, the wiki compiler

Pau and me have just released WiKo as sourceforge project. WiKo is a simple but powerful Python script that takes files with wiki content on a given directory and either builds a web, a LaTeX article or a blog.

The script has a very long history before being branded as WiKo. It has its roots on the simple script i did to build most of my websites (Can Voki, Codders, KKEPerians UNLTD...) The script eased keeping static a lot of pages sharing the same layout design. The script was later enhanced to read wiki pages and to build also LaTeX articles. Even I used it for my master thesis. Recently Pau also started to use it for his PhD thesis and he has added great improvements such as LaTeX formulas in HTML output or bibliography linking. And lately we started to use it as internal wiki for some projects we are involved within Barcelona Media. So we thought that such script deserved a sourceforge project, and that's how WiKo born. I hope it can be useful to you all.

WiKo might be useful to you if:

• You have an static website and you are considering to make it dynamic just to reuse design
• You have to write articles to be available in both html and pdf
• You are moving content from web, to LaTeX articles and to blogs and you are feed up of translating markup
• You love wiki syntax but you hate to edit wikis in web forms
• You like LaTeX output but you hate (or don't know) its markup
• You like colaborative environments such subversion and want to use it as base for a wiki
• You still are considering ikiwiki but you would like it was not written in Perl so you could contribute to it ;-)

Also I am currently working hard on being able to use WiKo to edit this blog. I am almost there so stay tunned.