Command of the day: git svn (Using git over a subversion repository)

This may be the end of a long quest for a tool to do 'offline commits on subversion'. That is, being able to work with small commits when you are not able to commit because you are off-line (on-flight or on-train hacking) or you are not a commiter yet (menthorship).

The fact is that, we can benefit of some nice features of git without having to migrate from subversion. I found a very interesting blog entry explaining that. I compiled my own notes for CLAM:

Install the required packages:
sudo apt-get install git-svn

Configure your user:
git config --global user.name 'David García Garzón'
git config --global user.email 'yo@alli.com'

Create a git clone of the svn:
git svn clone http://dgarcia@clam-project.org/clam --stdlayout -r HEAD
Notice that i skiped the 'trunk' part of the url we normally use. '--stdlayout' is to interpret trunk/ branches/ tags/ as such. '-rHEAD' avoids cloning the whole history which is faster.

Work with the repository as with a normal git one, avoiding remote operations (push/pull). Git can fustrate SVN users with a lack of shortcuts (ci, di, up...). You can define them at will:
git config --global alias.ci "commit -a"
git config --global alias.di diff
git config --global alias.stat status
git config --global alias.up "svn rebase"
git config --global alias.rci "svn dcommit"

To update your git repository to server changes:
git svn rebase
If you have uncommited local changes you have to 'stash' them first.

To apply all the local commits to subversion, once rebased:
git svn dcommit


To generate, instead, an incremental set of patches:
git format-patch -M -C -s --inline --stdout remotes/trunk > mypatch.mbox

Note that this is not a regular patch file but an mbox file containing several mails, one per commit, each with the patch attached and some aditional information such as the commit message.

Those commits can be sent to the menthor and applied to git sandbox with svn write access.
git am mypatch.mbox

An then the menthor can commit using previous command 'dcommit'. That easy... or not so.

Some other highlights i would do on git: Nice programmer oriented subcomands such as git bisect, to help bug hunting by doing a dicotomic search on revisions, or git stash, to discard temporarily uncommited changes and recovering them later, or 'git grep', which executes grep ignoring git control files and generated files. Tools like StGit and guilt that allows to edit the order of the local commits considering them a serie of patches similarly to quilt.

Command of the day: interdiff

Often, when working in code projects, I have to keep some local changes uncommited because I am offline or because i am doing some experiment and i am not sure that it will be successful. In those cases you end up doing a big commit with has very low grain to rollback any regressive change. Sure new distributed VCS allow offline commits but we are working with subversion. So I was glad when i recently discovered how to use interdiff command.

If you plan to make a set of offline changes, on each point you would commit, just take an 'svn diff' on a file. Given those accomulative diffs respect the BASE revision, interdiff knows how to extract the incremental ones from one state to the next one so. When you are back online, you revert all changes, apply and commit each patch separately.

Other interesting commands to manage patches (available in the patchutils package):

• combinediff: the reverse, joins two patches together
• recountdiff: very useful when you want to reapply a patch but the state of the code has changed.
• flipdiff: exchanges the order of two pages
• filterdiff: remove the changes affecting a set of files from an existing patch

No excuses for coarse commits after on-flight coding to Parma!

Command of the day: hotspot2calltree

Today, the command of the day is hotspot2calltree.

I do like a lot using kcachegrind to tune my C++ code. You can use KCacheGrind to navigate through the actual function calls in a given execution of your program and seeing very graphically where the time is spent.

Today i needed to optimize some python code but that's not C++ code. No problem. Add in your code this:

import hotshot
prof = hotshot.Profile("profile.hotspot")
prof.runcall(myFunction)
prof.close()

And then, at the shell:

sudo apt-get install kcachegrind-converters
hotspot2calltree -o profile.callgrind profile.hotspot
kcachegrind profile.callgrind

And now you get a nice kcachegrind profile you can navigate on.

Comand of the day: kig

Each task has a tool. With graphics and figures too. If i want to edit a photograph or to perform some nice effects on an existing image, that's a task for Gimp. If I need some cute artwork for icons, banners, web... I prefer to vector with my so loved Inkscape. When such drawings are not so artistical and need some related and formal figures and diagrams, i normally consider dia, limited but correct. If i want to plot a complex graph i let the task to graphviz's dot and i just declare the vertexs. If I want to plot some program data output, python plus matplotlib is the faster way to process and render it. Automating execution to visualization process is very convenient. Some time ago I used gnuplot for that but python is more flexible towards data formats. Often what you need is to explore such data, interactive visualization can be revealing with tools such as qtiplot.

But my problem today wasn't none of the above but drawing some geometric diagrams (angles, vectors, tangents...) to illustrate some trigonometric equations. I was thinking on QCad but Kig did perfectly. Kig is about doing geometrical manipulation: intersections, angle transportation, angle mesurements, solidarized elements, python scripting...



I was to integrate kig png exporting into WiKo figure generation but i found two show-stoper problems: Command line options for batch exporting seems not to work and there is no way to control the viewport but by resizing the windows and controlling the zoom. While the concept of limited size canvas exists, I didn't find a way to resize it :-( Anyway is worth to dig in such a tool even doing patches to have batch exporting working.

Command of the day: svk ls

Why not calling them command-of-the-year as the one and only command-of-the-day i posted until now was posted one year ago? Well, they are intended to be commands that saved me the day like the one i explain today.

On my previous entry, I explained that Pau and me were publishing on sourceforge a python script, WiKo, that we used to edit our latest articles and thesis. Because we used it in such context, latest version have been commited in a private repository we have for articles and the like.

I wanted to move the files to a public repository at sourceforge. Sadly, svnadmin dump just works when you are in the same host as the repository is and we have no regular ssh access to the subversion. We could bother our admin, Jordi Funollet, but he is always very busy and we are always requesting him. (Thanks, Jordi for your support!!)

Luckily i found a nice entry blog on how to use svk to remotely dump a svn repository.

You need to install svk, in debian based:

 sudo apt-get install svk 

Then we should create a mirror repository with svk just containing the revisions concerning the selected path

 svk ls svn+ssh://username@svn.myserver.com/path/to/repo/sub/path/to/wiko 

svk will ask you several questions:

• Choose a base URI to mirror from (press enter to use the full URI): To filter by path, i recommend to do such filtering on the command line by specifying the full path, so return.
• Depot path: [//mirror/wiko] Is very important here to change the depot name so you can skip that '//mirror' part, so answer //yourproject beginning with a double slash!!
• To end up, it asks which revisions do you want, normally the answer is 'a' (all)

Then dump the local svk repository:

 svnadmin dump ~/.svk/local > my-repository-dump 

And now you can import it into sourceforge as normal:

 svnadmin load [the-new-repository]  < my-reposotory-dump 

With sourceforge the process is more complex. You should upload the dumpfile to the shell server. For example, for wiko it was at: sftp://wiko.sourceforge.net/home/groups/w/wi/wiko/

Then you should specify the uploaded dump file as it was a cvs to svn migration. It last a while, after that you can checkout wiko code!

 svn co https://wiko.svn.sourceforge.net/svnroot/wiko/wiko 

I lost the history of the file before a move operation, but at least we recovered some of the history.

Taking a look at the man page you can see that svk is really helpfull for other purposes, but definitely, its ability of dumping a remote repository saved my day :-)

Command of the day: convmv

If you are moving around files from one system (ubuntu: default encoding is utf8) to another (debian: historical default encoding for catalonians who want to use the euro sign is latin1) or getting files from CD's built in windows (spanish windows encoding cp850), you surelly will get on the filenames strange signs instead accents, tilded n's and c's with cedillas, opening question marks... not talking about the weird (for us) nordic and eastern signs. You may say that using such names for files is getting in trouble, but i like them for mp3 in order to convert back and forward from id3 tags to filenames.

Now that i have to move to another office and another computer i had such filename encoding problems. I knew how to convert the encoding of the content of a file with iconv how to solve that problem on the file names. 'apt-cache search' gave me the solution: convmv.

You can use it recursively with an '-r' option, it detects which files are already on utf8 and you have to explicitly indicate the 'from' and 'to' encodings. It will be nice it to detect the original encoding when there is no conflict but, at least it lets you do a temptative conversion and see how it would look after conversion. So the way of working is:

Do a temptative conversion for all the files you need to convert:

convmv -f latin1 -t utf8 -r . | less

Check the output and see whether the non utf8 files have been properly converted. If some files were not latin1 neither utf8, the name will be not properly converted, just prove a different origin encoding for those concrete files. When you know which is the encoding to translate just execute:

convmv -f latin1 -t utf8 -r . --notest

If the applied conversion was not the proper one, just apply the conversion backwards and you'll have
the original one again.