[Frugalware-devel] Have you ever thought of changing the Frugalware SCM?

VMiklos vmiklos at frugalware.org
Wed Jun 13 23:42:05 CEST 2007


Na Wed, Jun 13, 2007 at 10:53:14PM +0200, Csaba Henk <csaba-ml at creo.hu> pisal(a):
> Frugalware uses Darcs. And Darcs apparently doesn't scale with such a
> volume.
> 
> As an end-user, the most annoying aspect of it is the unusuable slowness
> of the web interface, but it also tells something that syncing the tree
> happens via rsync instead of the native SCM pull operation these days.
> (This slightly reminds me to this page:
> http://xitami.org/xitami.org_on_apache ;) )
> You should know these things better than me, so I don't think more words are
> needed to describe the current situation.

voroskoi recently made a blog post about this:

http://blogs.frugalware.org/voroskoi/2007/06/09/problems_with_darcs_but_there_is_nothing

we realized that we should switch because our repo is too big

> (I also think that Darcs' version control semantics is flawed. I'm ready
> to go into details if anyone is interested, but it would be OT at this
> point. [And yes, I also think Darcs has some genious innovations.])

what we really can't give up is that darcs is distributed. on the other
hand, when playing with other scms, i had to realize that living without
cherry-picking (at least for record and revert) is not too good :)

> So I tried to convert frugalware-current to some other format via
> Tailor [http://progetti.arstecnica.it/tailor]. Practically, I targeted
> those two SCM-s which are known to be distributed and magnitudes faster
> than their competitors are: Mercurial (aka. Hg) and Git.

that's great, voroskoi mentions these too in his post, too :)

> First I tried Hg. It was not a trivial thing to do -- each changeset
> is converted via a darcs pull/hg commit and 
> 'darcs pull --match "hash <whatever>"' took _minutes_ for each changeset.
> This is no fun when you have over quarter-hundred thousand csets...
> 
> I succeeded to optimize this and get over the rest of the difficulties.
> Here is the result:
> 
>    http://mercurial.creo.hu/repos/frugalware-current-hg-experimental/
> 
> It's not perfect though. There is a little delta from the original one.

wow, it's nice. i know tailor and i used it already for converting
smaller repos. (there is a small script available at
http://darcs.frugalware.org/repos/vmexam/darcs/darcs2git which
simplifies the task). the first problem was the accents in the commit
messsages, i've created a path for this and it's alrady in tailor's
darcs. it's surprising that you were able to convert -current to hg as
the author said my problem affected any conversion where the source is
darcs

on the other side, it's interesting:

$ time lynx -dump
'http://mercurial.creo.hu/repos/frugalware-current-hg-experimental/?rev/fd74925ad115'
>/dev/null

real    0m13.247s

$ time lynx -dump
'http://darcs.frugalware.org/darcsweb/darcsweb.cgi?r=frugalware-current;a=darcs_commitdiff;h=20070507182839-f6986-80755f3e0bdef5d584a984e5c4cf86cb6a3fdee0.gz;'
>/dev/null

real    0m0.255s

i picked a random patch, maybe it's darcsweb's caching mechanism, who
knows. i know that hg should be faster than darcs, it's just interesting
:)

> Regarding Git: for some trivial Tailory problem I couldn't even start
> the conversion. It should not be hard to come over but I was really
> exhausted at this point and I need the time for other things in my life
> and I don't know if you were interested at all. So I stopped playing
> at this point.

i played with smaller repos, during the conversion tailor bails out with
an error when converting the first tag at some repos. i've mailed the
tailor maintainer, but this problem is not as clear to me as the accent
one was (there all the problem was that darcs did not generate a valid
xml)

> This is just a short summary but if you see some reason in switching
> the SCM, I can share my experience and I can go into details. (Given
> that I spent so much time on the conversion I thought I give a chance
> for these hours to not turn out to be a complete waste...)

:)

as voroskoi mentioned, darcs is fine, our only big problem with it is
its speed. as you and he mention, too the two promising alternatives are
hg and git. there were a benchmark about bzr, hg and git about
converting the mozilla repos and there it has been pointed out that git
is faster. since our problem was speed, i started to play with git.

due to the same reason (we like how darcs works), i've started to write
a wrapper around git, it's like cogito, but it's primary focus is
behaving like darcs. it's available at

http://darcs.frugalware.org/repos/vmexam/python/dg.py

currently what worth mentioning is that dg record and dg revert more or
less works like in darcs, with full cherry-pick support

again as voroskoi mentioned, we don't think that we're in a hurry, and
switching to anything without massive testing would be bad, but want to
change one day

finally two links which may be useful:

http://www.kernel.org/pub/software/scm/git/docs/everyday.html

and a table comparing git and darcs commands:

http://wiki.frugalware.org/Git_setup#Comparing_commands

it's far from complete but all i was found was a table listing ~5
commands in the darcs-users archive, it's much better than that is :)

so the current state:

1) we're working with the tailor author to be able to convert some of
our problematic repos from darcs to git

2) it seems that git has very lowlevel tools which worth a frontend, and
we'll see how easy/hard is to create one

and thanks for providing the hg repo, i'm sure some of us will do some
real benchmarks then regarding hg vs darcs (running darcs changes, darcs
what)

VMiklos

-- 
developer of Frugalware Linux - http://frugalware.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : /pipermail/attachments/20070613/50cedc5e/attachment.bin


More information about the Frugalware-devel mailing list