Perforce and Git : a synthetic benchmark

While at work, I needed to delete around 35000 files from a Perforce repository. Unfortunately for me (and all of us working over here), the main server is in Korea, which means that actually using the VCS is a giant PITA that most of us would like to avoid. Because of the fairly long response times, pretty much every action performed on the repository takes more or less one second. So you can easily calculate the whole time needed for deleting 35000 files : around 10 hours. I left my computer running overnight so it can peacefully complete the task.

That made me wonder : how long would that task take if Perforce was running on a local server. Also, I wanted to see how these times compare to the VCS I use more often, namely Git.

I created 35000 files containing (supposedly) random data with the following command :

for((i=0;i<35000;++i))
do
    dd if=/dev/urandom of=file$i.bin bs=$RANDOM count=1 >/dev/null 2>&1
done

$RANDOM is guaranteed to be between 0 and 32767 by Bash, so the files were maximum 32KB in size. Here are the benchmark results for git :

daniel@Jurij:~/test$ time git add .
real 0m17.467s
user 0m15.283s
sys 0m1.830s

daniel@Jurij:~/test$ time git commit -qam test
real 0m0.171s
user 0m0.097s
sys 0m0.070s

daniel@Jurij:~/test$ time git rm -rq .
real 3m42.355s
user 3m37.257s
sys 0m5.010s

daniel@Jurij:~/test$ time git commit -qam test
real 0m0.016s
user 0m0.013s
sys 0m0.000s

After seeing these numbers, I found them to be adequate. Then, I set up a Perforce server on my Ubuntu virtual machine, since Perforce offers trial versions of their software for installations which serve less than 20 users (and offer less than 20 workspaces, if you speak perforceish). After starting the server, setting up my user and the workspace, I ran these commands, of course having recreated the files beforehand.

daniel@daniel-VirtualBox:~/p4/test$ time find . -type f -exec p4 add -t binary '{}' >/dev/null +
real 0m5.572s
user 0m0.191s
sys 0m0.095s

daniel@daniel-VirtualBox:~/p4/tset$ time p4 submit -d test >/dev/null
real 1m19.138s
user 0m3.668s
sys 0m2.588s

daniel@daniel-VirtualBox:~/p4/test$ time find . -type f -exec p4 delete '{}' >/dev/null +
real 0m5.774s
user 0m0.600s
sys 0m0.528s

daniel@daniel-VirtualBox:~/p4/test$ time p4 submit -d test >/dev/null
real 0m2.817s
user 0m0.061s
sys 0m0.004s

The Perforce commandline client does not support passing wildcards, so I had to use find to help me in the task. The number of actual calls made to the application was minimized by the usage of ‚+’ instead of ‚;’ at the end of the invocation.

I did find the results quite surprising. I was expecting Git to easily trump Perforce in terms of performance, but this test quickly showed that this was clearly not the case. I still find Git much easier to use than Perforce, though. Especially considering the network lag introduced by my workplace environment, and the extra limitations put on the repository by corporate rules.

Informacje o Daniel

freezingly cold soul
Ten wpis został opublikowany w kategorii komputer, Uncategorized i oznaczony tagami . Dodaj zakładkę do bezpośredniego odnośnika.

4 odpowiedzi na „Perforce and Git : a synthetic benchmark

  1. Interesting results! To do the add without using find you could have used ‚p4 reconcile -a …’. Likewise with the submit you could have run ‚p4 submit -d ‚test’ and it would have submitted them all in one go.

  2. John Halbig pisze:

    Hello Daniel,

    From http://www.perforce.com/perforce/doc.current/manuals/cmdref/filespecs.html:

    Wildcards

    The Perforce system allows the use of three wildcards:

    Wildcard Meaning

    * Matches all characters except slashes within one directory.

    … Matches all files under the current working directory and all
    subdirectories. (matches anything, including slashes, and does
    so across subdirectories)

    %%1 – %%9 Positional specifiers for substring rearrangement in filenames,
    when used in views.

    Examples:

    Expression Matches

    J* Files in the current directory starting with J

    */help All files called help in current subdirectories
    ./… All files under the current directory and its subdirectories
    ./….c All files under the current directory and its subdirectories,
    that end in .c
    /usr/bruno/… All files under /usr/bruno
    //bruno_ws/… All files in the workspace or depot that is named bruno_ws
    //depot/… All files in the depot
    //… All files in all depots

    Please feel free to contact me at support@perforce.com if you have any questions.

    Cheers!

  3. stewartlord pisze:

    Hi Daniel, nice benchmarks. You can pass wildcards to p4. You can also setup your workspace to be writable like git/svn/hg, and use ‚p4 reconcile’ to have it detect your changes automatically. The submit time for the add doesn’t look too hot (commit rate of 443 files/second). We would hope to see more like 15k-25k files/second – which you are much closer to on the delete submit (12.5k files/second). You can add -Ztrack=1 to your command to get some detailed performance output if you are interested in what’s going on under the hood. For example: p4 -Ztrack=1 submit -d ‚test’. Later!

  4. Francois Hill pisze:

    I guess you did not do this performance test on an SSD harddrive? Because over 3 minutes far too long.

    Perhaps run the same benchmark on an SSD and you should see a massive difference. Because git is purely bottlenecked by your HDD mostly, whereas P4 has got the massive network bottleneck. Networks can be offline. Networks can get „drained”, etc.

Skomentuj

Wprowadź swoje dane lub kliknij jedną z tych ikon, aby się zalogować:

Logo WordPress.com

Komentujesz korzystając z konta WordPress.com. Log Out / Zmień )

Zdjęcie z Twittera

Komentujesz korzystając z konta Twitter. Log Out / Zmień )

Facebook photo

Komentujesz korzystając z konta Facebook. Log Out / Zmień )

Google+ photo

Komentujesz korzystając z konta Google+. Log Out / Zmień )

Connecting to %s