GNU Global benchmarks

Follow-up posts

The last post reports some performance numbers for cscope. There's another, similar tool that I've been curious about: GNU global. It's like cscope in many ways. It doesn't have some of the nicer cscope search types (caller, callee, assignment, etc), and cscope works fine so I've never felt the need to move. Since I just ran some cscope benchmarks, I thought it'd be interesting to run the exact same tests with GNU global. Here I use the gtags-cscope frontend. This is a compatibility layer in GNU global that has an identical interface to cscope (among other things this makes it trivial to use xcscope.el with GNU global).

Test description

The test conditions are the same as before. The testing in this and the previous post was performed by a script, which appears at the end of this post. gtags-cscope doesn't have a separate inverted-index mode, so just a single test appears here.

Here I'm using GNU global 6.2.10 built from source (upstream is in some sort of fight with the Debian maintainer, so the packages are ancient). Cscope is 15.8a.

Results

All timings in seconds. Timings from the previous post are re-iterated for easy comparison.

Cold disk cache

	Normal mode/ext3	Kernel mode/ext3	GNU Global/ext3	Normal mode/tmpfs	Kernel mode/tmpfs	GNU Global/tmpfs
Initial database build	45.9	80.2	84.1	14.0	44.2	14.0
Database re-build after touching a file	10.4	48.9	26.8	3.2	30.1	0.7
Initial search	7.5	3.0	23.3	0.8	31.2	0.2
Re-search after touching a file	12.7	43.7	28.4	3.5	32.1	0.7
Initial no-db-update search	5.3	0.8	0.1	0.8	0.8	0.0
No-db-update re-search after touching a file	5.1	0.8	0.1	0.7	0.8	0.0

Warm disk cache

	Normal mode/ext3	Kernel mode/ext3	GNU Global/ext3	Normal mode/tmpfs	Kernel mode/tmpfs	GNU Global/tmpfs
Initial database build	13.8	49.6	18.0	12.9	44.4	13.7
Database re-build after touching a file	3.5	35.5	1.3	2.7	30.8	0.6
Initial search	0.8	0.1	0.4	0.8	30.8	0.2
Re-search after touching a file	4.0	33.5	1.3	3.5	31.9	0.6
Initial no-db-update search	0.7	0.0	0.0	0.7	0.7	0.0
No-db-update re-search after touching a file	0.7	0.0	0.0	0.7	0.7	0.0

Conclusions

During normal use, we'd have a warm cache and we'd be using a real hard disk. This is the bottom-left area of the timing tables. Those timings indicate that GNU Global is much faster than cscope. Search performance appears to be on-par with with an inverted-index-enabled cscope, but database build times only suffer a little bit. This is interesting, and maybe would be worth switching to at some point.

Benchmark script

All the timings were performed with the following zsh script. It uses some zsh-isms, but could be converted to bash if somebody cares to do it.

#!/bin/zsh

# needed in cleandb()
setopt nonomatch

function dropcaches() {
    if [[ $warmcold == "cold" ]]; then
        sync ;
        sudo sysctl -w vm.drop_caches=3;
    fi
    sleep 2;
}

function cleandb() {
    # requires nonomatch option to ignore missing globs
    rm -f cscope.out* G*;
}

function touchfile() {
    sleep 2; # very important. cscope needs this to see the file update
    touch include/drm/drm_edid.h;
}

TIMEFMT='%E'

awktally='
BEGIN {
  skip = ENVIRON["skip"]
}

/^[0-9\.]+s$/ {
  gsub("s","");
  str = str " " $1
  if( n >= skip )
  {
    sum += $1;
  }
  n++;
}

END {
  print ENVIRON["name"] ": skipping: " skip " all: " str " mean: " sum/(n-skip)
}'

typeset -A skipcounts
skipcounts=(cold 2 warm 2)

typeset -A modeoptions
modeoptions=(normal "" kernel "-k -q")

cscope-indexer -l -r

Nrepeat=8

for mode (normal kernel global)
{
    if [[ $mode == "global" ]]; then
        cmd="gtags-cscope";
    else
        cmd="cscope $modeoptions[$mode]";
    fi

    for dotouch (0 1)
    {
        for warmcold (cold warm)
        {
            export name="$warmcold initial build; $mode mode; touching: $dotouch";
            export skip=$skipcounts[$warmcold];
            repeat $(($Nrepeat + $skip)) {
                if (($dotouch)); then
                    touchfile;
                else
                    cleandb;
                fi
                dropcaches;
                time ${(z)cmd} -b;
            } |& awk $awktally
        }
    }

    for dotouch (0 1)
    {
        for warmcold (cold warm)
        {
            export name="$warmcold initial search; $mode mode; touching: $dotouch";
            export skip=$skipcounts[$warmcold];
            repeat $(($Nrepeat + $skip)) {
                if (($dotouch)); then
                    touchfile;
                fi
                dropcaches;
                time ${(z)cmd} -L0 main > /dev/null;
            } |& awk $awktally
        }
    }

    for dotouch (0 1)
    {
        for warmcold (cold warm)
        {
            export name="$warmcold initial no-db search; $mode mode; touching: $dotouch";
            export skip=$skipcounts[$warmcold];
            repeat $(($Nrepeat + $skip)) {
                if (($dotouch)); then
                    touchfile;
                fi
                dropcaches;
                time ${(z)cmd} -d -L0 main > /dev/null;
            } |& awk $awktally
        }
    }
}