Here's yet another instance where the data processing I needed done could be acomplished entirely in the shell, with vnlog tools.
I have some time-series data in a text table. Via some join and filter operations, I have boiled down this table to a sequence of time indices where something interesting happened. For instance let's say it looks like this:
t.vnl
# time 1976 1977 1978 1979 1980 1986 1987 1988 1989 2011 2012 2013 2014 2015 4679 4680 4681 4682 4683 4684 4685 4686 4687 7281 7282 7283 7291 7292 7293
I'd like to find the longest contiguous chunk of time where the interesting thing kept happening. How? Like this!
$ < t.vnl vnl-filter -p 'time,d=diff(time)' |
vnl-uniq -c -f -1 |
vnl-filter 'd==1' -p 'count=count+1,time=time-1' |
vnl-sort -nrk count |
vnl-align
# count time
9 4679
5 2011
5 1976
4 1986
3 7291
3 7281
Bam! So the longest run was 9-frames-long, starting at time = 4679.