ACCBIN is a format invented by Wayne Davis in our lab. Here is his description of it.
Text files
The text format xdatplot understands is very simple. In fact, a
simple list of whitespace-separated decimal numbers will work fine, as
long as they are in the range [-32767, 32767]. (It
can easily be configured to accept a different range.) You can
also set parameter values in the text file. Parameters are set in
parameter lines as follows:
Parameter = valueThese are the parameters used by xdatplot:
Here is a simple text file that xdatplot can display:
# Sample xdatplot text file Comment = This is a comment VMul = 0.000610352 Gain = 0.002 # xdatplot ignores this Clock = 2000 -28 -25 -24 -23 -25 -28 -28 -27 -23 -21 -20 -17 -16 -14 -12 -13 -13 -12 -10 -10 -13 -9 -6 -6 -6 -8 -8 -10 -12 -14 -16 -18 -17 -20 -21 -19 -19 -18 -19 -21 -21 -20 -19 -17 -17 -13 -11 -10 -9 -6 -4 -6 -7 -8 -7 -3 -4 -4 -4 -5 -6 -9 -12 -15 -17 -17 -16 -16 -16 -15 -15 -16 -17 -18 -18 -16 -15 -14 -11 -7 -5 -4 -4 -3 -1 -2 -2 -2 -3 0 -1 -2 -3 -4 -5 -8 -10 -12 -14 -13 -14 -14 -14 -15 -16 -19 -20 -20 -17 -16 -15 -13 -10 -8 -7 -6 -7 -7 -5 -4 -4 -2 1 1 -1 -4 -5 -4 -9 -13 -14 -17 -17 -18 -17 -16 -15 -15 -17 -20 -20 -19 -18 -16 -12 -11 -9 -8 -6 -8 -8 -7 -8 -8 -9 -5 -3 -3 -5 -9 -10 -11 -14 -16 -16 -19 -20 -20 -19 -19 -17 -19 -22 -21 -21 -18 -18 -15 -12 -9 -8 -7 -7 -6 -5 -7 -9 -8 -7 -4 -4 -7 -7 -7 -9 -11 -14 -15 -16 -15 -16 -16 -16 -15 -20 -21 -23 -22 -21 -19 -17 -14 -11 -9 -7 -8 -7 -7 -8 -9 -10 -6 -3 -2 -4 -6 -9 -9 -12 -16 -16 -18 -18 -19 -18 -17 -18 -19 -23 -24 -23 -22 -19 -16 -15 -15 -12 -11 -11 -10 -8 -8 -8 -10 -7 -4 -4 -4 -5 -8 -9 -13 -17 -18 -19 -22 -22 -24 -24 -22 -20 -21 -22 -23 -23 -22 -19 -18 -16 -15 -13 -12 -11 -11 -11 -11 -11 -11 -10 -8 -7 -7 -9 -10 -12 -17 -19 -23 -24 -25 -27 -28 -29 -29 -30 -32 -35 -35 -34 -32 -29 -26 -21 -20 -22 -23 -23 -22 -20 -19 -19 -18 -18 -19 -19 -21 -22 -23 -27 -28 -29 -30 -31 -32 -32 -32 -32 -30 -31 -32 -32 -32 -30 -28 -26 -22 -21 -21 -21 -18 -18 -17 -16 -17 -15 -14 -16 -16 -18 -18 -20 -22 -23 -27 -28 -29 -30 -31 -31 -33 -34 -36 -36 -37 -36 -37 -36 -37 -33 -30 -29 -28 -27 -28 -28 -28 -27 -27 -25 -22 -23 -23 -22 -21 -24 -28 -30 -34 -34 -33 -33 -32 -33 -33 -34 -36 -36 -36 -35 -33 -33 -32 -31 -27 -26 -25 -25 -25 -27 -31 -31 -28 -25 -22 -20 -21 -21 -22 -24 -26 -27 -28 -29 -28 -30 -27 -26 -27 -29 -30 -30 -30 -26 -24 -23 -20 -18 -19 -20 -20 -17 -16 -16 -17 -14 -13 -12 -13 -15 -17 -19 -19 -21 -24 -25 -25 -26 -28 -28 -28 -26 -28 -30 -29 -28 -28 -28 -25 -22 -20 -19 -20 -18It is perfectly valid, if dull. It is supplied compressed as the sample file a1.asc.gz.
Compressed files
xdatplot will automatically uncompress compressed files, if they are
recognizable as such by their names. A file whose name ends in ".Z"
is uncompressed with the command "zcat". A file whose name ends in
".z" or ".gz" is uncompressed with the command "gunzip -qc". (The
".gz" extension is used by files compressed with gzip, the GNU
compression utility. It does a better job of compressing files than
the unix compress utility, and uncompression is faster, although
compression is slower. You can get gzip and gunzip free from prep.ai.mit.edu and other
anonymous FTP archives.) As supplied, xdatplot will recognize a
compressed ASCII file only if its name ends ".asc.Z", ".asc.z", or
".asc.gz". This can easily be changed.
Adding new formats: the easy way
OK, so your data acquistion sotware writes files that aren't text and
aren't ACCBIN, but you want xdatplot to understand them, anyway. How
do you do it?
There are two steps. First, you write a little program that translates a file in your format into an ACCBIN file. The program can be written in C, perl, FORTRAN (excuse me while I gag), Pascal (this is getting morbid) -- whatever you want. It can even be an absurd little shell script like this:
#!/bin/csh od -iv +500. | \ sed 's/^[0-9]* *([0-9]*) *//' | \ asc2bin -a 1This shell script works for files where the data start at position 500, and are stored as signed 16-bit integers. The sed command just strips off the annoying address labels that od puts at the front of each line. But what is this "asc2bin" thingy?
Glad you asked. asc2bin is a filter, supplied with xdatplot, that translates an ASCII file into an ACCBIN file. So you don't need to worry about understanding ACCBIN yourself: just turn your data into text, then let asc2bin take it from there.
Let's suppose this shell script is called my_convertor. Now you have to tell xdatplot about it. Create a file in your home directory called XDatplot that contains a line like this:
XDatplot*fileConvert: .myf: my_convertor < %s > %sThis tells xdatplot that whenever it sees a file whose name ends ".myf", it should try to convert it using the command "my_convertor infile.myf outfile". (By the way, it's not absolutely necessary that you put this information in a file called XDatplot in your home directory. You're just setting an X resource value -- see Customization for more information.
What if you don't want to name your files ".myf"? This will work:
XDatplot*fileConvert: : my_convertor < %s > %s(Yes, there are really two colons.) That tells xdatplot to try to convert anything but an ACCBIN file with my_convertor.
But wait, it doesn't work for compressed files! Can that be fixed? Yes, but you need to tell xdatplot how to uncompress them:
XDatplot*fileConvert: .myf.gz: gunzip -qc %s | my_convertor > %s\n\ .myf: my_convertor < %s > %sHere you've specified two conversions. xdatplot will use the first one that matches the filename. You can specify as many conversions as you like. You may need to be careful about the order, though, because a very general specification will shadow more specific ones (possibly including xdatplot's built-in conversions for text files). For instance, this will not work right:
XDatplot*fileConvert: : my_convertor < %s > %s\n\ .gz: gunzip -qc %s | my_convertor > %sThe first specification will match any filename, so the second will never be seen, and your compressed files won't be read. You need to put the ".gz" conversion first.
Wouldn't it be nice if xdatplot could look at the beginning of the file and figure out on that basis what convertor to use, instead opf being so hung up on filenames? This will probably be implemented in a future version. For now, the only way to do it is to write your own convertor that can recognize file types.
Tweaking the text file formats
You are now ready to learn a deep secret about xdatplot. xdatplot
actually can't read text files. Rather, it has built-in conversions
that tell it how to translate file types it can't understand. When
you set a file convertor as above, you're just adding new convertors
to the beginning of the list.
xdatplot uses asc2bin to convert textfiles. asc2bin has options that control how it does the translation. By default, the command "asc2bin -a 1 infile -o outfile" is used. "-a 1" just tells asc2bin to abort if 1 or more warning is generated. (Warnings happen when something that doesn't look like a parameter line or a valid number is read.)
Now, suppose you want to change how your text files are translated. Let's take a particular example. Suppose that all the numbers in your text files are between -1 and +1. asc2bin turns all numbers in its input into integers between -32767 and 32767, because ACCBIN format represents the data as 16-bit integers. So asc2bin, used with the default options, will round all the numbers in your text file to -1, 0, or 1, which of course will be disastrous.
You need to use the asc2bin scale option, which multiplies every input number by a scale factor before converting it to an integer. "asc2bin -s 30000 -a 1 infile -o outfile" will do the job perfectly. You do this by specifying a file convertor just as you did above:
XDatplot*fileConvert: .asc: asc2bin -s 30000 -a 1 %s -o %sxdatplot will now automatically use that command to convert any file whose name ends in ".asc".
For your information, here is xdatplot's built-in conversion list in full gory detail:
XDatplot*uncompress: \ .asc.Z: zcat %s | asc2bin -a 1 -o %s\n\ .asc.z: gunzip -qc %s | asc2bin -a 1 -o %s\n\ .asc.gz: gunzip -qc %s | asc2bin -a 1 -o %s\n\ .bin.Z: zcat %s > %s\n\ .bin.z: gunzip -qc %s > %s\n\ .bin.gz: gunzip -qc %s > %s\n\ .Z: zcat %s > %s\n\ .z: gunzip -qc %s > %s\n\ .gz: gunzip -qc %s > %s\n\ .asc: asc2bin -a 1 %s -o %s\n\ : asc2bin -a 1 %s -o %sThis list, by the way, is not really built-in. It is the uncompress resource, which you can change if you wish.
Adding new formats: the hard way
All this conversion has a down side: it's slow. For instance, asc2bin
takes 8 sec to convert a1.asc on my SPARCstation 2, even using the -i
option. Conversion from binary to binary is much faster if you don't
go through a text intermediate. But how can new binary formats be
added, that xdatplot can understand directly, without conversion?
At the moment, new native formats can be added only by editing the source code in Vt.c. There are two basic requirements:
There is a reason for these limitations. The magic number requirement is necessary if xdatplot is to be able to recognize different file formats. Magic numbers have become standard for most file formats. The requirement is reasonable and easy to fulfil. The second requirement: fixed data spacing, is more basic. It is fundamental to xdatplot's ease of operation that it be able to jump anywhere in the file at any time. It can't do that without a simple way of knowing where to find any datum. Thus there probably never will be a better strategy than conversion for variably spaced formats (such as text).
I am aware that some people might regard the creation of a brand-new format for this program strategically unsound, not to say idiotic. I would be very happy to make xdatplot understand a standard binary format for electrophysiology, if there is such a thing. I've never heard of one, but I'd like to.
xdp files
The File Save dialog allows you to save
files from xdatplot. The default extension for such files is .xdp.
These xdp files contain no data. Rather, they contain the name of the
datafile to which they refer. Therefore, even after a File Save in
xdatplot, you must keep the data file. The xdp file, however, can
contain new or changed information about the data file, such as a
changed comment, marks, or filters. Here is an xdp file:
! xdp-0.1 *dataFile: /home/eatworms/leon/src/xdatplot-0.1/a1.bin.gz *tUnits: sec *tMultiplier: NaN *tGain: 1 *vUnits: pA *vMultiplier: NaN *vGain: 0.002 *control*commentText.value: 8/10/93 20:51 *peakHeight: 20 *troughDepth: 20 *peakHalfWidth: 0.015 *troughHalfWidth: 0.015 *filterChain: hpfFilter 2 1e-05\n\ *color2Marks: e *color3Marks: r *color4Marks: dmv *markList: p 2247 t = 1.1235, V = 107.117, area = 0.704956, height = 76.5991\n\ t 2680 t = 1.34, V = -204.468, area = 0.936508, height = 162.811\n\ t 2771 t = 1.3855, V = -62.561, area = 0.133362, height = 21.9727\n\ p 6026 t = 3.013, V = 100.708, area = 0.736237, height = 74.7681\n\ t 6435 t = 3.2175, V = -210.266, area = 0.912323, height = 180.054\n\Most of the things in the file are pretty technical, but othes have obvious meanings. For instance, the line labeled "*tUnits" tells how to label the time axis. You may fearlessly edit xdp files with your favorite text editor to find out what the various things do. Nothing you put in the xdp file should crash xdatplot. (Let me know if you find an exception.)
Actually, if you are familiar with Xwindows, you will recognize this as an X resource file. In fact, xdp files are a good source of resource definitions for customizing xdatplot. Fiddle with xdatplot until you have the filtering you want, or the units are right, etc, save an xdp file, then plug the appropriate lines from the xdp file into your defaults file.
Leon Avery (leon@eatworms.swmed.edu)