YODL V1.14

Chapter 4: Technical information

The following information describes the YODL package from the point of the system administrator. Issues such as the installation of the package are addressed here.

4.1: Obtaining and installing YODL

The YODL program and the distributed macro package can be obtained at the ftp site ftp.icce.rug.nl in the directory /pub/unix. Look for the file yodl-1.14.tar.gz, which is a gzipped archive containing all sources, documentation and macro files. The version string (1.14 in this example) may have been upgraded by the time you read this; so always get the archive with the highest version number.

If you have an older yodl source tree, remove it first. Then uncompress and untar it in a `sources' directory, e.g., in /usr/local/src. This creates a subdirectory yodl-X.YY/ under which all necessary files are located.

Please note: YODL has evolved from a previous program dom, which is no longer supported or maintained. If you have a version of the predecessor program dom, then issue the following commands:

rm /usr/local/bin/dom /usr/local/bin/dom2* rm /var/man/cat3/yodl.3.gz rm -r /usr/local/lib/dom

Also note: Pre-1.03 versions of YOLD had a command yodl2less, which is now obsolete. This command has been replaced with yodl2manless and yodl2msless. If you have a pre-1.03 installation, run the command

rm /usr/local/bin/yodl2less

4.1.1: Configuring the yodl program

Once you unpack the archive, edit the Makefile in the yodl-X.YY/ subdirectory. At the top you will find a number of macros that may need adaptation (e.g., BINDIR is where make install puts the program). You can also set compiler options in this file.

Next, edit the file config.h. A number of relevant macros for the inner workings of yodl can be defined here. If you're unsure about the macros, leave them to their defaults.

4.1.2: Installing the yodl program

Once configured, type make install to build and install the program. The executable, which is built as src/yodl is created and copied to a system-wide program directory. The macro package from lib/yodl and divers post-processors are also placed in a public directory, which is /usr/local/lib/yodl by default (you can change this directory name in the Makefile too). Furthermore, a number of shell scripts (drivers of the yodl program) are copied to your programs directory.

The make install command only installs necessary executables and macrofiles. Use make installman to create and install manpages which go under /usr/man/man*, or use make installcat to create and install formatted manpages which go under /var/man/cat*. Available manual pages include descriptions of the executables, an abbreviated list of macros, and a description of the manpage document type. Optionally, you can install the whole YODL manual under /usr/man/man5/yodl.5 with make installmanual, or you can install the manual in preformatted form under /usr/man/cat5/yodl.5.gz using make installcatual. Note that all make commands that install documentation, also implicitely make install. This is not a bug! A working YODL set is required to create and install the documentation.

If you want yodl to create full documentation about itself, type make latexdoc (for LaTeX-style documentation) or make htmldoc (for HTML-style documentation). The documentation is written to doc/yodl.tex or doc/yodl.html respectively. You can also make flat ASCII documentation via the ms converter with

cd doc yodl2msless yodl > /tmp/yodl.txt

(Note that yodl.html already comes with the distribution, just start your favorite HTML browser on doc/yodl.html. Similarly, ASCII docs are provided in gzipped format as doc/yodl.txt.gz.)

If you can neither process LaTeX nor HTML nor ms documentation, try make txtdoc to produce a flat ASCII file yodl/yodl.txt (but then, if you can only view plain ASCII, why get YODL in the first place?).

4.1.2.1: Prerequisites for the installation

To successfully build and install the YODL package, the following tools must be present.

If you have problems building YODL on your particular system, please mail me the problem description (and, if you have it, the solution) -- I will adapt YODL for a wider variety of Unix blands.

4.2: Internal workings of the YODL package

This section describes the internal workings of the yodl package, for those who are interested (and of course, for the maintainer).

4.2.1: A single-pass interpreter

The yodl program is a single-pass interpreter of its input files. The program does not build an internal data representation of the text and commands that it encounters; therefore, all actions must be completed in one pass. The basic way by which yodl does this, is by reading input, taking actions and possibly `pushing back' information on the input.

This is best illustrated with an example. Given the code,

DEFINESYMBOL(sym) IFDEF(sym) (Symbol sym is defined! DEFINEMACRO(testmac)(0)(Testmac now stands for one thing.) ) (Symbol sym is not defined! DEFINEMACRO(testmac)(0)(Testmac now stands for another thing.) ) testmac()

yodl will take the following actions:

The most complex part of the program is therefore the tokenizer (lexical analyzer): that part is responsible for matching tokens such as identifiers in the input, and for pushing back text. The pushing back is implemented as a relocatable buffer, which is expanded as text is pushed into it and which shrinks as the lexical analyzer processes input. I already achieved about 100% speed gain by optimizing the tokenizer, but this implementation can still be somewhat of a CPU hog. (But hey, premature optimzation is the root of all evil, says D.E. Knuth. So why optimize, as long as you don't know whether you're being premature about it?)

The lexical analyzer is furthermore responsible for the line continuation feature (see section 2.2.2) and for the SUBST mechanism (see section 2.3.30). These mechanisms are also implemented with the push-back method.

4.2.2: Macros of the converters

The macro files for all converters are (by default) installed to a directory /usr/local/lib/yodl. This is a good place to look if you want to see how a converter works, or if you want to write a new converter.

The place to start looking is the file shared.yo, which is used by all converters. This file defines most commands that are `global' in the sense that all converters implement them: e.g., bf, em and so on. The implementation is per macro as follows, illustrated for e.g., bf (the actual implementation may be slightly different):

DEFINEMACRO(bf)(1)(\ whenlatex(latexcommand({\bf )ARG1+latexcommand(}))\ whenhtml(htmlcommand(<strong>)ARG1+htmlcommand(</strong>))\ whenman(mancommand(\fB)ARG1+mancommand(\fP))\ whensgml(sgmlcommand(<bf>)ARG1+sgmlcommand(</bf>))\ whentxt(ARG1))

The macro expands to a set of commands that are active per output format, using latexcommand, htmlcommand etc..

The file shared.yo is included in its turn by tex.yo, html.yo and so on. These separate files are very short; their function is to:

4.2.2.1: Adding a new macro

This section describes how to add a new user-defined macro. For the sake of simplicity, a macro fname(name) is defined to typeset a filename. The macro maps to em() in LaTeX output or to bf() in other formats.

The new macro is now in place. A new make job should install it all, and if necessary, make the appropriate new documentation files.

After this, please kindly mail me your addition to the YODL package. I will include it (if it's worth while), stating full credits too.