MGE System Utility Library - Full Internal Documentation  v1.5.1
Library of system utility functions.
AutoTools

Overview

This documentation covers some important aspects of the AutoTools project structure and build.

Bootstrapping The Project

A bash script called bootstrap.sh is provided in the project root directory. It is a helper script to build the project. Apart from the core configure and make it has options to run the analyzer, make check, run sparse, use debugging flags etc. Rather than duplicate documentation here, please see the acmbuild wiki. (acmbuild.sh is just a wrapper to bootstrap.sh and takes all the same arguments). The wiki can be found at:-

https://github.com/m-grant-prg/acmbuild/wiki

gnulib

The initial gnulib setup is performed by the command:-

gnulib-tool --import --source-base=src/prg/c/gen/lib \
--no-conditional-dependencies --no-libtool configmake

Files generated by the configmake modules are not tracked in git but the m4 macros in the m4 directory are. This allows people to work from a source tarball or a git clone without having to install gnulib and running gnulib-tool. This is described as option 3 on:-

https://www.gnu.org/software/gnulib/manual/html_node/VCS-Issues.html

The bootstrap.sh script has a -g option which will perform a:-

gnulib-tool --update    # If you have gnulib installed.

The gnulib files committed are updated by the repo owner from the latest version he has installed.

Temporary Headers and Libraries

Concept

The Temporary Header / Library concept is designed for use when an application (or library) is in concurrent development with a library on which it depends.

The library and headers which form the dependency can be copied here from an external project and linked against, thus testing the inter-action between the two.

Multiple libraries can be handled in this fashion with each library having it's own directory.

Header Components

Header Directory Structure

The header files should be located depending on how they are installed. So, if installed in includedir then the header will be in inc-tmp, if pkgincludedir then in a subdirectory named after the package. This is also how they are referenced in code, for example:-

#include <mge-portabilty.h>
vs
#include <libmgec/mge-portability.h>

The resulting structure would look something like:-

inc-tmp:
total 40
drwxr-xr-x 3 mgrantprg mgrantprg 4096 Sep 26 13:50 .
drwxr-xr-x 8 mgrantprg mgrantprg 4096 Aug 16 10:09 ..
-rw-r--r-- 1 mgrantprg mgrantprg 7 Sep 12 15:02 .gitignore
drwxr-xr-x 2 mgrantprg mgrantprg 4096 Sep 19 09:38 libmgec
-rw-r--r-- 1 mgrantprg mgrantprg 1470 Sep 12 15:02 Makefile.am
-rw-r--r-- 1 mgrantprg mgrantprg 16945 Sep 26 13:50 Makefile.in
inc-tmp/libmgec:
total 44
drwxr-xr-x 2 mgrantprg mgrantprg 4096 Sep 19 09:38 .
drwxr-xr-x 3 mgrantprg mgrantprg 4096 Sep 26 13:50 ..
-rw-r--r-- 1 mgrantprg mgrantprg 2361 Sep 16 14:54 dllist.h
-rw-r--r-- 1 mgrantprg mgrantprg 1447 Sep 16 14:54 libmgec.h
-rw-r--r-- 1 mgrantprg mgrantprg 3658 Sep 16 14:55 mge-bstree.h
-rw-r--r-- 1 mgrantprg mgrantprg 2469 Sep 16 14:56 mge-buffer.h
-rw-r--r-- 1 mgrantprg mgrantprg 2601 Sep 16 14:56 mge-errno.h
-rw-r--r-- 1 mgrantprg mgrantprg 1232 Sep 16 14:57 mge-memory.h
-rw-r--r-- 1 mgrantprg mgrantprg 2989 Sep 16 14:58 mge-message.h
-rw-r--r-- 1 mgrantprg mgrantprg 1948 Sep 16 14:58 mge-portability.h
-rw-r--r-- 1 mgrantprg mgrantprg 2571 Sep 16 15:19 sllist.h

Header Build

configure.ac provides the inc-tmp Makefile.am with the variable tmpheaders. This contains a space separated list of relative (to inc-tmp) paths to each temporary header file.

In addition, configure.ac checking of availability of such a header using AC_HEADER_CHECK is conditional on it not being present in inc-tmp.

Header Consumption

As mentioned above, the C include directives are the same for an installed package and for temporary headers.

Consumption is facilitated in each Makefile.am via the C pre-processor flags xxxxxx_CPPFLAGS. A typical example would be:-

if DEBUG
AM_CPPFLAGS = $(MG_DEBUG_CPPFLAGS)
else
AM_CPPFLAGS = $(MG_CPPFLAGS)
endif
AM_CPPFLAGS += -I$(srcdir)/../../inc-tmp -I$(srcdir)/../../inc
AM_CPPFLAGS += -I../../gen/inc -I../../gen/lib

Library Components

Library Directory Structure

The files for each library should be copied to their own directory under lib-tmp including the hidden .libs directory. The resulting directory structure would look something like:-

lib-tmp:
total 44
drwxr-xr-x 3 mgrantprg mgrantprg 4096 Sep 26 09:34 .
drwxr-xr-x 8 mgrantprg mgrantprg 4096 Aug 16 10:09 ..
drwxr-xr-x 8 mgrantprg mgrantprg 4096 Sep 19 15:30 libmgec
-rw-r--r-- 1 mgrantprg mgrantprg 4506 Aug 16 10:09 'lib-tmp usage.txt'
-rw-r--r-- 1 mgrantprg mgrantprg 1125 Aug 16 10:09 Makefile.am
-rw-r--r-- 1 mgrantprg mgrantprg 14486 Sep 26 09:17 Makefile.in
lib-tmp/libmgec:
total 100
drwxr-xr-x 8 mgrantprg mgrantprg 4096 Sep 19 15:30 .
drwxr-xr-x 3 mgrantprg mgrantprg 4096 Sep 26 09:34 ..
drwxr-xr-x 4 mgrantprg mgrantprg 4096 Sep 19 15:30 buf-msg
drwxr-xr-x 2 mgrantprg mgrantprg 4096 Sep 19 15:30 .deps
drwxr-xr-x 4 mgrantprg mgrantprg 4096 Sep 19 15:30 errors
-rw-r--r-- 1 mgrantprg mgrantprg 354 Sep 7 15:44 .gitignore
-rw-r--r-- 1 mgrantprg mgrantprg 931 Sep 19 15:24 libmgec.la
-rw-r--r-- 1 mgrantprg mgrantprg 322 Sep 19 15:24 libmgec_la-version.lo
-rw-r--r-- 1 mgrantprg mgrantprg 55384 Sep 19 15:24 libmgec_la-version.o
drwxr-xr-x 2 mgrantprg mgrantprg 4096 Sep 19 15:30 .libs
drwxr-xr-x 4 mgrantprg mgrantprg 4096 Sep 19 15:30 listsandsorts
drwxr-xr-x 4 mgrantprg mgrantprg 4096 Sep 19 15:30 memory
lib-tmp/libmgec/buf-msg:
total 200
drwxr-xr-x 4 mgrantprg mgrantprg 4096 Sep 19 15:30 .
drwxr-xr-x 8 mgrantprg mgrantprg 4096 Sep 19 15:30 ..
drwxr-xr-x 2 mgrantprg mgrantprg 4096 Sep 19 15:30 .deps
-rw-r--r-- 1 mgrantprg mgrantprg 0 Sep 19 15:24 .dirstamp
-rw-r--r-- 1 mgrantprg mgrantprg 327 Sep 19 15:24 libmgec_la-buffer.lo
-rw-r--r-- 1 mgrantprg mgrantprg 85776 Sep 19 15:24 libmgec_la-buffer.o
-rw-r--r-- 1 mgrantprg mgrantprg 330 Sep 19 15:24 libmgec_la-message.lo
-rw-r--r-- 1 mgrantprg mgrantprg 90944 Sep 19 15:24 libmgec_la-message.o
drwxr-xr-x 2 mgrantprg mgrantprg 4096 Sep 19 15:30 .libs
lib-tmp/libmgec/buf-msg/.deps:
total 24
drwxr-xr-x 2 mgrantprg mgrantprg 4096 Sep 19 15:30 .
drwxr-xr-x 4 mgrantprg mgrantprg 4096 Sep 19 15:30 ..
-rw-r--r-- 1 mgrantprg mgrantprg 0 Sep 19 15:24 .dirstamp
-rw-r--r-- 1 mgrantprg mgrantprg 6404 Sep 19 15:24 libmgec_la-buffer.Plo
-rw-r--r-- 1 mgrantprg mgrantprg 6943 Sep 19 15:24 libmgec_la-message.Plo
lib-tmp/libmgec/buf-msg/.libs:
total 184
drwxr-xr-x 2 mgrantprg mgrantprg 4096 Sep 19 15:30 .
drwxr-xr-x 4 mgrantprg mgrantprg 4096 Sep 19 15:30 ..
-rw-r--r-- 1 mgrantprg mgrantprg 85744 Sep 19 15:24 libmgec_la-buffer.o
-rw-r--r-- 1 mgrantprg mgrantprg 90920 Sep 19 15:24 libmgec_la-message.o
lib-tmp/libmgec/.deps:
total 12
drwxr-xr-x 2 mgrantprg mgrantprg 4096 Sep 19 15:30 .
drwxr-xr-x 8 mgrantprg mgrantprg 4096 Sep 19 15:30 ..
-rw-r--r-- 1 mgrantprg mgrantprg 2625 Sep 19 15:24 libmgec_la-version.Plo
lib-tmp/libmgec/errors:
total 188
drwxr-xr-x 4 mgrantprg mgrantprg 4096 Sep 19 15:30 .
drwxr-xr-x 8 mgrantprg mgrantprg 4096 Sep 19 15:30 ..
drwxr-xr-x 2 mgrantprg mgrantprg 4096 Sep 19 15:30 .deps
-rw-r--r-- 1 mgrantprg mgrantprg 0 Sep 19 15:24 .dirstamp
-rw-r--r-- 1 mgrantprg mgrantprg 323 Sep 19 15:24 libmgec_la-errno.lo
-rw-r--r-- 1 mgrantprg mgrantprg 60104 Sep 19 15:24 libmgec_la-errno.o
-rw-r--r-- 1 mgrantprg mgrantprg 323 Sep 19 15:24 libmgec_la-error.lo
-rw-r--r-- 1 mgrantprg mgrantprg 105648 Sep 19 15:24 libmgec_la-error.o
drwxr-xr-x 2 mgrantprg mgrantprg 4096 Sep 19 15:30 .libs
lib-tmp/libmgec/errors/.deps:
total 20
drwxr-xr-x 2 mgrantprg mgrantprg 4096 Sep 19 15:30 .
drwxr-xr-x 4 mgrantprg mgrantprg 4096 Sep 19 15:30 ..
-rw-r--r-- 1 mgrantprg mgrantprg 0 Sep 19 15:24 .dirstamp
-rw-r--r-- 1 mgrantprg mgrantprg 3948 Sep 19 15:24 libmgec_la-errno.Plo
-rw-r--r-- 1 mgrantprg mgrantprg 7346 Sep 19 15:24 libmgec_la-error.Plo
lib-tmp/libmgec/errors/.libs:
total 172
drwxr-xr-x 2 mgrantprg mgrantprg 4096 Sep 19 15:30 .
drwxr-xr-x 4 mgrantprg mgrantprg 4096 Sep 19 15:30 ..
-rw-r--r-- 1 mgrantprg mgrantprg 60072 Sep 19 15:24 libmgec_la-errno.o
-rw-r--r-- 1 mgrantprg mgrantprg 105616 Sep 19 15:24 libmgec_la-error.o
lib-tmp/libmgec/.libs:
total 1044
drwxr-xr-x 2 mgrantprg mgrantprg 4096 Sep 19 15:30 .
drwxr-xr-x 8 mgrantprg mgrantprg 4096 Sep 19 15:30 ..
-rw-r--r-- 1 mgrantprg mgrantprg 752278 Sep 19 15:24 libmgec.a
lrwxrwxrwx 1 mgrantprg mgrantprg 13 Sep 19 15:30 libmgec.la -> ../libmgec.la
-rw-r--r-- 1 mgrantprg mgrantprg 932 Sep 19 15:24 libmgec.lai
-rw-r--r-- 1 mgrantprg mgrantprg 55352 Sep 19 15:24 libmgec_la-version.o
lrwxrwxrwx 1 mgrantprg mgrantprg 16 Sep 19 15:30 libmgec.so -> libmgec.so.7.0.0
lrwxrwxrwx 1 mgrantprg mgrantprg 16 Sep 19 09:40 libmgec.so.2 -> libmgec.so.2.4.1
-rwxr-xr-x 1 mgrantprg mgrantprg 120168 Sep 19 09:36 libmgec.so.2.4.1
lrwxrwxrwx 1 mgrantprg mgrantprg 16 Sep 19 15:30 libmgec.so.7 -> libmgec.so.7.0.0
-rwxr-xr-x 1 mgrantprg mgrantprg 120168 Sep 19 15:24 libmgec.so.7.0.0
lib-tmp/libmgec/listsandsorts:
total 292
drwxr-xr-x 4 mgrantprg mgrantprg 4096 Sep 19 15:30 .
drwxr-xr-x 8 mgrantprg mgrantprg 4096 Sep 19 15:30 ..
drwxr-xr-x 2 mgrantprg mgrantprg 4096 Sep 19 15:30 .deps
-rw-r--r-- 1 mgrantprg mgrantprg 0 Sep 19 15:24 .dirstamp
-rw-r--r-- 1 mgrantprg mgrantprg 333 Sep 19 15:24 libmgec_la-bstree.lo
-rw-r--r-- 1 mgrantprg mgrantprg 97576 Sep 19 15:24 libmgec_la-bstree.o
-rw-r--r-- 1 mgrantprg mgrantprg 333 Sep 19 15:24 libmgec_la-dllist.lo
-rw-r--r-- 1 mgrantprg mgrantprg 84384 Sep 19 15:24 libmgec_la-dllist.o
-rw-r--r-- 1 mgrantprg mgrantprg 333 Sep 19 15:24 libmgec_la-sllist.lo
-rw-r--r-- 1 mgrantprg mgrantprg 85864 Sep 19 15:24 libmgec_la-sllist.o
drwxr-xr-x 2 mgrantprg mgrantprg 4096 Sep 19 15:30 .libs
lib-tmp/libmgec/listsandsorts/.deps:
total 32
drwxr-xr-x 2 mgrantprg mgrantprg 4096 Sep 19 15:30 .
drwxr-xr-x 4 mgrantprg mgrantprg 4096 Sep 19 15:30 ..
-rw-r--r-- 1 mgrantprg mgrantprg 0 Sep 19 15:24 .dirstamp
-rw-r--r-- 1 mgrantprg mgrantprg 6428 Sep 19 15:24 libmgec_la-bstree.Plo
-rw-r--r-- 1 mgrantprg mgrantprg 6286 Sep 19 15:24 libmgec_la-dllist.Plo
-rw-r--r-- 1 mgrantprg mgrantprg 6286 Sep 19 15:24 libmgec_la-sllist.Plo
lib-tmp/libmgec/listsandsorts/.libs:
total 272
drwxr-xr-x 2 mgrantprg mgrantprg 4096 Sep 19 15:30 .
drwxr-xr-x 4 mgrantprg mgrantprg 4096 Sep 19 15:30 ..
-rw-r--r-- 1 mgrantprg mgrantprg 97672 Sep 19 15:24 libmgec_la-bstree.o
-rw-r--r-- 1 mgrantprg mgrantprg 84408 Sep 19 15:24 libmgec_la-dllist.o
-rw-r--r-- 1 mgrantprg mgrantprg 85928 Sep 19 15:24 libmgec_la-sllist.o
lib-tmp/libmgec/memory:
total 104
drwxr-xr-x 4 mgrantprg mgrantprg 4096 Sep 19 15:30 .
drwxr-xr-x 8 mgrantprg mgrantprg 4096 Sep 19 15:30 ..
drwxr-xr-x 2 mgrantprg mgrantprg 4096 Sep 19 15:30 .deps
-rw-r--r-- 1 mgrantprg mgrantprg 0 Sep 19 15:24 .dirstamp
-rw-r--r-- 1 mgrantprg mgrantprg 326 Sep 19 15:24 libmgec_la-memory.lo
-rw-r--r-- 1 mgrantprg mgrantprg 85088 Sep 19 15:24 libmgec_la-memory.o
drwxr-xr-x 2 mgrantprg mgrantprg 4096 Sep 19 15:30 .libs
lib-tmp/libmgec/memory/.deps:
total 16
drwxr-xr-x 2 mgrantprg mgrantprg 4096 Sep 19 15:30 .
drwxr-xr-x 4 mgrantprg mgrantprg 4096 Sep 19 15:30 ..
-rw-r--r-- 1 mgrantprg mgrantprg 0 Sep 19 15:24 .dirstamp
-rw-r--r-- 1 mgrantprg mgrantprg 6517 Sep 19 15:24 libmgec_la-memory.Plo
lib-tmp/libmgec/memory/.libs:
total 92
drwxr-xr-x 2 mgrantprg mgrantprg 4096 Sep 19 15:30 .
drwxr-xr-x 4 mgrantprg mgrantprg 4096 Sep 19 15:30 ..
-rw-r--r-- 1 mgrantprg mgrantprg 85064 Sep 19 15:24 libmgec_la-memory.o

Library Build

configure.ac provides Makefile.am's with a variable tmplibraries. This variable contains a space separated list of the absolute paths of all libraries under lib-tmp.

In addition, configure.ac checks the availability of such libraries in strict order of lib-tmp, PKG_CHECK_MODULES and lastly AC_SEARCH_LIBS.

Library Consumption

To be able to LINK against any temporary libraries, the Makefile.am must have the following line:-

Usually:-
LDADD = $(tmplibraries)
or with the Product Option Variable can be:-
xxxxxxxxx_LDADD = $(tmplibraries)

N.B.

LDADD as opposed to xxxxxxxxx_LDFLAGS must be used as LDFLAGS does not necessarily set the search path variable LD_LIBRARY_PATH in the desired order, the standard location may precede our temporary library location in the search path.

Linking Verification

It can be useful to be able to prove which libraries are being used in a libtool environment. ldd may be the first thought however it can be quite misleading, as shown run against a project library project test program:-

mgrantprg@leonidas:~/SWDev/Projects/C/libmgesysutils/atbuild/src/prg/c/src/test-configfile/.libs$ ldd ./configfiletest
linux-vdso.so.1 (0x00007ffeecfcb000)
libmgesysutils.so.1 => /usr/lib/x86_64-linux-gnu/libmgesysutils.so.1 (0x00007f8ca17ca000)
libmgec.so.7 => not found
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8ca15f5000)
libmgec.so.2 => /usr/lib/x86_64-linux-gnu/libmgec.so.2 (0x00007f8ca15ee000)
/lib64/ld-linux-x86-64.so.2 (0x00007f8ca17ed000)
mgrantprg@leonidas:~/SWDev/Projects/C/libmgesysutils/atbuild/src/prg/c/src/test-configfile/.libs$

The answer is to switch on debugging. This produces voluminous output but near the end of one of the output files you will find the inits and finis for the libraries it is using. (See man 8 ld.so).

LD_DEBUG=all LD_DEBUG_OUTPUT=mg ./configfiletest
TL;DR
<snip>
10015: calling init: /home/mgrantprg/SWDev/Projects/C/libmgesysutils/src/prg/c/lib-tmp/libmgec/.libs/libmgec.so.7
10015:
10015:
10015: calling init: /home/mgrantprg/SWDev/Projects/C/libmgesysutils/atbuild/src/prg/c/src/libmgesysutils/.libs/libmgesysutils.so.1
10015:
10015:
10015: initialize program: /home/mgrantprg/SWDev/Projects/C/libmgesysutils/atbuild/src/prg/c/src/test-configfile/.libs/configfiletest
10015:
10015:
10015: transferring control: /home/mgrantprg/SWDev/Projects/C/libmgesysutils/atbuild/src/prg/c/src/test-configfile/.libs/configfiletest
10015:
10015:
10015: calling fini: /home/mgrantprg/SWDev/Projects/C/libmgesysutils/atbuild/src/prg/c/src/test-configfile/.libs/configfiletest [0]
10015:
10015:
10015: calling fini: /home/mgrantprg/SWDev/Projects/C/libmgesysutils/atbuild/src/prg/c/src/libmgesysutils/.libs/libmgesysutils.so.1 [0]
10015:
10015:
10015: calling fini: /home/mgrantprg/SWDev/Projects/C/libmgesysutils/src/prg/c/lib-tmp/libmgec/.libs/libmgec.so.7 [0]
</snip>

Simplify the search by:-

mgrantprg@leonidas:~/SWDev/Projects/C/libmgesysutils/atbuild/src/prg/c/src/test-configfile$ grep -hw 'init\|fini' mg.* | grep 'mge'
42577: calling init: /home/mgrantprg/SWDev/Projects/C/libmgesysutils/src/prg/c/lib-tmp/libmgec/.libs/libmgec.so.7
42577: calling init: /home/mgrantprg/SWDev/Projects/C/libmgesysutils/atbuild/src/prg/c/src/libmgesysutils/.libs/libmgesysutils.so.1
42577: calling fini: /home/mgrantprg/SWDev/Projects/C/libmgesysutils/atbuild/src/prg/c/src/test-configfile/.libs/configfiletest [0]
42577: calling fini: /home/mgrantprg/SWDev/Projects/C/libmgesysutils/atbuild/src/prg/c/src/libmgesysutils/.libs/libmgesysutils.so.1 [0]
42577: calling fini: /home/mgrantprg/SWDev/Projects/C/libmgesysutils/src/prg/c/lib-tmp/libmgec/.libs/libmgec.so.7 [0]
mgrantprg@leonidas:~/SWDev/Projects/C/libmgesysutils/atbuild/src/prg/c/src/test-configfile$

So, for libraries go to the .libs directory and use ldd ./library.so and for executables use the debugging shown above.

Linking Internal Libraries

Take the scenario where you have written a test program for a project library. Now the previous stable release may be installed on your machine. You do not want to link against that library but the one in development. To achieve this place the following in the test program Makefile.am:-

LDADD = ../libmgesysutils/libmgesysutils.la

Installation Considerations

Installation instructions can be found in the README file at:-

https://github.com/m-grant-prg/libmgesysutils

and on the wiki:-

https://github.com/m-grant-prg/libmgesysutils/wiki

Making Libraries Available

Having installed shared libraries it is necessary to run ldconfig to refresh links and cache. This cannot be automatically included in the installation as it will cause downstream package builds to fail when trying to access files (/etc) outside their chroot. It is easier to ask that this be run manually after 'make install'.

doc-base Documentation

This package produces documentation designed to be registered with scrollkeeper and dochelp. The problem arises when it is required to have an AT installation and a package manager installation residing in parallel. In the case of doc-base this would mean installing 2 sets of documentation with the same docid. Needless to say doc-base does not like this.

The solution is not to automatically install the doc-base documents unless specifically asked to do so with the –enable-atonly argument supplied to configure.