Next: 11. Software Organization, Up: CFHT Data Acquisition Previous: 9. Programming Languages   Contents

Subsections

# 10. Development Tools

## 10.1 GNU make and Make.Common

GNU make is freely available and works well on all of our Unix platforms, so we actually replace the vendor-provided make to avoid confusion and problems with Makefile compatibility. (The vendor-provided make with Linux systems is already the GNU version.) There are Info Pages and Man Pages installed on our Web site.

Make.Common is a Makefile intended to be included by all project directory Makefiles. It is designed to make the project Makefiles as simple as possible. To use it, include it twice, using the include directive, in your regular Makefile:


# Example Makefile
# Include Make.Common to set all make variables to defaults:
include ../Make.Common

# ... Project specific variable definitions and targets go here ...

# Include Make.Common again to define the targets for make:
include ../Make.Common
# End of Makefile.


(See Make.Common itself for more examples.)

You can only have one compiled executable or library per subdirectory and the executable or library must be named the same as the directory it is in. You can, however, create a directory structure as deep as is needed to group related programs and libraries together. Just remember to create a symbolic link to Make.Common in the level above whenever you make a new directory tree. (See how /cfht/src/pegasus/cli/ is set up for some examples.)

Shell scripts must all be named *.sh and appear in a subdirectory anywhere in your tree called ./scripts/. There can be multiple *.sh in a single ./scripts/ directory. When these are installed in the /cfht/bin/ directory, the .sh is stripped off. So they are invoked as their basename, i.e., without the .sh.

Similarly other non-compiled files like configuration files, parameter files, and bitmap files are kept together in a directory called ./confs/ and will get copied to /cfht/conf/ during a  make install'' if needed.

### 10.1.1 Standard Targets Provided by Make.Common

make momma
This target is only valid in the top-level source directory. It is intended to build and install all subdirectories. We recommend typing make first (without any explicit target names) to verify that files will be installed in the correct locations. At the top-level, make will only print a list of target directories.
make preinstall
This exists to make complete releases build properly. It is used mainly to install header files. For example, if library A and library B both need each other's header files in order to compile, this is the way to deal with this chicken-and-egg problem. The make momma target will take care of calling this at the right time.
make world
If you have a project that is divided into other subdirectories and you run  make world'' in the project directory, all subdirectories will be built with  make all install''.  make world'' in one of the subdirectories will change to the upper directory and run  make all install'' from there. So  make world'' can be used anywhere within a sub-project to build the whole thing.
make all
In a project directory  make all'' traverses into all the subdirectories, where it builds either the library or the executable for that subdirectory. Executables are not copied from /cfht/obs/project/* to /cfht/bin/, but some libraries may be installed /cfht/lib/ if another executable requires them.
make install
In a project directory, this runs  make install'' in all the subdirectories. In a subdirectory, it installs (and builds, if it is out of date) the library or executable in /cfht/lib/ or /cfht/bin/ respectively.
make clean
Removes the directory /cfht/obs/project/, where object files for the current project are kept.
make depend
The Makefile must not be checked into RCS. Running  make depend'' will append or replace a list of C source file dependencies (generated by  gcc -MM'') to the Makefile.
make execlist
Prints a list of all the programs with their runnable names (shell scripts will have the .sh removed) from this directory level down. This is the list of things that gets versioned and installed in /cfht/bin/ during a  make install''.
make titles
Generates emacs-compile-buffer-parsable output of the results of a search of your code for "===" and "%%%" markers. Files in each subdirectory are expected to be summarized in a file called "Index". (See /cfht/src/pegasus/cli/Index, for an example.) Copyright headers are automatically inserted into known file types, and DOS newlines (if found) are changed back to unix newlines. The modification time-stamp on the file is not altered.
make tar
Makes a tar file snapshot of the current directory tree, excluding RCS subdirectories and emacs backup files.

### 10.1.2 Ways to turn on other options from the command line

make TARGET=VXSPARC
Cross compilation! Assuming there's a block already in Make.Common for TARGET=VXSPARC, this will select a different compiler, linker, and obj.CROSSCC/ directory to place the binaries. No modifications needed to your Makefile. Add new cross-configurations to Make.Common.

make /cfht/obs/program/program-pure
Assuming  $(EXECNAME)$(EXECNAME)-pure: $(OBJS)...'' appears in the Makefile instead of $(EXECNAME): $(OBJS)'', then this will produce a purified copy of the program. make EXTRA_CFLAGS="-pg" If you have already used EXTRA_CFLAGS in the Makefile, this will override. The example above causes the program to be compiled with gprof profiling support. A  make clean must be done before the first pass to force all objects to be rebuild with the -pg option. make EXTRA_CFLAGS="-DDEBUG -DDEBUGDEBUG" Use this notation to include any arbitrary defines, for a particular invocation of make only. Don't worry about the fact that these extra flags get passed even at link time; gcc will simply ignore them where-ever they do not apply. make EXTRA_CFLAGS="-g" ????? Don't do this, and don't try to remove the -g' from the default options either. GCC can handle doing optimizations while also putting in debugging symbols. The installed versions are always stripped, however, so run the one in the project's ./obj/ directory if debugging symbols are required. It is even possible to analyze cores dumped by one of the stripped versions if you rebuild the exact same executable with symbols again (or if you still have the unstripped version in the ./obj/ directory. Here's an example of how gdb could be invoked on one of these cores:  gdb --symbols=/cfht/obs/dumper/dumper --exec=/cfht/bin/dumper --core=.../core  ### 10.1.3 Make Variables VERSION Normally this is the date, in the form YYMMDD, but you can override this with something like 1.1 if you prefer. CONFS Contains *.def *.par *.bm *.xbm *.rdb *.xrdb by default. Any matches which exist in a subdirectory called ./conf/ are copied to /cfht/conf/ during  make install''. Do not include the ./conf/ in the filenames or filename patterns here. SCRIPTS Contains *.sh by default. To limit the list to only certain .sh files within your scripts/ directory, list only the names of those files, but they must currently be .sh files. SUBDIRS If you define this, only SCRIPTS and CONFS are installed at this directory level, and then each of the directories named in SUBDIRS are traversed. Do not list scripts'' or conf'' in this list. They are handled automatically. Also, all built programs must live at the leaves'' of the directory tree, so there cannot be a program at a directory level which also defines SUBDIRS. SRCS Contains *.c and *.cc by default. OBJS (the list of *.o is generated from automatically SRCS. If you override SRCS, OBJS will be adjusted accordingly. HDRS Contains *.h and *.hh by default. If you have internal-only header files, then you can either place those in a subdirectory called ./internal/ and #include them with internal/foo.h, OR explicitly override HDRS in your Makefile. EXTRA_CFLAGS This variable is normally empty. Anything in it will be passed as extra options during the compile and link stages. If there are things that do not belong in CCDEFS or CCINCS (below) they can go here. CCWARN and WERROR The default for CCWARN is "-Wall -Wstrict-prototypes". The default for WERROR can be defined to -Werror in a separately included top-level Makefile, Make.Local. Once a project has compiles cleanly without any warnings, a good way to keep it that way is to make warnings equal errors for that project:  CCWARN +=$(WERROR)


If there is a Make.Local in the toplevel src/ directory which defines "WERROR = -Werror", as we have in our source tree at CFHT, then these projects will fail to build when warnings are introduced.

CCDEFS
This contains defines to be passed to the C source code. It can be added to using  CCDEFS+=...'' in the Makefile. Be sure to use += instead of just = because any -Dxxxx from Make.Common itself are important to the build process. Exactly one of the following will be defined, if a known system type was detected:

-D SUNOS
SunOS 4.1.x
-D HPUX
HP-UX 9 or 10
-D LINUX
Linux 1.x or 2.x
-D CYGWIN32
Cygnus win32 on Windows NT

So you can do things like this in the C code:


#include <sys/socket.h>

#ifdef SUNOS
/* Old Suns lack prototypes for these... */
int socket(int domain, int type, int protocol);
int listen(int s, int backlog);
#endif

. . .


It may also be useful to know the date of the most recently modified source file, the compilation date, and the version number from the Changes file inside the C source. All three of these can be added by putting the following in the Makefile:


CCDEFS+=$(VERSIONDEFS)  And then the C program would see the following three defines:  int main(int argc, const char* argv[]) { printf("Program FOO version %s %.2f %s (compiled %s)\n" VERSION, SOURCEDATE, BUILDDATE); . . . }  The above example assumes the version number should have two digits displayed after the decimal. Note that VERSION, SOURCEDATE, and BUILDDATE are only available if the project Makefile adds VERSIONSDEFS to CCDEFS. In addition, the following -D's may come from Make.Common under the following conditions:  -DNO_CFHTLOG - If /tmp/pipes/syslog.np was not found at compile-time. -DUSE_EPICS - If this machine is one on which we want EPICS channel access.  On our primary development system, HP-UX 10, only -DHPUX usually appears, and possibly -DUSE_EPICS. CCINCS Use "CCINCS+=-I..." to append to the list of directories for include files. CCLIBS Use "CCLIBS+=-L..." to append to the list of directories to search for libraries. CCLINK Set this to a list of "-llibname" options for linking. For example, if your program requires symbols from "libm.a", use  CCLINK += -lm  CCINCSX11, CCLIBSX11, CCLINKX11 These contain extra include paths, library paths, and library names, respectively, to be used with X-applications. Since vendors put these in different places, having these variables defined makes it easy within projects to indicate a program needs X by just saying:  CCINCS +=$(CCINCSX11)
CCLIBS += $(CCLIBSX11) CCLINK +=$(CCLINKX11)


Any application that uses unix networking stuff (sockets, hostname lookup, etc.) should add the following to their Makefile:


CCLINK += $(CCLINKNET)  Because some platforms (namely new Sun Solaris) need to add "-lsocket" and "-lnsl" in the link stage for the proper network routines. Other platforms may need a "-lresolv" here also for hostname lookup. Simply using CCLINKNET takes care of whatever might be needed here. ### 10.1.4 Variables Useful in the Makefile HOSTNAME Contains the name of the host that the project is currently being built on, as returned by the hostname command. TARGET This is the equivalent of the "SUNOS", "SOLARIS", "HPUX" symbols defined for C programs, for use within the Makefile. If you wanted certain projects to be built only on a certain architecture, you could use this variable. It contains the name of the operating system, as returned by "uname -s" followed by a dash (-) and the major revision of the OS. For SunOS, this is either a 4 or a 5, and for HP it is either an A (version 9) or a B (version 10). For example, to have certain projects build themselves only on old Suns:  SUBDIRS = libdet cdma chmem deti detio lu trafficoff ifeq ($(TARGET),SunOS-4)
SUBDIRS += detserver rdmem wrmem
endif


NO_CFHTLOG
If /tmp/pipes/syslog.np exists on the machine, this variable contains nothing. If /tmp/pipes/syslog.np is missing, this will contain "-DNO_CFHTLOG", and it will be passed to your C programs as well. If you want to do something differently depending on whether cfht_log facilities exist, you could do something like this in the Makefile, if you only need libcfht when cfht_log is used:


ifeq ($(NO_CFHTLOG),) CCLINK += -lcfht endif  And something like this in the C source code:  #ifdef NO_CFHTLOG syslog(syslog_type, message); #else cfht_log(CFHT_MAIN, cfht_log_type, message); #endif  ### 10.1.5 Variables That Depend on Local Installation Make.Common assumes that gcc is available. At CFHT, the following variables are set correctly for our installation, but at another site where gcc is not available (at least as a symbolic link) in /usr/local/bin, or in a case where gcc was not compiled to also use the Gnu linker, you may need to tweak some variables in Make.Common itself. (For all the variables mentioned previously, you override them in the individual project Makefiles, but for site installation stuff, you'll have to modify Make.Common). DIR_GNU If gcc is in /usr/local/bin or /usr/bin, there is no need to change this. Otherwise, you should set this to the path where gcc' can be found. INSTALL This must be set to the name of a BSD compatible install program. If you don't have GNU install on your system and you have a compatible install program, you can try putting it here (with full path). Otherwise it is recommended to download fileutils-X.YY.tar.gz'' from ftp.gnu.org or a mirror and install it. Be sure to configure it with  ./configure -program-prefix=g'' so it doesn't use names that conflict with the vendor-provided file utilities on your system. Although I wouldn't recommend it, it is possible to go through Make.Common and search for all$(INSTALL)'s and replace them with regular "cp" or "mkdir" commands (depending on whether it is installing a file or a directory.) GNU fileutils are not difficult to install, so try that first.

The default setting for CCLINK under Solaris may be problematic if you do not have a gcc that uses the GNU linker. Search for "-Wl,-rpath,". This garbage is needed to ensure that shared libraries will be found even if a user has an incorrect LD_LIBRARY_PATH or if the program needs to run without a complete environment (e.g. from a cgi script.) I think the equivalent of "-rpath" for the Sun linker is "-R", but you can comment out the entire CCLINK setting as long as you have the proper search paths in your LD_LIBRARY_PATH and/or the program is statically linked with any libraries that are in non-standard places.

## 10.5 Documentation

### 10.5.1 Web Manuals

Each document should have some kind of Web accessible version. A format which can be keyword-searched by standard search engines is also valuable. While many search engines can now scan PDF files, HTML is a much more widespread format, with built-in support in every Web browser client application. So HTML, or something which gets converted to HTML at the Web server end on the fly, is the most desirable format for the Web.

### 10.5.2 Printable Manuals

Since HTML Web pages do not contain page numbers, and usually also contain graphics that are already rendered at a low resolution (on the order of 100dpi for a typical monitor) they are generally not suitable for printing at 600+ dpi. A link to a hardcopy in either PostScript or PDF (preferably both) should be provided on the Web page.

### 10.5.3 Word Processors

MS Word or Frame files are not acceptable as distributable, printable copies. Many word processing applications have a save-to-HTML option, and also can create output for a PostScript printer or generate PDF format.

These give mixed results, but as long as the results are decent for both the HTML and printable versions, any of these are acceptable.

### 10.5.4 XFig, LATEX , LATEX2HTML , ps2pdf

Writing documentation can be a little like processing astronomy data (though less interesting). We typically come up with a combination of tools and recipes that work well. This section describes the particular recipe used to generate this document. It produces reasonably good results, and has the advantage that everything is in plain text source which can be generated, preprocessed, and re-used in the automated fashions to which programmers and astronomers are accustomed. There is no mandate to use this particular method.

Figures are generated with XFig, a simple 2D drawing program. They are saved as Encapsulated PostScript with an appropriate scaling factor. Both XFig and LATEX handle a variety of image formats, but a non bitmapped format like EPS gives the nicest results. (GIF, BMP, and JPEG are all bitmapped formats.)

The document itself is written as a LATEX article. The document has this basic structure:


\documentclass{article}
\usepackage{fullpage}
\usepackage{times}
\usepackage{html}
\usepackage{epsfig}

\title{DOCUMENT TITLE}
\date{DATE HERE}
\begin{document}
\maketitle
\begin{abstract}
. . . Abstract here . . .
\end{abstract}
\tableofcontents
\listoffigures
\section{FIRST SECTION}
. . .
\section{SECTION SECTION}
. . .
\end{document}


• Without the fullpage package, LATEX generates output with fairly large margins, intended for book printing and later trimming.
• The times package causes LATEX to use a standard PostScript font, eventually resulting in smaller and higher quality PostScript and PDF.
• The html package is part of the LATEX2HTML translator and allows, among other things, hyperlinks in the LATEX source:


• The epsfig package allows figures from XFig (or any other package which can export EPS) to be included in the following way:
• 
\begin{figure}[ht!]
\begin{center}
\caption{CAPTION FOR FIGURE GOES HERE}
\label{optional_label_for_references}
\epsfig{file=YOURFILE.eps}
\end{center}
\end{figure}


No matter what, you cannot assume that LATEX will place the figure at the exact location where these commands appear. Often, it will create a separate page and put all the figures together. If this is undesirable, you can tune this behavior a little with the following commands:


\renewcommand{\topfraction}{0.99}
\renewcommand{\textfraction}{0.1}
\renewcommand{\floatpagefraction}{0.99}


The LATEX source itself can be edited as raw mark-up in a text editor (similar to editing raw HTML) or a package like LyX can be used for this job. Given the .eps Encapsulated PostScript files and the tex file with the document, it is then possible to generate all the output formats in one step with a simple Makefile and our friend GNU make:


# -------- Makefile for a LaTeX document. ---------------

DOCUMENT=YOURDOC
FIGURES=FIGURE1.eps FIGURE2.eps FIGURE3.eps
L2HOPTIONS=-antialias -no_antialias_text -split +1 \
-show_section_numbers -long_titles 4

# ------- The rest of the Makefile is generic. ----------

all:	$(DOCUMENT).ps$(DOCUMENT).pdf $(DOCUMENT)$(DOCUMENT):	$(DOCUMENT).ps .latex2html-init latex2html$(L2HOPTIONS) $(DOCUMENT)$(DOCUMENT).pdf: $(DOCUMENT).ps ps2pdf$< $@$(DOCUMENT).ps:	$(DOCUMENT).tex$(FIGURES)
latex $(DOCUMENT).tex latex$(DOCUMENT).tex
dvips \$(DOCUMENT).dvi -o


The Makefile will automatically regenerate all formats (HTML, PS, and PDF) with the single command make'' whenever a figure or the document have been updated. To generate the PDF, it uses ps2pdf, and to generate the HTML it uses LATEX2HTML. The latter has many features which can be controlled from the LATEX source. Check the manual for more information.

Next: 11. Software Organization, Up: CFHT Data Acquisition Previous: 9. Programming Languages   Contents
Sidik Isani
2016-09-30