I spent the last couple weeks writing a lot of shell code and I thought I’d share a technique I learned some time ago that makes write cron jobs much easier. This is written in Bash 4 for Linux.
The Mother Script
If you’re writing a bunch of scripts that are going to run out of cron, you may find they share some basic structural needs:
- sending an email if something doesn’t work
- logging output to a file
- updating a database with their results
- cleaning up any temp files
- sharing common variable definitions and functions
You can satisfy that first bullet point by using the MAILTO directive in your crontab (e.g., MAILTO=you@example.com alone on a line). Any output (standard or error) will be sent via email to that address, though it’s rather crude (you can’t specify the subject, for example).
You can achieve all of those features by using a “mother” script. I’m not sure what the technical term is (I was listening to Parliament at the time), but it’s a script that does the following:
- Sets up all the common environment variables
- Points output to a log
- Waits to see what your script does
- Then cleans up, sends emails if appropriate, etc. automatically when your script is done
Here is a simplified version:
#!/bin/bash _when_done() { rc=$? printf "JOB RESULT: ${rc}\n" >> ${LOGFILE} if [ $? -ne 0 ] ; then mailx -s "JOB FAILED: ${0} ${rc}" you@example.com < ${LOGFILE} fi } some_common_func() { # anything you want } # MAIN export DBNAME="name" export DBPASSWORD="secret" export LOGFILE="/app/log/$(basename ${0}).$(date '+%Y%m%d').log" trap _when_done EXIT exec > ${LOGFILE} 2>&1
Let’s walk through this, starting where MAIN is marked. The variables DBNAME and DBPASSWORD are just example definitions. Any script that sources this mother script can use them. Same is true for LOGFILE, which will be something like:
your_script.sh.20230212.log
The “trap” command says “when I receive the EXIT signal, call the _when_done function”. A couple notes on this.
First, there is no “EXIT” signal in Linux. Bash provides this as a “pseudo signal” that means “whenever I am exiting”. Of course, if the KILL (9) signal is sent, that is untrappable, but every other signal (typically TERM) will be caught and the _when_done function executed.
Second, _when_done is called “_when_done” instead of “when_done” to reduce the chance of a namespace collision (the calling script might have a function called when_done). This is by convention rather than a language feature. Note some_common_func (any kind of function you want to share across scripts) is NOT underbarred because that’s intended for calling scripts to use, whereas _when_done is not.
The ‘exec’ call will put all output into the log file. It actually replaces the current running process with a copy of itself with stdout and stderr redirected.
When the calling script exits, _when_done is called. If the calling script exits normally, it notes the exit code in the log file and then sends mail if appropriate.
In Action
Here is a script that will succeed:
#!/bin/bash . /app/mother.sh printf "I am succeeding\n" exit 0
I called this script mom_succeed.sh and after it runs, the following log is created:
# ls -l /app/log/mom_succeed.sh.20230212.log -rw-r--r-- 1 root root 30 Feb 12 11:00 /app/log/mom_succeed.sh.20230212.log # cat log/mom_succeed.sh.20230212.log I am succeeding JOB RESULT: 0 #
Now let’s try a similar mom_fail.sh:
# ls -l /app/log/mom_fail.sh.20230212.log -rw-r--r-- 1 root root 26 Feb 12 11:01 /app/log/mom_fail.sh.20230212.log # cat /app/log/mom_fail.sh.20230212.log I will fail JOB RESULT: 1 #
Practicalities
Here is one very practical example of the benefits of a mother script: managing tempfiles.
As anyone who writes shell scripts knows, calling mktemp and forgetting to clean it up is a very easy mistake to make. Every point at which your script might exit unexpectedly is a point where you might leave linger tempfiles.
Rather than write a custom trap and cleanup setup in each of your scripts, do it all in the mother, painlessly. Let’s add some code to the mother script:
#!/bin/bash when_done() { rc=$? printf "JOB RESULT: ${rc}\n" >> ${LOGFILE} if [ $? -ne 0 ] ; then mailx -s "JOB FAILED: ${0} ${rc}" andrew@fabbro.org < ${LOGFILE} fi for tempfile in ${_cleanup} ; do rm -f ${tempfile} printf "removed tempfile: ${tempfile}\n" done } } export DBNAME="name" export DBPASSWORD="secret" export LOGFILE="/app/log/$(basename ${0}).$(date '+%Y%m%d').log" export _cleanup="" mother_mktemp() { _mother_mktemp=$(mktemp) _cleanup="${_cleanup} ${_mother_mktemp}" } trap when_done EXIT exec > ${LOGFILE} 2>&1
The new code is in bold.
The function mother_mktemp() creates a temp file and adds it to a global cleanup list. Then when the _when_done function is called, all tempfiles are removed.
Here is an example of a script called mom_temp.sh:
#!/bin/bash . /app/mother.sh printf "I am requesting three tempfiles...\n" for i in 1 2 3 ; do mother_mktemp this_tempfile=${_mother_mktemp} printf " #${i}: ${this_tempfile}\n" done
printf "goodbye\n" exit 0
Note that there are different ways to return values from functions but here I’m using a return variable (usually named after the name of the function). I learned this technique from Chris FA Johnson’s awesome book Shell Scripting Recipes and it is significantly faster than the other method of printing to stdout and capturing the result.
Execution:
# ./mom_temp.sh # cat /app/log/mom_temp.sh.20230212.log I am requesting three tempfiles... #1: /tmp/tmp.JUAyNOKELJ #2: /tmp/tmp.u7jVRUq6lv #3: /tmp/tmp.iXa1JW6yQT goodbye removed tempfile: /tmp/tmp.JUAyNOKELJ removed tempfile: /tmp/tmp.u7jVRUq6lv removed tempfile: /tmp/tmp.iXa1JW6yQT
Related Posts:
- eWallHost: 4GB VPS for Only $4.99/Month in Finland or Germany! - September 7, 2024
- Hostneva: Cheap Shared Hosting on Enterprise-Grade Fabric Around the World, Starting at Only $6/YEAR! - September 6, 2024
- Hudson Valley Host: Yearly VPS Offers That Make a 2GB VPS Cost Only $1.50/Month! - September 5, 2024
Leave a Reply