LowEndBox - Cheap VPS, Hosting and Dedicated Server Deals

Put a Glide in Your Stride and a Dip In Your Hip and Use a Mother Script!

Bash ShellI spent the last couple weeks writing a lot of shell code and I thought I’d share a technique I learned some time ago that makes write cron jobs much easier.  This is written in Bash 4 for Linux.

The Mother Script

If you’re writing a bunch of scripts that are going to run out of cron, you may find they share some basic structural needs:

  • sending an email if something doesn’t work
  • logging output to a file
  • updating a database with their results
  • cleaning up any temp files
  • sharing common variable definitions and functions

You can satisfy that first bullet point by using the MAILTO directive in your crontab (e.g., MAILTO=you@example.com alone on a line).  Any output (standard or error) will be sent via email to that address, though it’s rather crude (you can’t specify the subject, for example).

You can achieve all of those features by using a “mother” script.  I’m not sure what the technical term is (I was listening to Parliament at the time), but it’s a script that does the following:

  • Sets up all the common environment variables
  • Points output to a log
  • Waits to see what your script does
  • Then cleans up, sends emails if appropriate, etc. automatically when your script is done

Here is a simplified version:

#!/bin/bash

_when_done() {
  rc=$?
  printf "JOB RESULT: ${rc}\n" >> ${LOGFILE}
  if [ $? -ne 0 ] ; then
    mailx -s "JOB FAILED: ${0} ${rc}" you@example.com < ${LOGFILE}
  fi
}

some_common_func() {
  # anything you want
}

# MAIN
export DBNAME="name"
export DBPASSWORD="secret"
export LOGFILE="/app/log/$(basename ${0}).$(date '+%Y%m%d').log"

trap _when_done EXIT

exec > ${LOGFILE} 2>&1

Let’s walk through this, starting where MAIN is marked.  The variables DBNAME and DBPASSWORD are just example definitions.  Any script that sources this mother script can use them.  Same is true for LOGFILE, which will be something like:

your_script.sh.20230212.log

The “trap” command says “when I receive the EXIT signal, call the _when_done function”.  A couple notes on this.

First, there is no “EXIT” signal in Linux.  Bash provides this as a “pseudo signal” that means “whenever I am exiting”.  Of course, if the KILL (9) signal is sent, that is untrappable, but every other signal (typically TERM) will be caught and the _when_done function executed.

Second, _when_done is called “_when_done” instead of “when_done” to reduce the chance of a namespace collision (the calling script might have a function called when_done).  This is by convention rather than a language feature.  Note some_common_func (any kind of function you want to share across scripts) is NOT underbarred because that’s intended for calling scripts to use, whereas _when_done is not.

The ‘exec’ call will put all output into the log file.  It actually replaces the current running process with a copy of itself with stdout and stderr redirected.

When the calling script exits, _when_done is called.  If the calling script exits normally, it notes the exit code in the log file and then sends mail if appropriate.

In Action

Here is a script that will succeed:

#!/bin/bash

. /app/mother.sh

printf "I am succeeding\n"
exit 0

I called this script mom_succeed.sh and after it runs, the following log is created:

# ls -l /app/log/mom_succeed.sh.20230212.log 
-rw-r--r-- 1 root root 30 Feb 12 11:00 /app/log/mom_succeed.sh.20230212.log
# cat log/mom_succeed.sh.20230212.log 
I am succeeding
JOB RESULT: 0
#

Now let’s try a similar mom_fail.sh:

# ls -l /app/log/mom_fail.sh.20230212.log
-rw-r--r-- 1 root root 26 Feb 12 11:01 /app/log/mom_fail.sh.20230212.log
# cat /app/log/mom_fail.sh.20230212.log
I will fail
JOB RESULT: 1
#

Practicalities

Here is one very practical example of the benefits of a mother script: managing tempfiles.

As anyone who writes shell scripts knows, calling mktemp and forgetting to clean it up is a very easy mistake to make.  Every point at which your script might exit unexpectedly is a point where you might leave linger tempfiles.

Rather than write a custom trap and cleanup setup in each of your scripts, do it all in the mother, painlessly.  Let’s add some code to the mother script:

#!/bin/bash

when_done() {
  rc=$?
  printf "JOB RESULT: ${rc}\n" >> ${LOGFILE}
  if [ $? -ne 0 ] ; then
    mailx -s "JOB FAILED: ${0} ${rc}" andrew@fabbro.org < ${LOGFILE}
  fi

  for tempfile in ${_cleanup} ; do
    rm -f ${tempfile}
    printf "removed tempfile: ${tempfile}\n"
    done
  }
}

export DBNAME="name"
export DBPASSWORD="secret"
export LOGFILE="/app/log/$(basename ${0}).$(date '+%Y%m%d').log"
export _cleanup=""

mother_mktemp() {
  _mother_mktemp=$(mktemp)
  _cleanup="${_cleanup} ${_mother_mktemp}"
}

trap when_done EXIT

exec > ${LOGFILE} 2>&1

The new code is in bold.

The function mother_mktemp() creates a temp file and adds it to a global cleanup list.  Then when the _when_done function is called, all tempfiles are removed.

Here is an example of a script called mom_temp.sh:

#!/bin/bash 

. /app/mother.sh

printf "I am requesting three tempfiles...\n"
for i in 1 2 3 ; do
 mother_mktemp
 this_tempfile=${_mother_mktemp}
 printf "   #${i}: ${this_tempfile}\n"
done
printf "goodbye\n"
exit 0

Note that there are different ways to return values from functions but here I’m using a return variable (usually named after the name of the function).  I learned this technique from  Chris FA Johnson’s awesome book Shell Scripting Recipes and it is significantly faster than the other method of printing to stdout and capturing the result.

Execution:

# ./mom_temp.sh 
# cat /app/log/mom_temp.sh.20230212.log 
I am requesting three tempfiles...
#1: /tmp/tmp.JUAyNOKELJ
#2: /tmp/tmp.u7jVRUq6lv
#3: /tmp/tmp.iXa1JW6yQT
goodbye
removed tempfile: /tmp/tmp.JUAyNOKELJ
removed tempfile: /tmp/tmp.u7jVRUq6lv
removed tempfile: /tmp/tmp.iXa1JW6yQT

 

raindog308

No Comments

    Leave a Reply

    Some notes on commenting on LowEndBox:

    • Do not use LowEndBox for support issues. Go to your hosting provider and issue a ticket there. Coming here saying "my VPS is down, what do I do?!" will only have your comments removed.
    • Akismet is used for spam detection. Some comments may be held temporarily for manual approval.
    • Use <pre>...</pre> to quote the output from your terminal/console, or consider using a pastebin service.

    Your email address will not be published. Required fields are marked *