Put a Glide in Your Stride and a Dip In Your Hip and Use a Mother Script!

Feb 12, 2023 @ 3:09 pm

Bash Shell I spent the last couple weeks writing a lot of shell code and I thought I’d share a technique I learned some time ago that makes write cron jobs much easier. This is written in Bash 4 for Linux.

The Mother Script

If you’re writing a bunch of scripts that are going to run out of cron, you may find they share some basic structural needs:

sending an email if something doesn’t work
logging output to a file
updating a database with their results
cleaning up any temp files
sharing common variable definitions and functions

You can satisfy that first bullet point by using the MAILTO directive in your crontab (e.g., MAILTO=you@example.com alone on a line). Any output (standard or error) will be sent via email to that address, though it’s rather crude (you can’t specify the subject, for example).

You can achieve all of those features by using a “mother” script. I’m not sure what the technical term is (I was listening to Parliament at the time), but it’s a script that does the following:

Sets up all the common environment variables
Points output to a log
Waits to see what your script does
Then cleans up, sends emails if appropriate, etc. automatically when your script is done

Here is a simplified version:

#!/bin/bash

_when_done() {
  rc=$?
  printf "JOB RESULT: ${rc}\n" >> ${LOGFILE}
  if [ $? -ne 0 ] ; then
    mailx -s "JOB FAILED: ${0} ${rc}" you@example.com < ${LOGFILE}
  fi
}

some_common_func() {
  # anything you want
}

# MAIN
export DBNAME="name"
export DBPASSWORD="secret"
export LOGFILE="/app/log/$(basename ${0}).$(date '+%Y%m%d').log"

trap _when_done EXIT

exec > ${LOGFILE} 2>&1

Let’s walk through this, starting where MAIN is marked. The variables DBNAME and DBPASSWORD are just example definitions. Any script that sources this mother script can use them. Same is true for LOGFILE, which will be something like:

your_script.sh.20230212.log

The “trap” command says “when I receive the EXIT signal, call the _when_done function”. A couple notes on this.

First, there is no “EXIT” signal in Linux. Bash provides this as a “pseudo signal” that means “whenever I am exiting”. Of course, if the KILL (9) signal is sent, that is untrappable, but every other signal (typically TERM) will be caught and the _when_done function executed.

Second, _when_done is called “_when_done” instead of “when_done” to reduce the chance of a namespace collision (the calling script might have a function called when_done). This is by convention rather than a language feature. Note some_common_func (any kind of function you want to share across scripts) is NOT underbarred because that’s intended for calling scripts to use, whereas _when_done is not.

The ‘exec’ call will put all output into the log file. It actually replaces the current running process with a copy of itself with stdout and stderr redirected.

When the calling script exits, _when_done is called. If the calling script exits normally, it notes the exit code in the log file and then sends mail if appropriate.

In Action

Here is a script that will succeed:

#!/bin/bash

. /app/mother.sh

printf "I am succeeding\n"
exit 0

I called this script mom_succeed.sh and after it runs, the following log is created:

# ls -l /app/log/mom_succeed.sh.20230212.log 
-rw-r--r-- 1 root root 30 Feb 12 11:00 /app/log/mom_succeed.sh.20230212.log
# cat log/mom_succeed.sh.20230212.log 
I am succeeding
JOB RESULT: 0
#

Now let’s try a similar mom_fail.sh:

# ls -l /app/log/mom_fail.sh.20230212.log
-rw-r--r-- 1 root root 26 Feb 12 11:01 /app/log/mom_fail.sh.20230212.log
# cat /app/log/mom_fail.sh.20230212.log
I will fail
JOB RESULT: 1
#

Practicalities

Here is one very practical example of the benefits of a mother script: managing tempfiles.

As anyone who writes shell scripts knows, calling mktemp and forgetting to clean it up is a very easy mistake to make. Every point at which your script might exit unexpectedly is a point where you might leave linger tempfiles.

Rather than write a custom trap and cleanup setup in each of your scripts, do it all in the mother, painlessly. Let’s add some code to the mother script:

#!/bin/bash

when_done() {
  rc=$?
  printf "JOB RESULT: ${rc}\n" >> ${LOGFILE}
  if [ $? -ne 0 ] ; then
    mailx -s "JOB FAILED: ${0} ${rc}" andrew@fabbro.org < ${LOGFILE}
  fi

  for tempfile in ${_cleanup} ; do
    rm -f ${tempfile}
    printf "removed tempfile: ${tempfile}\n"
    done
  }
}

export DBNAME="name"
export DBPASSWORD="secret"
export LOGFILE="/app/log/$(basename ${0}).$(date '+%Y%m%d').log"
export _cleanup=""

mother_mktemp() {
  _mother_mktemp=$(mktemp)
  _cleanup="${_cleanup} ${_mother_mktemp}"
}

trap when_done EXIT

exec > ${LOGFILE} 2>&1

The new code is in bold.

The function mother_mktemp() creates a temp file and adds it to a global cleanup list. Then when the _when_done function is called, all tempfiles are removed.

Here is an example of a script called mom_temp.sh:

#!/bin/bash 

. /app/mother.sh

printf "I am requesting three tempfiles...\n"
for i in 1 2 3 ; do
 mother_mktemp
 this_tempfile=${_mother_mktemp}
 printf "   #${i}: ${this_tempfile}\n"
done

printf "goodbye\n"
exit 0

Note that there are different ways to return values from functions but here I’m using a return variable (usually named after the name of the function). I learned this technique from Chris FA Johnson’s awesome book Shell Scripting Recipes and it is significantly faster than the other method of printing to stdout and capturing the result.

Execution:

# ./mom_temp.sh 
# cat /app/log/mom_temp.sh.20230212.log 
I am requesting three tempfiles...
#1: /tmp/tmp.JUAyNOKELJ
#2: /tmp/tmp.u7jVRUq6lv
#3: /tmp/tmp.iXa1JW6yQT
goodbye
removed tempfile: /tmp/tmp.JUAyNOKELJ
removed tempfile: /tmp/tmp.u7jVRUq6lv
removed tempfile: /tmp/tmp.iXa1JW6yQT

Need a Laugh? Read the Linux Kernel List's Foam-Mouthed Responses to Russian Programmers Banned from...

Nontechnical Nonsense: Rust Stirs Up a Storm of Drama in the Linux Kernel: Ted T'so Shouting, Mainta...

RedHat Prepares to Give Bootloaders the Boot

Irritation Removed: You Do Everything As Root And Hate Manually Removing Sudo From Copy-Paste Comman...

Has the Itanic Finally Hit Its Last Iceberg? It's Been Removed From the Linux Kernel After a Little...

LowEndBoxTV: Is Windows or Linux Faster on a Hetzner Dedicated Server?

Author
Recent Posts

raindog308

Dread Lord of LowEnd Content at LowEndBox

I'm raindog308, techno polymath and long-time LowEndTalk community Administrator. My technical interests include all things Unix, perl, python, golang, shell scripting, vintage operating systems such as MVS, and relational database systems such as Oracle, PostgreSQL, and MySQL.

I also enjoy using LEBs! I have an empire of little guys for VPNs, larger guys for apps, and big guys for databases and storage. When I'm not in front a screen I'm into German Shepherd dogs, quality knives, target shooting, theology, tabletop roleplaying games, and forest hiking.

I enjoy writing technical articles here on LowEndBox to help people get more out of their systems. You can find me on LowEndTalk @raindog308