Untitled Document

Chapter - 19 Advanced Shell Scripting

We have already seen the basic features of the shell - both as an interpreter and as a acripting language .But shell is more than an interpreter and scripting language . It is necessary that one should understand the environmental chnages that take place when the shell executes a program , especially a shell script .

Advanced knowledge of shell programming is needed by the system administrator who has to constantly devise scripts that monitor and correct system functioning .A detailed knowledge of the shell's subtle features is also necessary if you aspire to be an expert script writer .

 

Shell and Sub - Shell - 

When the shell executes a shell script , it first spawns a sub-shell , which in turn executes the commands in the script . When script execution is complete , the child shell withers away and returns control to the parent shell . You can also explicitly invoke a sub-shell to execute a shell script . The command representing the shell itself ( sh,ksh and bash ) can be used to read the statement in join.sh

   sh join.sh             Shell also accepts script name as argument
   sh < join.sh           Its standard inout can also be redirected 

Thus a shell script run with sh,ksh or bash need not have execute permission . This technique , however , is applicable for executing only shell scripts and ot executables . You certainly can't use

   sh < a.out 

Even though the shell accepts a script name as argument , we generally don't run shell scripts in the way shown above . We simply key in the script name from the shell prompt and it as an executable. In this case , the current shell uses a sub-shell of the same type to execute it .However if this script contains the interpreter line in this form :

   #!/usr/bin/ksh 

then, eventhough the loginshell may be bourne , it will use the Korn shell to execute the script . Specification of the interpreter line also helps us identify the shell the script is meant to use .

( ) and { } : Sub Shell or Current Shell -

The shell uses two types of operators to group commands . You must understand clearly the cosequences of using one group in preference to the order :

  • The ( ) Statements enclosed within parentheses are executed in a sub-shell
  • The { } Statements enclosed within curly braces are executed in the current shell only .

We have seen the first type to collectively redirect the standard output of two commands with a single redirection symbol in a manner similar to this

     ( a.sh ; b.sh ; c.sh ) > d.sh 

There are some applications which requires a set of commands to be run without spawning a child shell .To consider an example , lets use both grouping operators with the cd and pwd commands . Check you current directory and then change it with cd ;

   $ pwd
   /home/sharma
   $ ( cd progs ; pwd ) 
   /home/sharma/progs
   $ pwd 
   /home/sharma             Back to orignal directory 

Working from a sub-shell , cd changed the working directory ( one of the environmental parameters ) to /hom/sharma/progs . The present ( login shell ) can't adopt this change , so the orignal directory is back in place . The same command group - this time using the { } operators - tells a different story :

   $ pwd
   /home/sharma
   $ { cd progs ; pwd }
   /home/sharma/progs
   $ pwd 
   /home/sharma/progs             Directory change is now permanent 

The two commands have now been executed without spawning a shell ; no separate environment was created and the change of directory became permanent even after the execution of the command group . Note that we need to precede the closing brace with a ; if both { and } appear in the same line . An -often used sequence used in many shell scripts check the number of command line arguments and terminates the script with exit if the test fails . For example

   if [ $# -ne 3 ] ; then 
      echo " You have not entered 3 arguments "
      exit 3 
   fi

can be easily replaced with this sequence using curly braces :

   [ $# -ne 3 ] && { echo " You have not entered 3 arguments" ; exit 3 ; }

why can't we use () instead of { } here ? The exit statement can terminate a script only if it runs in the same shell that's running the script . This is the case when exit runs inside the { } , but not when it runs inside ( ). An exit inside ( ) will stop executing the remaining statements in the group , but that won't automatically terminate a script .

export : Exporting Shell Variables -

By default , the values stored in shell variables are local to the shell and are not passed on to a child shell . But the shell can also export these variables recursively to all child processes so that once defined , they are available globally .

Consider a simple script whcih displays the value of a variable x , assigns a new value to it and then displays the new value again :

   $ cat var.sh
   echo the value of x is $x 
   x=20                                                    Now change the value of x 
   echo The new value of x is $x    

First assign 10 to x at the prompt and then execute the script

   $ x=10 ; var.sh
   The value of x is                                       value of x is not visible in a sub-shell 
   The new value of x is 20                                                   
   $ echo $x                                               Value set inside the script doesn't affect value outside script 
   10

Because x is alocal variable in the login shell , its value can't be accessed by echo in the script , which is run in a sub-shell . To make x available globally , you need to use export statement before the script is executed .

   $ x=10 ; export x 
   $ var.sh                                        
   The new value of x is 10                        value in parent shell now visible here                                                  
   The new value of x is 20                                
   $ echo $x                                       value reset inside script ( child shell )is not available outside it ( parent shell)
   10

When x is exported , its assigned value ( 10 ) is available in the script . But when you sxport a variable , it has another important cosequence ; a reassignment ( x=20) made in the script ( a sub -shell ) is not seen in the parent shell which executed the script .

You must export the variables you define unless you have strong reason not to let sub-shell inherit their values . To know whether you have alrready done so, use export without arguments . It lists all environment variables ( which are already exported ) and user - defined variables ( like x ) that you have exported . The env command also lists exported variables .

Running a Script in Current Shell -

Variable assignment made in the shell's startuo file ( .profile or .bash_profile ) are always seen in the login shell . It's obvious that the profile executed by the login shell without creating a sub-shell . But How ? If you thought that the .profile was executed by grouping it with the curly braces :

   { .profile ; }

then you will be disappointed to see that there is no execute permission for the file :

   $ ls -l .profile 
   -rw-r--r--     1     sharma      group    727   Feb  27   23:02  .profile  

There's a special command used to execute a shell script without creating a sub - shell - the . ( dot ) command . You can make changes to .profile and execute it with the . command without requiring to log out and login again :

   . .profile
The dot command executes a script without using a sub - shell . It also doesm't require the script to have execute permission .

let : Computation - A second Look ( Korn and Bash )

We have seen expr command for computation , Korn and Bash come with a built in integer handling facility that totally dispenses the need to use expr . We can compute with the let statement which is used here both with and without quotes :

   $ let sum=150+140                    No whitespace after variable 
   $ let sum="150 + 140"                No whitespace after variable

If we use whitespace for better readability , then quote the expression . In either case , sum is assigned the result of the expression :

   $ echo $sum 
   290

lets see how let handles variables . First define three variables ; a single let does it

   $ let x=12 y=8 z=9
   $ let z=x+y+$z             $ not required by let 
   $ echo $z
   29

Since this computataional feature is built in , scripts run much faster than when used with expr . Later we will be using let in place of expr in one of our scripts .

A Second Form of Computing wih ( ( and ) ) The Korn shell and Bash use the ( ( ) ) operators that replace the let statement itself :

   $ x=20 y=25 z=10
   $ z=$((x+y + z ))         Whitespace is unimportant
   $ echo $z
   55
   $ z=$((z+1))
   $ echo $z
   56

POSIX recommends the use of ( ( ) ) rather than let , and this form is mot likely used as standard feature . It's easier to use too because single dollar can replcae multiple ones .

Arrays ( Korn and Bash )

Korn and Bash support one-dimensional arrays where the first element has the index 0. Here's how you set and evaluate the value of the third element of the array prompt :

   $ prompt[2]="Enter Your name : "
   $ echo ${prompt[2]}
   Enter Your name :

Note that evaluation is done with curly braces ,and prompt[2] is treated just like a variable . It however doesn't conflict with a variable prompt that you may also define in the same shell . For assigning a group of elements , use a space-delimited list having either of these two forms ;

   set -A month_arr 0 31 29 31 30 31 30 31 31 30 31 30 31     Korn Only 

   month_ar=( 0 31 29 31 30 31 30 31 31 30 31 30 3)1          Bash Only 

In either case , the array stores the number of days available in each of the 12 months . The first element had to be deliberately assigned to zero for obvious reason . Finding out the number of days in June is simple :

   $ echo ${month_arr[6]}
   30

Using the @ or * as subscript , you can display all elements of the array as well as the number of elements . The forms are similar except for the presence of the # in one

   $ echo ${month_arr[@]}
   0 31 29 31 30 31 30 31 31 30 31 30 31
   $ echo ${#month_arr[@]}                        Length Of Array 
   13

Can we use arrays to validate an entered date ? The next script , dateval.sh does that . It takes into account the leap year changes .

   #!/usr/bin/ksh
   # script : dateval.sh - variables a date field using an array 

   IFS="/"
   n="[0-9] [0-9]"
   set -A month_arr 0 31 29 31 30 31 31 30 31 30 31

   while echo "Enter a date :/c" ; do
    read value 
    case "$value" in 
          "") echo "No date entered " ; continue;;
    $n/$n/$n) set value 
     let rem="$3 % 4 "
     if [ $2 -gt 12 -o $2 -eq 0 ] ; then 
         echo " Illegal Month" ; continue
     else
         case "$value" in 
     29/02/??) [ $rem -gt 0 ] &&

               { echo "20$3 is not a leap year " ; continue ;} ;;
            *) [ $1 -gt ${month_arr[$2] } -o $1 -eq 0 ] &&
               { echo " Illegal day" ; continue ; } ;;
         esac
    fi ;;
  *) echo "Invalid date" ; continue ;;
 esac

 echo "$1/$2/43 " is a valid date
 done

String Handling ( Korn and Bash ) -

Korn and Bash have adequate string handling function , lets see some of the features .

Length of a String -

The length of a stringis easily found by preceding the variable name with a #. Consider the following example .

   $ name="mangesh pande"
   $ echo ${#name}
   13

Now we can use this expression with if statement to check length of a string . This built-in feature is not only easier to use but also comparatively faster ;

   if [ ${#name} -gt 15 ]; then                        Korn and Bash 

Extracting a String by Pattern Matching -

We can extract a substring using special pattern matching feature .These functions make use of two characters - # and % . The selectio is based on mnemonic considerations . # is used to match at the begining and % at the end , and both are used inside curly braces when evaluating a variable .

To remove the extension from a filename , previously you had to use an external command - basename . This time , we can use a variable's ${variable%pattern} format to do that :

   $ filename=test.txt
   $ echo #{test%txt}
   test.                             txt stripped off 

The % symbol after the variable name deletes the shortest string that matches the variable's contents at the end . Had therre been two %s instead of one , the expression would have matched longest one . Let's now use %% with wild card to extracr the hostname from an FQDN :

   $ fqdn="java.sun.com"
   $ echo ${fqdn%%.*}
     java

Now lets match delete the longest string that matches the patern */ , but at the begining .

   $ filename="/var/mail/henry"
   $ echo  { $filename##*/ }
   henry

This deletes the segment /var/mail - the longest pattern that matches the pattern */ at the begining. The pattern matching forms of Korn ad Bash are listed in below table

Syntax Function
${var#pat} Shortest segment that matches the pat at begining of $var
${var##pat} Longest segment that matches pat at begining of $var
${var%pat} Shortest segment that matches pat at end of $var
${var%%pat} Longest segment that matches pat at end of $var

Conditional Parameter Substitution -

To continue on the subject of variable evaluation , we can evaluate a variable depending on whether it has a null or defined value .This feature is known as parameter substitution and is available in the shell . The general synatx is

   $ { <var>:<opt> <stg> }

This time , the variable var is followed by a colon and any of the symbol +,-,=,? as . The symbol is followed by the string stg .The evaluation can be done in four ways .

The + option - Here , var evaluates to stg if it is defined and assigned a nonnull string. This feature cann be used to set a variable to the output of a command andd echo a message if the variable is nonnull :

   found=`ls`
   echo ${found:+"This directory is not empty"}

ls displays nothing if it finds no files , in which case the variable found is set to a null string . However, the message is echoed if ls finds at least one file .

The - Option Here , var is evaluated to stg if it is undefined or assigned a null string ( the opposite of + option ). You can use this feature in a program which prompts for a filename , and then uses a default value when user simply press [Enter].

   echo "Enter the filename : \c"
   read flname 
   fname=${flname:-emp.lst}                       Instead of using if [ -z $flname ]

If flname is null or is not set , it evaluates to the string emp.lst . The value of flname , however , still rremains null . this compact assignment dispenses with the need for an if statement .

The = Option This also works similarly except that it goes a step further and make the assignment to the variable that is evaluated . With th = option , you can use parameter substitution with a command without making the intermediate assignment :

   echo "Enter the filename :\c"
   read flname
   grep $pattern ${flname:=emp.lst}            flname is now assigned 

Note that in the last statement , the variable flname itself got assigned . This feature is also useful in intializing a variable that is used in the control command of aloop.

   x=1 : while [ $x -le 10 ]
      can now be written as 
  while [ ${x:=1} -le 10 ]

The ? option It evaluates the parameter if the variable is assigned and nonnull . Otherwise , it echoes a string and kills the shell . This is quite useful in terminating a script if the user fails to respond properly to shell directives :

   echo "enter the filename : \c"
   read flname
   grep $pattern ${flname:?"No filename entered"}

If no filename is entered here , the message NO filename entered is displayed. The script is also aborted without the use of an explicit exit command . Apart from the * operator the other operators can also be used with positional parameters . We can now easily set a variable to some default if the script is invoked without an argument :

   flname=${1:-emp.lst}      $1 is null if script invoked without argument 

We can now compress some of the earlier scripts to even shorther command sequences . We will see some of these applciations later .

Only the = option actually assigns a value to a variable ; the others merely control the way the variable is evaluated .

Creating Function in Shell -

A shell function is like any other function ; it executes a group of statements as a bunch , its also returns a value . This is construct is available in most modern shells wherewe can use them to condense important routines to short sequnces . The synatx is as follows :

   function_name() {
      statements
      return value                             Optional 
    }

The function definition is followed by ( ) , and the body is enclosed within curly braces . On invocation , the function executes all statements in the body . The return statement , when present , returns a value representing the success or failure of the function .

Lets consider a simple application . When viewing the listing of a large number of files in a directory , you are often compelled to use ls -l | more . This command sequence is an ideal candidate for a shell unction , which we will name ll .

   $ ll ( ) {                                Function defined in the current command line is available in current shell only 
   > ls -l $* | more 
   > }

Like shell scripts , shell functions also use command line arguments ( like $1,$2, etc . ). $*,"$@" and $# also retain their usual significance in functions. Even though we need the ( ) in the function definition , you must not use them when invoking the function with or without arguments :

   ll            Executes ls -l | more
   ll us3rd??    Executes ls -l ux3rd?? | more 

We have used the ll function more like a procedure , but a shell function can also return its exit status with the return statement . The exit status can be gathered from the shell parameter, $?

Generating a Filename from the system date

As system administrator , you will often need to maintain separates files for each day of a specific activity . These filenames can be derived from the system date , so you can easily identify a file pertaining to a certain day . It's a good idea to have this value echoed by a shell function dated_fname , which in turn gets it from the date output

   dated_fname () {
    set -- `date`
    year=`expr $6 : '..\(..\)'`
    echo "$2$3_$year"
 }

When you invoke the function , echo displays a string derived from the current date :

   $ dated_fname

   May20_16

The above string output generated by function can be use to create the file with this name - May20_16.dmp . Next we will create one library file and we will add these fuctions to maifunc.ksh - so that we can directly call these functions in our shell script .

Add following topics from Book

1 To continue or Not to continue

2.One complete program using function '

 

Untitled Document Scroll To Top Untitled Document