Introduction to Shell

Shell gives you a quick and powerful way of having a programmable User Interface, where you can quickly write short programs to automate repetitive tasks.
I have had a slowly-changing collection of my own personal shell scripts moving with me from place to place and machine to machine for over 20 years.

Shell program = an interpreted program (i.e. not compiled). Also known as a "Shell script" or "batch file" of UNIX commands.
Like .BAT files in DOS/Windows.

Shell program can have any extension (typically no extension). Edit a file, put some UNIX commands in it, perhaps strung together with some logic:

if condition
then
 rm f1
else
 rm f2
fi

$ chmod +x file

$ file
$ file &

What shells are installed?

See what is installed:

ls -l /bin/*sh

-rwxr-xr-x 1 root root 663704 2008-06-06 21:04 /bin/bash
lrwxrwxrwx 1 root root      4 2008-09-11 12:47 /bin/csh -> tcsh
lrwxrwxrwx 1 root root     16 2008-09-11 12:48 /bin/ksh -> /lib/ast/bin/ksh
-rwxr-xr-x 1 root root 705552 2008-06-06 21:44 /bin/sash
lrwxrwxrwx 1 root root      4 2008-09-11 12:42 /bin/sh -> bash
-rwxr-xr-x 1 root root 334380 2008-06-06 20:48 /bin/tcsh
-rwxr-xr-x 1 root root 569692 2008-06-07 09:18 /bin/zsh

-r-xr-xr-x   1 root     bin       578964 Mar  2  2002 /bin/bash
-r-xr-xr-x   2 root     bin       159332 Apr  6  2002 /bin/csh
-r-xr-xr-x  17 root     bin          134 Apr  6  2002 /bin/hash
-r-xr-xr-x   4 root     root       95488 Apr  7  2002 /bin/jsh
-r-xr-xr-x   3 root     bin       201044 Mar 14  2003 /bin/ksh
-r-xr-xr-x   2 root     bin       159332 Apr  6  2002 /bin/pfcsh
-r-xr-xr-x   3 root     bin       201044 Mar 14  2003 /bin/pfksh
-r-xr-xr-x   4 root     root       95488 Apr  7  2002 /bin/pfsh
lrwxrwxrwx   1 root     root           5 Nov  7  2002 /bin/remsh -> ./rsh
-r-xr-xr-x   3 root     bin       201044 Mar 14  2003 /bin/rksh
-r-sr-xr-x   1 root     bin         9176 Apr  6  2002 /bin/rsh
-r-xr-xr-x   4 root     root       95488 Apr  7  2002 /bin/sh
-r-xr-xr-x   1 root     bin       737948 Apr  7  2002 /bin/ssh
-r-xr-xr-x   1 root     bin       358848 Mar  2  2002 /bin/tcsh
-r-xr-xr-x   1 root     bin       474736 Mar  2  2002 /bin/zsh

To start using one as the command-line, just type its name. e.g.

ksh

• Comparison of shells

The shebang line

The first line of a shell script should state which shell it is to run in, using syntax like:
#!/bin/shellname
This is called a "shebang" line.

For example:

• #!/bin/sh
  • Bourne shell sh
  • See sample program shtest.
  • Environment variables:
    • echo $HOME
    • echo $PATH

• #!/bin/csh
  • C Shell csh
  • Has C-like syntax with aliases (text substitutions analogous to #define macros) and functions (like C functions, recursion allowed).
  • See sample program cshtest.
  • Environment variables have slightly different syntax (see especially path):
    • echo $home
    • echo $path

• #!/usr/local/bin/perl  
  • Perl source

• #!/usr/local/bin/php  
  • PHP source

• #!/usr/local/bin/rexx  
  • Rexx source

Can have endless number of interpreters of source text. Can invent your own.

• Shebang

If no shebang line

But you cannot depend on this - exactly what happens depends on each system.
Your shell scripts may not be portable to different UNIX-family systems.

So it is good practice to define the shell in the first line.

Exercise - Get the test programs above to work, and see your environment variables:

1. shtest
   • on DCU Linux, runs in bash, because sh is bash (see above)
   • Some environment variables:

     HOME=/users/gdf1/mhtest09
     OSTYPE=linux
     PATH=/users/gdf1/mhtest09/bin:/usr/local/bin:/usr/bin:/bin:....
     PWD=/users/gdf1/mhtest09/bin
     SHELL=/bin/bash
     USER=mhtest09
     USERNAME=mhtest09
     http_proxy=http://wwwproxy.computing.dcu.ie:8000
     no_proxy='localhost, 127.0.0.1, dcu.ie'
     

   • To change them:
     PATH=$PATH:$HOME/bin:.

2. cshtest
   • on DCU Linux, runs in tcsh, because csh is tcsh (see above)
   • Some environment variables:

     cwd     /users/gdf1/mhtest09
     home    /users/gdf1/mhtest09
     path    (/users/gdf1/mhtest09/bin /usr/local/bin /usr/bin /bin ....)
     shell   /bin/tcsh
     user    mhtest09
     

   • To change them:
     set path = ( . $home/bin /bin /usr/local/bin .... )

Question

if test $1 = ""

Environment variables

env
printenv
set - may display shell functions too

Environment variables in parent and child processes

cd (change directory) in Shell prog only has effect within that prog.
In general, environment variables changed within Shell script (such as "cwd" or "PWD" - current working directory) are only changed for the duration of that script.

They are not even changed by default for children (scripts called by that script).
You need to "export" them to make them available to child processes:

export VAR

will make VAR available to a script called by this script, and any scripts called by that script, etc., without any further exporting needed. Only a single export statement is needed (in the top parent script).

You cannot, though, make changes in a script that take effect in the parent process. To write a "script" that changes the current process's environment variables you don't use a Shell program but rather write an alias, which is a command-line text substitution.

Note that environment variables are by default available to functions within the same Shell script without exporting.

Useful things for Shell programs (or "Shell scripts")

Remember pipes and redirection

UNIX separates "ordinary output" (standard output, stdout, 1>, >)
from "error output" (standard error, stderr, 2>)

prog > output 2> errors


> /dev/null		to get rid of some unwanted output
			e.g. error/warning messages from compilation/search
			(redirects it into a non-existent file)



#                       comment

$*           all arguments to the Shell program
$1           1st argument, etc.
$0		name of prog
$#		no. of args

shift		shift args leftwards 
		this is useful if you want to remove some of the 
		 first args, then "shift" a couple of times,
		 and then do "for i in $*" with the remaining args
		e.g. grep (switches) (string) file1 ... filen

exit		exit the Shell script
exit 0		exit with a return code that other progs can query

$?		return code of last prog executed
		e.g. quiet grep: grep > /dev/null and check $?
		though grep may have -q (quiet) option anyway

strings and echo

echo                    print something on screen, followed by new line
echo -n                 print with no new line

printf                  print with no new line
printf "\n"             print with new line

echo "string"
echo 'string'

	It is useful to have 2 choices for quotes.
	If using one for something else, surround with the other.
	e.g. To search for single quote in file:

	grep ' file        syntax error (why?)
	grep "'" file      surround with quotes and it works
	grep '"' file      to search for the double quote itself




The 2 quotes are not exactly equal:

echo "--$HOME--"       --/users/group/humphrys--
echo '--$HOME--'       --$HOME--
echo '--'$HOME'--'     --/users/group/humphrys--

Characters

echo '\0nn'       echo char by octal numeric code (on csh)
echo '\0nnn'       
echo -e (string)  on bash

echo "\0115"      echo "M"
echo "\0275"      echo "½"

ASCII chars
ISO/IEC 8859
To build list of chars

* (wildcards)

*                       all files
.*                      all hidden files

 "Hidden" in the sense that tools like ls won't list them by default.
 You can write your own tools of course to always list them by default.
 Not actually hidden in the security sense.

 Done for convenience not strict security, like write-protecting your own files
 (which is itself a kind of security since it stops some accidents).
 Doing things to  *  like  rm *  won't affect the  .*  files.
 Like the way  C:\Windows\  won't let you see the files first time.

 In fact  .*  files are probably only in your home directory,
 and your user files are strictly in sub-directories
 so the 2 never meet.
 Like how you strictly separate the files in  C:\Windows\
 from those in  C:\My Documents\

Q. Even on a 1-user system, why separate OS files from user files?

echo *                  echo all files
echo f*                 all files beginning with f
echo */*                files in next layer
*/*/*                   etc.

grep string *

# grep does not understand *

# but that's fine because grep does not actually RECEIVE *
# what happens is:
# the shell EXPANDS * to a list of files and passes these to grep
# so grep actually receives:

grep string f1 f2 .. fn

x=`echo *`
echo $x

Q. Why have the shell interpret "*"? Why not just pass "*" as argument to progs?

Environment variables

Like global vars for all programs.
Note any environment vars that are declared within a program are local to that program only.

var=value               set environment variable
echo var                print the string "var"
echo $var               print value of environment variable
set                     show all environment variables

echo $home              get into the habit of using these
                        instead of the actual hard-coded values,
                        - makes scripts more portable
echo path is $path
echo $user
echo `hostname`
echo `arch`             on DCU Linux you get i686
                        on DCU Solaris you get sun4

set path = ( $home/bin/`arch` ... )

date

date                         looks like: "Tue Feb 17 16:28:33 GMT 2009"
CURRENTDATE=`date`           remember backquotes
echo $CURRENTDATE             

date "+%b %e"                looks like: "Jan 21" (see "man date")
CURRENTDATE=`date "+%b %e"`    
echo $CURRENTDATE             

1. We want a random temporary filename:

   filename="/tmp/random.`date +%H%M%S`.txt"
   

   Perhaps not random enough. e.g. 2 clients accessing server at same second. A totally unique, random temporary name can be got from   $$   which is the process ID of this invocation of the shell script:

   filename=/tmp/random.$$.txt
   

2. Or want a different seed each time for random-number generator.

set | grep -i random
echo $RANDOM  
set | grep -i random
echo $RANDOM            
echo $RANDOM            

Sample Shell program - rmifexists

e.g. Want to be able to repeatedly run:
rmifexists *.bak
and not get error message if no *.bak files found.

for i in $* do if test -f $i then rm $i fi done

• in fact, rm -f will do the job with no warnings
• rm

• rm -r (recursive rm)
• Don't type rm -rf /

Exercise

Why not:

        if test -f $*
        then
         rm $*
        fi

How to pass information to a program

In general, how to pass information to a program:

1. Set environment variables. All programs can read these.

2. Command-line arguments.
   Program icon - Properties.

3. Pipe to the program, where the program expects to read from standard input.

4. Configuration files.
   See   .* files in home directory on UNIX.
   e.g. See:
   • $HOME/.mozilla/firefox/id/prefs.js
   Equivalents exist on Windows. e.g. See:
   • C:\Program Files\Mozilla Firefox\defaults\pref\firefox.js
   Edit - Preferences
   Tools - Options

5. Talk to the program while it is running. How to do this depends on the OS.

Shell Tutorials

• shelldorado.com
  • Shell and UNIX tutorials

• Shell tutorial
• Shell tutorial

• Shell sample scripts