% Topics in Linux Shell Scripting % Ian! D. Allen -- -- [www.idallen.com](http://www.idallen.com/) % 7pm Thursday December 2 2021 Ian! D. Allen: Who am I? ======================== - BA Psychology 1980 (Waterloo) - MMath Computer Science 1985 (Waterloo) - Three years as head sysadmin of the National Capital Free-Net - one sysadmin, 50,000 users in the password file! - 21 years as professor of Computer Studies, mostly intro to Linux shell - last teaching page for using Linux: Why a shell? ============ - to make it possible and easy for humans to find and run commands - many (most?) shells do not do math very well - shell syntax is kept short to minimize typing, e.g. rather than: FILE *f = fopen("filename","w"); /* ...check return status and maybe print error msg... */ int ret = fputs("Hello World!\n",f); /* ...check return status and maybe print error msg... */ ret = fclose(f); /* ...check return status and maybe print error msg... */ Using the shell, we write simply: $ echo "Hello World!" >filename Running commands ================ - shells pass individual pieces of text to commands as separate "arguments" - special "meta" characters interpreted by the shell to do neat things - quote characters and backslashes to hide and turn off meta meaning - GLOB patterns for file names: $ rm tmp* - spaces to separate arguments $ rm one two three - angle brackets to redirect input output: $ date >file - "OR" bar is a "pipe" symbol to connect commands: $ ls dir | wc - semi-colon allows multiple commands: $ date ; ls ; who Keeping pathnames short using a current directory ================================================= - Unix/Linux maintain a "current directory" and the shell uses it - /etc/passwd is the file "passwd" in the /etc/ directory - just "passwd" is the file "passwd" in the "current directory", whatever that is Finding commands - $PATH ======================== - commands are programs stored in files; type the name to execute the file - most commands are separate files; some commands are built-in to the shell - e.g. changing current directory must be built-in to the shell - to keep command names short, rather than typing the absolute paths of file names, use $PATH as a list of places to look for command names - first match in PATH wins, even if multiple matches - you can have your own directory of commands, usually named $HOME/bin/ - your current directory is not automatically searched for commands - do NOT put "." (your current directory) in your PATH - see the mystery/ directory Shell command history - keep lots of it ======================================= - remembers everything I've ever done (or as much as I choose) - my current history extends back to April 2009 - has about 800,000 command lines in it (many duplicates) - 800,000 commands in .bash_history is only 26MB (text is small) - I only have to look up a command line once in the manual - next time I need it, I get it by searching my history with ^R - to protect the history file from truncation or overwriting: - run (as root): chattr +a .bash_history - If you have a large BASH history file, it's much faster to load the whole thing than to load only half of it. (To load half of it, BASH parses and reads the whole thing but has to throw half of it away.) Shell aliases ============= - aliases for common options: - alias cp='cp -i -p' - alias mv='mv -i' - but aliases are not usually loaded in remote shells: - e.g. no alias used in $ ssh localhost cp foo bar - better to write small shell wrapper programs for your bin/ directory - but don't alias rm to 'rm -i' - your muscle memory will quickly render the -i useless - only use -i for *infrequent* operations that you rarely confirm - aliases to further abbreviate commands - alias a='alias' - alias b='popd' - alias pd='pushd' - alias d='dirs -v' - alias h='history' - alias j='jobs -l' - alias l='less' - aliases hide your frequent typing errors: alias gerp=grep ; alias mial=mail - aliases for commands on other O/S: alias type='less' ; alias dir='ls' Shell redirection follies ========================= - redirection syntax is removed from the command line before the command runs - echo a b c # echo has three arguments - echo a b c >file # echo still has three arguments - redirection syntax may appear anywhere in the command line - $ echo a b c d >foo $ echo a b >foo c d $ echo >foo a b c d $ >foo echo a b c d # "reverse Polish command line" - redirection is done first, before the command is run - cp /etc/paswd foo ; wc foo ; sort foo >foo ; wc foo $ cp /etc/passwd foo ; cat .bashrc foo > foo cat: foo: input file is output file $ cat foo .bashrc > foo - no warning Unlike the awful C Shells, Bourne shells use real parsers, so this works: for arg do wc "$arg" done >outfile Scripts can specify their interpreters ====================================== Shebang ("#!") lines for scripts: - #!/bin/sh -u - #!/bin/cat - #!/bin/wc - #!/bin/rm - see the shebang directory for examples Shell scripts can include embedded Perl, PHP, or AWK scripts ============================================================ - examples/perl.sh - examples/sendsms.php - examples/cpuusechecklinux.sh - examples/mbox2maildir.sh Better script practices ======================= Code it well now so that you don't get surprised later. Unexpected GLOB expansions can happen years after you wrote the script. - do not use C shells to write scripts; too many bugs - set PATH at the start of your script so you get the right programs - set umask so you get the right file permissions - set LOCALE if you dare to sort or wc anything - after 30 years of ASCII-only Unix, sort order changed with i18n - see - check the error return codes of commands in your scripts and exit on fail - cd "$newdir" || exit $? - cp "$old" "$new" || exit $? - quote all uses of variable expansions and all strings with GLOB characters - "wc $file" works fine until $file has a blank or GLOB char in it - "grep ***Error*** file" works fine until someone does "touch Error" - send error messages to stderr, not stdout (use: echo 1>&2 "$0: Error....") - error messages must use $0; do not hard-code the script name - error messages say what was expected and show was received: - echo 1>&2 "$0: Expecting one file name, found $# ($*)" Fun with ASCII Art shell scripts ================================ - see the fun/ directory for some ASCII Art shell scripts