While example scripts

Topics:

while1
while2
while3
while4
while5

while1:

#!/bin/bash

while read -r x y etc; do
  if [ -z "$y" ]; then
    echo "line too short" >&2
  elif [ -n "$etc" ]; then
    echo "line too long, unexpected $etc" >&2
  fi
  echo "$x, $y"
done

All the commands in the do block are executed once for every line with x, y and etc assigned to the first, second and remainder tokens in the file.

testing$ cat badfile
1 1.8
2 3.2
3 
4 12.6
5 31.5 32 
6 60.5
testing$ ./while1 <badfile >fig1.data
line too short
line too long, unexpected: 32
testing$ cat fig1.data
1, 1.8
2, 3.2
3, 
4, 12.6
5, 31.5
6, 60.5

The while1 command reads all the lines badfile and writes to fig1.data. The sample file, badfile, has two errors, one too short and one two long. Thee error message appear on the termnal, since these messages are writed to FD 2 (STDERR).

while2:

#!/bin/bash

let lineNo=0
while read -r x y etc; do
  let lineNo+=1
  if [ "$x" -a ! "$y" ]; then
    echo "line $lineNo too short" >&2
    errCode=1
  elif [ "$etc" ]; then
    echo "line $lineNo too long, unexpected $etc" >&2
  fi
  echo $x${y:+, }$y
done
[ ! "$errCode" ]

To make the error and warning messages more informative we have added two variables.

The integer variable lineNo was added to count lines in the file. The let shell command assigns integer variables to integer expressions.)
The variable erroCode was added to keep track of a error conditions. Any line too short.

The let shell command assigns integer variables to integer expressions. Here we start by assigning lineNo to 0 and then increasing it by 1 as the very first command in the do block. Any variable which is unassigned will expand as a null string. So we expect $errCode to be null after the while loop is completed when no errors were encountered. Since the implied exit and the end of the script will exit with status of the last command, this script with have a success exit if the errCoded is never assigned. A blank line is will not be considered and error.

There are two minor improvements to the script. The first test is a compound and "$x" -a ! "$y" ($x has content and "$y" is empty) and this cause the error if x is present, but y is missing. The comma is only put in the output if it is needed with the ${y:+, }. This means blank lines stay as blank lines with no error condition.

testing$ ./while2 <badfile >fig1.data && echo "good data file"
line 3 too short
line 5 too long, unexpected 32
testing$ ./while2 <warningfile >fig1.data && echo "good data file"
line 3 too long, unexpected 8
line 5 too long, unexpected 32
good data file
testing$ ./while2 <goodfile >fig1.data && echo "good data file"
good data file

To test while2 we have three sample files a badfile, warningfile and goodfile. We test three times, the badfile finds two bad lines and returns a failed status, and that is why the good data file is not echoed. The test with the warning file also finds two bad lines, but long lines are only warnings. After the too warning, the good data file appears, as it does for the goodfile. The good file returns nothing. (This is typical of UNIX - quiet is good)

while3:

#!/bin/bash
# input: a whitespace (one tab or spaces) separated data 
# output:  a comma separated data of x,y pairs
#  Ignores comments and allows \ logical continuation

while read x y etc; do
  [ "${x:0:1}" != '#' ] || continue
  [ "$x" -a "$y" ] && echo "$x, $y" 
done

reading a data file with comments and a logical continuation

$ cat commentfile
# Data collected 12/7/10
1 1.8
2 3.2
3\
 7.5 data collecte at noon
4 12.6
5 31.5 
6 60.5
$ ./while3 <commentfile
1, 1.8
2, 3.2
3, 7.5
4, 12.6
5, 31.5
6, 60.5
$

while4:

#!/bin/bash
# input: comma separted data pairs 
# outpt: tab separated data pairs 

IFS=', '
while read x y etc; do
  [ "$x" -a "$y" ] || continue
  echo -e "$x\t$y"
done

Example using expand to expand the tabs

$ ./while4 <fig2.data
1       1.8
2       3.2
3       7.5
4       12.6
5       31.5
6       60.5
$ ./while4 <fig2.data | expand -4
1   1.8
2   3.2
3   7.5
4   12.6
5   31.5
6   60.5
$

while5:

#!/bin/bash
# input: comma separated xy pairs 
# output: mathemataic findfit function
# positional porameter: model function with variables an or bn
#  For example: a0 + a1*x + a2*x**2

fun=${1//\*\*/^}
for v in {a,b}{0..9}; do
   [ "${fun/*$v*/}" ] || vars="$vars${vars:+,}$v"
done
[ $vars ] || exit 1

while read -r xy; do
  pts="$pts${pts:+,}{$xy}"
done
echo "FindFit[{$pts}, $fun, {$vars}, x]"

A mathematica list of points uses nested braces. Such as

{{x1,y1}, ... {xn,yn}}

The while5 version reads through a file of comma separate pairs and puts puts the braces to make a mathematica list of points. Then the\ Mathematica function is construction to find fit with the model function a*b**x and convert it to Fortan form (which is used by gnuplot).

Mathematica is not installed on this machine, but we can pipe to another machine which has Mathematica installed. Here we pipe it to the command line version on strauss.udel.edu. The output is the normal banner and prompts you would see it you ran it on strauss. Mathematica does not pause for input. It comes back the the bash prompt, after displaying the Fortran form of the fitted function.

Piping output to mathematica on strauss

$ ./while5 <fig1.data | ssh strauss.udel.edu math
dnairn@strauss.udel.edu's password: 
Mathematica 6.0 for Sun Solaris SPARC (64-bit)
Copyright 1988-2008 Wolfram Research, Inc.

In[1]:= 
Out[1]//FortranForm= 0.8303701122106603*2.0459945103037094**x

In[2]:=
$ tail -2 fig1rc
function_x='0.83037*2.04599**x'
figTitle='data with fitted exponential'
$

This is the fitted function used for figure 1.