Thursday, March 21, 2013

The return of Super Sed and Wonder Awk...

I needed to compare the tabbed separated data of a file (file A) to expected data (file B).  However, the contents of file A contained the processing date in each line of output in the file.

To handle the issue of non-matching dates, the "processing date" in file B was updated to be the string "PROCESSING_DATE".  That just left the date in file A to contend with.

Here is where sed and awk came to the rescue.  I used head -n1 to get the first line of file A, and used awk to get the processing date (which appeared in the 11th column).  The processing date was stored in a variable named target_date. Next, I used sed to do a replacement on all instances of target_date in file A.  After which I was able to do a diff on the two files to see if the output was as expected.

Here is how it looked in the shell script:

# get the target date
target_date=`head -n1 fileA.txt |  awk '{print $11}'`
# get the sed argument using the target date
sed_args="s/$target_date/PROCESSED_DATE/g"

# do an inline replacement on the target file
sed -i $sed_args fileA.txt

# check for differences
diffs=`diff fileA.txt fileB.txt`

if [ -n $diffs ]; then
    echo "There were differences between the files."
else
    echo "No differences were found."
fi

No comments:

Post a Comment