This will be brief introduction to awk and it’s many uses. I will show a few brief segments about various uses of awk and when it comes in handy for certain tasks. For more detail one should read the Awk User’s Manual.

Awk is powerful text processing tool. It has been featured in Unix like operating systems from the beginning. It provides tools to process column and line oriented data files with filtering or pasting.

Line Fields

Awk reads lines in from either standard input or a file specified on the command line. It assumes that this files is text, and has columns. The columns are separated by whitespace by default, but this can be changed with the -F flag on the command line. The fields are available as variables $1 up to and including $n where n is the number of the last field on the line. $0 is the whole line unaltered or split.

So if we wanted to extract out fields 1, 2, and 4 from a csv one could compose a command like so:

awk -F, '{print $1 $2 $4}' mydata.csv

Blocks

Awk has various blocks that can be used in your program. I rarely use these but they come in handy when accumulating values. These blocks tell the program to do something at various points during execution, like END { } which tells the program to do these things once all input has been read. Another block is the default one which is just { }, and contains anything you want to do with each line.

A brief example would be when I want to sum a column of numbers which takes the form of:

awk '{s += $1}END{print s}' mydata

Filtering

Awk also has the ability to filter data files in a way that is similar to grep but actually understands numbers. I used this during NCL doing log processing. It comes in handy when you don’t want to deal with databases and massaging your log file into one. An example of this would be if you wanted to find all lines where a certain field is greater than 30 and less than 2000.

awk '$3 > 30 && $3 < 2000' mydata

Wrapping Up

Awk offers a lot of cool line oriented data processing tools, and is an essential tool for system administrator to use and understand. I hope this has been a useful introduction to Awk for you.