AWK is a data driven programming language designed for processing text-based data, either in files or data streams. Here only examples that can be useful for basic operations are given. Awk language has lots of other features which are not included here.

Syntax: awk [flags] [select pattern/find(sort)/commands] [input file]

  • "BEGIN {awk-commands} END {awk-commands}"

Example:

awk "BEGIN {FS='o'} {print $1,$3} END{print 'Total Rows=',NR}"
# Print every line in a file:	
awk '{print}' test.txt

# Print the lines which contains the given pattern:
awk '/test/ {print}' test.txt
awk '/test/' test.txt

# Print the fields 1 and 4 with delimeter whitespace:
awk '{print $1, $4}' test.txt
awk '{print $1 $4}' test.txt

# Display a block of test starts with the word start and ends with the word end:
awk '/start/,/stop/' test.txt

-f flag will execute specifying the name of the script file:
awk -f script.awk test.txt

Number the lines:
awk '{print NR, $0}' test.txt

Built-In Variables In Awk

NR command keeps a current count of the number of input records. Remember that records are usually lines. Awk command performs the pattern/action statements once for each record in a file.

# Use of NR built-in variables (Display Line Number):
awk '{print NR,$0}' test.txt

# Another use of NR built-in variables (Display Line From 3 to 6):
awk 'NR==3, NR==6 {print NR,$0}' test.txt

# Print the first item with its row number (NR), separated by "– " from each line:
awk '{print NR "- " $1 }' test.txt

NF command keeps a count of the number of fields within the current input record.

# Use of NF built-in variables (Display Last Field):
awk '{print $1,$NF}' test.txt 

# Print any non empty line if present:
awk 'NF < 0' test.txt
awk 'NF == 0 {print NR}' test.txt
awk 'NF <= 0 {print NR}' test.txt

FS command contains the field separator character which is used to divide fields on the input line. The default is “white space”, meaning space and tab characters. FS can be reassigned to another character (typically in BEGIN) to change the field separator.

awk 'BEGIN {print "FS = " FS}' | cat -vte

RS command stores the current record separator character. Since, by default, an input line is the input record, the default record separator character is a newline.

awk 'BEGIN {print "RS = " RS}' | cat -vte

OFS command stores the output field separator, which separates the fields when Awk prints them. The default is a blank space. Whenever print has several parameters separated with commas, it will print the value of OFS in between each parameter.

awk 'BEGIN {print "OFS = " OFS}' | cat -vte

ORS command stores the output record separator, which separates the output lines when Awk prints them. The default is a newline character. print automatically outputs the contents of ORS at the end of whatever it is given Print.

awk 'BEGIN {print "ORS = " ORS}' | cat -vte

Other Exp:

# To find the length of the longest line present in the file:  
awk '{ if (length($0) > max) max = length($0) } END { print max }' test.txt

# To count the lines in a file:  
awk 'END { print NR }' test.txt 

# Printing lines with more than 10 characters:  
awk 'length($0) > 10' test.txt 

# To find/check for any string in any specific column:  
awk '{ if($3 == "B6") print $0;}' test.txt

# Print the squares of first numbers from 1 to n say 6:  
awk 'BEGIN { for(i=1;i<=6;i++) print "square of", i, "is",i*i; }'

awk printf

CommandsOutput
printf "%d", 99/249
printf "%e", 99/24.950000e+01
printf "%f", 99/249.500000
printf "%6.2f", 99/249.50
printf "%g", 99/249.5
printf "%o", 99/261
printf "%06o", 99/2000061
printf "%x", 99/231
`printf "%s
`printf "%10s
`printf "%-10s
`printf "%.3s
`printf "%10.3s
`printf "%-10.3s
printf "%%"%

Important Flags

FlagsDescription
-FWith this flag you can specify FIELD SEPARATOR (FS), and thus don't need to use the BEGIN rule
-vCan be used to specify variables(like we did in BEGIN{OFS=":"}
-DYou can debug your .awk scripts specifying this flag(awk -D script.awk)
-oTo specify the output file (if no name is given after the flag, the output is defaulted to awkprof.out)

Resources: