ELA_11_AWK_Usage

##AWK INTRO##

Features:

1. Reporter

2. Field Processor

3. Supports Scripting

4. Programming Constructs

5. Defaults delimiter is whitespace (space or tab)

6. Supports: pipes, Files, and STDIN as sources of input

Usage:

awk 'instructions' file(s)

awk '/pattern/{ procedure }' file

awk -f script_file file(s)

Tasks:

Note: $0 represents the current record or row

1. Print entire row at a time, from an input file (example.txt)

a. awk '{ print $0 }' example.txt or awk '{ print }' example.txt

2. Print specific columnn from example.txt

a. awk '{ print $1 }' example.txt - this prints first columns

3. Print multiple columns from example.txt

a. awk '{ print $1,$2 }' first print 1st column then 2nd column(same line)

b. awk '{ print $2,$1 }' first print 2nd column then 1st column(same line)

c. awk '{print $1; print $2 }' first print 1st column then 2nd column (separate lines)

4. Print columns from lines containing king

a. awk '/king/{ print $0 }' example.txt

5. Print columns from lines containing numbers

a. awk '/[0-9]/{ print $0 }' example.txt

6. Removing blank lines with sed and pipe output to awk for processing

a. sed -e '/^$/d' example.txt | awk '/^[0-9]*$/{ print $0 }'

7. Print Blank lines

a. awk '/^$/{ print }' example.txt or

b. awk '/^$/{print $0 }' example.txt

8. Print all lines having king/KING/King (case insensitive)

a. awk '/dog/I{ print }' example.txt

### Delimiters ###

Default delimiter: whitespace (space,tabs)

For Changing it user -F

Task:

1. Parse /etc/passwd using awk

a. awk -F: '{print $3,$1}' /etc/passwd -- prints uid followed by username

2. Support for character classes in setting the default delimiter

a. awk -F"[:;,.\t]" -- defining range of delimiter

### Awk Script ###

Features:

1. Ability to organize patterns and procedures into a script file

2. The patterns/procedures are much neater and easier to read

3. Less information is placed on the command-line

4. By default, loops through lines of input from vafious sources STDIN, Pipe, files

5. # mark is the default comment charecter

Awk Scripts contains of 3 parts:

1. Before (denoted using: BEGIN) - Executed prior to FIRST line of input being read

2. During (main Awk loop) - Focuses on looping through lines of input

3. After (denoted using: END) - Executed after the LAST line of input has been processed

Note: BEGIN and END components af Awk scripts are optional

Task:

1. Print some useful information using awk insteed echo

a. awk 'BEGIN { print "Testing Awk without input file,pipe or STDIN" }'

b. awk 'BEGIN { FS = ":" print FS}'

c. awk 'BEGIN { FS = ":"; print "New Field Separator is"; print FS; }'

2. Creating separate script file which will contain all 3 parts of awk
vim awk_script

# This script parses document for items containing deer

#Componet 1 - BEGIN

BEGIN { print "Begin Processing of various records" }

#Component 2 - Main Loop

/dea r/{print} ## Only print those line which contain deer

# Component 3 - END

END { print "Process Complete" }

wq!

3. parse ur awk asript file fith script file

a. awk -f awk_script example.txt

4. Parse /etc/passwd

a. print entire line

awk -F: '{ print }' /etc/passwd

or

{ print }

b. print specific cloumns

awk -F: '{ print $3,$1 }' /etc/passwd

or

{ print $3,$1 }

c. print specific column of specific users

awk -F: '/root/{ print $3,$1}' /etc/passwd

or

/root/{ print $3,$1}

d. print specific column of specific users matching a given column $1

awk -F: '$1 ~ /root/ { print $3,$1 }' /etc/passwd

or

$1 ~ /root/ { print $3,$1 }

e. print only those user info who can login

awk -F: '$7 ~ /root/ { print }' /etc/passwd

or

$7 ~ /root/ { print }

### Awk Varibles ##

Features 3 Types of Variables

1. System Variables -i.e. FILENAME, RS(record separator), ORS(Output record sepatater)

2. Scalars - i.e. a = 3

3. Arrays - i.e. variable_name[n]

Note : Variable do not need to be declared. Awk, automatically registers them in memory

Note : variable name are case sentive

System Variable:

1. FILENAME - name of the current input file

2. FNR - used when multiple input files are used

3. FS - field separater - default to whitespace - can be a single charecter, including via RegEx

4. OFS - output field separator - default is whitspace

5. NF - number of fields in the current record

6. NR - current record number(it is autosumed when referenced in END section)

7. RS - record separator - default to newline

8. ORS - output record separator - default to newline

9. ARGV - array of comman line arguments - indexed at 0, begins with $1

10. ARGC - total no to command line arguments

11. ENVIRON - array of enviroment variables for the current user

Task:

1. print key system variables

a. print FILENAME --- which will print filename which is parsed

Lets modify our awk_script file

# vim awk_script

This script parses document for items containing deer

#Componet 1 - BEGIN

BEGIN { print "Begin Processing of various records" }

#Component 2 - Main Loop

/root/{print $1$7} ## Only print those line which contain deer

# Component 3 - END

END { print "Process Complete"

print "Filename:" FILENAME}

wq!

###OUTPUT

$ awk -f awk_script /etc/passwd

Begin Processing of various records

root:x:0:0:root:/root:/bin/bash
Process Complete

Filename:/etc/passwd ### it has printed file name in the END section

Task:

1. Lets print different awk variables

a. Lets print number of fields per record (NF)

#Component 2 - Main Loop

{print $1,$7,NF} ## it also print no of field per record

b. Lets print record number (NR)

#Component 2 - Main Loop

{print RN,$1,$7,NF} ## it print record no and also print no of field per record

c. Print ARGC which keeps number of argument pased it should not be includend in BEGIN or MAin Loop section

# Component 3 - END

END { print "Process Complete"

print "Filename:" FILENAME

print ARGC}

#### Scalar Variables #########

1. Always has to be set in BEGIN section

variable = value

eg--

#Componet 1 - BEGIN

BEGIN { FS = ":" ; age = 50 ; print "Begin Processing of various records" }

2. We can can call it or increment it from Main loop section

#Component 2 - Main Loop

/root/{print NR, $1, $7, NF } ## Only print those line which contain root

{ print "Age is: "age }

{ ++age }