See All Traffic by a Referrer in access.log

See All Traffic by a Referrer in access.log

In your web-stats, you may sometimes find an unknown referrer, which might have sent you a few visitors. Intrigued by it you go and grep Apache’s access.log to see what those visitors came to read:

grep "referrer" access.log | less

That line is enough in most situations, but what if you get intrigued yet further and want to know if these visitors stayed for anything more at your site?

That is also doable in one line:

grep `awk '/guestbook/ {print "-e "$1}' < access.log` < access.log | less

Here awk extracts the list of hosts which sport a certain string in their HTTP requests (“guestbook” e.g. could be a part of referring URL of some online guestbook). awk also prepends each host with “-e ” – the flag grep understands. The external grep finds all the hits by those hosts.

For that command to be easily accessible in future the following function definition may be added to .bashrc.

rt ()
{
ref=$1
log=$2
grep `awk "/$ref/"' {print "-e "$1}' <$log` <$log | less
}

Or, in a one-line form to be activated on-demand from shell history (Control-R) without modifying .bashrc:

rt () { ref=$1; log=$2; grep `awk "/$ref/"' {print "-e "$1}' <$log` <$log | less; }

Note the use of single- and double-quotes in awk: double-quotes allow for shell parameter substitution ($ref), while single-quotes escape awk’s own variable reference ($1)

With this shell function defined just type at the command prompt:

rt referrer-substring path/to/access.log