linux - Find number of occurrences of keyword in log file within last minute -

for purposes of publishing metrics aws cloudwatch information of number of occurrences of keyword (eg., error, exception) within last minute (from current system time) in application logs.

following commands have tried far based on answers related thread ( filter log file entries based on date range):

awk -vdate=`date -d'now-1 minutes' +["%y-%m-%d %h:%m:%s"` '($1 fs $2) > date {print $3}' application.log | grep "error" | uniq -c  awk -vdate=`date -d'now-1 minutes' +["%y-%m-%d %h:%m:%s"` '{if ($1 > date) {print $3}}' application.log | grep "error" | uniq -c  awk -vdate=`date -d'now-1 minutes' +["%y-%m-%d %h:%m:%s"` '{if ($1 == $date) {print $3}}' application.log | grep "error" | uniq -c 

but error when try this:

awk: cmd. line:1: 13:06:17 awk: cmd. line:1:   ^ syntax error 

following format of log file:

2016-02-05 12:10:48,761 [info] org.xxx 2016-02-05 12:10:48,761 [info] org.xxx 2016-02-05 12:10:48,763 [info] org.xxx 2016-02-05 12:10:48,763 [info] org.xxx 2016-02-05 12:10:48,763 [error] org.xxx 2016-02-05 12:10:48,763 [info] org.xxx 2016-02-05 12:10:48,764 [info] ffrom org.xxx 2016-02-05 12:10:48,773 [warn] org.xxx 2016-02-05 12:10:48,777 [info] org.xxx 2016-02-05 12:10:48,778 [info] org.xxx 

stuck on quite while. help!

you're using deprecated backticks , not quoting date output. instead:

awk -vdate="$(date -d'now-1 minutes' +"%y-%m-%d %h:%m:%s")" '($1 fs $2) > date { if ($3~/error/) print $3}' file 

note don't need pipe grep , not having space between -v , date script gawk-specific , if it's gawk-specific don't need external call date since gawk has it's own builtin time functions (hint: begin{date=strftime("%y-%m-%d %h:%m:%s",systime()-60)}).

you don't need uniq -c without seeing real input , expected output (doing uniq -c given input wouldn't make sense vs wc -l) i'm not going guess more.

oh heck, here's whole script in gawk:

$ cat tst.awk begin {     #date = strftime("%y-%m-%d %h:%m:%s",systime()-60)     date = "2016-02-05 12:10:48" } ($1" "$2) > date {     if ($3 ~ /error/) {         cnt[$3]++     } } end {     (err in cnt) {         print err, cnt[err]     } } $ $ awk -f tst.awk file [error] 1 

i assume in reality have various flavors of "error" , that's why want count of each. uncomment strftime line , delete hard-coded timestamp line run on real data.