[logstash-users] Logstash - Cannot update @timestamp field

[boardman411]

Hi, I'm new to logstash and I'm trying to take the date field from a XML
Log and update the @timestamp field with the date of the log.. I can grab
the current log time using GROK.

grok {
pattern =>
"(?<logdate>[0-9]{4}[0-9]{2}[0-9]{2}\w[0-9]{2}:[0-9]{2}:[0-9]{2})"


I see 'logdate' in my output when processing the file.

"message" => "20150425T16:39:00 <stats> (Removed rest of message)
"@timestamp" => "2015-05-22T13:16:19.172Z",
"host" => "nykdsr000000680.intranet.barcapint.com",
"logdate" => "20150425T16:39:00",


What I can't seem to do, is update the @timestamp field with the logdate..
Below is my FILTER, can anyone see what I'm doing incorrect here..

filter {
grok {
pattern =>
"(?<logdate>[0-9]{4}[0-9]{2}[0-9]{2}\w[0-9]{2}:[0-9]{2}:[0-9]{2})"
}

xml {
source => "message"
target => "message_parsed"
add_tag => ["xml_parsed"]
}

date {
match => ["logdate", "yyyy-MM-dd HH:mm:ss"]
target => "@timestamp"
}
}

Is my Match syntax incorrect... Thanks for looking...

[Peter Paul Holzhauer]

I would try this grok-filter:

grok {
match => [ "message",
"(?<logdate>[0-9]{4}[0-9]{2}[0-9]{2}\w[0-9]{2}:[0-9]{2}:[0-9]{2})" ]
}

Peter Paul

Post by boardman411
Hi, I'm new to logstash and I'm trying to take the date field from a XML
the current log time using GROK.
grok {
pattern =>
"(?<logdate>[0-9]{4}[0-9]{2}[0-9]{2}\w[0-9]{2}:[0-9]{2}:[0-9]{2})"
I see 'logdate' in my output when processing the file.
"message" => "20150425T16:39:00 <stats> (Removed rest of message)
"host" => "nykdsr000000680.intranet.barcapint.com",
"logdate" => "20150425T16:39:00",
Below is my FILTER, can anyone see what I'm doing incorrect here..
filter {
grok {
pattern =>
"(?<logdate>[0-9]{4}[0-9]{2}[0-9]{2}\w[0-9]{2}:[0-9]{2}:[0-9]{2})"
}
xml {
source => "message"
target => "message_parsed"
add_tag => ["xml_parsed"]
}
date {
match => ["logdate", "yyyy-MM-dd HH:mm:ss"]
}
}
Is my Match syntax incorrect... Thanks for looking...

--
The information contained in this e-mail communication is solely intended
for the person/legal person to whom it has been sent, and as it may contain
information of a personal or confidential nature, it may not be made public
by virtue of law, regulations or agreement. If someone other than the
intended recipient should receive or come into possession of this e-mail
communication, he/she will not be entitled to read, disseminate, disclose
or duplicate it. If you are not the intended recipient, you are requested
to inform the sender of this e-mail message of this immediately, and to
destroy the original e-mail communication. Neither Randstad Holding nv nor
its subsidiaries accept any liability for incorrect and incomplete
transmission or delayed receipt of this e-mail.

[boardman411]

I have finally got around the issue and it's with the Regex, as well as the
way I've configured some of the parameters in the date filter. The main
issue is with how the 'T' parsed..

Correct Format

grok {
pattern =>
"^(?<logdate>[0-9]{4}[0-9]{2}[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}).*"

date {
match => ["logdate", "yyyyMMdd'T'HH:mm:ss"]

Need to put quotes around the 'T'..

Works after this..