[logstash-users] csv-filter error

[A***@gmx.de]

Hello,

I have a problem with my csv filter. I receive the following error
Trouble parsing csv {:source=>"message", :raw=>"",
:exception=>#<NoMethodError: undefined method `each_index' for
nil:NilClass>, :level=>:warn}
{
"message" => "Requested slot for webservice communication
for user: OO2.",
"@version" => "1",
"@timestamp" => "2015-06-01T08:32:17.734Z",
"host" => "server1800",
"path" => "/tmp/test_oo.log",
"errorid" => "191",
"month" => "Jun",
"day" => "01",
"time" => "05:11:08",
"server" => "Server32",
"logfile" => "DEVTEST-OO-Logs",
"severity" => "Debug",
"level" => "16",
"logger" => "ThreadDispatcher",
"aspnet-sessionid" => nil,
"gdc" => "de",
"event-context" => nil,
"aspnet-user-identity" => nil,
"exception" => nil
}
{
"message" => [
[0] ""
],
"@version" => "1",
"@timestamp" => "2015-06-01T08:32:17.735Z",
"host" => "server1800",
"path" => "/tmp/test_oo.log",
"tags" => [
[0] "_grokparsefailure",
[1] "_csvparsefailure"
]
}

and I don't know exactly why I am receiving this error.

My logfile is looking like:

<191>Jun 01 05:11:08 Server32 DEVTEST-OO-Logs:
Debug,16,ThreadDispatcher,,de,,,Requested slot for webservice communication
for user: OO2.,

And my filter part looks like:

filter {
grok {
match => [ "message",
"<%{DATA:errorid}>%{MONTH:month}%{SPACE}%{MONTHDAY:day}%{SPACE}%{TIME:time}%{SPACE}%{HOSTNAME:server}%{SPACE}%{DATA:logfile}:%{SPACE}%{DATA:severity},%{GREEDYDATA:message}"
]
overwrite => [ "message" ]
}
csv {
columns => ["level", "logger", "aspnet-sessionid", "gdc",
"event-context", "aspnet-user-identity", "message", "exception"]
separator => ","
}
}

I am using the logstash version 1.4.2


Has anybody an idea why I am receiving the csv error?

Thank you!

Greetz

[Magnus Bäck]

Post by A***@gmx.de
I have a problem with my csv filter. I receive the following error
Trouble parsing csv {:source=>"message", :raw=>"",
:exception=>#<NoMethodError: undefined method `each_index' for
nil:NilClass>, :level=>:warn}
{
"message" => "Requested slot for webservice
communication for user: OO2.",
"host" => "server1800",
"path" => "/tmp/test_oo.log",
"errorid" => "191",
"month" => "Jun",
"day" => "01",
"time" => "05:11:08",
"server" => "Server32",
"logfile" => "DEVTEST-OO-Logs",
"severity" => "Debug",
"level" => "16",
"logger" => "ThreadDispatcher",
"aspnet-sessionid" => nil,
"gdc" => "de",
"event-context" => nil,
"aspnet-user-identity" => nil,
"exception" => nil
}

I can't reproduce this with the input you provided. I get the 'nil'
fields but no NoMethodError message. Do you have any other filters
or outputs?

[...]
--
Magnus Bäck | Software Engineer, Development Tools
***@sonymobile.com | Sony Mobile Communications

[A***@gmx.de]

Hello Magnus,

no this is my whole config file. I would say it is a standard file without
any complicated things.

input {
file {
path => "/tmp/test_oo.log"
}
}

filter {
grok {
match => [ "message",
"<%{DATA:errorid}>%{MONTH:month}%{SPACE}%{MONTHDAY:day}%{SPACE}%{TIME:time}%{SPACE}%{HOSTNAME:server}%{SPACE}%{DATA:logfile}:%{SPACE}%{DATA:severity},%{GREEDYDATA:message}"
]
overwrite => [ "message" ]
}
csv {
columns => ["level", "logger", "aspnet-sessionid", "gdc",
"event-context", "aspnet-user-identity", "message", "exception"]
separator => ","
}
}

output {
stdout {
codec => rubydebug
}
}

This is really strange.

Post by Magnus Bäck

Post by A***@gmx.de
I have a problem with my csv filter. I receive the following error
Trouble parsing csv {:source=>"message", :raw=>"",
:exception=>#<NoMethodError: undefined method `each_index' for
nil:NilClass>, :level=>:warn}
{
"message" => "Requested slot for webservice
communication for user: OO2.",
"host" => "server1800",
"path" => "/tmp/test_oo.log",
"errorid" => "191",
"month" => "Jun",
"day" => "01",
"time" => "05:11:08",
"server" => "Server32",
"logfile" => "DEVTEST-OO-Logs",
"severity" => "Debug",
"level" => "16",
"logger" => "ThreadDispatcher",
"aspnet-sessionid" => nil,
"gdc" => "de",
"event-context" => nil,
"aspnet-user-identity" => nil,
"exception" => nil
}

I can't reproduce this with the input you provided. I get the 'nil'
fields but no NoMethodError message. Do you have any other filters
or outputs?
[...]
--
Magnus BÀck | Software Engineer, Development Tools