[elasticsearch][logstash] logstash 中 使用某欄位當作 elasticsearch 的 _id

一般的 elasticsearch ouput 如下 可以使用 document_id 來指定 event 的 _id

output { elasticsearch { index => "test1" document_type => "message_logs" document_id => "%{type}-%{id}" action => index hosts => "myhost" }}

一個更完整的例子如下 使用 mutate 從json裏面取值，增加一個 gid 的欄位。如果，當json 沒有id這個欄位，就捨去這筆 event 。
input { #beats { # port => 5044 #} stdin { codec => json_lines } } filter { if [type] == "my_report_text" { json { source => "message" } date { match => ["server_timestamp", "UNIX"] remove_field => ["server_timestamp"] } mutate { remove_field => ["message","source","input_type","offset","count","fields"] add_field => { "gid" => "%{[text_v1][id]}" } } if ![text_v1][id] { drop {} } } } output { if [type] == "my_report_text" { elasticsearch { hosts => ["127.0.0.1","10.2.101.163","10.2.101.164"] index => "report_text-%{+YYYY.MM.dd}" workers => 5 document_id => "%{gid}" } } }