elasticsearch - Logstash with queue enabled not ack http input events after jdbc input runs -


i’m using logstash queuing enabled.

i’ve setup logstash inject rows mysql via mysql input plugin on startup. injecting 1846 rows.

i have http input.

when take down es , restart logstash expected errors

logstash_1 warn logstash.outputs.amazones - failed flush outgoing items {:outgoing_count=>1, :exception=>“faraday::connectionfailed”, :backtrace=>nil} logstash_1 error logstash.outputs.amazones - attempted send bulk request elasticsearch configured @ … i’d expect when in situation hitting logstash http input result in ack.

actually http post not return , injection not seen in logstash logs.

my logstash.yaml looks like

queue {   type: persisted   checkpoint.writes: 1   queue.max_bytes: 8gb   queue.page_capacity: 512mb } 

and logstash.conf

input {    jdbc {     jdbc_connection_string => "${jdbc_connection_string}"     jdbc_user => "${jdbc_user}"     jdbc_password => "${jdbc_password}"     jdbc_driver_library => "/home/logstash/jdbc_driver.jar"     jdbc_driver_class => "com.mysql.jdbc.driver"     statement => "         select blah blah blah     "   }    http {     host => "0.0.0.0"     port => 31311   } }  output {   stdout { codec => json_lines }   amazon_es {     hosts => ["${aws_es_host}"]     region => "${aws_region}"     aws_access_key_id => '${aws_access_key_id}'     aws_secret_access_key => '${aws_secret_access_key}'     "index" => "${index_name}"     "document_type" => "data"     "document_id" => "%{documentid}"   } } 

is possible http input still ack events i’m pretty sure queue cannot full each event payload 850 characters?

thanks in advance


Comments

Popular posts from this blog

android - InAppBilling registering BroadcastReceiver in AndroidManifest -

python Tkinter Capturing keyboard events save as one single string -

sql server - Why does Linq-to-SQL add unnecessary COUNT()? -