elasticsearch - MySQL Kafka ELK Stack Date Issue -
i creating data ingestion from:
mysql --> kafka --> logstash --> elasticsearch --> kibana
i able pipeline data mysql till kibana, facing issues while dealing "date" column in mysql table.
the date column appearing epoch date in kafka topic, , in es document.
can convert date format "yyyy-mm-dd" in elasticsearch/logstash part?
to answer question directly: use elasticsearch dynamic template define date field date
, elasticsearch interpret epoch value correctly.
do know don't need logstash in there @ all? can use elasticsearch sink kafka connect. can separately, or part of confluent platform.
you'll want follow blog series i'm writing here, builds out pipeline (mysql -> kafka -> elasticsearch) : https://www.confluent.io/blog/simplest-useful-kafka-connect-data-pipeline-world-thereabouts-part-1/
Comments
Post a Comment