Hi Bharath, We did not deploy logstash as part of OOM deployment in Casablanca because some issues needed to be resolved:
* We had a use-case specific solution, not a general solution. * Want to evaluate whether logtstash or Kafka mirror-maker should be used. * It was not clear what topics should be stored in PNDA – the consensus is settling on just the VES topic. * We need a reliable way of checking PNDA readiness before running log stash Cheers, Donald. On 27 Nov 2018, at 15:05, Thiruveedula, Bharath <[email protected]<mailto:[email protected]>> wrote: Hi Donald, Thanks a lot for the reply and providing the configuration. So logstash is not deployed as part of OOM deployment? Best Regards Bharath T On Mon, Nov 26, 2018 at 10:05 PM Donald Hunter via Lists.Onap.Org<http://lists.onap.org/> <[email protected]<mailto:[email protected]>> wrote: Hi Bharath, The DMaaP topic is unauthenticated.VES_MEASUREMENT_OUTPUT and the PNDA topic is ves.avro. The logstash configuration would look like this: input { kafka { topics => ["unauthenticated.VES_MEASUREMENT_OUTPUT"] bootstrap_servers => "dmaap-host:9092" } } filter { mutate { add_field => { "src" => "ves" "host_ip" => "onap.demon" } rename => { "message" => "rawdata"} } ruby { # Convert the Logstash timestamp to a milliseconds timestamp integer # You can use whatever makes sense to you as long as the timestamp is a valid timestamp integer in milliseconds code => "event.set('timestamp', (event.get('@timestamp').to_f * 1000).to_i)" } } output { kafka { topic_id => "ves.avro" bootstrap_servers => "pnda-kafka-host:9094" value_serializer => "org.apache.kafka.common.serialization.ByteArraySerializer" codec => pnda-avro{ schema_uri => "dataplatform-raw.avsc" } } stdout { codec => "json" } } Note that this will also display the messages to stdout, which is helpful for debugging. You can remove the stdout section if you don't want that. Cheers, Donald. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14119): https://lists.onap.org/g/onap-discuss/message/14119 Mute This Topic: https://lists.onap.org/mt/28430704/21656 Group Owner: [email protected] Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
