Thank you Mathias for this. While looking around, I've stumbled upon Telegraf <https://github.com/influxdata/telegraf> (an agent for collecting, processing, aggregating, and writing metrics) that has a Kinesis input plugin and a Warp 10 outplut plugin <https://github.com/influxdata/telegraf/blob/release-1.15/plugins/outputs/warp10/README.md>. If you have any comment on this solution, I'll happily hear it!
Thanks again! Adam On Friday, August 28, 2020 at 2:02:23 PM UTC+2 [email protected] wrote: > Warp 10 has plugins for consuming data from an MQTT or Kafka message > broker, if such protocol is not an option for the data flowing through > Kinesis then maybe you can create a Kinesis plugin using the Java SDK ( > https://docs.aws.amazon.com/streams/latest/dev/developing-consumers-with-sdk.html), > > it would probably be rather simple to do following the same logic present > in the MQTT or Kafka plugins. > > Kafka Plugin: https://github.com/senx/warp10-plugin-kafka > MQTT Plugin: https://github.com/senx/warp10-plugin-mqtt > > On Thursday, August 27, 2020 at 10:59:11 PM UTC+2 [email protected] > wrote: > >> I have continued my research and unfortunately it appears Kinesis Data >> Firehose will not do the work as it is not process data in real time. It >> processes data every 60s which is not enough for my use case. >> >> Still looking for a solution, if you have an idea please share! :) >> >> On Thursday, August 27, 2020 at 6:35:05 PM UTC+2 Adam Makhlouf wrote: >> >>> Hi everyone, >>> >>> I am considering using Warp 10 (standalone) in the AWS ecosystem on 2 >>> EC2 instances (with datalog replication). >>> >>> I am getting sensor data from Kinesis Data Stream that I plan on >>> ingesting in Warp 10. I have found no reference on how that would work. >>> I can see that Amazon Kinesis Data Firehose can now stream data to an >>> HTTP endpoint >>> <https://aws.amazon.com/blogs/big-data/stream-data-to-an-http-endpoint-with-amazon-kinesis-data-firehose/>. >>> >>> Would that be a correct approach? >>> >>> Thanks for your help! >>> >>> >>> >>> >>> -- You received this message because you are subscribed to the Google Groups "Warp 10 users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/warp10-users/29ccf69e-6a13-4207-9d36-bd846020c212n%40googlegroups.com.
