Hi guys, I have a stream of sensor data coming from rabbitmq. For each sensor message, it is of the JSON format and have the following fields:
deviceId: "BOT-N3" reading0: 2.25 reading1: 3.78 .... readingN: -1.35 each float number of readingN represents a sensor reading on a specific field location. Now for each incoming message, I want to do a query which gives me the average and standard deviation of a certain 'deviceId' 's 'readingN' over a custom time range (a year ago to now, a month ago to now, etc). So if N=28, for each incoming message I will need to do 28 queries on the historic data at almost the same time. I need the query results to be returned in near real time so the other incoming messages won't get blocked. Is STORM a good solution to this issue? I have tried Elasticsearch-Logstash-Kibana stack already, It seems that when the incoming message rates are high, The messages will be blocked since the ES server can't correspond to hundreds of query requesst at the same time. Will STORM help me in this case? What is the common use case of STORM in processing real-time sensor data (coming from sensor network specifically)? Thanks! best Yuheng
