The storm metron-writer bolt has to write to many different sensor's HDFS files. The hdfs bolts provided by storm only support writing to one file (or set of files). The alternative would have been a bolt per sensor, which did not support our architecture.
In short, the hdfs bolt from storm would have necessitated a bolt per sensor and adding new sensors would have required topology restart and recreation; this isn't something that we wanted. On Mon, Aug 28, 2017 at 11:25 AM, Otto Fowler <[email protected]> wrote: > What is the reasoning behind not using the Storm provided hdfs bolts for > metron-writer? Anyone remember? >
