Dear Praveen, The questions aren't silly, but it is rather tough to understand what you are trying to convey. When you say "omit", do you mean "emit"? Bolts can emit data even without having to write to disk (I think there's a 2MB limit to the size of that data that can be emitted, because Thrift can't handle more than that). If you want one bolt to write to disk and then want another bolt to read from disk, then that's also possible. The first bolt can just send to the second bolt, whatever information is necessary to read from file. As of what I know, basic datatypes will automatically get serialized. If you have a more complex class, then serialize it with Serializable.
If you could re-phrase your question and make it clearer, people here would be able to help you better. On Sat, Jul 2, 2016 at 7:16 AM, praveen reddy < [email protected]> wrote: > Hi All, > > i am new to Storm and Kafka and working on POC. > > my requirement is get a message from Kafka in json format, spout reading > that message and firts bolt converting the json message to different format > like csv and the second bolt saving it to hadoop. > > now i came up with initial design where i can use kafkaspout to read kafka > topics and bolt converting it to csv file and next bolt saving in hadoop. > > i have following questions > can the first bold which coverts the message to csv file can omit it? the > file would be saving on disk. can a file which is saved on disk can be > omitted. > how does the second bolt read the file which is saved on disk by first > bolt? > do we need to serialize message ommitted by spout and/or bolt? > > sorry if the questions sound silly, this is my first topology with minimum > knowledge of storm. > > if you guys think of proper design how to implement the my requirement can > you please let me know > > thanks in advance > > -Praveen > -- Regards, Navin
