Correct, a function would be the way to go. Filtering and aggregating is very simple in Trident.
From: clay teahouse [mailto:[email protected]] Sent: Monday, November 03, 2014 12:35 PM To: [email protected] Subject: Re: emitting batches of tuples Hello Bill, But piping is different from joining and merging. I don't want to join/merge two streams. I want to pass, pipe the output of one stream to another processor for further processing, as you can with bolts. I suppose I could in case of trident, just apply a function to the stream and change the stream the way I want. thanks, Clay On Mon, Nov 3, 2014 at 10:26 AM, Brunner, Bill <[email protected]<mailto:[email protected]>> wrote: Yes, you can use merge or join. Merge combines streams assuming they have the same output tuple signature. Join combines them using tuple values, similar to a db column join. https://storm.incubator.apache.org/documentation/Trident-API-Overview.html From: clay teahouse [mailto:[email protected]<mailto:[email protected]>] Sent: Monday, November 03, 2014 9:41 AM To: [email protected]<mailto:[email protected]> Subject: Re: emitting batches of tuples Thanks Andrew. How would I chain the streams in trident? I want to pipe the output of one stream to another stream. Can I have an hierarchy of streams with trident? Clay On Mon, Nov 3, 2014 at 8:03 AM, Brunner, Bill <[email protected]<mailto:[email protected]>> wrote: In Trident, every .each() call is a stream object. So from your spout “A”, you can just do val stream1 = A.each() val stream2 = A.each() and now you have 2 streams from your spout. You can then join or merge the streams later. From: clay teahouse [mailto:[email protected]<mailto:[email protected]>] Sent: Monday, November 03, 2014 7:37 AM To: [email protected]<mailto:[email protected]> Subject: Re: emitting batches of tuples But I need to be able to chain multiple streams with different type of records and need to be able to emit multiple streams from a single bolt. I am not sure if the same can be done as easily with trident. Are there examples of chaining and branching tridents out there? thanks Clay On Mon, Nov 3, 2014 at 5:45 AM, Andrew Xor <[email protected]<mailto:[email protected]>> wrote: Hi, I think you should take a look at Trident API here<http://storm.incubator.apache.org/documentation/Trident-API-Overview.html> if you want an easy way to process tuples in batches... let me know if this is what you are looking for. Cheers. Kindly yours, Andrew Grammenos -- PGP PKey -- <https://www.dropbox.com/s/2kcxe59zsi9nrdt/pgpsig.txt> https://www.dropbox.com/s/ei2nqsen641daei/pgpsig.txt On Mon, Nov 3, 2014 at 1:42 PM, clay teahouse <[email protected]<mailto:[email protected]>> wrote: Hello All, Is it possible emit batches of tuples, as opposed to one tuple at a time? In other word, is it possible to batch the tuples before emitting them? An application for batching the tuples is for example for writing the tuples to a tcp socket but not wanting to do a flush after each tuple is written to the socket. Everything runs locally. Sorry if the answer is obvious. thanks, Clay ________________________________ This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message. ________________________________ This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message. ---------------------------------------------------------------------- This message, and any attachments, is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/emaildisclaimer. If you are not the intended recipient, please delete this message.
