Maybe this email got lost among a bunch of commits. But I hope that you maybe able to give us some insight where you intend to take Storm.
Thank you, Peter On Wed, Mar 26, 2014 at 4:30 PM, David Smith <[email protected]>wrote: > Hello, > > I would like some clarity from committers about certain features and your > intentions about their future direction going forward. > > 1. Transactional topologie / Linear DRPC Topologies. For the longest time, > this has been informally described as deprecated. But the codebase for this > is still not marked as deprecated. Can we have some guidance on whether > this code will live on and if for how long and when it should be preferred > over Trident (what Trident cannot accomplish that this can)? If it's not > intended to live on, then may I suggested that you deprecate the code > initially and mark it for removal during subsequent release? > > 2. Trident. There's a lot of good documentation on how to write a Trident > topology but there's hardly any documentation on how to write a Trident > spout. People waste a lot of time attempting to write a Trident spout for > their data sources and give up because it's unclear and all their > assumptions they will bring forward from their knowledge of standard Storm > Spouts are not valid for Trident Spouts. Bringing the Kafka spout into > storm project will address some of these concerns as Trident is fully > supported under Kafka. But it will be useful to know the operational > characteristics that need to be supported by the data source in order to > support a Trident spout. It would be quite helpful to see a simple > implementation of a Trident Spout to serve as guidance for writing a > Trident spout. > > 3. Trident currently supports building Trident spout based on an existing > IRichSpout's with the understanding that they will only be > non-transactional within Trident. But lot of IRichSpouts cannot be used in > this manner as they expect to receive ack's and fail's. That IRichSpout > contract is not enforced by Trident (understandably) but it would be much > easier for developers to grok this if there was a common API contract that > IRichSpouts that work with Trident can implement (even if only in a > non-transactional manner). > > 3. DRPC support in regular Storm without Trident. From what I understand > this is not possible right now without using the deprecated > LinearDRPCTopologyBuilder. > > 4. Tick Tuple support in Trident > > 5. Storm internals commonly being used by users to develop features. Like > CoordinatedBolt, RotatingMap, etc. These can be highly valuable and we > might want to consider making them a part of the core API for advanced > users and document how they should be used. > > 6. IBatchBolt. Can this be used without Transactional Topologies? > > 7. Different stream groupings other than shuffle, field may require better > documentation before users can understand how to use them. > > Once again, this is not a complaint but a plea for clarity about future > direction. I understand that some of these are not currently supported and > it may be while before they will be (if ever). That is fine, but now that > Storm is incubating under Apache, it would be nice to have a roadmap that > indicates the direction that Storm is heading towards. > > Thank you, > David >
