Hello, I would like some clarity from committers about certain features and your intentions about their future direction going forward.
1. Transactional topologie / Linear DRPC Topologies. For the longest time, this has been informally described as deprecated. But the codebase for this is still not marked as deprecated. Can we have some guidance on whether this code will live on and if for how long and when it should be preferred over Trident (what Trident cannot accomplish that this can)? If it's not intended to live on, then may I suggested that you deprecate the code initially and mark it for removal during subsequent release? 2. Trident. There's a lot of good documentation on how to write a Trident topology but there's hardly any documentation on how to write a Trident spout. People waste a lot of time attempting to write a Trident spout for their data sources and give up because it's unclear and all their assumptions they will bring forward from their knowledge of standard Storm Spouts are not valid for Trident Spouts. Bringing the Kafka spout into storm project will address some of these concerns as Trident is fully supported under Kafka. But it will be useful to know the operational characteristics that need to be supported by the data source in order to support a Trident spout. It would be quite helpful to see a simple implementation of a Trident Spout to serve as guidance for writing a Trident spout. 3. Trident currently supports building Trident spout based on an existing IRichSpout's with the understanding that they will only be non-transactional within Trident. But lot of IRichSpouts cannot be used in this manner as they expect to receive ack's and fail's. That IRichSpout contract is not enforced by Trident (understandably) but it would be much easier for developers to grok this if there was a common API contract that IRichSpouts that work with Trident can implement (even if only in a non-transactional manner). 3. DRPC support in regular Storm without Trident. From what I understand this is not possible right now without using the deprecated LinearDRPCTopologyBuilder. 4. Tick Tuple support in Trident 5. Storm internals commonly being used by users to develop features. Like CoordinatedBolt, RotatingMap, etc. These can be highly valuable and we might want to consider making them a part of the core API for advanced users and document how they should be used. 6. IBatchBolt. Can this be used without Transactional Topologies? 7. Different stream groupings other than shuffle, field may require better documentation before users can understand how to use them. Once again, this is not a complaint but a plea for clarity about future direction. I understand that some of these are not currently supported and it may be while before they will be (if ever). That is fine, but now that Storm is incubating under Apache, it would be nice to have a roadmap that indicates the direction that Storm is heading towards. Thank you, David
