I think we have 'aggregate' in Trident, where the function output fields replace the input fields.
Thanks, Manu Zhang On Tue, Aug 30, 2016 at 4:17 PM Jungtaek Lim <[email protected]> wrote: > Hi devs, > > While implementing some features of Storm SQL on top of Trident, I realized > that there's no Trident operation which provides a way to add fields from > some Function and remove existing fields within one operation. > 'each' appends the function output fields to origin input fields, and 'map' > and 'flatMap' use same output fields, so no luck. > > Is this a missing feature, or there's a reason for doing so? Trident uses > TridentTupleView so I guess this is related but I didn't have a deep look > for Trident itself. > If anyone knows about the history please let me know. > > Thanks, > Jungtaek Lim (HeartSaVioR) > > ps. It might be better to explain why I'm finding that feature. I'm trying > to minimize below each -> project -> each -> project chain into one. > > https://github.com/apache/storm/blob/master/external/sql/storm-sql-core/src/jvm/org/apache/storm/sql/compiler/backends/trident/TridentLogicalPlanCompiler.java#L164-L167 > Please note that Trident doesn't allow duplicated field name so using > temporary field name is necessary if we should preserve existing fields. >
