FYI: I just found that Stream class has TODO comment above of class: *// TODO: need to be able to replace existing fields with the function fields (like Cascading Fields.REPLACE)* which is written at Aug. 2012, initial importing Trident source code.
According to the TODO comment, it seems to be a just missing feature. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 8월 31일 (수) 오전 7:29, Jungtaek Lim <[email protected]>님이 작성: > Hi Manu, > > I'm finding for 1:1 tuple transition, not aggregation. In other words, I'm > finding like 'V map(T, Function<T, V>)' which seems fit for many use case > but Trident doesn't support this yet so I'd like to find out why. > > Thanks, > Jungtaek Lim (HeartSaVioR) > > 2016년 8월 31일 (수) 오전 7:09, Manu Zhang <[email protected]>님이 작성: > >> I think we have 'aggregate' in Trident, where the function output fields >> replace the input fields. >> >> Thanks, >> Manu Zhang >> >> On Tue, Aug 30, 2016 at 4:17 PM Jungtaek Lim <[email protected]> wrote: >> >> > Hi devs, >> > >> > While implementing some features of Storm SQL on top of Trident, I >> realized >> > that there's no Trident operation which provides a way to add fields >> from >> > some Function and remove existing fields within one operation. >> > 'each' appends the function output fields to origin input fields, and >> 'map' >> > and 'flatMap' use same output fields, so no luck. >> > >> > Is this a missing feature, or there's a reason for doing so? Trident >> uses >> > TridentTupleView so I guess this is related but I didn't have a deep >> look >> > for Trident itself. >> > If anyone knows about the history please let me know. >> > >> > Thanks, >> > Jungtaek Lim (HeartSaVioR) >> > >> > ps. It might be better to explain why I'm finding that feature. I'm >> trying >> > to minimize below each -> project -> each -> project chain into one. >> > >> > >> https://github.com/apache/storm/blob/master/external/sql/storm-sql-core/src/jvm/org/apache/storm/sql/compiler/backends/trident/TridentLogicalPlanCompiler.java#L164-L167 >> > Please note that Trident doesn't allow duplicated field name so using >> > temporary field name is necessary if we should preserve existing fields. >> > >> >
