FYI: Just worked to have one - http://issues.apache.org/jira/browse/STORM-2072
This adds map and flatMap to have additional parameter: outputFields. If outputFields are specified, it replaces origin fields with output fields. - Jungtaek Lim (HeartSaVioR) 2016년 8월 31일 (수) 오전 10:27, Jungtaek Lim <[email protected]>님이 작성: > FYI: I just found that Stream class has TODO comment above of class: > *// TODO: need to be able to replace existing fields with the function > fields (like Cascading Fields.REPLACE)* > which is written at Aug. 2012, initial importing Trident source code. > > According to the TODO comment, it seems to be a just missing feature. > > Thanks, > Jungtaek Lim (HeartSaVioR) > > > 2016년 8월 31일 (수) 오전 7:29, Jungtaek Lim <[email protected]>님이 작성: > >> Hi Manu, >> >> I'm finding for 1:1 tuple transition, not aggregation. In other words, >> I'm finding like 'V map(T, Function<T, V>)' which seems fit for many use >> case but Trident doesn't support this yet so I'd like to find out why. >> >> Thanks, >> Jungtaek Lim (HeartSaVioR) >> >> 2016년 8월 31일 (수) 오전 7:09, Manu Zhang <[email protected]>님이 작성: >> >>> I think we have 'aggregate' in Trident, where the function output fields >>> replace the input fields. >>> >>> Thanks, >>> Manu Zhang >>> >>> On Tue, Aug 30, 2016 at 4:17 PM Jungtaek Lim <[email protected]> wrote: >>> >>> > Hi devs, >>> > >>> > While implementing some features of Storm SQL on top of Trident, I >>> realized >>> > that there's no Trident operation which provides a way to add fields >>> from >>> > some Function and remove existing fields within one operation. >>> > 'each' appends the function output fields to origin input fields, and >>> 'map' >>> > and 'flatMap' use same output fields, so no luck. >>> > >>> > Is this a missing feature, or there's a reason for doing so? Trident >>> uses >>> > TridentTupleView so I guess this is related but I didn't have a deep >>> look >>> > for Trident itself. >>> > If anyone knows about the history please let me know. >>> > >>> > Thanks, >>> > Jungtaek Lim (HeartSaVioR) >>> > >>> > ps. It might be better to explain why I'm finding that feature. I'm >>> trying >>> > to minimize below each -> project -> each -> project chain into one. >>> > >>> > >>> https://github.com/apache/storm/blob/master/external/sql/storm-sql-core/src/jvm/org/apache/storm/sql/compiler/backends/trident/TridentLogicalPlanCompiler.java#L164-L167 >>> > Please note that Trident doesn't allow duplicated field name so using >>> > temporary field name is necessary if we should preserve existing >>> fields. >>> > >>> >>
