Hi Ran,

GroupStep in OLTP does use "on the fly"-reduction.

        
https://github.com/apache/incubator-tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/GroupStep.java#L176-L182
                
https://github.com/apache/incubator-tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/util/ReducingBarrierStep.java#L89-L92

That is, if the amount of objects getting grouped becomes to large, a "midway" 
reduction takes place. 

Thoughts?,
Marko.

http://markorodriguez.com

On Feb 10, 2016, at 10:35 AM, Ran Magen <rma...@gmail.com> wrote:

> I'm not sure if its relevant for this ticket, but I have another use-case
> for combiners in an OLTP use-case.
> In Unipop (to which I'll hopefully have documentation very soon and be able
> to share with you guys) we have GroupStep optimizations. But since we can
> have results for the same step coming from different back-ends, we need
> some way to combine them to one result.
> If we had a simple way to combine the results, that would be great. If not,
> we'll just implement something by ourselves...
> 
> On Mon, 8 Feb 2016 at 17:29 Marko A. Rodriguez (JIRA) <j...@apache.org>
> wrote:
> 
>> Marko A. Rodriguez created TINKERPOP-1136:
>> ---------------------------------------------
>> 
>>             Summary: GroupStep OLAP does not combine, only reduces.
>>                 Key: TINKERPOP-1136
>>                 URL: https://issues.apache.org/jira/browse/TINKERPOP-1136
>>             Project: TinkerPop
>>          Issue Type: Improvement
>>          Components: process
>>    Affects Versions: 3.1.1-incubating
>>            Reporter: Marko A. Rodriguez
>>             Fix For: 3.2.0-incubating
>> 
>> 
>> Right now, {{GroupStep}} does not support a combiner. Unfortunately, this
>> is a necessary consequence of not knowing the semantics of the reduction
>> :|. This is analogous to why Spark has {{reduceByKey}} vs. {{groupByKey}}.
>> HOWEVER, if a {{Barrier}} is used for reduction in the {{reduceTraversal}}
>> and that {{Barrier}} is a {{ReducingBarrierStep}}, then a combiner can be
>> used. I suspect that 99% of the time, the {{Barrier}} is a
>> {{ReducingBarrierStep}}. Once TINKERPOP-1131 is merged, it should be
>> possible to make {{GroupStep}} more intelligent by doing an analysis of its
>> {{Barrier}}-step.
>> 
>> 
>> 
>> --
>> This message was sent by Atlassian JIRA
>> (v6.3.4#6332)
>> 

Reply via email to