Cole Greer created TINKERPOP-3210:
-------------------------------------
Summary: cap() has strange interactions with barriers and side
effects in OLAP
Key: TINKERPOP-3210
URL: https://issues.apache.org/jira/browse/TINKERPOP-3210
Project: TinkerPop
Issue Type: Improvement
Components: process
Affects Versions: 3.7.4, 3.2.0-incubating
Reporter: Cole Greer
There's an old bug in cap() stemming from how it updates side effects.
`TraversalSideEffects.set()` should never be called from distributed
environments (stated clearly in the
[javadocs|https://tinkerpop.apache.org/javadocs/3.7.4/full/org/apache/tinkerpop/gremlin/process/traversal/TraversalSideEffects.html#set(java.lang.String,java.lang.Object)]).
This means that for OLAP, this should never be called by workers.
The [current implementation of
cap()|https://github.com/apache/tinkerpop/blob/fa698ba2aba8967dcd17eb61cb13648b934fab5b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/sideEffect/SideEffectCapStep.java#L95]
uses `TraversalSideEffects.set()` to update the side-effect with the final
post-processed result. There are no controls here which verify it is safe to
call `TraversalSideEffects.set()`, and while this does not lead to issues for
most queries, there are quite a few combinations of side-effect steps, cap, and
barriers which result in exceptions. This bug dates back to at least
3.2.0-incubating, which included massive changes to the implementation of side
effects and barriers in OLAP, and remains present in 3.7.4.
{code:java}
gremlin> Gremlin.version()
==>3.2.0-incubating
gremlin> g = TinkerFactory.createModern().traversal().withComputer()
==>graphtraversalsource[tinkergraph[vertices:6 edges:6], graphcomputer]
gremlin> g.V().aggregate("a").cap("a").unfold().barrier().both()
java.lang.RuntimeException: java.lang.IllegalStateException:
java.lang.IllegalArgumentException: The memory can only be set() during vertex
program setup and terminate: a
Display stack trace? [yN] n
gremlin>
g.V().both().groupCount("a").out().fold().cap("a").select(Column.keys).unfold().both()
java.lang.RuntimeException: nava.lang.IllegalStateException:
java.lang.IllegalArgumentException: The memory can only be set() during vertex
program setup and terminate: a
{code}
{code:java}
gremlin> Gremlin.version()
==>3.7.4
gremlin> g = TinkerFactory.createModern().traversal().withComputer()
==>graphtraversalsource[tinkergraph[vertices:6 edges:6], graphcomputer]
gremlin> g.V().aggregate("a").cap("a").unfold().barrier().both()
java.lang.RuntimeException: java.lang.IllegalStateException:
java.lang.IllegalArgumentException: The memory can only be set() during vertex
program setup and terminate: a
Type ':help' or ':h' for help.
Display stack trace? [yN]n
gremlin>
g.V().both().groupCount("a").out().fold().cap("a").select(Column.keys).unfold().both()
java.lang.RuntimeException: java.lang.IllegalStateException:
java.lang.IllegalArgumentException: The memory can only be set() during vertex
program setup and terminate: a
Type ':help' or ':h' for help.
Display stack trace? [yN]y
java.lang.IllegalStateException: java.lang.RuntimeException:
java.lang.IllegalStateException: java.lang.IllegalArgumentException: The memory
can only be set() during vertex program setup and terminate: a
at
org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.VertexProgramStep.processNextStart(VertexProgramStep.java:88)
at
org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:155)
at
org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:55)
at
org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.ComputerResultStep.processNextStart(ComputerResultStep.java:68)
at
org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:155)
at
org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.hasNext(DefaultTraversal.java:192)
at
org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
at
org.apache.tinkerpop.gremlin.console.Console$_closure3.doCall(Console.groovy:268)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:343)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:328)
at
org.codehaus.groovy.runtime.metaclass.ClosureMetaClass.invokeMethod(ClosureMetaClass.java:280)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1007)
at groovy.lang.Closure.call(Closure.java:433)
at groovy.lang.Closure.call(Closure.java:422)
at
org.apache.groovy.groovysh.Groovysh.setLastResult(Groovysh.groovy:483)
at org.apache.groovy.groovysh.Groovysh.execute(Groovysh.groovy:209)
at
org.apache.tinkerpop.gremlin.console.GremlinGroovysh.super$3$execute(GremlinGroovysh.groovy)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:343)
at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:328)
at groovy.lang.MetaClassImpl.doInvokeMethod(MetaClassImpl.java:1333)
at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1088)
at
org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:146)
at
org.apache.tinkerpop.gremlin.console.GremlinGroovysh.execute(GremlinGroovysh.groovy:87)
at
org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
at org.apache.groovy.groovysh.Shell.leftShift(Shell.groovy:121)
at
org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
at org.apache.groovy.groovysh.ShellRunner.work(ShellRunner.groovy:93)
at
org.apache.groovy.groovysh.InteractiveShellRunner.work(InteractiveShellRunner.groovy:124)
at
org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
at org.apache.groovy.groovysh.ShellRunner.run(ShellRunner.groovy:57)
at
org.apache.groovy.groovysh.InteractiveShellRunner.run(InteractiveShellRunner.groovy:96)
at
org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
at
org.apache.tinkerpop.gremlin.console.Console.<init>(Console.groovy:181)
at
org.codehaus.groovy.vmplugin.v8.IndyInterface.fromCache(IndyInterface.java:321)
at org.apache.tinkerpop.gremlin.console.Console.main(Console.groovy:580)
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException:
java.lang.IllegalStateException: java.lang.IllegalArgumentException: The memory
can only be set() during vertex program setup and terminate: a
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at
org.apache.tinkerpop.gremlin.process.computer.traversal.step.map.VertexProgramStep.processNextStart(VertexProgramStep.java:68)
... 42 more
Caused by: java.lang.RuntimeException: java.lang.IllegalStateException:
java.lang.IllegalArgumentException: The memory can only be set() during vertex
program setup and terminate: a
at
org.apache.tinkerpop.gremlin.tinkergraph.process.computer.TinkerGraphComputer.lambda$submit$3(TinkerGraphComputer.java:252)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalStateException: java.lang.IllegalArgumentException:
The memory can only be set() during vertex program setup and terminate: a
at
org.apache.tinkerpop.gremlin.tinkergraph.process.computer.TinkerWorkerPool.executeVertexProgram(TinkerWorkerPool.java:115)
at
org.apache.tinkerpop.gremlin.tinkergraph.process.computer.TinkerGraphComputer.lambda$submit$3(TinkerGraphComputer.java:177)
... 4 more
Caused by: java.util.concurrent.ExecutionException:
java.lang.IllegalArgumentException: The memory can only be set() during vertex
program setup and terminate: a
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at
org.apache.tinkerpop.gremlin.tinkergraph.process.computer.TinkerWorkerPool.executeVertexProgram(TinkerWorkerPool.java:111)
... 5 more
Caused by: java.lang.IllegalArgumentException: The memory can only be set()
during vertex program setup and terminate: a
at
org.apache.tinkerpop.gremlin.process.computer.Memory$Exceptions.memorySetOnlyDuringVertexProgramSetUpAndTerminate(Memory.java:159)
at
org.apache.tinkerpop.gremlin.tinkergraph.process.computer.TinkerMemory.set(TinkerMemory.java:127)
at
org.apache.tinkerpop.gremlin.tinkergraph.process.computer.TinkerWorkerMemory.set(TinkerWorkerMemory.java:88)
at
org.apache.tinkerpop.gremlin.process.computer.traversal.MemoryTraversalSideEffects.set(MemoryTraversalSideEffects.java:61)
at
org.apache.tinkerpop.gremlin.process.traversal.step.sideEffect.SideEffectCapStep.supply(SideEffectCapStep.java:95)
at
org.apache.tinkerpop.gremlin.process.traversal.step.util.SupplyingBarrierStep.processNextStart(SupplyingBarrierStep.java:72)
at
org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:155)
at
org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:55)
at
org.apache.tinkerpop.gremlin.process.traversal.step.map.TraversalMapStep.processNextStart(TraversalMapStep.java:50)
at
org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:155)
at
org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:55)
at
org.apache.tinkerpop.gremlin.process.traversal.step.map.FlatMapStep.processNextStart(FlatMapStep.java:48)
at
org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:155)
at
org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:55)
at
org.apache.tinkerpop.gremlin.process.traversal.step.map.FlatMapStep.processNextStart(FlatMapStep.java:48)
at
org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:155)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:132)
at
org.apache.tinkerpop.gremlin.process.computer.traversal.WorkerExecutor.drainStep(WorkerExecutor.java:231)
at
org.apache.tinkerpop.gremlin.process.computer.traversal.WorkerExecutor.execute(WorkerExecutor.java:152)
at
org.apache.tinkerpop.gremlin.process.computer.traversal.TraversalVertexProgram.execute(TraversalVertexProgram.java:307)
at
org.apache.tinkerpop.gremlin.tinkergraph.process.computer.TinkerGraphComputer.lambda$submit$0(TinkerGraphComputer.java:182)
at
org.apache.tinkerpop.gremlin.tinkergraph.process.computer.TinkerWorkerPool.lambda$executeVertexProgram$0(TinkerWorkerPool.java:103)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
... 4 more
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
