[
https://issues.apache.org/jira/browse/TINKERPOP-1153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15150924#comment-15150924
]
ASF GitHub Bot commented on TINKERPOP-1153:
-------------------------------------------
GitHub user okram opened a pull request:
https://github.com/apache/incubator-tinkerpop/pull/230
TINKERPOP-1153 & TINKERPOP-1154: (Summary) Multi-OLAP compilation in a
single Traversal (hardened)
https://issues.apache.org/jira/browse/TINKERPOP-1154
https://issues.apache.org/jira/browse/TINKERPOP-1153
There is a lot in this pull request. While creating the infrastructure to
allow us to do multi-OLAP jobs in a single Traversal compilation, I ended up
cleaning numerous ancient areas of the codebase that were awkward with respects
to latest developments. Everything is backwards compatible though there are
lots of Deprecations. However, there is a new `GraphComptuerTest` that requires
that `GraphComputer` providers support "compute key revival." I had to do a
small tweak to both `TinkerGraphComputer` and `SparkGraphComputer` for them to
pass the test. `GiraphGraphComputer` just handled it. For users, this ticket
basically provides them:
```
g.V().hasLabel('person').
pageRank(0.85).
by('friendRank').
by(outE('knows')).
times(10).
out('worksFor').order().by('friendRank').limit(10).valueMap('name','friendRank')
```
CHANGELOG
```
* Added `PeerPressureVertexProgramStep` and `GraphTraversal.peerPressure()`.
* Added `PureTraversal` for handling pure and compiled versions of a
`Traversal`. Useful in OLAP.
* Added `ScriptTraversal` which allows for delayed compilation of
script-based `Traversals`.
* Simplified `VertexProgram` implementations with a `PureTraversal`-model
and deprecated `ConfigurationTraversal`.
* Simplified script-based `Traversals` via `ScriptTraversal` and deprecated
`TraversalScriptFunction` and `TraversalScriptHelper`.
* Added `ByModulating` interface which allows the `Step` to decide how a
`by()`-modulation should be handled. (*breaking*)
* Simplified the `by()`-modulation patterns of `OrderGlobalStep` and
`OrderLocalStep`.
* Added `GraphComputerTest.shouldSupportPreExistingComputeKeys()` to ensure
existing compute keys are "revived." (*breaking*)
```
UPDATE DOCS
```
For providers that support their own `GraphComputer` implementation, note
that there is a new `GraphComputerTest.shouldSupportPreExistingComputeKeys()`.
When chaining OLAP jobs together, if an OLAP job requires the compute keys of a
previous OLAP job, then the existing compute keys must be accessible. A simple
2 line change to `SparkGraphComputer` and `TinkerGraphComputer` solved this for
TinkerPop. `GiraphGraphComputer` did not need an update as this feature was
already naturally supported.
If the provider has custom steps that leverage `by()`-modulation, those
still will now need to implement `ByModulating`. Most of the methods in
`ByModulating` are `default` and, for most situations, only
`ByModulating.modulateBy(Traversal)` should be implemented. Note that this
method's body will most like be identical to `TraversalParent.addLocalChild()`.
Providers that have custom Gremlin language implementations (e.g.
Gremlin-Scala), there is a new class called `ScriptTraversal` which will handle
script-based processing of traversals. The entire `GroovyXXXTest`-suite was
updated to use this new class. The previous `TraversalScriptHelper` class has
been deprecated so immediate upgrading is not required.
```
REFERENCE DOCS
*** I will do these at the time of merge.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/apache/incubator-tinkerpop TINKERPOP-1154
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-tinkerpop/pull/230.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #230
----
commit f2bec67b97a6d0b1dad1392bbdd6f51525686da8
Author: Marko A. Rodriguez <okramma...@gmail.com>
Date: 2016-02-15T15:30:46Z
Added ScriptTraversal which contains a String script and reference to a
traversal source and script engine. This greatly simplifies serializing
traversals and will enable us to just talk always in terms of Traversal and not
worry about if its a serializable or non-serializable traversal.
commit 44f78159496737b8c0d7d714fe8dc747bfa5842a
Author: Marko A. Rodriguez <okramma...@gmail.com>
Date: 2016-02-15T15:31:49Z
revived a deprecated method for backwards compatibility.
commit 94ffac91b39340b7ae7f2c3609c8715339183f7d
Author: Marko A. Rodriguez <okramma...@gmail.com>
Date: 2016-02-15T18:25:01Z
Added ByModulating interface which has numerous modulateBy() overloads.
Interestingly, all the default call the respective Traversal-form. This makes
it much easier for steps to reason about incoming traversals from
by()-modulation and do something different with it. GraphTraversal no longer
reasons about by(). It simply calls
((ByModulator)getEndStep()).modulateBy(...). Clean logic. Added more test cases
to OrderTest as we didn't have tests for a few of the by() modulators. I
believe we will be able to get rid of
TraversalParent.addXXXChild()/removeXXXChild() as these were provided so that
Traversal could add modulation. TINKERPOP-1153
commit 55e5559d0ea68843a2dff4d0c3f286a6a6f29f12
Author: Marko A. Rodriguez <okramma...@gmail.com>
Date: 2016-02-15T19:29:56Z
Fixed TINKERPOP-1105. DetachedVertexProperties were being serialized as
DetachedProperty by GryoSerializers.
commit 45b097552710a4884af89bcd67868b689a918a9f
Author: Marko A. Rodriguez <okramma...@gmail.com>
Date: 2016-02-15T19:48:04Z
Merge branch 'master' into TINKERPOP-1154
commit c3eb653cbd3cb38e0925f0c1c21f4e5761637b79
Author: Marko A. Rodriguez <okramma...@gmail.com>
Date: 2016-02-16T16:31:22Z
Simplified the configuration internals of TraversalVertexProgram,
PageRankVertexProgram, and PeerPressureVertexProgram. Stubbed the ability to
allow these vertex programs determine which source vertices to use for their
computation. Added another PageRankTest which verifies the new ByModulating
behavior for PageRankVertexProgramStep --
g.V().pageRank().by('friendRank').by(outE('knows')). Introduced PureTraversal
which contains an uncompiled and compiled form of a Traversal. Useful for OLAP
based VertexPrograms and Steps.
commit d06329eef60bf01da072f107b411d8554ec79203
Author: Marko A. Rodriguez <okramma...@gmail.com>
Date: 2016-02-16T16:42:57Z
added ColumnTraversal for Order-based column selection. Added more test
cases to LambdaRestrictionStrategyTest. Split the test suite into 'allowed' and
'not allowed' traversals so its easier to test the scope of what is considered
'lambda'.
commit 010c9f343df16d09266560631471147edb4f442f
Author: Marko A. Rodriguez <okramma...@gmail.com>
Date: 2016-02-16T18:56:37Z
little more work. saving what I have for a big push next.
commit 88f24c7f5a889520655fb645010eead3e2815078
Author: Marko A. Rodriguez <okramma...@gmail.com>
Date: 2016-02-16T22:15:50Z
Through hell and back. You can now go
TraversalVertexProgram->PageRankVertexProgramStep->TraversalVertexPrograph->OLTP.
I had to create the concept of 'reving and old view.' VertexProgramStrategy is
much more flexible. Still needs alot of work, but we are getting there.
commit 01b4a539b9a5a2cd28ac66adcc2bf5843d5e1769
Author: Marko A. Rodriguez <okramma...@gmail.com>
Date: 2016-02-16T23:17:09Z
added GraphComputerTest.shouldSupportPreExistingComputeKeys(). This is
necessary when job chaining and HALTED_TRAVERSERS are later required by another
TraversalVertexProgram.
commit 86284f2c8f6c13782e5165c37e35c7a56a66192a
Author: Marko A. Rodriguez <okramma...@gmail.com>
Date: 2016-02-17T03:45:15Z
TraversalVertexProgram now uses the GraphStep instead of reasoning on its
internal state. This simplifies the code and stages us for mid-traversal
V()/E(). Added a new GraphTest. Fixed an old problem with
TraversalVertexProgram and headless-traversals. I just changed and old
GraphComputerTest and it worked. Sweet.
commit 969472f065f8d4a6792a512f9c3bbcc9d2c3ceb6
Author: Marko A. Rodriguez <okramma...@gmail.com>
Date: 2016-02-17T12:43:40Z
GraphTraversalSource, for all its withXXX-methods, simply calls
TraversalSource.super.withXXX. Now that we have
TraversalSource.getStrategies(), its super easy.
commit 11238b630bfc9af71f0a443eada36381c9f5fa15
Author: Marko A. Rodriguez <okramma...@gmail.com>
Date: 2016-02-17T16:01:47Z
Created VertexProgramStep which is abstract and provides
processNextStarts() functionality to TraversalVertexProgramStep and
PageRankVertexProgramStep. Moved the PureTraversal model into the
VertexProgramSteps. Added TimesModulating which allows steps to do what they
wish with times(int). pageRank().times(20) for example. General cleanup.
commit 2d5cd1c0d8ec3f687f237cdf9be4b0b74a73af56
Author: Marko A. Rodriguez <okramma...@gmail.com>
Date: 2016-02-17T17:04:22Z
VertexProgramStrategy is now smart about GraphStep.PageRank. It will just
propagate the GraphStep forward to reduce the number of OLAP jobs required.
Added VertexProgramStrategyTest which verifies the compilation of a OLAP
traversal.
commit 622f84b9dc94ae5ef1b6c174fcf360cef2366d37
Author: Marko A. Rodriguez <okramma...@gmail.com>
Date: 2016-02-17T18:04:27Z
Added PeerPressureVertexProgramStep and GraphTraversal.peerPressure().
Added GraphTraversal.pageRank(alpha). Ensured that VertexComputing steps are
NOT used in StandardTraversal.
----
> Add ByModulating and TimesModulating interfaces.
> ------------------------------------------------
>
> Key: TINKERPOP-1153
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1153
> Project: TinkerPop
> Issue Type: Improvement
> Components: process
> Affects Versions: 3.1.2-incubating
> Reporter: Marko A. Rodriguez
> Fix For: 3.2.0-incubating
>
>
> Currently {{GraphTraversal}} is responsible for the logic for {{by()}}- and
> {{times()}}-modulators. For instance, if you do:
> {code}
> ...by('name')
> {code}
> It is {{GraphTraversal}} that will compile that to
> {{__.values('name').limit(1)}}. This should not be the logic of
> {{GraphTraversal}}, but instead, logic within the step being modulated. Thus,
> I propose:
> {code}
> public interface ByModulating {
> public void add(String string) throws UnsupportedOperationException
> public void add(Traversal traversal) throws UnsupportedOperationException
> public void add(T t) throws UnsupportedOperationException
> }
> {code}
> Likewise for {{TimesModulating}} ... and any other modulators down the road.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
