[jira] [Commented] (TINKERPOP-1578) Support writable distributed graphs using spark as the underlying engine

2016-12-07 Thread Dovid Kopel (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15728804#comment-15728804
 ] 

Dovid Kopel commented on TINKERPOP-1578:


My approach added on the `spark-gremlin` module. I provided mutations using 
spark as the underlying storage mechanism. If one wants to use the `SparkGraph` 
they may do so without using the graph computer directly. If you look at my 
`SparkTest#testSparkGraph()` test you can see this functional.

> Support writable distributed graphs using spark as the underlying engine
> 
>
> Key: TINKERPOP-1578
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1578
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: hadoop
>Affects Versions: 3.2.3
>Reporter: Dovid Kopel
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TINKERPOP-1578) Support writable distributed graphs using spark as the underlying engine

2016-12-07 Thread Marko A. Rodriguez (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15728642#comment-15728642
 ] 

Marko A. Rodriguez commented on TINKERPOP-1578:
---

Your work only works for {{SparkGraphComputer}}. You made your changes at the 
Spark-level, not at the {{GraphComputer}}-level. Thus, its not general. We need 
to be able to have OLAP mutations working for ANY {{GraphComputer}} 
implementation: {{GiraphGraphComputer}}, {{TinkerGraphComputer}}, 
{{SparkGraphComputer}}, etc. This means figuring out the interface additions 
and {{GraphComputerTest}} infrastructure needed to validate the semantics of 
OLAP mutations across all {{GraphComputer}} implementations.

Its a massive amount of work, but would be a huge contribution. As your 
contribution stands right now, it can't be merged as its not provider-agnostic.

> Support writable distributed graphs using spark as the underlying engine
> 
>
> Key: TINKERPOP-1578
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1578
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: hadoop
>Affects Versions: 3.2.3
>Reporter: Dovid Kopel
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TINKERPOP-1578) Support writable distributed graphs using spark as the underlying engine

2016-12-07 Thread Dovid Kopel (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15728436#comment-15728436
 ] 

Dovid Kopel commented on TINKERPOP-1578:


The changes I made didn't affect the `GraphComputer`...can you explain what you 
mean?

> Support writable distributed graphs using spark as the underlying engine
> 
>
> Key: TINKERPOP-1578
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1578
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: hadoop
>Affects Versions: 3.2.3
>Reporter: Dovid Kopel
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TINKERPOP-1578) Support writable distributed graphs using spark as the underlying engine

2016-12-01 Thread Dovid Kopel (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15712441#comment-15712441
 ] 

Dovid Kopel commented on TINKERPOP-1578:


So far I haven't touched graph computer at allit's the original one

Dovid Kopel
do...@dovidkopel.com
443-291-9212
Sent from my Android.




> Support writable distributed graphs using spark as the underlying engine
> 
>
> Key: TINKERPOP-1578
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1578
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: hadoop
>Affects Versions: 3.2.3
>Reporter: Dovid Kopel
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TINKERPOP-1578) Support writable distributed graphs using spark as the underlying engine

2016-12-01 Thread Marko A. Rodriguez (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15711896#comment-15711896
 ] 

Marko A. Rodriguez commented on TINKERPOP-1578:
---

Can you generalize to make work for {{GraphComputer}} and not just 
{{SparkGraphComputer}} ? This way its not a Spark-specific thing?

> Support writable distributed graphs using spark as the underlying engine
> 
>
> Key: TINKERPOP-1578
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1578
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: hadoop
>Affects Versions: 3.2.3
>Reporter: Dovid Kopel
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TINKERPOP-1578) Support writable distributed graphs using spark as the underlying engine

2016-11-30 Thread Dovid Kopel (JIRA) 

[ 
https://issues.apache.org/jira/browse/TINKERPOP-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15709687#comment-15709687
 ] 

Dovid Kopel commented on TINKERPOP-1578:


https://github.com/dovidkopel/tinkerpop/tree/TINKERPOP-1578

I spent around two days and have basic functionality. I did not make a pull 
request yet. I haven't created ample tests and coverage.

I intentionally abstracted the RDD storage from the graph to allow for future 
changes.

I had fully functional traversals using the built-in strategies.

Please let me know what you think so far. If you think this is a reasonable 
approach I can clean things up and make an official pull request.

> Support writable distributed graphs using spark as the underlying engine
> 
>
> Key: TINKERPOP-1578
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1578
> Project: TinkerPop
>  Issue Type: New Feature
>Reporter: Dovid Kopel
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)