Re: Im starting a new project, but tinkerpop3 seems incapable, considering forking tinkerpop2
Hi Jeffrey, I agree that simplicity and hackability were a big plus for TinkerPop2, and that these have taken a bit of a back seat, in TP3, to powerful but heavyweight features like backend support for OLAP. The stackable/pluggable nature of graph implementations took a a hit, as the OOP-friendly graph interfaces (TransactionalGraph, KeyIndexableGraph, IndexableGraph, etc.) were replaced with the more nuanced GraphFeatures, traversal strategies, etc. I wouldn't say that you have to "implement Gremlin" to implement an OLTP graph, though. You get GraphTraversalSource for free. A summer intern I had the pleasure of working with recently wrote a Graph implementation (as a wrapper for another, non-TP graph store) in 660 lines of code. It's not so hard that one would need to revert to TP2. With TP4 on the horizon, it might be worth making a list of "nice to have (again)"s. There are some features I would like to help to bring back, as well. Josh On Tue, Oct 3, 2017 at 5:49 PM, Jeffrey Freeman < jeffrey.free...@syncleus.com> wrote: > Hi, Some of you may already know me as the author of Ferma. This thread is > unrelated to that project, it will continue to support TP2 and TP3 as is. > > So here's the thing, I sued to use TP2 a lot as part of some frameworks I > was working on building (evolutionary algorithms, big data processing, > extremely massive datasets, etc). One technique i was leveraging is how > easy it was for TP2 to support a backend system as a graph. I could take > almost any existing system completely unrelated to graph databases and by > simply implementing edges and vertex in blueprints in a few minutes i could > have the entire TP2 ecosystem working on it and performing traversals. Some > examples of ways i leveraged this in TP2: > > Fusion Graph - A graph driver that allowed me to take two completely > different graph systems (say neo4j and titan) and fuse them so they look > like one graph. I could even connect edges from a vertex in titan with a > vertex in neo4j. > > Recursive graphs - I could make it so edges could contain clusters of edges > and vertexes could contain complete graphs embedded inside them, they could > even be defined by completely different underlying systems. This gave me a > sort of hierarchical graph. > > Apache Storm graphs - I was able to encapsulate the topology from apache > storm as a graph so one could perform traversals across a storm topology as > it is running to produce statistics or to effect its behavior based on > traffic or usage > > MapDB graphs - using MapDB as a graph backend or even a traditional > database or any other storage system not usually seen a a graph. > > The list is really endless. But the problem I'm facing with TP3 is that it > is no longer trivial to implement a Graph. Now you have to pretty much > implement Gremlin for your graph and countless other methods. I get why > this is done, from a performance standpoint if your going to view gremlin > as a query language for graph databases it is needed. But what i need is > some middle ground where I can still implement a Graph as easy as i could > in TP2 even if the end result is rather poor performance on the gremlin > queries (which can be optimized later in some cases as the development > matures). > > As far as i can tell this just isnt possible in TP3, correct me if im wrong > because I'd love to use it for these use cases. If that turns out to be > true and no one here has any better ideas (which id very much welcome) my > next resort would be to revive TP2, fork it as a new project under a new > name, and continue to maintain it as a solution that addresses some of > these needs. i welcome any ideas or inputs from the community on this for > me. >
Im starting a new project, but tinkerpop3 seems incapable, considering forking tinkerpop2
Hi, Some of you may already know me as the author of Ferma. This thread is unrelated to that project, it will continue to support TP2 and TP3 as is. So here's the thing, I sued to use TP2 a lot as part of some frameworks I was working on building (evolutionary algorithms, big data processing, extremely massive datasets, etc). One technique i was leveraging is how easy it was for TP2 to support a backend system as a graph. I could take almost any existing system completely unrelated to graph databases and by simply implementing edges and vertex in blueprints in a few minutes i could have the entire TP2 ecosystem working on it and performing traversals. Some examples of ways i leveraged this in TP2: Fusion Graph - A graph driver that allowed me to take two completely different graph systems (say neo4j and titan) and fuse them so they look like one graph. I could even connect edges from a vertex in titan with a vertex in neo4j. Recursive graphs - I could make it so edges could contain clusters of edges and vertexes could contain complete graphs embedded inside them, they could even be defined by completely different underlying systems. This gave me a sort of hierarchical graph. Apache Storm graphs - I was able to encapsulate the topology from apache storm as a graph so one could perform traversals across a storm topology as it is running to produce statistics or to effect its behavior based on traffic or usage MapDB graphs - using MapDB as a graph backend or even a traditional database or any other storage system not usually seen a a graph. The list is really endless. But the problem I'm facing with TP3 is that it is no longer trivial to implement a Graph. Now you have to pretty much implement Gremlin for your graph and countless other methods. I get why this is done, from a performance standpoint if your going to view gremlin as a query language for graph databases it is needed. But what i need is some middle ground where I can still implement a Graph as easy as i could in TP2 even if the end result is rather poor performance on the gremlin queries (which can be optimized later in some cases as the development matures). As far as i can tell this just isnt possible in TP3, correct me if im wrong because I'd love to use it for these use cases. If that turns out to be true and no one here has any better ideas (which id very much welcome) my next resort would be to revive TP2, fork it as a new project under a new name, and continue to maintain it as a solution that addresses some of these needs. i welcome any ideas or inputs from the community on this for me.
[jira] [Commented] (TINKERPOP-1796) Driver connection pool SSL properties missing
[ https://issues.apache.org/jira/browse/TINKERPOP-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16190264#comment-16190264 ] ASF GitHub Bot commented on TINKERPOP-1796: --- Github user robertdale commented on the issue: https://github.com/apache/tinkerpop/pull/727 Thanks. VOTE +1 > Driver connection pool SSL properties missing > - > > Key: TINKERPOP-1796 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1796 > Project: TinkerPop > Issue Type: Bug > Components: driver >Affects Versions: 3.3.0, 3.2.6 >Reporter: Robert Dale >Assignee: Robert Dale > Fix For: 3.2.7, 3.3.1 > > > When using `Configuration` properties, the keys keyCertChainFile, keyFile, > keyPassword are not picked up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] tinkerpop issue #727: TINKERPOP-1796: Driver connection pool SSL properties ...
Github user robertdale commented on the issue: https://github.com/apache/tinkerpop/pull/727 Thanks. VOTE +1 ---
Re: [DISCUSS] Add Ferma to "Community Contributions" Listing
Thanks a lot I see it. Much appreciated. On Tue, Oct 3, 2017 at 2:18 PM, Stephen Mallette wrote: > Ferma has been added and the site published. Most of the time the site > updates pretty quickly but we're subject to whatever apache infrastrucutre > allows, but I assume it will show up there soon enough. > > On Tue, Oct 3, 2017 at 8:30 AM, Jeffrey Freeman < > jeffrey.free...@syncleus.com> wrote: > > > So its been 72 hours since this was posted. Good it add it now? > > > > On Mon, Oct 2, 2017 at 6:24 AM, Stephen Mallette > > wrote: > > > > > Looks fine to me. > > > > > > On Sun, Oct 1, 2017 at 12:40 PM, Jeffrey Freeman < > > > jeffrey.free...@syncleus.com> wrote: > > > > > > > OK so I changed the reference in the README and the docs as you > > > requested. > > > > Please let me know if that is sufficient or if you need me to change > it > > > > everywhere the word tinkerpop is used (including phrases like > > > "Tinkerpop3" > > > > > > > > On Sun, Oct 1, 2017 at 12:33 PM, Jeffrey Freeman < > > > > jeffrey.free...@syncleus.com> wrote: > > > > > > > > > One question. In some places i reference the specific version as > > > > > "Tinkerpop3" can those references remain as-is, if they are to be > > > > reworded > > > > > whats the proper format for those? > > > > > > > > > > On Sun, Oct 1, 2017 at 12:32 PM, Jeffrey Freeman < > > > > > jeffrey.free...@syncleus.com> wrote: > > > > > > > > > >> Happy to make those changed. If I understand you correct the issue > > is > > > > >> just with naming "Tinkerpop" to "Apache TinkerPop™" to make the > > > > >> trademark more clear? I'm happy to do that, assuming markdown can > > > > support > > > > >> the tm mark, if not ill add it as (tm). > > > > >> > > > > >> On Sun, Oct 1, 2017 at 11:23 AM, Stephen Mallette < > > > spmalle...@gmail.com > > > > > > > > > >> wrote: > > > > >> > > > > >>> +1 from me - pending some fixes in initial references to > > "TinkerPop" > > > in > > > > >>> Ferma README and web site/docs to become "Apache TinkerPop™". > Other > > > > than > > > > >>> that it looks good to me. > > > > >>> > > > > >>> We'll give this 72 hours to see if anyone else has comments - > > > assuming > > > > no > > > > >>> objections we can get Ferma into the listing after that. Thanks > for > > > > your > > > > >>> continued contribution in this area. > > > > >>> > > > > >>> On Sun, Oct 1, 2017 at 11:11 AM, Jeffrey Freeman < > > > > >>> jeffrey.free...@syncleus.com> wrote: > > > > >>> > > > > >>> > Ferma should be listed in the "Community Contributions" section > > of > > > > the > > > > >>> main > > > > >>> > tinkerpop3 apache site ( http://tinkerpop.apache.org ) > > > > >>> > > > > > >>> > Their project/github link: https://github.com/Syncleus/Ferma > > > > >>> > Their website and documentation: http://syncleus.com/Ferma > > > > >>> > > > > > >>> > They meet all requirements in the policy document ( > > > > >>> > http://tinkerpop.apache.org/policy.html ) including full > support > > > for > > > > >>> > Tinkerpop3, their own website, many releases, and > documentation. > > > > >>> > > > > > >>> > I've also created the following ticket to help speed this > along: > > > > >>> > https://issues.apache.org/jira/browse/TINKERPOP-1794 > > > > >>> > > > > > >>> > (FYI I know i sent a similar email earlier, this new e-mail is > > in a > > > > >>> format > > > > >>> > as suggested from a member of the community in that thread) > > > > >>> > > > > > >>> > > > > >> > > > > >> > > > > > > > > > > > > > > >
[GitHub] tinkerpop pull request #728: Do not strong-freeze dependencies
GitHub user aboudreault opened a pull request: https://github.com/apache/tinkerpop/pull/728 Do not strong-freeze dependencies In setup.py, strongly freezing deps is not recommended since it can easily create a conflict with other packages in a user environment. Also, If you know that gremlinpython works well with a lower package version (e.g six 1.6), I recommend to just change it to the minimum required to be more flexible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/aboudreault/tinkerpop gremlinpython-deps Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tinkerpop/pull/728.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #728 commit 64479b3a6c70a006a3493153525c22e271a34b53 Author: Alan Boudreault Date: 2017-10-03T18:12:48Z Do not strong-freeze dependencies ---
Re: [DISCUSS] Add Ferma to "Community Contributions" Listing
Ferma has been added and the site published. Most of the time the site updates pretty quickly but we're subject to whatever apache infrastrucutre allows, but I assume it will show up there soon enough. On Tue, Oct 3, 2017 at 8:30 AM, Jeffrey Freeman < jeffrey.free...@syncleus.com> wrote: > So its been 72 hours since this was posted. Good it add it now? > > On Mon, Oct 2, 2017 at 6:24 AM, Stephen Mallette > wrote: > > > Looks fine to me. > > > > On Sun, Oct 1, 2017 at 12:40 PM, Jeffrey Freeman < > > jeffrey.free...@syncleus.com> wrote: > > > > > OK so I changed the reference in the README and the docs as you > > requested. > > > Please let me know if that is sufficient or if you need me to change it > > > everywhere the word tinkerpop is used (including phrases like > > "Tinkerpop3" > > > > > > On Sun, Oct 1, 2017 at 12:33 PM, Jeffrey Freeman < > > > jeffrey.free...@syncleus.com> wrote: > > > > > > > One question. In some places i reference the specific version as > > > > "Tinkerpop3" can those references remain as-is, if they are to be > > > reworded > > > > whats the proper format for those? > > > > > > > > On Sun, Oct 1, 2017 at 12:32 PM, Jeffrey Freeman < > > > > jeffrey.free...@syncleus.com> wrote: > > > > > > > >> Happy to make those changed. If I understand you correct the issue > is > > > >> just with naming "Tinkerpop" to "Apache TinkerPop™" to make the > > > >> trademark more clear? I'm happy to do that, assuming markdown can > > > support > > > >> the tm mark, if not ill add it as (tm). > > > >> > > > >> On Sun, Oct 1, 2017 at 11:23 AM, Stephen Mallette < > > spmalle...@gmail.com > > > > > > > >> wrote: > > > >> > > > >>> +1 from me - pending some fixes in initial references to > "TinkerPop" > > in > > > >>> Ferma README and web site/docs to become "Apache TinkerPop™". Other > > > than > > > >>> that it looks good to me. > > > >>> > > > >>> We'll give this 72 hours to see if anyone else has comments - > > assuming > > > no > > > >>> objections we can get Ferma into the listing after that. Thanks for > > > your > > > >>> continued contribution in this area. > > > >>> > > > >>> On Sun, Oct 1, 2017 at 11:11 AM, Jeffrey Freeman < > > > >>> jeffrey.free...@syncleus.com> wrote: > > > >>> > > > >>> > Ferma should be listed in the "Community Contributions" section > of > > > the > > > >>> main > > > >>> > tinkerpop3 apache site ( http://tinkerpop.apache.org ) > > > >>> > > > > >>> > Their project/github link: https://github.com/Syncleus/Ferma > > > >>> > Their website and documentation: http://syncleus.com/Ferma > > > >>> > > > > >>> > They meet all requirements in the policy document ( > > > >>> > http://tinkerpop.apache.org/policy.html ) including full support > > for > > > >>> > Tinkerpop3, their own website, many releases, and documentation. > > > >>> > > > > >>> > I've also created the following ticket to help speed this along: > > > >>> > https://issues.apache.org/jira/browse/TINKERPOP-1794 > > > >>> > > > > >>> > (FYI I know i sent a similar email earlier, this new e-mail is > in a > > > >>> format > > > >>> > as suggested from a member of the community in that thread) > > > >>> > > > > >>> > > > >> > > > >> > > > > > > > > > >
[jira] [Closed] (TINKERPOP-1794) Popular OGM Ferma should be listed in "Community Contributions"
[ https://issues.apache.org/jira/browse/TINKERPOP-1794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stephen mallette closed TINKERPOP-1794. --- Resolution: Invalid closing as invalid as this issue was sorted out on the dev list: https://lists.apache.org/thread.html/ead4a156a9981e4bd17a8920bad60ea97e22620f60c2ba63eed4a7a5@%3Cdev.tinkerpop.apache.org%3E > Popular OGM Ferma should be listed in "Community Contributions" > --- > > Key: TINKERPOP-1794 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1794 > Project: TinkerPop > Issue Type: Task > Components: documentation >Affects Versions: 3.2.6 >Reporter: Jeffrey Freeman > Labels: documentation, easyfix > Original Estimate: 10m > Remaining Estimate: 10m > > Ferma should be listed in the "Community Contributions" section of the main > tinkerpop3 apache site ( http://tinkerpop.apache.org ) > Their project/github link: https://github.com/Syncleus/Ferma > Their website and documentation: http://syncleus.com/Ferma > They meet all requirements in the policy document ( > http://tinkerpop.apache.org/policy.html ) including full support for > Tinkerpop3, their own website, many releases, and documentation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (TINKERPOP-1796) Driver connection pool SSL properties missing
[ https://issues.apache.org/jira/browse/TINKERPOP-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16189850#comment-16189850 ] ASF GitHub Bot commented on TINKERPOP-1796: --- Github user boliza commented on the issue: https://github.com/apache/tinkerpop/pull/727 @robertdale done. > Driver connection pool SSL properties missing > - > > Key: TINKERPOP-1796 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1796 > Project: TinkerPop > Issue Type: Bug > Components: driver >Affects Versions: 3.3.0, 3.2.6 >Reporter: Robert Dale >Assignee: Robert Dale > Fix For: 3.2.7, 3.3.1 > > > When using `Configuration` properties, the keys keyCertChainFile, keyFile, > keyPassword are not picked up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] tinkerpop issue #727: TINKERPOP-1796: Driver connection pool SSL properties ...
Github user boliza commented on the issue: https://github.com/apache/tinkerpop/pull/727 @robertdale done. ---
[jira] [Closed] (TINKERPOP-1661) Docker-built documentation does not always point locally
[
https://issues.apache.org/jira/browse/TINKERPOP-1661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Kuppitz closed TINKERPOP-1661.
-
Resolution: Fixed
Fix Version/s: 3.3.1
3.2.7
> Docker-built documentation does not always point locally
>
>
> Key: TINKERPOP-1661
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1661
> Project: TinkerPop
> Issue Type: Improvement
> Components: build-release
>Affects Versions: 3.2.4
>Reporter: Robert Dale
>Assignee: Daniel Kuppitz
>Priority: Minor
> Fix For: 3.2.7, 3.3.1
>
>
> After performing {{docker/build.sh -d}}
> When I pull up the initial URL to look at the docs, it's local -
> http://172.17.0.2/ .
> Anchor links to the same page are also local - http://172.17.0.2/#tutorials .
> However, all links to other pages are remote! Also, you can't just replace
> the server name with the local IP, the URL is slightly different. e.g.
> http://tinkerpop.apache.org/docs/3.3.0-SNAPSHOT/reference
> must be changed to http://172.17.0.2/reference/ in order to view it locally.
> If the reviewer isn't paying attention to the url, s/he might be reviewing
> the wrong content. It would be better if it were all local and/or relative
> paths.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (TINKERPOP-1661) Docker-built documentation does not always point locally
[
https://issues.apache.org/jira/browse/TINKERPOP-1661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16189840#comment-16189840
]
ASF GitHub Bot commented on TINKERPOP-1661:
---
Github user asfgit closed the pull request at:
https://github.com/apache/tinkerpop/pull/725
> Docker-built documentation does not always point locally
>
>
> Key: TINKERPOP-1661
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1661
> Project: TinkerPop
> Issue Type: Improvement
> Components: build-release
>Affects Versions: 3.2.4
>Reporter: Robert Dale
>Assignee: Daniel Kuppitz
>Priority: Minor
> Fix For: 3.2.7, 3.3.1
>
>
> After performing {{docker/build.sh -d}}
> When I pull up the initial URL to look at the docs, it's local -
> http://172.17.0.2/ .
> Anchor links to the same page are also local - http://172.17.0.2/#tutorials .
> However, all links to other pages are remote! Also, you can't just replace
> the server name with the local IP, the URL is slightly different. e.g.
> http://tinkerpop.apache.org/docs/3.3.0-SNAPSHOT/reference
> must be changed to http://172.17.0.2/reference/ in order to view it locally.
> If the reviewer isn't paying attention to the url, s/he might be reviewing
> the wrong content. It would be better if it were all local and/or relative
> paths.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] tinkerpop pull request #725: TINKERPOP-1661 Docker-built documentation does ...
Github user asfgit closed the pull request at: https://github.com/apache/tinkerpop/pull/725 ---
[jira] [Commented] (TINKERPOP-1796) Driver connection pool SSL properties missing
[ https://issues.apache.org/jira/browse/TINKERPOP-1796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16189838#comment-16189838 ] ASF GitHub Bot commented on TINKERPOP-1796: --- Github user robertdale commented on the issue: https://github.com/apache/tinkerpop/pull/727 @boliza I created a JIRA issue. Please review it. Can you change the title to include 'TINKERPOP-1796' and put the link 'https://issues.apache.org/jira/browse/TINKERPOP-1796' in the first comment? > Driver connection pool SSL properties missing > - > > Key: TINKERPOP-1796 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1796 > Project: TinkerPop > Issue Type: Bug > Components: driver >Affects Versions: 3.3.0, 3.2.6 >Reporter: Robert Dale >Assignee: Robert Dale > Fix For: 3.2.7, 3.3.1 > > > When using `Configuration` properties, the keys keyCertChainFile, keyFile, > keyPassword are not picked up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] tinkerpop issue #727: add three missing config in Driver Settings
Github user robertdale commented on the issue: https://github.com/apache/tinkerpop/pull/727 @boliza I created a JIRA issue. Please review it. Can you change the title to include 'TINKERPOP-1796' and put the link 'https://issues.apache.org/jira/browse/TINKERPOP-1796' in the first comment? ---
[jira] [Created] (TINKERPOP-1796) Driver connection pool SSL properties missing
Robert Dale created TINKERPOP-1796: -- Summary: Driver connection pool SSL properties missing Key: TINKERPOP-1796 URL: https://issues.apache.org/jira/browse/TINKERPOP-1796 Project: TinkerPop Issue Type: Bug Components: driver Affects Versions: 3.2.6, 3.3.0 Reporter: Robert Dale Assignee: Robert Dale Fix For: 3.2.7, 3.3.1 When using `Configuration` properties, the keys keyCertChainFile, keyFile, keyPassword are not picked up. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (TINKERPOP-1661) Docker-built documentation does not always point locally
[
https://issues.apache.org/jira/browse/TINKERPOP-1661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16189831#comment-16189831
]
ASF GitHub Bot commented on TINKERPOP-1661:
---
Github user twilmes commented on the issue:
https://github.com/apache/tinkerpop/pull/725
I cleared out my images and rebuilt and all was good.
VOTE: +1
> Docker-built documentation does not always point locally
>
>
> Key: TINKERPOP-1661
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1661
> Project: TinkerPop
> Issue Type: Improvement
> Components: build-release
>Affects Versions: 3.2.4
>Reporter: Robert Dale
>Assignee: Daniel Kuppitz
>Priority: Minor
>
> After performing {{docker/build.sh -d}}
> When I pull up the initial URL to look at the docs, it's local -
> http://172.17.0.2/ .
> Anchor links to the same page are also local - http://172.17.0.2/#tutorials .
> However, all links to other pages are remote! Also, you can't just replace
> the server name with the local IP, the URL is slightly different. e.g.
> http://tinkerpop.apache.org/docs/3.3.0-SNAPSHOT/reference
> must be changed to http://172.17.0.2/reference/ in order to view it locally.
> If the reviewer isn't paying attention to the url, s/he might be reviewing
> the wrong content. It would be better if it were all local and/or relative
> paths.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] tinkerpop issue #725: TINKERPOP-1661 Docker-built documentation does not alw...
Github user twilmes commented on the issue: https://github.com/apache/tinkerpop/pull/725 I cleared out my images and rebuilt and all was good. VOTE: +1 ---
[jira] [Commented] (TINKERPOP-1786) Recipe and missing manifest items for Spark on Yarn
[ https://issues.apache.org/jira/browse/TINKERPOP-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16189804#comment-16189804 ] ASF GitHub Bot commented on TINKERPOP-1786: --- Github user okram commented on a diff in the pull request: https://github.com/apache/tinkerpop/pull/721#discussion_r142426106 --- Diff: hadoop-gremlin/conf/hadoop-gryo.properties --- @@ -29,8 +29,8 @@ gremlin.hadoop.outputLocation=output spark.master=local[4] spark.executor.memory=1g spark.serializer=org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoSerializer +gremlin.spark.persistContext=true --- End diff -- Yes, we know what it does, but by default we have it set to `false` and we don't want to create a backwards breaking usage. > Recipe and missing manifest items for Spark on Yarn > --- > > Key: TINKERPOP-1786 > URL: https://issues.apache.org/jira/browse/TINKERPOP-1786 > Project: TinkerPop > Issue Type: Improvement > Components: hadoop >Affects Versions: 3.3.0, 3.1.8, 3.2.6 > Environment: gremlin-console >Reporter: Marc de Lignie >Priority: Minor > Fix For: 3.2.7, 3.3.1 > > > Thorough documentation for running OLAP queries on Spark on Yarn has been > missing, keeping some users from getting the benefits of this nice feature of > the Tinkerpop stack and resulting in a significant number of questions on the > gremlin users list. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] tinkerpop pull request #721: TINKERPOP-1786 Recipe and missing manifest item...
Github user okram commented on a diff in the pull request: https://github.com/apache/tinkerpop/pull/721#discussion_r142426106 --- Diff: hadoop-gremlin/conf/hadoop-gryo.properties --- @@ -29,8 +29,8 @@ gremlin.hadoop.outputLocation=output spark.master=local[4] spark.executor.memory=1g spark.serializer=org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoSerializer +gremlin.spark.persistContext=true --- End diff -- Yes, we know what it does, but by default we have it set to `false` and we don't want to create a backwards breaking usage. ---
[GitHub] tinkerpop pull request #727: add three missing config in Driver Settings
GitHub user boliza opened a pull request: https://github.com/apache/tinkerpop/pull/727 add three missing config in Driver Settings The configs is: 1. keyCertChainFile 2. keyFile 3. keyPassword You can merge this pull request into a Git repository by running: $ git pull https://github.com/boliza/tinkerpop add_missing_config Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tinkerpop/pull/727.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #727 commit 3cdaaea5ee192b1875dd86217f864d30df287243 Author: Ranger Tsao Date: 2017-10-03T14:44:12Z add three missing config 1. keyCertChainFile 2. keyFile 3. keyPassword ---
Re: [DISCUSS] Gremlin Server Security
Note that there is some discussion on the "math" issue on the gremlin-users mailing list: https://groups.google.com/d/msg/gremlin-users/EFog3lhh4Xw/1VVi2c9J On Wed, Sep 20, 2017 at 11:03 AM, Keith Lohnes wrote: > I agree with Robert that the scripting-off option would work, but would > have to be the default. I've run into people who aren't particularly happy > about having to `def` variables because we've enabled a sandbox. I think > saying, "These are not enabled by default" in the docs would be a > necessity. Being cut off from a feature like lambdas, I'd imagine I'd be > getting quite a few complaints/support tickets with "Why don't my lambdas > work?" That being said I also try to recommend not using lambdas when I > can. > > Aside from math, I wonder a bit about type conversion and string/array > concatenation also come to mind as things users may be doing that I'm not > aware of a Gremlin specific work around for. > > Overall, I really like the idea as I think it forces the user into a better > mindset about access patterns for their data. I think if there's a > reasonable way to address having the flexibility of lambdas, while offering > the security of GLV only execution, that would be ideal. Either the > restrictive subset of that functionality is allowed or, as Robert > mentioned, making some of these things Gremlin steps, such as a `math` step > that evaluates, `plus`, `minus` etc. steps. Maybe a `cast` step and a > `concat` step. > > -Keith Lohnes > > On Wed, Sep 20, 2017 at 9:21 AM Robert Dale wrote: > > > Network security is out of scope. It's generally applicable. There's > > nothing peculiar to Gremlin Server that would warrant its own section on > > network security. > > > > I could see where a security-first approach is taken and scripting would > be > > off by default. This forces the user to be somewhat more aware of > opening a > > potential vulnerability vs. "I don't use scripting so I didn't know to > > disable it". > > > > Ideally, many of the functions used in lambdas would become first-class > > citizens in gremlin. Then there is a much higher potential for turning > (or > > leaving) off scripting. > > > > > > Robert Dale > > > > On Wed, Sep 20, 2017 at 8:23 AM, Stephen Mallette > > wrote: > > > > > It would be an option to turn off script processing, but if you're > saying > > > that you believe that would almost never be possible (as most > production > > > systems will use lambdas) then there probably isn't much point to > adding > > > the option. Scripts will always just be there and users will have to > > > appropriately secure their systems. > > > > > > What patterns do you normally see in the kinds of lambdas you are > > writing? > > > I'm wondering if there were more coarse grained patterns in lambdas > that > > > could be solved with a highly restrictive subset of the scripting > > language. > > > Like a good example of where I tend to get stuck with Gremlin is > "math" - > > > if "math" was a general thing that needed lambdas we could probably > make > > it > > > so that the scriptengine was locked down to only include math > operations. > > > > > > On Tue, Sep 19, 2017 at 3:17 PM, Daniel Kuppitz > wrote: > > > > > > > > > > > > > Obviously lambdas wouldn't work but that might be fine for many > > > > > applications. > > > > > > > > > > > > I would make a bet that lambdas are used in most production systems. > > > When I > > > > work with customers, I usually try to rewrite queries and prevent the > > use > > > > of lambdas, but often there's simply no way to do the same thing > > without > > > > lambdas without taking a performance hit. So if you're going to make > it > > > an > > > > option in Gremlin Server, cool, but we shouldn't disallow lambdas > > > > altogether. That's probably what you meant, but I thought it might be > > > good > > > > to emphasize it. > > > > > > > > Cheers, > > > > Daniel > > > > > > > > > > > > On Tue, Sep 19, 2017 at 11:05 AM, Stephen Mallette < > > spmalle...@gmail.com > > > > > > > > wrote: > > > > > > > > > I just updated the reference docs for Gremlin Server to include > some > > > more > > > > > wording on security. I just wanted to make it more clear that > Gremlin > > > > > Server executes arbitrary code. I like to think people get that and > > > > > understand the implications of what that means from a security > > > > perspective, > > > > > but.. > > > > > > > > > > I didn't add much more on "how to secure Gremlin Server" because I > > > think > > > > > what we allow for is pretty much well documented: > > > > > > > > > > 1. Authentication > > > > > 2. Encryption > > > > > 3. Script Execution Management > > > > > > > > > > I feel like there might be a fourth category that involves > discussing > > > how > > > > > to physically protect Gremlin Server with firewall/network stuff, > but > > > I'm > > > > > probably not the best person to write that (or it's simply out of > > scope > > > > for > > > > > our refere
Re: [DISCUSS] Add Ferma to "Community Contributions" Listing
So its been 72 hours since this was posted. Good it add it now? On Mon, Oct 2, 2017 at 6:24 AM, Stephen Mallette wrote: > Looks fine to me. > > On Sun, Oct 1, 2017 at 12:40 PM, Jeffrey Freeman < > jeffrey.free...@syncleus.com> wrote: > > > OK so I changed the reference in the README and the docs as you > requested. > > Please let me know if that is sufficient or if you need me to change it > > everywhere the word tinkerpop is used (including phrases like > "Tinkerpop3" > > > > On Sun, Oct 1, 2017 at 12:33 PM, Jeffrey Freeman < > > jeffrey.free...@syncleus.com> wrote: > > > > > One question. In some places i reference the specific version as > > > "Tinkerpop3" can those references remain as-is, if they are to be > > reworded > > > whats the proper format for those? > > > > > > On Sun, Oct 1, 2017 at 12:32 PM, Jeffrey Freeman < > > > jeffrey.free...@syncleus.com> wrote: > > > > > >> Happy to make those changed. If I understand you correct the issue is > > >> just with naming "Tinkerpop" to "Apache TinkerPop™" to make the > > >> trademark more clear? I'm happy to do that, assuming markdown can > > support > > >> the tm mark, if not ill add it as (tm). > > >> > > >> On Sun, Oct 1, 2017 at 11:23 AM, Stephen Mallette < > spmalle...@gmail.com > > > > > >> wrote: > > >> > > >>> +1 from me - pending some fixes in initial references to "TinkerPop" > in > > >>> Ferma README and web site/docs to become "Apache TinkerPop™". Other > > than > > >>> that it looks good to me. > > >>> > > >>> We'll give this 72 hours to see if anyone else has comments - > assuming > > no > > >>> objections we can get Ferma into the listing after that. Thanks for > > your > > >>> continued contribution in this area. > > >>> > > >>> On Sun, Oct 1, 2017 at 11:11 AM, Jeffrey Freeman < > > >>> jeffrey.free...@syncleus.com> wrote: > > >>> > > >>> > Ferma should be listed in the "Community Contributions" section of > > the > > >>> main > > >>> > tinkerpop3 apache site ( http://tinkerpop.apache.org ) > > >>> > > > >>> > Their project/github link: https://github.com/Syncleus/Ferma > > >>> > Their website and documentation: http://syncleus.com/Ferma > > >>> > > > >>> > They meet all requirements in the policy document ( > > >>> > http://tinkerpop.apache.org/policy.html ) including full support > for > > >>> > Tinkerpop3, their own website, many releases, and documentation. > > >>> > > > >>> > I've also created the following ticket to help speed this along: > > >>> > https://issues.apache.org/jira/browse/TINKERPOP-1794 > > >>> > > > >>> > (FYI I know i sent a similar email earlier, this new e-mail is in a > > >>> format > > >>> > as suggested from a member of the community in that thread) > > >>> > > > >>> > > >> > > >> > > > > > >
