date:20160830

Re: Building gremlin-python

2016-08-30 Thread David Brown

Nice work with this Stephen! One thing: if I am thinking about this
correctly, pip isn't a requirement for building. I think we are only
using pip inside the virtualenv, and since the virtualenv binary
installs pip and setuptools by default when in creates the
environment, I don't think we need it to be installed on the system.
So basically, if you have Python(2) on on your machine, you only need
to make sure that you install virtualenv before building...make sense?
or am I missing something?

On Tue, Aug 30, 2016 at 5:32 PM, Stephen Mallette  wrote:
> With some help from Dave I've tweaked up the build for python some more -
> the command to build is still:
>
> mvn clean install -pl gremlin-python -DglvPython
>
> but now:
>
> + The prereqs for building are just python 2.7, pip and virtualenv
> + The build uses virtualenv to isolate the python environment to the
> /target directories
> + -DskipTests is now better respected by Gremlin Server - won't bother to
> start it, if tests are skipped
> + Minor adjustments to the deploy phase for python. We can't really deploy
> to pypi on deploy in our release process because pypi has no staging
> environment (like sonatype does) so the release would be immediately public
> prior to vote on the release. So i made a special "-Dpypi" that activates a
> profile that will hook into "mvn deploy" and push the artifacts to pypi
> with twine.
>
> Note that these changes are all in the "virtualenv" branch at the moment. I
> suspect I will merge it to master tomorrow after some more tests.
>
>
>
> On Mon, Aug 29, 2016 at 5:45 PM, Dylan Millikin 
> wrote:
>
>> Yeah this is great. Will be useful for all GLVs.
>>
>> On Mon, Aug 29, 2016 at 6:13 PM, David Brown  wrote:
>>
>> > Wow Stephen thanks for all your hard work! This will really make
>> > driver development a lot easier.
>> >
>> > On Sun, Aug 28, 2016 at 6:48 PM, Stephen Mallette 
>> > wrote:
>> > > Took me half of my Sunday, but I just got Gremlin Server to start up
>> and
>> > > shutdown in the standard maven integration-test phase of
>> gremlin-python.
>> > > That means we can now write native python tests that round-trip to
>> > Gremlin
>> > > Server in addition to the regular unit tests with pytest we already
>> > have! I
>> > > actual  Note that I actually start two gremlin servers one with auth on
>> > and
>> > > one without (ports 8183/8182 respectively) so that we can test
>> > > authentication features. Further note that native tests are bound to
>> the
>> > > integration-test phase of maven and run automatically on -glvPython.
>> > that's
>> > > a bit different from our other projects where integration test phases
>> > only
>> > > run if you -DskipIntegrationTests=false. gremlin-python won't obey that
>> > > setting.
>> > >
>> > >
>> > >
>> > > On Sat, Aug 27, 2016 at 6:05 AM, Stephen Mallette <
>> spmalle...@gmail.com>
>> > > wrote:
>> > >
>> > >> Now that gremlin-python has been merged to master, you might wonder
>> > about
>> > >> what this has done to our build/release process. Well, not very much.
>> If
>> > >> you do your standard
>> > >>
>> > >> mvn clean install
>> > >>
>> > >> on master right now you should see that everything builds and is good
>> to
>> > >> go - even gremlin-python. So, everything is good, right? right? well,
>> > yes
>> > >> and no.
>> > >>
>> > >> The "yes" aspect here is that the entire project still builds with
>> maven
>> > >> which keeps our build toolchain simple. Users can just have java
>> > installed
>> > >> as they always did and still get a clean build of TinkerPop. The "no"
>> > >> aspect is that native python tests (and if you were deploy,
>> > >> packaging/deployment tasks) did not execute.
>> > >>
>> > >> What's good however is that even the native python build tasks are
>> still
>> > >> just part of the maven toolchain. You just need to have python 2.x
>> > >> installed and, if you do, build with:
>> > >>
>> > >> mvn clean install -DglvPython
>> > >>
>> > >> You will now see in your output the results of native pytest
>> execution.
>> > I
>> > >> think this approach almost sets the basic pattern for future GLVs. I'd
>> > >> prefer to not have -glvPython to some degree and simply detect python
>> on
>> > >> the system and then execute natively if it can, but then i think about
>> > what
>> > >> happens as we add more GLVs and then i sorta like the idea of having
>> the
>> > >> specific option to turn things on and off.
>> > >>
>> > >>
>> > >>
>> >
>> >
>> >
>> > --
>> > David M. Brown
>> > R.A. CulturePlex Lab, Western University
>> >
>>



-- 
David M. Brown
R.A. CulturePlex Lab, Western University

[jira] [Created] (TINKERPOP-1423) IncidentToAdjacentStrategy should be disabled for tree steps

2016-08-30 Thread Carl LeCompte (JIRA)

Carl LeCompte created TINKERPOP-1423:


 Summary: IncidentToAdjacentStrategy should be disabled for tree 
steps
 Key: TINKERPOP-1423
 URL: https://issues.apache.org/jira/browse/TINKERPOP-1423
 Project: TinkerPop
  Issue Type: Bug
  Components: process
Affects Versions: 3.2.0-incubating
Reporter: Carl LeCompte
Priority: Minor


IncidentToAdjacentStrategy causes edges to be dropped from the result of a 
.tree() step, even if explicitly traversed. This is conceptually equivalent to 
obtaining traversed edges via a .path() step, therefore the behavior should be 
the same (in other words, the traversal should not be modified if a .tree() 
step is present).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TINKERPOP-1230) Serialising lambdas for RemoteGraph

2016-08-30 Thread Michael Pollmeier (JIRA)


[ 
https://issues.apache.org/jira/browse/TINKERPOP-1230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449827#comment-15449827
 ] 

Michael Pollmeier commented on TINKERPOP-1230:
--

Sorry for mixing up the versions of documentation. The 'lambda' solution is 
sending code as strings over the wire, that's not quite what I had in mind. It 
doesn't really matter for dynamic languages, but if used with a statically 
compiled language like Scala or Java you lose all the help from the compiler / 
IDE.

I see what you're saying re my lambda serialisation project - not ideal. 
Another option would be to allow to extend gremlin-server with some endpoints 
and additional libraries - the current plugin system is not powerful enough for 
that task, is it? 

One of the things I'm trying to figure out is how I can use Gremlin-Scala with 
DSE graph. I'm having a hard time to see how that could work, since DSE is 
closed source and doesn't really allow to extend it with additional libraries. 
But since DSE doesn't have a remote protocol (unlike Titan) and I can't really 
send remote queries to the integrated Gremlin Server (at least not without 
losing all the help from the compiler), I don't currently see a way to 
integrate with it. If this is not the right place to discuss such matters, just 
let me know - maybe chat is better?

> Serialising lambdas for RemoteGraph
> ---
>
> Key: TINKERPOP-1230
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1230
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: driver, server
>Affects Versions: 3.1.1-incubating
>Reporter: Michael Pollmeier
>Assignee: Marko A. Rodriguez
>Priority: Minor
> Fix For: 3.2.2
>
>
> I just made an attempt to serialise lambdas and send them via the 
> RemoteGraph. I didn't quite get there, but wanted to share my findings: 
> * it's possible to serialise lambdas on the jvm by just extending 
> `Serializable`:
> http://stackoverflow.com/questions/22807912/how-to-serialize-a-lambda/22808112#22808112
> * sending a normal predicate doesn't work (this is a Scala REPL but it should 
> be pretty easy to convert this to java/groovy)
>   val g = RemoteGraph.open("conf/remote-graph.properties").traversal()
>   val pred1 = new java.util.function.Predicate[Traverser[Vertex]] { def 
> test(v: Traverser[Vertex]) = true }
>   g.V().filter(pred1).toList 
> // java.lang.RuntimeException: java.io.NotSerializableException: $anon$1
>  // on server: nothing
>  
> * simply adding Serializable let's us send it over the wire, but the server 
> doesn't deserialise it
>   val pred2 = new java.util.function.Predicate[Traverser[Vertex]] with 
> Serializable { def test(v: Traverser[Vertex]) = true }
>   g.V().filter(pred2).toList 
>   // on server: [WARN] OpExecutorHandler - Could not deserialize the 
> Traversal instance
> org.apache.tinkerpop.gremlin.server.op.OpProcessorException: Could 
> not deserialize the Traversal instance
> at 
> org.apache.tinkerpop.gremlin.server.op.traversal.TraversalOpProcessor.iterateOp(TraversalOpProcessor.java:135)
> at 
> org.apache.tinkerpop.gremlin.server.handler.OpExecutorHandler.channelRead0(OpExecutorHandler.java:68)
>   // on client: 
> org.apache.tinkerpop.gremlin.driver.exception.ResponseException: $anon$1 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TINKERPOP-1278) Implement Gremlin-Python and general purpose language variant test infrastructure

2016-08-30 Thread stephen mallette (JIRA)


[ 
https://issues.apache.org/jira/browse/TINKERPOP-1278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449778#comment-15449778
 ] 

stephen mallette commented on TINKERPOP-1278:
-

[~stamhankar999] thanks for your thoughts on the testing aspects of things. i 
think we have some work to do before we can call the testing model completely 
settled for all GLVs. I suspect some changes will occur. I think we need a good 
mix of native tests as well as those that can be executed via the process 
suites. We will try to come to a full understanding of that model for 3.3.x.

[~aholmber] i remember your commit comment. i think that happened when [~okram] 
was away on vacation. since he wrote the code generation stuff for that, 
perhaps he'd be best to have a look and address your concerns.

> Implement Gremlin-Python and general purpose language variant test 
> infrastructure
> -
>
> Key: TINKERPOP-1278
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1278
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: language-variant
>Affects Versions: 3.2.0-incubating
>Reporter: Marko A. Rodriguez
>Assignee: Marko A. Rodriguez
> Fix For: 3.2.2
>
>
> As discussed on dev@...
> Apache TinkerPop should provide, out-of-the-box, at least 3 Gremlin language 
> variants. It would be cool if these were:
> * Python (Mark Henderson)
> * PHP ([~PommeVerte])
> * Ruby (?[~okram])
> I think each of these should be generated using the reflection-model 
> presented in 
> http://tinkerpop.apache.org/docs/3.2.1-SNAPSHOT/tutorials/gremlin-language-variants/.
>  Moreover, on every {{mvn clean install}}, the code for these variants is 
> generated.
> Given the desire to separate language variants from language drivers, I think 
> that a language driver for each variant above should be "plugable." Moreover, 
> we should provide one driver implementation for each -- simple GremlinServer 
> REST.
> {code}
> gremlin-variants/
>   gremlin-ruby/
> gremlin_ruby.rb
> gremlin_ruby_rest_driver.rb
>   gremlin-php/
> Gremlin_PHP.php
> Gremlin_PHP_REST_Driver.php
>   gremlin-python/
> gremlin-python.py
> gremlin-python-rest-driver.py
> {code}
> Next, each variant implementation should be testable. This is PAINFUL if we 
> have to implement each {{g_V_out_repeatXasXaXX}} test case in 
> {{ProcessXXXSuite}}. Perhaps some RegEx transducer magic could be used to 
> convert all those tests from Gremlin-Java to the respective host language? 
> However, even if we do that, we still have the problem of how to test the 
> returned results. 
> I think what we should test the returned results using the JVM. For instance, 
> JRuby, Jython, JPHP (does it exist?). If we do this, we will save ourselves a 
> massive headache. All we have to do is create a {{GraphProvider}} that uses 
> {{TinkerGraph}} and whose {{TraversalSource}} is some sort of wrapper around 
> reflection-generated Ruby (e.g.).
> {code}
> g.V.out_e("knows") // returns a Ruby iterator
> {code}
> That Ruby iterator should be converted to a Java iterator and then the 
> {{ProcessXXXSuite}} can verify the results.
> With this, most everything is reflectively constructed.
> {code}
> gremlin_ruby.rb // generated via Java reflection
> gremlin_ruby_rest_driver.rb // manually coded
> match_test.rb   // generated via RegEx transducer
> has_test.rb // generated via RegEx transducer
> ...
> RubyGraphProvider.java// manually coded
> RubyProcessStandardSuite.java // manually coded
> RubyProcessComputerSuite.java // manually coded
> {code}
> Thus, the testing data flow would be:
> {code}
> MatchTest.Traversals.java --transducer-> match_test.rb
> match-test.rb --REST--> GremlinServer
> GremlinServer --GraphSON-->match-test.rb
> GraphSON --JRuby/GraphSONReader-->Java objects
> Java objects --JRuby-->MatchTest.java 
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TINKERPOP-1278) Implement Gremlin-Python and general purpose language variant test infrastructure

2016-08-30 Thread Adam Holmberg (JIRA)


[ 
https://issues.apache.org/jira/browse/TINKERPOP-1278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449751#comment-15449751
 ] 

Adam Holmberg commented on TINKERPOP-1278:
--

I see this is merged now. I wanted to follow up on some comments I had on one 
commit pre-merge:
https://github.com/apache/tinkerpop/commit/6ed7edc0b4ad2abf933e917812d49ad92230c8d1#commitcomment-18473940

Is there still a chance to make this generate more idiomatic Python as 
suggested in the GLV docs? Was this discussed anywhere else?

> Implement Gremlin-Python and general purpose language variant test 
> infrastructure
> -
>
> Key: TINKERPOP-1278
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1278
> Project: TinkerPop
>  Issue Type: Improvement
>  Components: language-variant
>Affects Versions: 3.2.0-incubating
>Reporter: Marko A. Rodriguez
>Assignee: Marko A. Rodriguez
> Fix For: 3.2.2
>
>
> As discussed on dev@...
> Apache TinkerPop should provide, out-of-the-box, at least 3 Gremlin language 
> variants. It would be cool if these were:
> * Python (Mark Henderson)
> * PHP ([~PommeVerte])
> * Ruby (?[~okram])
> I think each of these should be generated using the reflection-model 
> presented in 
> http://tinkerpop.apache.org/docs/3.2.1-SNAPSHOT/tutorials/gremlin-language-variants/.
>  Moreover, on every {{mvn clean install}}, the code for these variants is 
> generated.
> Given the desire to separate language variants from language drivers, I think 
> that a language driver for each variant above should be "plugable." Moreover, 
> we should provide one driver implementation for each -- simple GremlinServer 
> REST.
> {code}
> gremlin-variants/
>   gremlin-ruby/
> gremlin_ruby.rb
> gremlin_ruby_rest_driver.rb
>   gremlin-php/
> Gremlin_PHP.php
> Gremlin_PHP_REST_Driver.php
>   gremlin-python/
> gremlin-python.py
> gremlin-python-rest-driver.py
> {code}
> Next, each variant implementation should be testable. This is PAINFUL if we 
> have to implement each {{g_V_out_repeatXasXaXX}} test case in 
> {{ProcessXXXSuite}}. Perhaps some RegEx transducer magic could be used to 
> convert all those tests from Gremlin-Java to the respective host language? 
> However, even if we do that, we still have the problem of how to test the 
> returned results. 
> I think what we should test the returned results using the JVM. For instance, 
> JRuby, Jython, JPHP (does it exist?). If we do this, we will save ourselves a 
> massive headache. All we have to do is create a {{GraphProvider}} that uses 
> {{TinkerGraph}} and whose {{TraversalSource}} is some sort of wrapper around 
> reflection-generated Ruby (e.g.).
> {code}
> g.V.out_e("knows") // returns a Ruby iterator
> {code}
> That Ruby iterator should be converted to a Java iterator and then the 
> {{ProcessXXXSuite}} can verify the results.
> With this, most everything is reflectively constructed.
> {code}
> gremlin_ruby.rb // generated via Java reflection
> gremlin_ruby_rest_driver.rb // manually coded
> match_test.rb   // generated via RegEx transducer
> has_test.rb // generated via RegEx transducer
> ...
> RubyGraphProvider.java// manually coded
> RubyProcessStandardSuite.java // manually coded
> RubyProcessComputerSuite.java // manually coded
> {code}
> Thus, the testing data flow would be:
> {code}
> MatchTest.Traversals.java --transducer-> match_test.rb
> match-test.rb --REST--> GremlinServer
> GremlinServer --GraphSON-->match-test.rb
> GraphSON --JRuby/GraphSONReader-->Java objects
> Java objects --JRuby-->MatchTest.java 
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] tinkerpop pull request #394: Server missing scripts directory

2016-08-30 Thread robertdale

Github user robertdale closed the pull request at:

https://github.com/apache/tinkerpop/pull/394


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] tinkerpop issue #394: Server missing scripts directory

2016-08-30 Thread spmallette

Github user spmallette commented on the issue:

https://github.com/apache/tinkerpop/pull/394
  
i merged #395 from tp31 to master - feel free to close this one. thanks 
again


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] tinkerpop issue #395: TP31: Server missing scripts directory

2016-08-30 Thread robertdale

Github user robertdale commented on the issue:

https://github.com/apache/tinkerpop/pull/395
  
OK.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] tinkerpop issue #395: TP31: Server missing scripts directory

2016-08-30 Thread robertdale

Github user robertdale commented on the issue:

https://github.com/apache/tinkerpop/pull/395
  
Hold on.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] tinkerpop pull request #395: TP31: Server missing scripts directory

2016-08-30 Thread robertdale

GitHub user robertdale opened a pull request:

https://github.com/apache/tinkerpop/pull/395

TP31: Server missing scripts directory 

Fixed distribution scripts target directory.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/robertdale/tinkerpop 
server-missing-scripts-dir-tp31

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/tinkerpop/pull/395.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #395


commit db8ee8a2c3c5472fec6f6560817548d3b015fc2e
Author: Robert Dale 
Date:   2016-08-30T10:50:28Z

Fixed scripts target directory




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Building gremlin-python

[jira] [Created] (TINKERPOP-1423) IncidentToAdjacentStrategy should be disabled for tree steps

[jira] [Commented] (TINKERPOP-1230) Serialising lambdas for RemoteGraph

[jira] [Commented] (TINKERPOP-1278) Implement Gremlin-Python and general purpose language variant test infrastructure

[jira] [Commented] (TINKERPOP-1278) Implement Gremlin-Python and general purpose language variant test infrastructure

[GitHub] tinkerpop pull request #394: Server missing scripts directory

[GitHub] tinkerpop issue #394: Server missing scripts directory

[GitHub] tinkerpop issue #395: TP31: Server missing scripts directory

[GitHub] tinkerpop issue #395: TP31: Server missing scripts directory

[GitHub] tinkerpop pull request #395: TP31: Server missing scripts directory

10 matches

Site Navigation

Mail list logo

Footer information