Re: [Neo4j] Feedback after evaluation
Hmm I think that works, but it won't help with the OOM, as groovy compares the script contents to check if it is the same script Michael Am 10.12.2011 um 03:13 schrieb espeed: > Michael - > > What if each Gremlin script was scoped inside a Groovy function? > > Example: https://gist.github.com/1454298 > > Would that help keep things clean? > > - James > > -- > View this message in context: > http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-Feedback-after-evaluation-tp3569774p3574631.html > Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. > ___ > NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register > and consider posting at https://groups.google.com/forum/#!forum/neo4j > > Neo4j mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Feedback after evaluation
Michael - What if each Gremlin script was scoped inside a Groovy function? Example: https://gist.github.com/1454298 Would that help keep things clean? - James -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-Feedback-after-evaluation-tp3569774p3574631.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Feedback after evaluation
Exactly, looks good. Thanks James I looked again into the source code of the GroovyScriptEngine and so far there is no public way of removing older scripts. They also use a HashMap and not a LinkedHashMap with LRU enabled for storing them. One solution I could think of is to have two script engines, one for one-off shots while will be thrown away regularly. And another which will contain the scripts that have been used at least twice (or x-times) and will stay around forever (or probably throw an PermGen OOM on usage). The only other option would be to duplicate the GroovyScriptEngine functionality and handle everything ourselves. I don't think that's a suitable way to go. I like neither. And have no resources right now to update the GremlinPlugin, feel free to fork update and issue and pull request, or at least a GitHub issue. Cheers Michael Am 10.12.2011 um 01:26 schrieb espeed: > Michael - > > If I understand you correctly, then this modified Gremlin script and request > format should solve "problem 2": > > Gremlin Script: > https://gist.github.com/1453964 > > Script Engine REST Request that uses "params": > https://gist.github.com/1453966 > > - James > > -- > View this message in context: > http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-Feedback-after-evaluation-tp3569774p3574507.html > Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. > ___ > NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register > and consider posting at https://groups.google.com/forum/#!forum/neo4j > > Neo4j mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Feedback after evaluation
Michael - If I understand you correctly, then this modified Gremlin script and request format should solve "problem 2": Gremlin Script: https://gist.github.com/1453964 Script Engine REST Request that uses "params": https://gist.github.com/1453966 - James -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-Feedback-after-evaluation-tp3569774p3574507.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Feedback after evaluation
On 10/12/2011, at 10:12 AM, Michael Hunger wrote: > And can't be garbage collected as it is still around and holds handles to > those classes. > Believe me I would love if there was another way. It should really be addressed properly. It just sounds like "it's too hard, so we we'll just restart everything". Not the way to go, not. I'm sure there are lots of issues, but dropping all the optimisations is just not good enough IMO. ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Feedback after evaluation
Because they are internal to the groovy script engine? And can't be garbage collected as it is still around and holds handles to those classes. Believe me I would love if there was another way. Cheers Michael Am 09.12.2011 um 23:53 schrieb Dmytrii Nagirniak: > > > On 10/12/2011, at 9:03, Michael Hunger > wrote: >> For the first issue we have a work-around in the plugin which recreates the >> script-engine every 500 requests (should probably be configurable) but this >> is less than optimal. > > That is just rediculous workaround as to me! You drop everything that the > engine carefully tried to compile and optimize! This defeats the point of the > optimization. > Why not use a priority queue/list/cache and drop the entries that are least > used only? > > ___ > NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register > and consider posting at https://groups.google.com/forum/#!forum/neo4j > > Neo4j mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Feedback after evaluation
Michael Hunger wrote > > The gremlin plugin (as does the cypher plugin) take an map (a json map) as > "params", each of which which you then can refer to everywhere in the > gremlin script (key == variable name, value == value). So no need for > JSONSlurper. > Ahh, that's different than Rexster's Gremlin extension (https://github.com/tinkerpop/rexster/wiki/Gremlin-Extension). That makes things easy. Michael Hunger wrote > > The real problem here lies in the fact that each of your statements (with > different parameters) will cause the groovy-scripting engine to generate a > new groovy class, which will in turn > * cause PermGen OutOfMemory errors as those can't be garbage collected as > long as the script engine is around > * will take about factor 100-1000 longer to execute (parsing, class > generation, loading and such). > > For the first issue we have a work-around in the plugin which recreates > the script-engine every 500 requests (should probably be configurable) but > this is less than optimal. The second problem will hit you all the time. > What do you recommend for the second problem? And are you saying it will be 100-1000 longer than a normal Groovy or 100-1000 longer than Java? Last week when I emailed you, you were looking into a way to store a custom, server-side Gremlin library. Would that help? - James -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-Feedback-after-evaluation-tp3569774p3574372.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Feedback after evaluation
On 10/12/2011, at 9:03, Michael Hunger wrote: > For the first issue we have a work-around in the plugin which recreates the > script-engine every 500 requests (should probably be configurable) but this > is less than optimal. That is just rediculous workaround as to me! You drop everything that the engine carefully tried to compile and optimize! This defeats the point of the optimization. Why not use a priority queue/list/cache and drop the entries that are least used only? ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Feedback after evaluation
Ouch :)
The gremlin plugin (as does the cypher plugin) take an map (a json map) as
"params", each of which which you then can refer to everywhere in the gremlin
script (key == variable name, value == value).
So no need for JSONSlurper.
The real problem here lies in the fact that each of your statements (with
different parameters) will cause the groovy-scripting engine to generate a new
groovy class, which will in turn
* cause PermGen OutOfMemory errors as those can't be garbage collected as long
as the script engine is around
* will take about factor 100-1000 longer to execute (parsing, class generation,
loading and such).
For the first issue we have a work-around in the plugin which recreates the
script-engine every 500 requests (should probably be configurable) but this is
less than optimal.
The second problem will hit you all the time.
Michael
See:
http://docs.neo4j.org/chunked/milestone/gremlin-plugin.html#rest-api-send-a-gremlin-script-with-variables-in-a-json-map
Am 09.12.2011 um 20:35 schrieb espeed:
>
> Michael Hunger wrote
>>
>> I understood you were just templating the params in there (string
>> replacement) which would result in different groovy strings for every set
>> of different parameters. Is this correct?
>>
>
> The string replacement is done on the client side. When the script is
> presented to the extension, it looks like a normal Gremlin script.
>
> For example, the update_indexed_vertex YAML script has three params: nodeId,
> properties, indexName. But all the string replacement is done by the client.
>
> By the time the Gremlin extension gets it, the nodeId is an integer, which
> is used to look up the node object:
>
> node = g.getRawGraph().getNodeById(5)
>
> The properties are a JSON string, which are immediately converted to a
> Groovy map so they can be iterated upon:
>
> Map properties = slurper.parseText('{"age":35,"name":"James
> Thornton"}')
>
> The indexName is a string, which is used to look up the actual index:
>
> index = manager.forNodes('people')
>
> Here is what the Gremlin extension actually sees:
> https://gist.github.com/1452942
>
> Do see an issue with that?
>
> - James
>
>
>
> --
> View this message in context:
> http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-Feedback-after-evaluation-tp3569774p3573842.html
> Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
> ___
> NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register
> and consider posting at https://groups.google.com/forum/#!forum/neo4j
>
> Neo4j mailing list
> [email protected]
> https://lists.neo4j.org/mailman/listinfo/user
___
NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register
and consider posting at https://groups.google.com/forum/#!forum/neo4j
Neo4j mailing list
[email protected]
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Feedback after evaluation
Michael Hunger wrote
>
> I understood you were just templating the params in there (string
> replacement) which would result in different groovy strings for every set
> of different parameters. Is this correct?
>
The string replacement is done on the client side. When the script is
presented to the extension, it looks like a normal Gremlin script.
For example, the update_indexed_vertex YAML script has three params: nodeId,
properties, indexName. But all the string replacement is done by the client.
By the time the Gremlin extension gets it, the nodeId is an integer, which
is used to look up the node object:
node = g.getRawGraph().getNodeById(5)
The properties are a JSON string, which are immediately converted to a
Groovy map so they can be iterated upon:
Map properties = slurper.parseText('{"age":35,"name":"James
Thornton"}')
The indexName is a string, which is used to look up the actual index:
index = manager.forNodes('people')
Here is what the Gremlin extension actually sees:
https://gist.github.com/1452942
Do see an issue with that?
- James
--
View this message in context:
http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-Feedback-after-evaluation-tp3569774p3573842.html
Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
___
NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register
and consider posting at https://groups.google.com/forum/#!forum/neo4j
Neo4j mailing list
[email protected]
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Feedback after evaluation
I understood you were just templating the params in there (string replacement) which would result in different groovy strings for every set of different parameters. Is this correct? Michael Am 09.12.2011 um 11:44 schrieb espeed: > > Michael Hunger wrote >> >> you should use native gremlin params where they can be used. otherwise >> you'll blow the scriptengine in the plugin and loose lots of performance >> > > Hi Michael - > > What do you mean exactly? After the JSON param converts to a map, everything > is a native Groovy param. > > - James > > -- > View this message in context: > http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-Feedback-after-evaluation-tp3569774p3572589.html > Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. > ___ > NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register > and consider posting at https://groups.google.com/forum/#!forum/neo4j > > Neo4j mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Feedback after evaluation
Michael Hunger wrote > > you should use native gremlin params where they can be used. otherwise > you'll blow the scriptengine in the plugin and loose lots of performance > Hi Michael - What do you mean exactly? After the JSON param converts to a map, everything is a native Groovy param. - James -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-Feedback-after-evaluation-tp3569774p3572589.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Feedback after evaluation
you should use native gremlin params where they can be used. otherwise you'll blow the scriptengine in the plugin and loose lots of performance M mobile mail please excuse brevity and typos Am 09.12.2011 um 11:16 schrieb espeed : > On Thursday, December 8, 2011 2:48:59 AM UTC-6, Dmytrii Nagirniak wrote: > > Unfortunately I couldn't see a lot of value in the REST API either. > The core operations that are taken for granted with native bindings > (traversals using poor Ruby constructs) would require to execute HTTP > request (that's what SELECT N+1 in SQL world is). > Or otherwise I would have to wrap all the logic in the traversal queries. It > would significantly overcomplicate the system with HTTP handling logic. > > > Hi Dmytrii - > > Neo4j Server has a built-in Gremlin scripting engine that enables REST > clients to execute transactions in a single HTTP request. > > Gremlin is a domain-specific language for graphs written in Groovy. If you > were using a relational database, you would use its domain-specific > language, which is SQL. Same idea. > > For the last few weeks, I have been working on Bulbs 0.3, which is a Python > REST client for Neo4j Server, and it has a library of Gremlin templates in a > YAML file. > > The Python methods do named variable substitution on the Gremlin templates > and then execute them via the Neo4j Server Gremlin extension. > > Here's an example: > > gremlin.yaml > https://gist.github.com/1450859 > > element.py > https://gist.github.com/1450871 > > You can see the create_indexed_vertex Gremlin script has JSON args. Python > lists and dicts are converted to JSON, and then on the server side, the > Gremlin script converts them into Groovy maps and lists. > > Marko is working on adding the JSONSlurper library import to Gremlin so you > won't have to do the import each time > (https://github.com/tinkerpop/gremlin/issues/259). > > >> So I decided to write another REST library >> (http:://github.com/dnagir/morpheus), >> but then gave up realising that you just cannot have a proper abstraction >> over HTTP. >> (I'll probably kill off that repo). > > Groovy is pretty simple. Consider reviving Morpheus and using Gremlin for > scripting -- you'll get all the power of native Ruby and Neo4j without the > Java. > > When Bulbs 0.3 is released, I'll post the full gremlin.yaml, and you should > be able to use it in Ruby without any mods since it's just YAML. > > - James > > -- > View this message in context: > http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-Feedback-after-evaluation-tp3569774p3572548.html > Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. > ___ > NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register > and consider posting at https://groups.google.com/forum/#!forum/neo4j > > Neo4j mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Feedback after evaluation
Yes, I agree. Basically, with Cypher for declarative and optimizable queries, and Gremlin/Groovy for the power user or fine-tuned traversals, the REST API could possibly be very minimalistic. Just my 2c. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer brew install neo4j && neo4j start heroku addons:add neo4j On Fri, Dec 9, 2011 at 11:03 AM, James Thornton wrote: > > > > > > On Thursday, December 8, 2011 2:48:59 AM UTC-6, Dmytrii Nagirniak wrote: >> >> >> Unfortunately I couldn't see a lot of value in the REST API either. >> The core operations that are taken for granted with native bindings >> (traversals using poor Ruby constructs) would require to execute HTTP >> request (that's what SELECT N+1 in SQL world is). >> Or otherwise I would have to wrap all the logic in the traversal queries. >> It would significantly overcomplicate the system with HTTP handling logic. > > > Hi Dmytrii - > > Neo4j Server has a built-in Gremlin scripting engine that enables REST > clients execute transactions in a single HTTP request. > > Gremlin is a domain-specific language for graphs written in Groovy. If you > were using a relational database, you would use its domain-specific > languages, which is SQL. Same idea. > > For the last few weeks, I have been working on Bulbs 0.3, which is a Python > REST client for Neo4j Server, and I has a library of Gremlin templates in a > YAML file. The Python methods do named variable substitution on the Gremlin > templates and then execute them via the Neo4j Server Gremlin extension. > > Here's an example: > > gremlin.yaml > https://gist.github.com/1450859 > > element.py > https://gist.github.com/1450871 > > You can see the create_indexed_vertex Gremlin script has JSON args. Python > lists and dicts are converted to JSON, and then on the server side, the > Gremlin script converts them into Groovy maps and lists. > > Marko is working on adding the JSONSlurper library import to Gremlin so you > won't have to do the import each time > (https://github.com/tinkerpop/gremlin/issues/259). > > >> So I decided to write another REST library >> (http:://github.com/dnagir/morpheus), >> but then gave up realising that you just cannot have a proper abstraction >> over HTTP. >> (I'll probably kill off that repo). > > Groovy is pretty simple. Consider reviving Morpheus and using Gremlin for > scripting -- you'll get all the power of native Ruby and Neo4j without the > Java. > > When Bulbs 0.3 is released, I'll post the full gremlin.yaml, and you should > be able to use it in Ruby without any mods since it's just YAML. > > - James > > > > > > > >> >> Also there are no decent HTTP restful clients. The only one is neography - >> that works pretty well, but doesn't give me any abstraction similar to >> neo4j.rb. >> There is also neology. I declare it dead, I couldn't even run tests >> because a dependent gem was removed from the author's own github repository. >> So I even had no way to fix any issues there. >> >> Last one - architect4r - good idea behind. But the abstractions are leaky. >> You can't make system more or less performant without resorting to HTTP. >> I also did minor contribution to it (accepted PR). But since then the >> author never replied to my tweets, neither he replied to emails. >> Maybe he's just sick or something else, but that's what we have. >> And there were failing specs all over the place. >> >> So I decided to write another REST library >> (http:://github.com/dnagir/morpheus), but then gave up realising that you >> just cannot have a proper abstraction over HTTP. >> (I'll probably kill off that repo). >> >> >> So all in all, to summarise: I am giving up on neo4j because it forces me >> into Java world to leverage its full power. >> I could have agreed on that if I would be a Java dev. But there is nothing >> in this world that can convince me to choose Java instead of Ruby (maybe >> other langs in the future). >> >> But what DO have to mention is the dedication of people around neo4j. >> Everybody tried their best to help. >> And that feels like everybody within Neo Technologies has common vision >> and is really passionate and keen to help. >> I can't remember any other company that would be so dedicated. >> >> I would really love to use neo4j, but unfortunately I can't do that until >> it will be available as native binding for C Ruby. >> >> And as a lost note, I want to say THANKS to the neo4j community for the >> great and amazing support you all guys give. >> >> Cheers, >> Dmytrii >> http://www.ApproachE.com >> >> >> ___ >> NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please >> register and consider posting at >> https://groups.google.com/forum/#!forum/neo4j >> >> Neo4j mailing list >> [email protected] >> https://lists.neo4j.org/mailman/listinfo/user ___
Re: [Neo4j] Feedback after evaluation
On Thursday, December 8, 2011 2:48:59 AM UTC-6, Dmytrii Nagirniak wrote: Unfortunately I couldn't see a lot of value in the REST API either. The core operations that are taken for granted with native bindings (traversals using poor Ruby constructs) would require to execute HTTP request (that's what SELECT N+1 in SQL world is). Or otherwise I would have to wrap all the logic in the traversal queries. It would significantly overcomplicate the system with HTTP handling logic. Hi Dmytrii - Neo4j Server has a built-in Gremlin scripting engine that enables REST clients to execute transactions in a single HTTP request. Gremlin is a domain-specific language for graphs written in Groovy. If you were using a relational database, you would use its domain-specific language, which is SQL. Same idea. For the last few weeks, I have been working on Bulbs 0.3, which is a Python REST client for Neo4j Server, and it has a library of Gremlin templates in a YAML file. The Python methods do named variable substitution on the Gremlin templates and then execute them via the Neo4j Server Gremlin extension. Here's an example: gremlin.yaml https://gist.github.com/1450859 element.py https://gist.github.com/1450871 You can see the create_indexed_vertex Gremlin script has JSON args. Python lists and dicts are converted to JSON, and then on the server side, the Gremlin script converts them into Groovy maps and lists. Marko is working on adding the JSONSlurper library import to Gremlin so you won't have to do the import each time (https://github.com/tinkerpop/gremlin/issues/259). > So I decided to write another REST library > (http:://github.com/dnagir/morpheus), > but then gave up realising that you just cannot have a proper abstraction > over HTTP. > (I'll probably kill off that repo). Groovy is pretty simple. Consider reviving Morpheus and using Gremlin for scripting -- you'll get all the power of native Ruby and Neo4j without the Java. When Bulbs 0.3 is released, I'll post the full gremlin.yaml, and you should be able to use it in Ruby without any mods since it's just YAML. - James -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-Feedback-after-evaluation-tp3569774p3572548.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Feedback after evaluation
On 09/12/2011, at 2:44 PM, Vivek Prahlad wrote: > I would like to point out that spork does actually work with JRuby. From > what I can see, guard and guard-spork are now supported with JRuby as well. Obviously, it doesn't. I don't even get to guard. And of course not with neo4j. This video shows everything (3x sped up to save you time not watching how jruby starts and I type). I would love to see the same video where it actually works. http://www.youtube.com/watch?v=buJvDN5rg28 And this is the problem. I am wasting my time to fix some weird issues with JRuby. All that "just works" on MRI. Anyway, I hope someday I'll be able to make it all work. ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Feedback after evaluation
Hi Dmytrii, I would like to point out that spork does actually work with JRuby. From what I can see, guard and guard-spork are now supported with JRuby as well. Please take a look at the supported platform list here: https://github.com/guard/guard-spork and here: https://github.com/guard/guard Here's my Gemfile entry: gem 'spork', '~> 0.9.0.rc' I'm using spork, but not guard, BTW. Cheers, Vivek On Fri, Dec 9, 2011 at 5:00 AM, Dmytrii Nagirniak wrote: > > On 09/12/2011, at 8:15 AM, Andreas Ronge wrote: > > > Yes there is - nailgun. > > Charles has already been involved in many of the problems that Dmytrii > > had with JRuby. > > For a good summery of Dmytrii's problem, check his twitter feed. > > For RSpec JRuby performance and Charles response, see > > https://gist.github.com/1423288 > > > Yes, that's true. > The summary is that JRuby will never be on par with MRI in terms of > startup time (Even with nailgun). > It also doesn't support Spork because JRuby can't fork processes. > > And this is extremely valuable and important. > > ___ > NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please > register and consider posting at > https://groups.google.com/forum/#!forum/neo4j > > Neo4j mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user > ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Feedback after evaluation
On 09/12/2011, at 9:22 AM, Max De Marzi Jr. wrote:
> Is HTTP as a protocol the problem? Maybe, it does have some advantages though.
Yes. Definitely. There are always pros/cons for everything.
> I think we all agree the REST API is not finished yet.
> We talked last week about Batch operations as a poor-man's replacement for
> Transactions and the concerns that brought up.
>
> I think the REST API will get there eventually... or (since it had a short
> brush with death two months ago) be replaced with a full Cypher language
> with data operations (INSERT, UPDATE, DELETE SQL equivalents)
>
> In the mean time, my solution has been Polyglot Persistence using
> ActiveRecord callbacks.
Yes, that is something I may consider. That's probably the most pragmatic
approach.
But currently simple WITH RECURSIVE
(http://www.postgresql.org/docs/8.4/static/queries-with.html) SQL may do the
job for me.
> after_create :create_node
>
> def create_node
> if self.valid?
> self.node = $neo.create_node("identity_id" =>
> self.id)["self"].split("/).last
> self.save
> end
>
> after_save :create_relationship
>
> def create_relationship
> from = self.grantor.node
> to = self.requester.node
> $neo.create_relationship("vouched", from, to)
> end
>
I thought about something like this and it definitely may work. But I will try
to use single DB as long as possible.
> You don't have to limit yourself to one database.
> Chances are you'll need Redis anyway, so you might as well think in terms of
> multiple storage units from the beginning.
>
You definitely have good point here.
___
NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register
and consider posting at https://groups.google.com/forum/#!forum/neo4j
Neo4j mailing list
[email protected]
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Feedback after evaluation
On 09/12/2011, at 7:43 AM, Michael Hunger wrote: > What happens if your run the 1400 rpec tests several times? I don't get the point of running it several times. I need to run one/couple of spec as soon as possible and see the feedback. > Which older gems were problematic for you? It's not about OLDER gems. We are talking about existing and well maintained. Here is a quick recap of just migrating over to JRuby using the most standard stack of technologies. http://blog.approache.com/2011/11/issues-switching-to-jruby-from-mri-19.html (Although I made it work at the end of the day, should write follow up post). And here you can read the discussion of more people having issues with JRuby. https://groups.google.com/group/rails-oceania/browse_thread/thread/1e382d367c1c55f7?hl=en The bottom line is that it is pain and way too much time is spent on dealing with issues that you just don't have at all in MRI Ruby. ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Feedback after evaluation
On 09/12/2011, at 8:15 AM, Andreas Ronge wrote: > Yes there is - nailgun. > Charles has already been involved in many of the problems that Dmytrii > had with JRuby. > For a good summery of Dmytrii's problem, check his twitter feed. > For RSpec JRuby performance and Charles response, see > https://gist.github.com/1423288 Yes, that's true. The summary is that JRuby will never be on par with MRI in terms of startup time (Even with nailgun). It also doesn't support Spork because JRuby can't fork processes. And this is extremely valuable and important. ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Feedback after evaluation
Is HTTP as a protocol the problem? Maybe, it does have some advantages
though.
I think we all agree the REST API is not finished yet.
We talked last week about Batch operations as a poor-man's replacement for
Transactions and the concerns that brought up.
I think the REST API will get there eventually... or (since it had a short
brush with death two months ago) be replaced with a full Cypher language
with data operations (INSERT, UPDATE, DELETE SQL equivalents)
In the mean time, my solution has been Polyglot Persistence using
ActiveRecord callbacks.
after_create :create_node
def create_node
if self.valid?
self.node = $neo.create_node("identity_id" =>
self.id)["self"].split("/).last
self.save
end
after_save :create_relationship
def create_relationship
from = self.grantor.node
to = self.requester.node
$neo.create_relationship("vouched", from, to)
end
You don't have to limit yourself to one database.
Chances are you'll need Redis anyway, so you might as well think in terms of
multiple storage units from the beginning.
___
NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register
and consider posting at https://groups.google.com/forum/#!forum/neo4j
Neo4j mailing list
[email protected]
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Feedback after evaluation
Yes there is - nailgun. Charles has already been involved in many of the problems that Dmytrii had with JRuby. For a good summery of Dmytrii's problem, check his twitter feed. For RSpec JRuby performance and Charles response, see https://gist.github.com/1423288 /Andreas On Thu, Dec 8, 2011 at 9:43 PM, Michael Hunger wrote: > Isn't there something like the background jvm thingy that exists for groovy, > scala and other languages? A server that the current process connects to, > sends code over and runs it in the JVM? > > What happens if your run the 1400 rpec tests several times? > > Perhaps we should ask Charles Nutter for his feedback on these issues? > > Which older gems were problematic for you? > > Michael > > Am 08.12.2011 um 21:30 schrieb Andreas Ronge: > >> Hi >> >> Thanks for your great feedback ! >> The tooling support for JRuby has worked well for me and would >> probably still use JRuby even if there was a good native MRI neo4j >> wrapper. >> But I have a java background and might not be spoiled with that >> instant feedback loop of doing behaviour driven development using MRI >> Ruby. The JVM never has time to warm up when running the RSpec tests. >> I guess it will always be slower than MRI for running tests (but not >> otherwise). Neo4j.rb has 1400 RSpecs (with very few mocks). JRuby will >> run them in about 1-2 minutes. >> I guess the other problems like having a rails console with write >> access to the database is solvable. >> Thanks again for your feedback and I do agree with your criticism but >> for me it's not that problematic. >> >> Cheers >> Andreas >> >> On Thu, Dec 8, 2011 at 9:48 AM, Dmytrii Nagirniak wrote: >>> Hi Guys, >>> >>> Just want to quickly give some feedback on the neo4j after some evaluation. >>> >>> Overall, I like the neo4j a lot, but have decided not to use it. >>> >>> Probably the primary reason is the tooling around Ruby. Don't get me wrong. >>> The neo4j.rb is just amazing. It really is. >>> I even contributed couple of Pull Requests (and those were accepted). >>> >>> The problem is in neo4j Java roots. The only option for me was to use JRuby >>> (will say a word on REST later). >>> But unfortunately choosing JRuby is just too troublesome and give much more >>> headache comparing to "normal" C/MRI Ruby. >>> >>> Everything is so much harder (even speed is x times slower). >>> >>> On "normal" (MRI 1.9.3) Ruby I run all the specs immediately, after saving >>> a file. Immediate feedback. >>> With JRuby I'd have to wait for almost half a minute. TDD is gone. Not good >>> enough. But this is just the first "issues" that I faced. >>> >>> A lot of other libraries just don't work with. >>> There are always small walls on my way that I have to break through, that >>> would never happen with normal Ruby. >>> It just gives me a lot of pain. >>> >>> Unfortunately I couldn't see a lot of value in the REST API either. >>> The core operations that are taken for granted with native bindings >>> (traversals using poor Ruby constructs) would require to execute HTTP >>> request (that's what SELECT N+1 in SQL world is). >>> Or otherwise I would have to wrap all the logic in the traversal queries. >>> It would significantly overcomplicate the system with HTTP handling logic. >>> >>> Also there are no decent HTTP restful clients. The only one is neography - >>> that works pretty well, but doesn't give me any abstraction similar to >>> neo4j.rb. >>> There is also neology. I declare it dead, I couldn't even run tests because >>> a dependent gem was removed from the author's own github repository. >>> So I even had no way to fix any issues there. >>> >>> Last one - architect4r - good idea behind. But the abstractions are leaky. >>> You can't make system more or less performant without resorting to HTTP. >>> I also did minor contribution to it (accepted PR). But since then the >>> author never replied to my tweets, neither he replied to emails. >>> Maybe he's just sick or something else, but that's what we have. >>> And there were failing specs all over the place. >>> >>> So I decided to write another REST library >>> (http:://github.com/dnagir/morpheus), but then gave up realising that you >>> just cannot have a proper abstraction over HTTP. >>> (I'll probably kill off that repo). >>> >>> >>> So all in all, to summarise: I am giving up on neo4j because it forces me >>> into Java world to leverage its full power. >>> I could have agreed on that if I would be a Java dev. But there is nothing >>> in this world that can convince me to choose Java instead of Ruby (maybe >>> other langs in the future). >>> >>> But what DO have to mention is the dedication of people around neo4j. >>> Everybody tried their best to help. >>> And that feels like everybody within Neo Technologies has common vision and >>> is really passionate and keen to help. >>> I can't remember any other company that would be so dedicated. >>> >>> I would really love to use neo4j, but unfortunately I ca
Re: [Neo4j] Feedback after evaluation
Isn't there something like the background jvm thingy that exists for groovy, scala and other languages? A server that the current process connects to, sends code over and runs it in the JVM? What happens if your run the 1400 rpec tests several times? Perhaps we should ask Charles Nutter for his feedback on these issues? Which older gems were problematic for you? Michael Am 08.12.2011 um 21:30 schrieb Andreas Ronge: > Hi > > Thanks for your great feedback ! > The tooling support for JRuby has worked well for me and would > probably still use JRuby even if there was a good native MRI neo4j > wrapper. > But I have a java background and might not be spoiled with that > instant feedback loop of doing behaviour driven development using MRI > Ruby. The JVM never has time to warm up when running the RSpec tests. > I guess it will always be slower than MRI for running tests (but not > otherwise). Neo4j.rb has 1400 RSpecs (with very few mocks). JRuby will > run them in about 1-2 minutes. > I guess the other problems like having a rails console with write > access to the database is solvable. > Thanks again for your feedback and I do agree with your criticism but > for me it's not that problematic. > > Cheers > Andreas > > On Thu, Dec 8, 2011 at 9:48 AM, Dmytrii Nagirniak wrote: >> Hi Guys, >> >> Just want to quickly give some feedback on the neo4j after some evaluation. >> >> Overall, I like the neo4j a lot, but have decided not to use it. >> >> Probably the primary reason is the tooling around Ruby. Don't get me wrong. >> The neo4j.rb is just amazing. It really is. >> I even contributed couple of Pull Requests (and those were accepted). >> >> The problem is in neo4j Java roots. The only option for me was to use JRuby >> (will say a word on REST later). >> But unfortunately choosing JRuby is just too troublesome and give much more >> headache comparing to "normal" C/MRI Ruby. >> >> Everything is so much harder (even speed is x times slower). >> >> On "normal" (MRI 1.9.3) Ruby I run all the specs immediately, after saving a >> file. Immediate feedback. >> With JRuby I'd have to wait for almost half a minute. TDD is gone. Not good >> enough. But this is just the first "issues" that I faced. >> >> A lot of other libraries just don't work with. >> There are always small walls on my way that I have to break through, that >> would never happen with normal Ruby. >> It just gives me a lot of pain. >> >> Unfortunately I couldn't see a lot of value in the REST API either. >> The core operations that are taken for granted with native bindings >> (traversals using poor Ruby constructs) would require to execute HTTP >> request (that's what SELECT N+1 in SQL world is). >> Or otherwise I would have to wrap all the logic in the traversal queries. It >> would significantly overcomplicate the system with HTTP handling logic. >> >> Also there are no decent HTTP restful clients. The only one is neography - >> that works pretty well, but doesn't give me any abstraction similar to >> neo4j.rb. >> There is also neology. I declare it dead, I couldn't even run tests because >> a dependent gem was removed from the author's own github repository. >> So I even had no way to fix any issues there. >> >> Last one - architect4r - good idea behind. But the abstractions are leaky. >> You can't make system more or less performant without resorting to HTTP. >> I also did minor contribution to it (accepted PR). But since then the author >> never replied to my tweets, neither he replied to emails. >> Maybe he's just sick or something else, but that's what we have. >> And there were failing specs all over the place. >> >> So I decided to write another REST library >> (http:://github.com/dnagir/morpheus), but then gave up realising that you >> just cannot have a proper abstraction over HTTP. >> (I'll probably kill off that repo). >> >> >> So all in all, to summarise: I am giving up on neo4j because it forces me >> into Java world to leverage its full power. >> I could have agreed on that if I would be a Java dev. But there is nothing >> in this world that can convince me to choose Java instead of Ruby (maybe >> other langs in the future). >> >> But what DO have to mention is the dedication of people around neo4j. >> Everybody tried their best to help. >> And that feels like everybody within Neo Technologies has common vision and >> is really passionate and keen to help. >> I can't remember any other company that would be so dedicated. >> >> I would really love to use neo4j, but unfortunately I can't do that until it >> will be available as native binding for C Ruby. >> >> And as a lost note, I want to say THANKS to the neo4j community for the >> great and amazing support you all guys give. >> >> Cheers, >> Dmytrii >> http://www.ApproachE.com >> >> >> >> >> ___ >> NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please >> register and consider
Re: [Neo4j] Feedback after evaluation
Hi Thanks for your great feedback ! The tooling support for JRuby has worked well for me and would probably still use JRuby even if there was a good native MRI neo4j wrapper. But I have a java background and might not be spoiled with that instant feedback loop of doing behaviour driven development using MRI Ruby. The JVM never has time to warm up when running the RSpec tests. I guess it will always be slower than MRI for running tests (but not otherwise). Neo4j.rb has 1400 RSpecs (with very few mocks). JRuby will run them in about 1-2 minutes. I guess the other problems like having a rails console with write access to the database is solvable. Thanks again for your feedback and I do agree with your criticism but for me it's not that problematic. Cheers Andreas On Thu, Dec 8, 2011 at 9:48 AM, Dmytrii Nagirniak wrote: > Hi Guys, > > Just want to quickly give some feedback on the neo4j after some evaluation. > > Overall, I like the neo4j a lot, but have decided not to use it. > > Probably the primary reason is the tooling around Ruby. Don't get me wrong. > The neo4j.rb is just amazing. It really is. > I even contributed couple of Pull Requests (and those were accepted). > > The problem is in neo4j Java roots. The only option for me was to use JRuby > (will say a word on REST later). > But unfortunately choosing JRuby is just too troublesome and give much more > headache comparing to "normal" C/MRI Ruby. > > Everything is so much harder (even speed is x times slower). > > On "normal" (MRI 1.9.3) Ruby I run all the specs immediately, after saving a > file. Immediate feedback. > With JRuby I'd have to wait for almost half a minute. TDD is gone. Not good > enough. But this is just the first "issues" that I faced. > > A lot of other libraries just don't work with. > There are always small walls on my way that I have to break through, that > would never happen with normal Ruby. > It just gives me a lot of pain. > > Unfortunately I couldn't see a lot of value in the REST API either. > The core operations that are taken for granted with native bindings > (traversals using poor Ruby constructs) would require to execute HTTP request > (that's what SELECT N+1 in SQL world is). > Or otherwise I would have to wrap all the logic in the traversal queries. It > would significantly overcomplicate the system with HTTP handling logic. > > Also there are no decent HTTP restful clients. The only one is neography - > that works pretty well, but doesn't give me any abstraction similar to > neo4j.rb. > There is also neology. I declare it dead, I couldn't even run tests because a > dependent gem was removed from the author's own github repository. > So I even had no way to fix any issues there. > > Last one - architect4r - good idea behind. But the abstractions are leaky. > You can't make system more or less performant without resorting to HTTP. > I also did minor contribution to it (accepted PR). But since then the author > never replied to my tweets, neither he replied to emails. > Maybe he's just sick or something else, but that's what we have. > And there were failing specs all over the place. > > So I decided to write another REST library > (http:://github.com/dnagir/morpheus), but then gave up realising that you > just cannot have a proper abstraction over HTTP. > (I'll probably kill off that repo). > > > So all in all, to summarise: I am giving up on neo4j because it forces me > into Java world to leverage its full power. > I could have agreed on that if I would be a Java dev. But there is nothing in > this world that can convince me to choose Java instead of Ruby (maybe other > langs in the future). > > But what DO have to mention is the dedication of people around neo4j. > Everybody tried their best to help. > And that feels like everybody within Neo Technologies has common vision and > is really passionate and keen to help. > I can't remember any other company that would be so dedicated. > > I would really love to use neo4j, but unfortunately I can't do that until it > will be available as native binding for C Ruby. > > And as a lost note, I want to say THANKS to the neo4j community for the great > and amazing support you all guys give. > > Cheers, > Dmytrii > http://www.ApproachE.com > > > > > ___ > NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register > and consider posting at https://groups.google.com/forum/#!forum/neo4j > > Neo4j mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
