Re: [Neo4j] New user...
Welcome to the list :) /jake On Dec 6, 2011 1:54 PM, Lawrence lawrence.bande...@gmail.com wrote: Just sending my email -- Lawrence C. Bandeira ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Controlling Traversal Depth with Python Neo4j Bindings
On Sat, Dec 3, 2011 at 1:08 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Chris, Sounds like a case for Cypher, Jake just added on support for it? I don't think that has been released yet though. Chris: I will be adding tested and documented support for this as soon as I have time, hopefully next week. In the meantime, know that the traversal description builder stuff is an augmented version of the Java traversal description. Generally, any method available in the java traversal description API should be available in the python version. Until it's in the python documentation though, I can't guarantee it will work properly. /jake /peter Sent from my phone, please excuse typos and autocorrection. On Dec 2, 2011 2:40 AM, Chris Diehl di...@alumni.cmu.edu wrote: Hi Jake, I have a situation where I want to start at a given node in my graph and traverse down a linear chain of nodes a specified # of hops. Is there a way for me to specify the traversal depth using the Python bindings? If not, what would you suggest as the best way to achieve that goal? Cheers, Chris ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Standalone server and transactions
On Fri, Dec 2, 2011 at 3:45 PM, Dmytrii Nagirniak dna...@gmail.com wrote:
On 03/12/2011, at 12:50 AM, Rick Bullotta wrote:
I also agree with you that it would be nice to have transaction support
over a remote API, but that's not a trivial thing to accomplish in a web
scenario when there is not any thread affinity or even necessarily
server affinity between requests that occur within a transaction.
Knowing when to abandon a pending transaction in a scenario where the
client(s) do not maintain a persistent connection is also quite challenging.
You just meant to say that HTTP is stateless? :)
I proposed one solution in the other thread.
Basically:
1. you POST to /transactions?timeout=1000 to request a start of a
transaction.
2. The server returns a transaction id for the client to use.
3. Client can PUT /transactions/1234/commit|rollback
4. Otherwise server rolls it back after timeout (which may default to 1
second and can't be longer that configured).
It has additional benefit that you can combine transaction with batching:
1. GET /nodes/123
2. POST /transactions/
3. POST /batch - update everything in a batch, including the transaction
itself.
Thus additional request to commit/rollback the transaction is not even
necessary as in can be batched.
This is similar to the approach we're looking at, except interaction
currently looks like:
POST /tx
200: 1
POST /tx/1/node {name:Steven}
201
POST /tx/1/batch [blah blah]
200
PUT /tx/1/state COMMITTED
/jake
One possibility might be to leverage another RPC mechanism or perhaps
even something like Web Sockets in the not too distant future. But I think
the goal of a language neutral binding also needs to be preserved if
possible.
HTTP has its advantages. It can be heavily cached by reverse-proxy for
example :)
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Jacob Hansson
Phone: +46 (0) 763503395
Twitter: @jakewins
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Standalone server and transactions
On Fri, Dec 2, 2011 at 4:53 PM, Jacob Hansson
jacob.hans...@neotechnology.com wrote:
On Fri, Dec 2, 2011 at 3:45 PM, Dmytrii Nagirniak dna...@gmail.comwrote:
On 03/12/2011, at 12:50 AM, Rick Bullotta wrote:
I also agree with you that it would be nice to have transaction support
over a remote API, but that's not a trivial thing to accomplish in a web
scenario when there is not any thread affinity or even necessarily
server affinity between requests that occur within a transaction.
Knowing when to abandon a pending transaction in a scenario where the
client(s) do not maintain a persistent connection is also quite challenging.
You just meant to say that HTTP is stateless? :)
I proposed one solution in the other thread.
Basically:
1. you POST to /transactions?timeout=1000 to request a start of a
transaction.
2. The server returns a transaction id for the client to use.
3. Client can PUT /transactions/1234/commit|rollback
4. Otherwise server rolls it back after timeout (which may default to 1
second and can't be longer that configured).
It has additional benefit that you can combine transaction with batching:
1. GET /nodes/123
2. POST /transactions/
3. POST /batch - update everything in a batch, including the transaction
itself.
Thus additional request to commit/rollback the transaction is not even
necessary as in can be batched.
This is similar to the approach we're looking at, except interaction
currently looks like:
POST /tx
200: 1
Sorry, this should've been 201, of course.
POST /tx/1/node {name:Steven}
201
POST /tx/1/batch [blah blah]
200
PUT /tx/1/state COMMITTED
/jake
One possibility might be to leverage another RPC mechanism or perhaps
even something like Web Sockets in the not too distant future. But I think
the goal of a language neutral binding also needs to be preserved if
possible.
HTTP has its advantages. It can be heavily cached by reverse-proxy for
example :)
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Jacob Hansson
Phone: +46 (0) 763503395
Twitter: @jakewins
--
Jacob Hansson
Phone: +46 (0) 763503395
Twitter: @jakewins
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Standalone server and transactions
On Fri, Dec 2, 2011 at 5:12 PM, Dmytrii Nagirniak dna...@gmail.com wrote:
On 03/12/2011, at 2:53 AM, Jacob Hansson wrote:
This is similar to the approach we're looking at, except interaction
currently looks like:
POST /tx
200: 1
POST /tx/1/node {name:Steven}
201
POST /tx/1/batch [blah blah]
200
PUT /tx/1/state COMMITTED
Yeah. Right. We do need the Trn Id for each request. Missed that part.
I just wonder maybe there is a way to do it without modifying existing
API? (HTTP header is going to be harder).
The idea with this approach is that the current API would remain exactly
the way it is (and each request to the old endpoints would be within a
single transaction, just like today). We just add this tx resource to do
transactional operations across multiple requests.
Also I think I'll start to work on a new Ruby REST client (
https://github.com/dnagir/morpheus) and it would be good to know the API
in advance if possible (or maybe how to discover it).
So none of this is decided on, the approach I posted is one direction we're
experimenting in. The reason I can't give you a commitment to some
particular direction is that we're still debating and testing approaches.
It gets complex, because the issue is not just transactions, but thoughts
about throughput and latency as well.
Figuring this out is a high-priority issue, but we are currently spread too
thin to put a full time developer on it. Me and Michael Hunger are working
at it in our spare time, and are making some progress. Hopefully we will be
able to assign a full time dev on it before the end of the year, but there
are absolutely no promises.
I promise though that as soon as we have a workable version to show off, we
will post about it on the list to get feedback on it :)
/jake
Cheers,
Dmytrii.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Jacob Hansson
Phone: +46 (0) 763503395
Twitter: @jakewins
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher REST endpoint
On Fri, Dec 2, 2011 at 3:55 PM, jadell josh.ad...@gmail.com wrote: Quick question: Just saw this commit from Jake: http://github.com/neo4j/community/commit/f011c51fcbd5f12236fc633f57dcec9fb8cade83 Will the old Cypher plugin endpoint be going away, or will it remain for back-compat? Which release will have the new endpoint? This will be helpful info for library authors. Damn Josh, you're quick :) We only talked briefly about how long we will be supporting the old plugin, but it will be *at least* through 1.6, and probably 1.7. The new endpoint will most likely be available as of the next milestone release. We've been told we're bad about making sure changelogs are properly updated, and we had a long talk about that today. We will be making sure that info on this, as well as deprecation plans for the old plugin, are properly included in the changelog for whatever release it becomes a part of. The new endpoint is in fact the same code as the plugin, so it should be reasonably easy to make the switch (should just be the URI change). The only caveat is that it does not currently support google table output like the plugin does. I haven't talked to Peter about if that should be included or not. /jake Thanks, -- Josh -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Cypher-REST-endpoint-tp3554828p3554828.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST API - always batch?
On Thu, Dec 1, 2011 at 9:03 AM, Dmytrii Nagirniak dna...@gmail.com wrote: Thanks Peter. I was asking because of the request to support batching in neography: https://github.com/maxdemarzi/neography/issues/16 I sounds like the normal API is obsolete then :) It encourages non-transactional operations. I think the batch-ing should be promoted a little bit more. Check the availability and place the reservation if something is still available is pretty hard to implement without it. Not sure if batching supports conditional operations though. Does it? Is there anything that would be impossible/slow etc to do with batching? I mean, there must be some drawbacks :) There are a few reasons we aren't pushing the batch API as the primary way to work with REST. One is that it's a bit more complex to get started with, but the two primary reasons are these: 1. There are limits to how large the results can be from a batch API call. The batch API supports streaming deserialization of it's input, but output is still created as a giant string in memory. It is reasonably easy to hit the limits of how large the batch API results can be when you start batching requests. 2. We don't want it to be the primary API. We have lots of ideas for how we can improve the current REST API (add full transactional support, massively improve throughput and latency), and so we would rather go down that road. The batch API is a pragmatic solution to a big problem, but I don't think it is a good long-term path to take. /jake Cheers. On 01/12/2011, at 6:08 PM, Peter Neubauer wrote: Dmytri, sounds like an interesting plan! I don't think there is anything directly that I can see, except a bug, https://github.com/neo4j/community/issues/113 with \\ which you might watch our for, reported by Josh Adell. Let us know how it works out! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer brew install neo4j neo4j start heroku addons:add neo4j On Thu, Dec 1, 2011 at 5:53 AM, Dmytrii Nagirniak dna...@gmail.com wrote: Hi, I wonder what the disadvantages would be if every request to the server would always be a batch (even for simple GETs)? This would make it easier to use the API from the application. How bad is this idea? Cheers, Dima http://www.ApproachE.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j Tutorial
If you are interested in one-on-one help, take a look at the neo4j events calendar. Joining a local meetup or a webinar could be a good route to go. Such Neo4j events are listed here: http://plancast.com/neo4j Also, take a look at the neo4j koans, a set of unit tests that walk you through how to work with the neo4j java API: https://github.com/jimwebber/neo4j-tutorial/ Cheers! /jake On Wed, Nov 30, 2011 at 3:42 PM, bigToch arsene.toche...@corenett.comwrote: Hello geeks, I would like if possible that somebody walks me through a complete tutorial on Neo4j. I am a newbie of Noe4j and I have read some stuffs on the wiki web page but still I have not caught yet the procedure. I just need a simple scenario and a step-by-step guide to implement the scenario. I know that it may sound weird but I would like to have an aid because I will be building an app using the Neo4j. Thank you guys, -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-Tutorial-tp3548519p3548519.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Batch URIs
On Tue, Nov 29, 2011 at 9:28 AM, Nigel Small ni...@nigelsmall.name wrote: Following on from the relative URI work that Jim is looking at, I was wondering if there was any way that the entity URIs used for RESTful batch requests could be made more discoverable or elegantly derivable? Currently the only way I'm aware of to calculate a relative node URI, for example, is to hack around parts of the full URI: self._relative_uri = .join(self._uri.partition(/node)[1:]) Basically here I'm looking for the substring /node within the full URI and taking everything from that point onwards as the relative part which I use to inject into a batch payload. This is obviously somewhat dependent on this hardcoded part never changing and doesn't feel particularly like the most elegant way to achieve this. Thoughts welcomed! Yeah, I think you mentioned this in the discussion on that ticket as well, so this has been in my head a bit.. The thinking I had when writing the batch API was that URLs should be relative as a means of forcing requests to stay inside the /db/data resource. The reason for that is the transactionality of the batch API. The idea is that the db part of /db/data at some future point will allow pointing to other databases inside the same server (e.g. /myfirstdb/data /myotherdb/data), and the transactionality of the batch API would only apply within one database. I do agree that it is an unelegant solution tho.. Perhaps we should allow full URLs (or at least full paths), and instead throw a securityexception or something if you don't include the /db/data path. /jake *Nigel Small* Phone: +44 7814 638 246 Blog: http://nigelsmall.name/ GTalk: ni...@nigelsmall.name MSN: nasm...@live.co.uk Skype: technige Twitter: @technige https://twitter.com/#%21/technige LinkedIn: http://uk.linkedin.com/in/nigelsmall ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j Python Index Iteration
Hey Chris, I'm happy to inform you that this is already the case :) If you loop through the entire result, it automatically gets closed. /jake On Wed, Nov 23, 2011 at 8:42 PM, Chris Diehl cpdi...@gmail.com wrote: Hi Jake, As you know, with the current Neo4j Python bindings, one needs to close the index results after doing a lookup and iterating through the results. hits = idx['akey']['avalue'] for item in hits: pass # Always close index results when you are # done, to free up resources. hits.close() Would it be possible to return an object that automatically closes hits after iteration is complete? I'd love to encapsulate the index in a class that takes care of the close for me so I don't have to worry about inserting that somewhere later in my code. Not exactly sure how to pull that off... Chris -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] some questions
://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] the social graph is neither (blog post)
On Tue, Nov 22, 2011 at 10:46 AM, serge s.fedoro...@gmail.com wrote: what do you think of this post ? http://blog.pinboard.in/2011/11/the_social_graph_is_neither/ His first point, that modeling the social graph accurately is impossible is of course true. It is impossible to model reality exactly like it is, because we are already using all the matter in the universe to model reality in the first place. And the social act aspect of declaring a relationship to someone is very true as well. I think he is correct in saying that the we're gonna accurately model all social relationships on the planet plan is bound to fail. Relationships on facebook and twitter and so on don't actually *model* social relationships, but rather *are* social relationships. We as users define the meaning of them, we say things like don't know him really, but I think I'm Facebook friends with him. Facebook friend is not the same thing as being friends, it is a social relationship of its own, which we have collectively defined. The social graph does not accurately model the intricacies of our social life, in fact, it increases the complexity of it. -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/the-social-graph-is-neither-blog-post-tp3527242p3527242.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Lab day: Cypher queries in embedded python bindings
Hey all,
Like we've mentioned before, we have lab-day fridays at Neo4j, and today I
hacked some stuff together that landed directly in trunk for the embedded
python bindings.
As of 1 minute ago, the following operations are now possible with the
embedded python API:
from neo4j import GraphDatabase
db = GraphDatabase(/home/jake/db)
# Plain query
result = db.query(START n=node(0) RETURN n)
# Parameterized query
result = db.query(START n=node({id}) RETURN n, id=0)
# Pre-parsed query
get_node_by_id = db.prepare_query(START n=node({id}) RETURN n)
result = db.query(get_node_by_id, id=0)
# Read the result
for row in result:
print row['n']
for value in result['n']:
print value
node = db.query(get_node_by_id, id=0)['n'].single
Lemme know what you think :)
This is not available on Pypi yet (will be when the first 1.6 milestone is
released) but you can build it super-easily yourself, instructions are in
the readme at github: https://github.com/neo4j/python-embedded
Cheers,
--
Jacob Hansson
Phone: +46 (0) 763503395
Twitter: @jakewins
--
Jacob Hansson
Phone: +46 (0) 763503395
Twitter: @jakewins
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] [URGENT] Recommended server configurations
On Fri, Nov 18, 2011 at 7:21 PM, gustavoboby gustavob...@gmail.com wrote: Hi people's, I'm creating a social network with a larg number of expected hits and i need help with the server recommended configurations: 1 - Operating system (Linux or Windows? What specifically?) If you have the choice, Linux is preferable. We fully support both platforms, but generally get higher performance on Linux, and less problems. 2 - Hardware (How much Memory necessary?) This completely depends on how much data you intend to store. Can you provide an estimation of how big your dataset would be? Number of nodes, number of relationships per nodes, and how many properties (on both nodes and relationships), and what types of property values. You think the use of Neo4j REST API will cause problem? I use it to develop my Asp.Net applications It depends on how you use it. Generally, you will get reasonable insert speed if the client you use supports the batch operations part of the REST API, query speed will depend on the query of course. You will get significantly better performance with the embedded database right now, but that is only available in JVM languages and Python. I am open to suggestions!! I thank the help. -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/URGENT-Recommended-server-configurations-tp3519328p3519328.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Python embedding: problems with shutdown and threads
I've just pushed an update to handle the attachThreadToJVM stuff automatically, I ran your test, Michael, on the DB after that, and it seems to shut down properly. You may want to double check though. You can build this now, by checking out https://github.com/neo4j/python-embedded/ and following the instructions in the readme. This will be part of the 1.5 release, which is currently a delayed, but can probably be expected within a week or two. /Jake On Thu, Nov 3, 2011 at 4:48 PM, Johan Lundberg lundb...@nordu.net wrote: On 2011-11-03 11:08, Jacob Hansson wrote: On Thu, Nov 3, 2011 at 6:55 AM, Tim McNamara paperl...@timmcnamara.co.nzwrote: Here are a few notes of my own from using Neo4j with Python embedded. I know that this isn't the right place . Ubuntu doesn't set $JAVA_HOME. I ended up hard coding the location of my JVM. I seem to encounter unclean shutdowns all the time when running with the iPython REPL. Is there a db.shutdown() command that I've been neglecting to use? Yeah, as long as you call db.shutdown(), things should be ok. See here: http://docs.neo4j.org/chunked/snapshot/python-embedded-reference-core.html We should perhaps set up the db to shut itself down properly automatically if it detects that python is shutting down.. We decided against that before, since we felt like we don't own the application lifecycle. WDYT? I found this snippet to work great for my modules: def _close_db(): try: neo4jdb.shutdown() except NameError: print 'Could not shutdown Neo4j database. Is it open in \ another process?' import atexit atexit.register(_close_db) This looks like a good option, I'll create an issue for adding this. /Jake Tim On 3 November 2011 17:28, Michael Rene Armida m...@marmida.com wrote: I'm having some weird problems with threading on shutdown. As I previously discovered, using neo4j in a multi-threaded Python app requires complete segregation of neo4j into a single thread. That all works fine for creating a database, however, it appears that shutting one down is a bit of a mystery. I wrote a test program to minimally illustrate the difficulty: http://pastebin.com/gg204kae Threading is always easy to get wrong, so I added a simple switch to remove neo4j from the picture. When I disable neo, the program starts up and shuts down cleanly. When I re-enable neo, some of the print statements never get run, and the app mysteriously exits. I am testing this by saving it into a script and running it on the command line. Any time after the worker thread finishes creating the neo db, I hit ctrl-c, and the two scenarios play out as above. It seems that the embedded Python API is not exactly at the forefront of popular use, and so I understand if nobody knows what's up with this. I just thought I should add something, for the record. I hope my script doesn't have an obvious bug. Thanks, Michael I had a discussion with Tobias this morning about threading with JPype, and am currently writing a fix that will allow full threading support for neo4j-embedded. I'll make sure to run this script as well after that fix is implemented, to make sure it gets fixed as well. As far as the forefront of use goes, there is a surprising amount of pypy downloads of neo4j-embedded (4000 new installs in the last 30 days), so the feedback you are providing here is gonna end up helping a lot of people :) Thank you so much for taking the time to document the issues you are having. /jake ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Johan Lundberg NORDUnet A/S lundb...@nordu.net +46730714375 Tulegatan 11 113 53 Stockholm Sweden ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Python embedding: problems with shutdown and threads
On Thu, Nov 3, 2011 at 6:55 AM, Tim McNamara paperl...@timmcnamara.co.nzwrote: Here are a few notes of my own from using Neo4j with Python embedded. I know that this isn't the right place . Ubuntu doesn't set $JAVA_HOME. I ended up hard coding the location of my JVM. I seem to encounter unclean shutdowns all the time when running with the iPython REPL. Is there a db.shutdown() command that I've been neglecting to use? Yeah, as long as you call db.shutdown(), things should be ok. See here: http://docs.neo4j.org/chunked/snapshot/python-embedded-reference-core.html We should perhaps set up the db to shut itself down properly automatically if it detects that python is shutting down.. We decided against that before, since we felt like we don't own the application lifecycle. WDYT? Tim On 3 November 2011 17:28, Michael Rene Armida m...@marmida.com wrote: I'm having some weird problems with threading on shutdown. As I previously discovered, using neo4j in a multi-threaded Python app requires complete segregation of neo4j into a single thread. That all works fine for creating a database, however, it appears that shutting one down is a bit of a mystery. I wrote a test program to minimally illustrate the difficulty: http://pastebin.com/gg204kae Threading is always easy to get wrong, so I added a simple switch to remove neo4j from the picture. When I disable neo, the program starts up and shuts down cleanly. When I re-enable neo, some of the print statements never get run, and the app mysteriously exits. I am testing this by saving it into a script and running it on the command line. Any time after the worker thread finishes creating the neo db, I hit ctrl-c, and the two scenarios play out as above. It seems that the embedded Python API is not exactly at the forefront of popular use, and so I understand if nobody knows what's up with this. I just thought I should add something, for the record. I hope my script doesn't have an obvious bug. Thanks, Michael I had a discussion with Tobias this morning about threading with JPype, and am currently writing a fix that will allow full threading support for neo4j-embedded. I'll make sure to run this script as well after that fix is implemented, to make sure it gets fixed as well. As far as the forefront of use goes, there is a surprising amount of pypy downloads of neo4j-embedded (4000 new installs in the last 30 days), so the feedback you are providing here is gonna end up helping a lot of people :) Thank you so much for taking the time to document the issues you are having. /jake ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Python embedding: problems with shutdown and threads
On Thu, Nov 3, 2011 at 3:24 PM, Michael Rene Armida m...@marmida.com wrote: That's awesome news. I will check it out when it's ready. I would suggest wrapping up the lifecycle management in a context manager. You're already using the GraphDatabase object's __enter__ and __exit__ to manage transactions, so you'd probably have to add a second object. Would you like me to add a bug somewhere? I could turn that script into a unit test, which should make it easier to evaluate. It's fine as it is for now, I've written a unit test that checks that threading works (which it seems to do, although there are still a few issues), and I'll add your script to that test suite as well once I get this to pass. Unless I hit any snags, should be pushed later today or tomorrow. There is no need for it now, but if you report bugs in the future, there is a bug tracker active for the project on github. Reporting bugs there is probably preferable to reporting them here, since that makes it easy to keep track of and prioritize work. https://github.com/neo4j/python-embedded /jake On Thu, Nov 3, 2011 at 3:08 AM, Jacob Hansson jacob.hans...@neotechnology.com wrote: On Thu, Nov 3, 2011 at 6:55 AM, Tim McNamara paperl...@timmcnamara.co.nz wrote: Here are a few notes of my own from using Neo4j with Python embedded. I know that this isn't the right place . Ubuntu doesn't set $JAVA_HOME. I ended up hard coding the location of my JVM. I seem to encounter unclean shutdowns all the time when running with the iPython REPL. Is there a db.shutdown() command that I've been neglecting to use? Yeah, as long as you call db.shutdown(), things should be ok. See here: http://docs.neo4j.org/chunked/snapshot/python-embedded-reference-core.html We should perhaps set up the db to shut itself down properly automatically if it detects that python is shutting down.. We decided against that before, since we felt like we don't own the application lifecycle. WDYT? Tim On 3 November 2011 17:28, Michael Rene Armida m...@marmida.com wrote: I'm having some weird problems with threading on shutdown. As I previously discovered, using neo4j in a multi-threaded Python app requires complete segregation of neo4j into a single thread. That all works fine for creating a database, however, it appears that shutting one down is a bit of a mystery. I wrote a test program to minimally illustrate the difficulty: http://pastebin.com/gg204kae Threading is always easy to get wrong, so I added a simple switch to remove neo4j from the picture. When I disable neo, the program starts up and shuts down cleanly. When I re-enable neo, some of the print statements never get run, and the app mysteriously exits. I am testing this by saving it into a script and running it on the command line. Any time after the worker thread finishes creating the neo db, I hit ctrl-c, and the two scenarios play out as above. It seems that the embedded Python API is not exactly at the forefront of popular use, and so I understand if nobody knows what's up with this. I just thought I should add something, for the record. I hope my script doesn't have an obvious bug. Thanks, Michael I had a discussion with Tobias this morning about threading with JPype, and am currently writing a fix that will allow full threading support for neo4j-embedded. I'll make sure to run this script as well after that fix is implemented, to make sure it gets fixed as well. As far as the forefront of use goes, there is a surprising amount of pypy downloads of neo4j-embedded (4000 new installs in the last 30 days), so the feedback you are providing here is gonna end up helping a lot of people :) Thank you so much for taking the time to document the issues you are having. /jake ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST traverse deprecation
On Tue, Nov 1, 2011 at 5:37 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Well, I think that Cypher is not yes taking into account Uniqueness, or is it Andres? Otherwise, most of the normal traversals are covered. To keep things consistent and give people a way of doing fine grained traversals, I would then suggest to document all the existing examples using Cypher or Groovy code, so we can document that things are working as expected and not loose functionality. Sounds like a really good idea. The JavaScript exposure then would totally go away from the REST API and possibly be factored out into a plugin like Groovy/Gremlin. Or dropped. I vote dropped :) WDYT? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/- Öresund - Innovation happens HERE. On Tue, Nov 1, 2011 at 9:31 AM, Jacob Hansson jacob.hans...@neotechnology.com wrote: On Tue, Nov 1, 2011 at 5:22 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Nope, I mean a server side script execution (which we already are doing in http://docs.neo4j.org/chunked/snapshot/rest-api-traverse.html) to execute traversal API code on the server. Neo4js is a client driver. Remember that adding yet another way to talk to the server comes at a cost of confusion and spreading engineering focus thin. I don't think we should do this unless there is a good answer to the question: *Is there anything* *that this solves that is not solved by Cypher or Gremlin, and if so, why don't we expand Cypher or Gremlin to incorporate that use case instead? *Collecting our efforts in fewer buckets means that improvements will benefit everyone, not just the group that uses the Groovy endpoint, or the Traversal API, or the JS endpoint and so on. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/- Öresund - Innovation happens HERE. On Tue, Nov 1, 2011 at 9:19 AM, Andres Taylor andres.tay...@neotechnology.com wrote: You, something like Neo4js https://github.com/neo4j/neo4js? We've had that since forever. Or do you mean something different? Andrés On Tue, Nov 1, 2011 at 5:06 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Also, there is another possibility: We are exposing a ECMA script execution engine via the Traverse enpoints. Instead of trying to shoehorn the traversal API into REST calls and then anyway executing script snippets, Why not exposing a JavaScript plugin along exactly the same lines as the Groovy plugin, and provide examples on how to use the same usecases via that plugin, so we cover the traversal API that way? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/- Öresund - Innovation happens HERE. On Tue, Nov 1, 2011 at 9:00 AM, Jacob Hansson jacob.hans...@neotechnology.com wrote: It would be interesting to rephrase the question like this: Are there any use cases that the REST traversal API covers, that we currently cannot replicate in Cypher or Gremlin? On Sun, Oct 30, 2011 at 2:45 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Well, Andres has been putting the first algo into cypher and we are thinking of exposing them there so thru can be used as path functions in an expression. However, that will be step 2. WDYT? On Oct 29, 2011 6:47 PM, maxdemarzi maxdema...@gmail.com wrote: I think the traversal rest api is fine for my purposes, but if cypher is where we want to go long term, then we need to deprecate it in 1.6 and drop it in 1.7. This gives us some time to update our libraries and for cypher to settle down. Are you guys thinking of getting rid of the rest built in graph algorithms or are those there to stay? -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-REST-traverse-deprecation-tp3460396p3464979.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User
Re: [Neo4j] REST traverse deprecation
On Wed, Nov 2, 2011 at 10:20 AM, Michael Hunger michael.hun...@neotechnology.com wrote: I vote not dropped, as there are lots of non-java developers out there which would want to use REST server dynamically. I am completely open to the notion that I am wrong here. But I want us to have answered the questions I was asking before properly, if we are going to embrace all these ways to talk to the server. What use cases does dynamic script execution cover that cypher or gremlin do not? Why should those use cases not be covered by cypher or gremlin? That was the reason why we wrote the ruby-script-extension support. Michael Am 02.11.2011 um 10:08 schrieb Jacob Hansson: On Tue, Nov 1, 2011 at 5:37 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Well, I think that Cypher is not yes taking into account Uniqueness, or is it Andres? Otherwise, most of the normal traversals are covered. To keep things consistent and give people a way of doing fine grained traversals, I would then suggest to document all the existing examples using Cypher or Groovy code, so we can document that things are working as expected and not loose functionality. Sounds like a really good idea. The JavaScript exposure then would totally go away from the REST API and possibly be factored out into a plugin like Groovy/Gremlin. Or dropped. I vote dropped :) WDYT? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/- Öresund - Innovation happens HERE. On Tue, Nov 1, 2011 at 9:31 AM, Jacob Hansson jacob.hans...@neotechnology.com wrote: On Tue, Nov 1, 2011 at 5:22 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Nope, I mean a server side script execution (which we already are doing in http://docs.neo4j.org/chunked/snapshot/rest-api-traverse.html) to execute traversal API code on the server. Neo4js is a client driver. Remember that adding yet another way to talk to the server comes at a cost of confusion and spreading engineering focus thin. I don't think we should do this unless there is a good answer to the question: *Is there anything* *that this solves that is not solved by Cypher or Gremlin, and if so, why don't we expand Cypher or Gremlin to incorporate that use case instead? *Collecting our efforts in fewer buckets means that improvements will benefit everyone, not just the group that uses the Groovy endpoint, or the Traversal API, or the JS endpoint and so on. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/- Öresund - Innovation happens HERE. On Tue, Nov 1, 2011 at 9:19 AM, Andres Taylor andres.tay...@neotechnology.com wrote: You, something like Neo4js https://github.com/neo4j/neo4js? We've had that since forever. Or do you mean something different? Andrés On Tue, Nov 1, 2011 at 5:06 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Also, there is another possibility: We are exposing a ECMA script execution engine via the Traverse enpoints. Instead of trying to shoehorn the traversal API into REST calls and then anyway executing script snippets, Why not exposing a JavaScript plugin along exactly the same lines as the Groovy plugin, and provide examples on how to use the same usecases via that plugin, so we cover the traversal API that way? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/- Öresund - Innovation happens HERE. On Tue, Nov 1, 2011 at 9:00 AM, Jacob Hansson jacob.hans...@neotechnology.com wrote: It would be interesting to rephrase the question like this: Are there any use cases that the REST traversal API covers, that we currently cannot replicate in Cypher or Gremlin? On Sun, Oct 30, 2011 at 2:45 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Well, Andres has been putting the first algo into cypher and we are thinking of exposing them there so thru can be used as path functions in an expression. However, that will be step 2. WDYT? On Oct 29, 2011 6:47 PM, maxdemarzi maxdema...@gmail.com wrote: I think the traversal rest api is fine for my purposes, but if cypher
Re: [Neo4j] Cypher: including start in result
On Wed, Nov 2, 2011 at 11:18 AM, Michael Hunger michael.hun...@neotechnology.com wrote: Hi, in the pattern matching employed by cypher nodes already bound to an identifier won't be bound to another identifier. I was thinking this was a bug when I saw Didis mail here, but wasn't sure so I didn't answer. Is there a complex reason why this is the case, and is it going to stay this way? It feels a bit counter-intuitive.. That is the reason why all nodes that already belong to principals don't appear in n. In your case it might be easiest to just return n,principals Cheers Michael Am 02.11.2011 um 10:56 schrieb D. Frej: Hi, I have a very simply graph: UserGroup 2 -[:IS_MEMBER_OF]- UserGroup 1, User 3 UserGroup 1 -[:IS_MEMBER_OF]- User 1, User 2 I use the following cypher to find all nodes that are below UserGroup 2 (I start at node User 3 and go one level up in the tree which equals UserGroup 2): START principals=node:nodes(NAME='User 3') MATCH (principals)-[:IS_MEMBER_OF]-()-[:IS_MEMBER_OF*0..]-(n) RETURN n The result does not contain User 3 even though it is a child of UserGroup2 and connected to it via IS_MEMBER_OF relationship. Is there any way to also include it in the result? Thanks, - Didi ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST traverse deprecation
On Wed, Nov 2, 2011 at 11:11 AM, Michael Hunger michael.hun...@neotechnology.com wrote: custom use-cases like the ones implemented by our customers with plugins / extensions and in the case of the ruby script extensions a way I'd like to promote to talk to the server that is 1) create you own RESTful endpoints that are focused on a protocol speaking your domain and not nodes + rels Isn't running domain logic inside a database server a well known anti-pattern? My experience with companies building lots of stuff with stored procedures is that it is *really* hard to maintain, hard to scale, hard to test, hard to tweak and impossible to get an overview of. See for instance how much pain Etsy.com had getting off this type of architecture: http://arstechnica.com/business/news/2011/10/when-clever-goes-wrong-how-etsy-overcame-poor-architectural-choices.ars 2) the protocol is use-case oriented This also implies running domain logic in the database server. 3) it only transmits the information necessary (not more). This is a really good point. Wouldn't it be better though, if we pulled some use cases like this out and said we need Cypher to be able to do this, and thus make Cypher better, rather than saying screw it, lets just let clients use the database as an application server. What are some use cases where this occurs? Lets look at them, and see if it would be possible to make cypher or gremlin do it. 4) The code of these endpoints would be transactionally executed This is a bug, in my mind, of the REST API. Solving it in roundabout ways like this and the batch-operations-API is not actually solving the problem. Use-Cases with customers so far have been: * unique nodes with locking + index lookup This is an *excellent* use case. Compared to the scripting solution, wouldn't it be better if we supported upserts directly in the REST API, as a core feature? * batch data insertion (also unique stuff see above) This is mainly because the REST API is slow, no? If so, spending time working around the problem with arbitrary script execution is beating around the bush. We should fix the real problem here, not hack around it. * custom traversals with complicated callbacks This is really what my original question to Peter was about - what are some traversals like this, that we currently cannot do with Cypher or Gremlin? Can we extend Cypher to accomodate those use cases? * aggregating multiple gremlin / cypher queries in the server and returning only a partial result-set/projection of those executions This also seems like a limitation of Cypher. Can you give an example of when this would be necessary, and we can talk about how that could be integrated into Cypher, if possible? * With the ruby script extension what we've done there is to leverage the web/REST frameworks of the platform and allow rack-apps to expose their REST endpoints within the neo4j-server. Thomas and I also talked about a more generic way to allow any language supported by the javax.script extension to register endpoints with the neo4j-server. A generic javax.scripting languages plugin would accommodate for more ad-hoc execution mode for which writing an extension is too much ceremony. Michael Am 02.11.2011 um 10:54 schrieb Jacob Hansson: On Wed, Nov 2, 2011 at 10:20 AM, Michael Hunger michael.hun...@neotechnology.com wrote: I vote not dropped, as there are lots of non-java developers out there which would want to use REST server dynamically. I am completely open to the notion that I am wrong here. But I want us to have answered the questions I was asking before properly, if we are going to embrace all these ways to talk to the server. What use cases does dynamic script execution cover that cypher or gremlin do not? Why should those use cases not be covered by cypher or gremlin? That was the reason why we wrote the ruby-script-extension support. Michael Am 02.11.2011 um 10:08 schrieb Jacob Hansson: On Tue, Nov 1, 2011 at 5:37 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Well, I think that Cypher is not yes taking into account Uniqueness, or is it Andres? Otherwise, most of the normal traversals are covered. To keep things consistent and give people a way of doing fine grained traversals, I would then suggest to document all the existing examples using Cypher or Groovy code, so we can document that things are working as expected and not loose functionality. Sounds like a really good idea. The JavaScript exposure then would totally go away from the REST API and possibly be factored out into a plugin like Groovy/Gremlin. Or dropped. I vote dropped :) WDYT? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com
Re: [Neo4j] Cypher: including start in result
On Wed, Nov 2, 2011 at 12:26 PM, Andres Taylor andres.tay...@neotechnology.com wrote: On Wed, Nov 2, 2011 at 12:01 PM, Jacob Hansson jacob.hans...@neotechnology.com wrote: Is there a complex reason why this is the case, and is it going to stay this way? It feels a bit counter-intuitive.. Cypher has no way of expressing uniqueness today. I think it should. I think the current scheme is least surprising: it allows the foaf thing to work without surprises: MATCH me-[:FRIEND]-friend-[:FRIEND]-foaf This will not return yourself as a foaf. I think I would like to be able to say: MATCH me-[:FRIEND]-friend-[:FRIEND]-foaf WITH NO UNIQUENESS/NODE UNIQUENESS/RELATIONSHIP UNIQUENESS/IDENTIFIER UNIQUENESS Or something like that. Identifier uniqueness would be the default. It's a difficult concept, so I want the default to be as non-surprising as possible. Makes sense. My two cents on this: The foaf example would have surprised me if it returned me the first time I ran it, but now, looking at the query, I'm actually surprised it does not. It would be cool to list some use cases, and try to determine which is the least surprising default. Also, the word unique could be confusing as well, since there could be lots of different nodes being referenced by the me identifier. Really, what you are saying is Guarantee that any node referenced by this identifier is not referenced by any other. This is one of the things that keep me up at night. Andrés ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST traverse deprecation
On Wed, Nov 2, 2011 at 12:34 PM, Andres Taylor andres.tay...@neotechnology.com wrote: On Wed, Nov 2, 2011 at 11:56 AM, Jacob Hansson jacob.hans...@neotechnology.com wrote: On Wed, Nov 2, 2011 at 11:11 AM, Michael Hunger michael.hun...@neotechnology.com wrote: * custom traversals with complicated callbacks This is really what my original question to Peter was about - what are some traversals like this, that we currently cannot do with Cypher or Gremlin? Can we extend Cypher to accomodate those use cases? Well, Gremlin is just Groovy+Pipes, so there's nothing you can't do in Gremlin. If we're talking about Cypher, I have one for you: PageRank. PageRank iterates over the data set a specified number of times, marking and updating stuff on the fly, until it's happy and returns a result. It's not hard at all to write using Gremlin, but impossible with Cypher. I can't see a way to make that nice and easy with Cypher. START a=node(1) MATCH p = a-[1..100]-other RETURN pageRank(a via p using weights blah saving state in property x) :) But seriously, aren't you alluding to a key distinction here - Graph Querying vs. Graph Processing? For processing, cypher is not your friend. But neither is writing jobs from scratch in your language of choice - at least not compared to writing a gremlin script. In this space (processing-style traversals), are there any use cases that are better served by arbitrary script execution than gremlin queries? And is catering to those use cases, if they exist, worth the cost in confusion, diffusion of effort and security problems? Andrés ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST traverse deprecation
On Wed, Nov 2, 2011 at 3:44 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Mmh, this thread is just about deprecating the travers() section of the REST API, not take away general scripting form the server. I have not seen one single case where people have exploited the JavaScript exposure in the return/stop evaluators to do anything like you say Michael. Extensions and plugins are not going away for the time being, I am only concerned about http://docs.neo4j.org/chunked/snapshot/rest-api-traverse.html not adding value to the API, and being able to be scripted in a more concise way via the existing extensions (which I want to make examples of). I haven't heard any argument for this case, so the question is whether to document the usecases in other ways (Groovy, JavaScript or Cypher plugins) and deprecate this section, since it doesn't add functionality, just exposes scripting in a hacked-up way. Good that you point this out, this thread is a bit confusing since we're talking about a lot of different things. If we can't find use cases that warrants the REST traversal API, we should deprecate it. It would be great to get the input of some of the people working on clients for the REST server before a decision is made though. I would also *really* like to continue the discussion about arbitrary script execution, because I want to understand the arguments for it. Perhaps we should break that discussion into a separate thread, to keep things concise. Imma start a new thread :) Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/- Öresund - Innovation happens HERE. On Wed, Nov 2, 2011 at 3:11 AM, Michael Hunger michael.hun...@neotechnology.com wrote: custom use-cases like the ones implemented by our customers with plugins / extensions and in the case of the ruby script extensions a way I'd like to promote to talk to the server that is 1) create you own RESTful endpoints that are focused on a protocol speaking your domain and not nodes + rels 2) the protocol is use-case oriented 3) it only transmits the information necessary (not more). 4) The code of these endpoints would be transactionally executed Use-Cases with customers so far have been: * unique nodes with locking + index lookup * batch data insertion (also unique stuff see above) * custom traversals with complicated callbacks * aggregating multiple gremlin / cypher queries in the server and returning only a partial result-set/projection of those executions * With the ruby script extension what we've done there is to leverage the web/REST frameworks of the platform and allow rack-apps to expose their REST endpoints within the neo4j-server. Thomas and I also talked about a more generic way to allow any language supported by the javax.script extension to register endpoints with the neo4j-server. A generic javax.scripting languages plugin would accommodate for more ad-hoc execution mode for which writing an extension is too much ceremony. Michael Am 02.11.2011 um 10:54 schrieb Jacob Hansson: On Wed, Nov 2, 2011 at 10:20 AM, Michael Hunger michael.hun...@neotechnology.com wrote: I vote not dropped, as there are lots of non-java developers out there which would want to use REST server dynamically. I am completely open to the notion that I am wrong here. But I want us to have answered the questions I was asking before properly, if we are going to embrace all these ways to talk to the server. What use cases does dynamic script execution cover that cypher or gremlin do not? Why should those use cases not be covered by cypher or gremlin? That was the reason why we wrote the ruby-script-extension support. Michael Am 02.11.2011 um 10:08 schrieb Jacob Hansson: On Tue, Nov 1, 2011 at 5:37 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Well, I think that Cypher is not yes taking into account Uniqueness, or is it Andres? Otherwise, most of the normal traversals are covered. To keep things consistent and give people a way of doing fine grained traversals, I would then suggest to document all the existing examples using Cypher or Groovy code, so we can document that things are working as expected and not loose functionality. Sounds like a really good idea. The JavaScript exposure then would totally go away from the REST API and possibly be factored out into a plugin like Groovy/Gremlin. Or dropped. I vote dropped :) WDYT? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer
[Neo4j] Running arbitrary code in the server
Breaking this out of the talk about deprecating REST traversals ( http://neo4j.org/forums/#nabble-td3460396). So, I'm trying to understand the arguments pro and con *POSTing arbitrary code to the server*, as well as *writing server plugins for the sake of adding domain logic inside the database*. To be clear, I'm not debating plugins that extend general server functionality, like spatial, gremlin, cypher and so on. It sounds to me like the main argument for both of these is you use it to build SOA by deploying code into the database. I think SOA is kick-ass, but I'm not sure I understand why running your domain logic inside your database is a good way to go about building SOA systems. Is the SOA thing the main argument for supporting arbitrary code execution? if answer is True: *What are the arguments for doing SOA this way?* else: *What is the main argument for arbitrary code execution then?* The reason I want this discussion is because I'm worried that there are problems with this approach that will bite us later, and I want to know that the arguments pro are worth that pain. I also think that a lot of the arguments for arbitrary script execution are related to shortcomings in the server and in Cypher, and I don't think hacking around those shortcomings is a good solution. -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST traverse deprecation
It would be interesting to rephrase the question like this: Are there any use cases that the REST traversal API covers, that we currently cannot replicate in Cypher or Gremlin? On Sun, Oct 30, 2011 at 2:45 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Well, Andres has been putting the first algo into cypher and we are thinking of exposing them there so thru can be used as path functions in an expression. However, that will be step 2. WDYT? On Oct 29, 2011 6:47 PM, maxdemarzi maxdema...@gmail.com wrote: I think the traversal rest api is fine for my purposes, but if cypher is where we want to go long term, then we need to deprecate it in 1.6 and drop it in 1.7. This gives us some time to update our libraries and for cypher to settle down. Are you guys thinking of getting rid of the rest built in graph algorithms or are those there to stay? -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-REST-traverse-deprecation-tp3460396p3464979.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Java segfault using paste and Python bindings
Hey Michael, thanks for the investigative work you put into this, and sorry
for not replying earlier.
I was not aware of the thread limitations of JPype, that's quite
dissapointing.. Overall, it feels like the JPype solution is not a
manageable one in the long term. The embedded use case is cool, because
it's so freaking convenient, but using JPype to embed the JVM is painful.
There are a few other options for JVM-python integration that are starting
to look interesting, perhaps we can look around a little bit and see what
is available. It would be especially cool if we can use something that
works across CPython, Jython as well as PyPy. I know Tobias was thinking
about something that would make that possible.
In the short to medium term though, I'd like to focus on two things (please
note that this is *my opinion*, not a product roadmap):
*Getting cypher and gremlin into the embedded python API*
Eventually, I think we should deprecate the traversal API in the embedded
client in favor of Cypher. Traversals need callbacks and shared state and
so on between the two VMs, and its gonna hurt, I think, if we depend on
that further down the road.
Doing this will also make it simple to keep full 1-1 API compatibility
between the embedded client and REST clients, making it easy for end users
to switch between the two.
*Making the REST API fast enough to make that use case more appealing.*
JPype is really just a cheap way of implementing a form of remoting API.
It's currently worth the pain, since it is a ton faster than our REST API,
but it's a band-aid, not a proper fix. We should focus on speeding up the
REST API.
WDYT?
/Jake
On Tue, Nov 1, 2011 at 5:00 PM, Michael Rene Armida m...@marmida.com wrote:
I solved my problem by ensuring that I don't 'import neo4j' until I'm in a
dedicated thread. So long as all neo4j interaction is handled within that
thread, I don't crash.
The Python neo4j bindings use JPype; I noticed this note from the JPype
docs (http://jpype.sourceforge.net/doc/user-guide/userguide.html#threading
):
For the most part, python threads based on OS level threads (i.e posix
threads), will work without problem. The only thing to remember is to
call jpype.attachThreadToJVM() in the thread body to make the JVM usable
from that thread. For threads that you do not start yourself, you can
call isThreadAttachedToJVM() to check.
I couldn't find any evidence for this, but I think that some of the Java
code in the neo4j bindings may call attachThreadToJVM at import time.
Unfortunately, this means that even though neo4j is thread-safe, it must be
constrained to a single thread when used from Python. But that's not too
disappointing, considering.
On Sat, Oct 29, 2011 at 4:25 PM, Michael Rene Armida m...@marmida.com
wrote:
I am attempting to use Paste to serve a small web app that uses neo4j via
the Python bindings. I get a segfault whenever I try to create a new
database via 'neo4j.GraphDatabase'. I have narrowed this down to the
following minimal sample:
http://pastebin.com/MkNRvLR6
The top of the stack trace looks like this:
Stack: [0xb42e7000,0xb4ae8000], sp=0xb4ae44f0, free space=8181k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
code)
C [_jpype.so+0x26497] JPJavaEnv::NewObjectA(_jclass*, _jmethodID*,
jvalue*)+0x37
C [_jpype.so+0x3c0e8] JPMethodOverload::invokeConstructor(_jclass*,
std::vectorHostRef*, std::allocatorHostRef* )+0x178
C [_jpype.so+0x3a417] JPMethod::invokeConstructor(std::vectorHostRef*,
std::allocatorHostRef* )+0x47
C [_jpype.so+0x1beba] JPClass::newInstance(std::vectorHostRef*,
std::allocatorHostRef* )+0x2a
C [_jpype.so+0x67b9c] PyJPClass::newClassInstance(_object*,
_object*)+0xfc
C [python+0x96822] PyEval_EvalFrameEx+0x4332
C [python+0x991e7] PyEval_EvalCodeEx+0x127
I believe that is JPype trying to create an EmbeddedGraphDatabase via
neo4j.GraphDatabase() in python.
I noticed this previous thread:
http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-CPython-Pylons-and-threading-td942435.html
That looks like it might be related, except this problem occurs whenever
I
try to create a database, not on shutdown. The issue definitely has
something to do with using neo4j under Paste; here's a Python shell
session
which shows neo4j working:
import webob
import app
debug_app = app.DebugApp()
response = debug_app(webob.Request.blank('/'))
response.body
'it worked'
Relevant version info:
- ubuntu 11.04 (same crash under 32- and 64-bit systems)
- Python 2.7.1-0ubuntu5
- neo4j_embedded-1.5.b2
- jpype 0.5.4.1-3
- paste 1.7.5.1
- OpenJDK 6b22-1.10.2 (same crash under the Sun JRE)
Thanks for your time and any clues you might have.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Jacob Hansson
Phone
Re: [Neo4j] REST traverse deprecation
On Tue, Nov 1, 2011 at 5:22 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Nope, I mean a server side script execution (which we already are doing in http://docs.neo4j.org/chunked/snapshot/rest-api-traverse.html) to execute traversal API code on the server. Neo4js is a client driver. Remember that adding yet another way to talk to the server comes at a cost of confusion and spreading engineering focus thin. I don't think we should do this unless there is a good answer to the question: *Is there anything* *that this solves that is not solved by Cypher or Gremlin, and if so, why don't we expand Cypher or Gremlin to incorporate that use case instead? *Collecting our efforts in fewer buckets means that improvements will benefit everyone, not just the group that uses the Groovy endpoint, or the Traversal API, or the JS endpoint and so on. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/- Öresund - Innovation happens HERE. On Tue, Nov 1, 2011 at 9:19 AM, Andres Taylor andres.tay...@neotechnology.com wrote: You, something like Neo4js https://github.com/neo4j/neo4js? We've had that since forever. Or do you mean something different? Andrés On Tue, Nov 1, 2011 at 5:06 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Also, there is another possibility: We are exposing a ECMA script execution engine via the Traverse enpoints. Instead of trying to shoehorn the traversal API into REST calls and then anyway executing script snippets, Why not exposing a JavaScript plugin along exactly the same lines as the Groovy plugin, and provide examples on how to use the same usecases via that plugin, so we cover the traversal API that way? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/- Öresund - Innovation happens HERE. On Tue, Nov 1, 2011 at 9:00 AM, Jacob Hansson jacob.hans...@neotechnology.com wrote: It would be interesting to rephrase the question like this: Are there any use cases that the REST traversal API covers, that we currently cannot replicate in Cypher or Gremlin? On Sun, Oct 30, 2011 at 2:45 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Well, Andres has been putting the first algo into cypher and we are thinking of exposing them there so thru can be used as path functions in an expression. However, that will be step 2. WDYT? On Oct 29, 2011 6:47 PM, maxdemarzi maxdema...@gmail.com wrote: I think the traversal rest api is fine for my purposes, but if cypher is where we want to go long term, then we need to deprecate it in 1.6 and drop it in 1.7. This gives us some time to update our libraries and for cypher to settle down. Are you guys thinking of getting rid of the rest built in graph algorithms or are those there to stay? -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-REST-traverse-deprecation-tp3460396p3464979.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Delete a graph in Neo4J REST server
On Fri, Oct 21, 2011 at 2:14 AM, andrew ton andrewt...@yahoo.com wrote: Hello, Is there a API or a way to delete a graph in the Neo4J REST server? For example, I can send a REST request to delete the root of the graph then all nodes under that root are also removed. Thanks, Hi Andrew, deleting the root node will only remove the root node, the rest of the graph will remain intact. The server on it's own can't delete the full database in one go, but you can use this server plugin to add that functionality: https://github.com/jexp/neo4j-clean-remote-db-addon /jake ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Configuring Neo4J via Python bindings
It looks like our build pipeline is clogged up right now. We're looking into it, but it might take a while.. /jake On Mon, Oct 17, 2011 at 9:56 PM, Chris Diehl di...@alumni.cmu.edu wrote: Hey Jake, Awesome! Thanks so much for making this happen! On my end, it doesn't look like the docs are up yet. I'll check back... Thanks again, Chris On Mon, Oct 17, 2011 at 4:12 AM, Jacob Hansson jacob.hans...@neotechnology.com wrote: Hey Chris, I've just added support for passing in extra JVM args, it's available as of this commit: https://github.com/neo4j/python-embedded/commit/d41950f8c7e6c65d67f60868976e1864a496a2ea Documentation will pop up in an hour or so here: http://docs.neo4j.org/chunked/snapshot/python-embedded-reference-core.html This feature will be available as of the 1.5 release, due later this week. If you want to get access to it right away, you can build neo4j-embedded directly from source, the code is available here: https://github.com/neo4j/python-embedded See the readme for how to build from source. Thanks for bringing this up! Jake On Fri, Oct 14, 2011 at 7:47 PM, Chris Diehl di...@alumni.cmu.edu wrote: Hi Michael, I added a print statement to my code to output how many nodes have been inserted. The script zips along until it gets up around 18.6K nodes. Then the program halts when a Java exception is thrown, as shown below. Inserting user 18629 Inserting user 18630 Inserting user 18631 Inserting user 18632 Inserting user 18633 Inserting user 18634 Inserting user 18635 Traceback (most recent call last): File jive_app_db_graph_construction.py, line 53, in module node_idx['userid'][user_node['userid']] = user_node File /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/contextlib.py, line 34, in __exit__ self.gen.throw(type, value, traceback) File string, line 11, in transaction RuntimeError: An unknown error occured while handling a Java Exception Strangely enough, the script returned a more explicit Out of Heap Memory error once when I reran the script. It's bizarre that the failure output is not consistent, given I start with a clean DB each time. Increasing the amount of available heap memory seems like the reasonable place to start. If you can let me know how to specify the amount of heap memory for the JVM instantiated by the bindings, that'd be great. I'm running on a MacBook Pro with 8 gigs of memory running OSX 10.6.8. Here's the contents of the messages.log file. 2011-10-14 10:34:17.336-0700: Creating new db @ /Users/chris.diehl/Graphs/ JiveCommunityV5.0.ng/neostore 2011-10-14 10:34:17.419-0700: Opened [/Users/chris.diehl/Graphs/ JiveCommunityV5.0.ng/nioneo_logical.log.1] clean empty log, version=0, lastTxId=1 2011-10-14 10:34:17.441-0700: Opened [/Users/chris.diehl/Graphs/ JiveCommunityV5.0.ng/index/lucene.log.1] clean empty log, version=0, lastTxId=1 2011-10-14 10:34:17.443-0700: TM new log: tm_tx_log.1 2011-10-14 10:34:17.451-0700: --- CONFIGURATION START --- 2011-10-14 10:34:17.451-0700: Physical mem: 8192MB, Heap size: 123MB 2011-10-14 10:34:17.452-0700: Kernel version: Neo4j - Graph Database Kernel 1.5-SNAPSHOT (revision: 1.5.M02-4-g3d1e9d8) 2011-10-14 10:34:17.453-0700: Neo4j - Graph Database Kernel 1.5-SNAPSHOT (revision: 1.5.M02-4-g3d1e9d8) 2011-10-14 10:34:17.456-0700: Operating System: Mac OS X; version: 10.6.8; arch: x86_64; cpus: 8 2011-10-14 10:34:17.456-0700: VM Name: Java HotSpot(TM) 64-Bit Server VM 2011-10-14 10:34:17.456-0700: VM Vendor: Apple Inc. 2011-10-14 10:34:17.456-0700: VM Version: 20.1-b02-384 2011-10-14 10:34:17.456-0700: Boot Class Path: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/jsfd.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/classes.jar:/System/Library/Frameworks/JavaVM.framework/Frameworks/JavaRuntimeSupport.framework/Resources/Java/JavaRuntimeSupport.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/ui.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/laf.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/sunrsasign.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/jsse.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/jce.jar:/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Classes/charsets.jar 2011-10-14 10:34:17.457-0700: Class Path: /Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/antlr-2.7.7.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/asm-3.2.jar:/Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/neo4j/javalib/asm-analysis-3.2.jar:/Users/chris.diehl
Re: [Neo4j] WebAdmin visualization (are rels. labels configurable?)
They can't currently, but it is actually the feature I'm currently working on, so it's coming :) On Mon, Oct 17, 2011 at 10:03 PM, Pablo Pareja ppar...@era7.com wrote: Hi! I just tried the WebAdmin visualization time for the first time and it seems pretty cool. I have one question though, can be relationships labels be different than the rel. name? I've been playing a bit with the display options for nodes but didn't find anything for relationhips. Cheers, Pablo -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Python embedded and Java batch inserter
I think this might be another case of a problem that would be a lot easier
to solve if the python bindings pushed out full stack traces. It's in my
backlog to fix that.
I'm gonna *guess* that the problem has to do with store upgrades. Are you
using the 1.5.M02 version of neo4j embedded?
If so, you need to tell the database it's ok to upgrade the store, like
this:
from neo4j import GraphDatabase
db = new GraphDatabase('somefolder', allow_store_upgrade=true)
On Mon, Oct 17, 2011 at 4:19 PM, Jacopo Farina jacopo1.far...@gmail.comwrote:
Hi all,
I'm trying to work with Neo4j embedded with Python, but, if I try to open a
database created with the Java batch inserter (neo4j 1.4) by using
Python-embedded, I get the error described here
https://trac.neo4j.org/ticket/275
The database works correctly when opened in Neoclipse or with neo4j
stand-alone server (by replacing the data folder), and if I create a new
database directly with Python embedded it works too. Is there a way to
solve
this problem?
Then, how much is fast the neo4j-REST traverser compared with the Java
embedded traverser on a 3 million nodes graph?
As far as I know, we don't have any benchmarks that compare this. The speed
difference should not be huge though. The server is a lot slower for reads
and writes, but for traversals, the overhead of the server plays a much
smaller part.
Sidenote: The size of your graph should generally not affect traversal speed
(unless the graph does not fit in memory), what matters is how much of the
graph you traverse. In java embedded, we used to say that you get maybe 1
000 000 relationships traversed per second (although speed can be both
higher and much lower, depending on what your traversal does).
Cheers!
Jacopo Farinat
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
/Jake
--
Jacob Hansson
Phone: +46 (0) 763503395
Twitter: @jakewins
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Index Creation via REST API
It seems that what is happening is that there already is an index with the
name you specify created, and for some reason, it has a boolean value as its
type.
We will have to dig into how that happened (do you think you could have
passed something like type:true during your experiments?), but it may be
possible to fix it without a patch, if you can delete the existing index.
If you open up webadmin, there is a tab for index management. See if your
index shows up there, and if it does, delete it. Then try running the curl
query you describe below again.
/Jake
On Mon, Oct 17, 2011 at 6:35 PM, Christian Scharr
christian.sch...@hsh-berlin.com wrote:
curl -X POST http://localhost:7474/db/data/index/node/ -H Content-Type:
application/json --data-binary
'{name:address,config:{provider:lucene,type:fulltext}}'
Greets,
Christian Scharr
-Original Message-
From: user-boun...@lists.neo4j.org [mailto:user-
boun...@lists.neo4j.org] On Behalf Of Peter Neubauer
Sent: Monday, October 17, 2011 6:11 PM
To: Neo4j user discussions
Subject: Re: [Neo4j] Index Creation via REST API
This looks right. Can you create the error with a curl call so I can
recreate it?
/peter
Sent from my phone.
On Oct 17, 2011 5:58 PM, Christian Scharr Christian.Scharr@hsh-
berlin.com
wrote:
Hello,
trying to work RESTfully with Neo4j via Standalone Server (1.5.M01)
I'm
about to become desperate.
I'm sending an POST-Request to
http://localhost:7474/db/data/index/node/, getting status-code 500
with
the response-text java.lang.Boolean cannot be cast to
java.lang.String.
The indexes I've wanted to create are the following one:
{name:address,config:{provider:lucene,type:fulltext}}
{name:user,config:{provider:lucene,type:fulltext}}
Neo4j log contains the following line:
17.10.2011 16:56:10 org.mortbay.log.Slf4jLog warn
FATAL: /db/data/index/node/
java.lang.ClassCastException: java.lang.Boolean cannot be cast to
java.lang.String
at
org.neo4j.index.impl.lucene.LuceneIndexImplementation.match(LuceneIndex
I
mplementation.java:165)
at
org.neo4j.index.impl.lucene.LuceneIndexImplementation.configMatches(Luc
e
neIndexImplementation.java:156)
at
org.neo4j.kernel.IndexManagerImpl.findIndexConfig(IndexManagerImpl.java
:
128)
at
org.neo4j.kernel.IndexManagerImpl.getOrCreateIndexConfig(IndexManagerIm
p
l.java:183)
at
org.neo4j.kernel.IndexManagerImpl.getOrCreateNodeIndex(IndexManagerImpl
.
java:272)
at
org.neo4j.kernel.IndexManagerImpl.forNodes(IndexManagerImpl.java:260)
at
org.neo4j.server.rest.web.DatabaseActions.createNodeIndex(DatabaseActio
n
s.java:338)
at
org.neo4j.server.rest.web.RestfulGraphDatabase.jsonCreateNodeIndex(Rest
f
ulGraphDatabase.java:589) [...]
Whats wrong with this requests and in case this is an known bug at
the
REST server-component when will it be fixed?
Greets,
Christian Scharr
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Jacob Hansson
Phone: +46 (0) 763503395
Twitter: @jakewins
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Set property on node in Neo4j REST server
On Mon, Oct 17, 2011 at 8:05 AM, Jim Webber j...@neotechnology.com wrote:
Hi Peter,
For the specific case of updating a single property then:
PUT node/{node_id}/property/{property_key}
with an entity body like:
some text
or
27
would be OK I think
But not until the 1.6 M01 release :-)
But this is exactly what we do today, no?
Jim
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Jacob Hansson
Phone: +46 (0) 763503395
Twitter: @jakewins
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Deprecation Warning with Python Neo4j Bindings
Glad you're enjoying it :) The warning seems to be coming from JPype itself, we'll see if they end up pushing an update for it. If it ends up becoming a problem for supporting newer python versions, we could potentially write a patch for JPype.. /Jake On Wed, Oct 12, 2011 at 10:42 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Neo folks, Just wanted to pass along a deprecation warning I'm getting after installing the latest versions of JPype and Neo4j under Python 2.6. from neo4j import GraphDatabase /Users/chris.diehl/.virtualenvs/py26-neo4j-embedded/lib/python2.6/site-packages/jpype/_pykeywords.py:18: DeprecationWarning: the sets module is deprecated import sets Stoked to use this for pushing data into Neo4j. Thanks again! Chris Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Embedded neo4j in Python up to date again
Agreed. It was Tobias that came up with that solution, I quite like it as
well, clean and simple.
/Jake
On Tue, Oct 11, 2011 at 6:00 AM, Tim McNamara
paperl...@timmcnamara.co.nzwrote:
This is great news. I love that it uses context managers for transactions.
On 7 October 2011 02:06, Jacob Hansson jacob.hans...@neotechnology.com
wrote:
On Thu, Oct 6, 2011 at 1:57 PM, Peter Neubauer
peter.neuba...@neotechnology.com wrote:
Yes Javier,
that would be great! Do you think you could open an issue and provide
some details on how to go about it?
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph
database.
http://startupbootcamp.org/- Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Oct 5, 2011 at 5:15 PM, Javier de la Rosa ver...@gmail.com
wrote:
On Wed, Oct 5, 2011 at 07:48, Jacob Hansson
jacob.hans...@neotechnology.com wrote:
there is now a first stable release of the new neo4j-embedded
python package available from PyPi.
This news is really great! And I'm very glad to see that some methods
are seemed to the neo4j-rest-client [1]. Is there any chance to find
some day from neo4j import restgraphdatabase and to make both
API's totally equal?
Thank you very much for all the work involved.
Regards!
[1] https://github.com/versae/neo4j-rest-client
--
Javier de la Rosa
http://versae.es
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Javier,
it would be really awesome to combine the two, or at least make sure the
APIs are completely compatible.
I'm hesitant to bundle them together, because the REST client is pure
python, and can keep a very small footprint. The embedded package, on the
other hand, bundles the full Neo4j distribution, and so it's a full 10MB
large.
In most settings, this won't matter much, but it still feels unnecessary
to
force people who just want the REST client to include the full DB
distribution in their projects.
Architecture-wise, perhaps we can have a common front for them, that can
then be powered either by the embedded implementation, or by the REST
client, and then use something like the db path to determine which one to
use:
from neo4j import GraphDatabase
embeddeddb = GraphDatabase('/home/data')
serverdb = GraphDatabase('http://localhost:7474')
There is one major hurdle we need to get past first, we need to implement
transaction support in the REST API that is feature-compatible with the
embedded transaction API. When that is in place, the REST client will be
able to behave exactly like the embedded client, and we can combine them.
/Jake
--
Jacob Hansson
Phone: +46 (0) 763503395
Twitter: @jakewins
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Jacob Hansson
Phone: +46 (0) 763503395
Twitter: @jakewins
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j embedded
If you choose to use the server or if you choose the embedded database will have a significant impact on how those results turn out. For performing large queries, the server should not be much slower than the embedded database. It will be significantly slower (more than two orders of magnitude) on single read/write operations. Like Jim mentioned, we are looking at a new REST implementation that essentially nullifies this speed difference. It is currently at an experimental stage, but we will make sure to share details and get feedback on it from y'all as it progresses, stay tuned. /Jake On Fri, Oct 7, 2011 at 3:15 PM, Antriani Stylianou antrian...@gmail.comwrote: Thanks again for the answer but my system is actually a benchmarking system! In other words I am trying to find out if there are performance difference between the 2 DBs not through an application. On 10/7/11 1:36 PM, Jim Webber wrote: Hi Antriani, Ok, but the REST API doesn't produce more network overhead (i.e. HTTP...) compared to a driver implementation using sockets for access for example MySQL JDBC ? I have no data to support this, but I'm going to make an educated guess and say that the MySQL JDBC driver is overall faster than our REST API :-) But my point stands: measure your system, not its components. Neo4j might well have a slower network API than MySQL, but you might find that because of the data model, you ship less data over the network, or the queries run so much faster that network differences become negligible. Something will be your bottleneck in either the Neo4j case or the MySQL case, but if the Neo4j case is faster despite having a slower network interface, then you might not care. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Embedded neo4j in Python up to date again
On Thu, Oct 6, 2011 at 1:57 PM, Peter Neubauer
peter.neuba...@neotechnology.com wrote:
Yes Javier,
that would be great! Do you think you could open an issue and provide
some details on how to go about it?
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/- Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Oct 5, 2011 at 5:15 PM, Javier de la Rosa ver...@gmail.com
wrote:
On Wed, Oct 5, 2011 at 07:48, Jacob Hansson
jacob.hans...@neotechnology.com wrote:
there is now a first stable release of the new neo4j-embedded
python package available from PyPi.
This news is really great! And I'm very glad to see that some methods
are seemed to the neo4j-rest-client [1]. Is there any chance to find
some day from neo4j import restgraphdatabase and to make both
API's totally equal?
Thank you very much for all the work involved.
Regards!
[1] https://github.com/versae/neo4j-rest-client
--
Javier de la Rosa
http://versae.es
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Javier,
it would be really awesome to combine the two, or at least make sure the
APIs are completely compatible.
I'm hesitant to bundle them together, because the REST client is pure
python, and can keep a very small footprint. The embedded package, on the
other hand, bundles the full Neo4j distribution, and so it's a full 10MB
large.
In most settings, this won't matter much, but it still feels unnecessary to
force people who just want the REST client to include the full DB
distribution in their projects.
Architecture-wise, perhaps we can have a common front for them, that can
then be powered either by the embedded implementation, or by the REST
client, and then use something like the db path to determine which one to
use:
from neo4j import GraphDatabase
embeddeddb = GraphDatabase('/home/data')
serverdb = GraphDatabase('http://localhost:7474')
There is one major hurdle we need to get past first, we need to implement
transaction support in the REST API that is feature-compatible with the
embedded transaction API. When that is in place, the REST client will be
able to behave exactly like the embedded client, and we can combine them.
/Jake
--
Jacob Hansson
Phone: +46 (0) 763503395
Twitter: @jakewins
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Embedded neo4j in Python up to date again
Hey everybody, a few hundred of you have already noticed that we snuck out a release of Neo4j embedded for python a few weeks back, but for those of you that haven't, there is now a first stable release of the new neo4j-embedded python package available from PyPi. Read all about it here: http://blog.neo4j.org/2011/10/from-neo4j-import-graphdatabase.html Documentation is available in the manual: http://docs.neo4j.org/chunked/snapshot/python-embedded.html We're really sorry it took so long :( Please let us know what you think, and if you find any issues! -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Fastest way to load large dataset using new python embedded mode
Hey Sean, we haven't added support for the batch inserter in the python bindings yet, I can think of two things you can do: a) Use the batch inserter in java land like you say to create your db, and then just set embedded python to use that db location. b) Just use the normal API, depending on how interconnected your data is, if you do transactions of say 100 000 inserts per TX, it shouldn't take that long to insert 70M nodes. Pure insert of 1 node with one property clocks in on my machine at about 30 000 inserts per second. It might be interesting to add support for the GEOFF ( http://py2neo.org/geoff/) import/export format that the cool kids behind py2neo has developed.. Add a ticket to the github page for neo4j-embedded if you'd like to see any of that happen :) /jake On Wed, Oct 5, 2011 at 4:03 PM, Sean Davis sdav...@mail.nih.gov wrote: I have a few datasets that contain about 70M nodes. Relationships between these sets will be sparse and will be added over time. What is the fastest way to load these nodes into neo4j? I can work with java (http://wiki.neo4j.org/content/Batch_Insert) if necessary, but I'd be interested to hear if there is a way to use this API in the new embedded python mode. Thanks, Sean ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST Performance
On Sun, Oct 2, 2011 at 10:36 PM, Mattias Persson matt...@neotechnology.comwrote: 2011/10/2 Jacob Hansson jacob.hans...@neotechnology.com Hey Christian, we don't have any proper benchmarks for the server currently, but it is, unfortunately, still quite a bit slower than the core embedded database. There are a few things you can do to get a faster experience: - Use persistent HTTP connections - Use HTTP pipelining (this has, in my late-night-hacky tests, given significant performance boosts, close to twice the speed) Any code/config/hints as to how to achieve this? Well, most modern HTTP implementations will use persistent connections if they can, but it's implementation dependent. In Java, the built in UrlConnection (or whatever the name of the class is, I don't remember. The thing you get when you do URL#getConnection) reuses TCP connections per default, on the precondition that you read the full response before sending the next request. As for pipelining, this is (supposedly) supported by Jetty, and so should work on the neo4j server (my experiments did not use Jetty to serve our API, but the principle for why it is faster still applies, and the same performance gains should be seen). So really, it's just a matter of using a HTTP client that supports pipelining, unfortunately, there are very few to choose from. Here is one for Java: http://hotpotato.biasedbit.com/ /Jake On Sun, Oct 2, 2011 at 6:37 AM, Christian Straight protosap...@mightyelephant.com wrote: I'm evaluating using REST-ful calls to neo4j stand-alone servermainly I would like to understand the performance issues better. Can anyone on this list direct me to some neo4j benchmarks? many thanks, Christian Straight @ WildGigs .:. Typed with thumbs and sent from my HTC Evo ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST API Base URI
On Tue, Sep 27, 2011 at 11:29 PM, Nuo Yan yan@gmail.com wrote: Hi Jake, Thanks for your reply. Sorry if my previous email was misleading, but I don't absolutely need key-based authentication; that just turned out to be the what I first thought of. I was trying to figure out what authentication methods were supported so that I can evaluate. HTTPS + basic auth with strong passwords do sound good to me at this moment. However, at this moment, instead of using authentication, I turned out configured security rules on my the neo4j server to only take connections from my app server so I temporarily don't have a problem in the near future. But it's good to know about the authentication extension. By the way, do you have any data about the performance impact of using the authentication extension to do basic auth + HTTPS compare to talking to neo4j without authentication (both through REST)? If you have such data, please let me know. That would be very helpful information. I'm afraid not. However, as long as you make sure to reuse TCP connections (Connection:keep-alive), the overhead of https should be minimal. While the same does not apply for basic auth, it should not add a significant performance overhead compared to normal REST. /Jake Thanks, Nuo On Tue, Sep 27, 2011 at 6:10 AM, Jacob Hansson jacob.hans...@neotechnology.com wrote: As far as I know, the auth plugin currently only supports HTTP BASIC authentication. But it would be interesting to expand it to allow other authentication methods, for instance by allowing pluggable authentication providers. So if I understand your question correctly, yes, it does support authentication beyond just limiting access by IP, but it does not support authentication using keys or certificates. Out of curiosity, what is the driver behind needing key-based authentication? The combination of HTTP BASIC, strong passwords and HTTPS seems like it would supply quite strong security, no? /Jake On Wed, Sep 21, 2011 at 9:16 PM, Nuo Yan yan@gmail.com wrote: Limiting access to certain URIs or IPs is great, but ideally at this moment I also need something like a key-based authentication so I can send a private key in the headers from my restclient and neo4j would authenticate and only take authenticated requests. Does the authentication extension support key-based authentication? If not, what forms of authentication does it currently support (user/password, token-based, shared-secret)? Thanks, Nuo On Wed, Sep 21, 2011 at 11:24 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Nuo, Thomas Baum has been working on an authentication extension for Neo4j Server, see https://github.com/neo4j/authentication-extension. He could help you set it up and test it - it's used in hosting scenarios and should do what you want in terms of limiting access to certain URIs if that is what you want? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, Sep 21, 2011 at 6:14 PM, Nuo Yan yan@gmail.com wrote: For my scenario, I currently only use the LB for request authentication, so that not everyone on the network can access the neo4j server. In my use case, I think making the base uri configurable (including the protocol, e.g. https://foobar.com/) is enough. There are a couple other approaches that may also solve my problem. For example, having a configurable option to make the REST APIs return only relative paths in the result. Or, if neo4j could have a built-in request authentication mechanism for the REST interface, I may be able to get rid of the lb completely. On Wed, Sep 21, 2011 at 7:04 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Nuo, right now the server is discovering its own IP upon startup. However, it sounds reasonable to return configurable base URL. I am just wondering if that really is enough to support a LB scenario? Do you need more stuff there? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE
Re: [Neo4j] REST API Base URI
||types} Is there a config on the neo4j server that I can set to make it either return the lb URI https://10.0.0.1; as the base uri or return relative path in the result? ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] neo4j python import error
For the DLL issues, see the windows section of the installation
instructions, here:
http://docs.neo4j.org/chunked/1.5-SNAPSHOT/python-embedded-installation.html
Keep me posted!
Jake
On Fri, Sep 23, 2011 at 10:37 AM, bmeagle dewald.piete...@gmail.com wrote:
On Thu, Sep 22, 2011 at 3:37 PM, Jacob Hansson-2 [via Neo4j Community
Discussions] ml-node+s438527n3358782...@n3.nabble.com wrote:
Update on this:
I just pushed this to master, you will be able to follow the build via
our
build control panel. When the deploy Embedded Python project has run
successfully, the updated version should be available on pypi for
download.
There is currently almost 200 builds queued up, so it may take a while..
http://builder.neo4j.org/project.html?projectId=project34tab=projectOverview
(just click login as guest)
Thanks for reporting this, it turned out to be a proper bug (using :
instead of ; as classpath separators on windows). Let me know if the
fix
worked!
/Jake
I downloaded and installed the newest version, but I still got the same
error, classpath was still incorrect.
print os.environ['CLASSPATH']
.;C:\Program
Files\Java\jre1.6.0_07\lib\ext\QTJava.zip;C:\Python27\Lib\site-packages\neo4j\javalib\
import neo4j
Traceback (most recent call last):
File pyshell#3, line 1, in module
import neo4j
File C:\Python27\lib\site-packages\neo4j\__init__.py, line 29, in
module
from neo4j.core import GraphDatabase, Direction, NotFoundException, BOTH,
ANY, INCOMING, OUTGOING
File C:\Python27\lib\site-packages\neo4j\core.py, line 19, in module
from _backend import *
File C:\Python27\lib\site-packages\neo4j\_backend.py, line 158, in
module
raise ImportError(Cannot find Neo4j java classes, used classpath: %s %
classpath)
ImportError: Cannot find Neo4j java classes, used classpath: .;C:\Program
Files\Java\jre1.6.0_07\lib\ext\QTJava.zip;C:\Python27\Lib\site-packages\neo4j\javalib\
I then updated the classpath with a modified version of your code:
libpath = 'C:\\Python27\\Lib\\site-packages\\neo4j\\javalib\\'
os.environ['CLASSPATH']=';'.join(libpath+p for p in
os.listdir(libpath))
print os.environ['CLASSPATH']
C:\Python27\Lib\site-packages\neo4j\javalib\antlr-2.7.7.jar;C:\Python27\Lib\site-packages\neo4j\javalib\asm-3.2.jar;C:\Python27\Lib\site-packages\neo4j\javalib\asm-analysis-3.2.jar;C:\Python27\Lib\site-packages\neo4j\javalib\asm-commons-3.2.jar;C:\Python27\Lib\site-packages\neo4j\javalib\asm-tree-3.2.jar;C:\Python27\Lib\site-packages\neo4j\javalib\asm-util-3.2.jar;C:\Python27\Lib\site-packages\neo4j\javalib\blueprints-core-0.9.jar;C:\Python27\Lib\site-packages\neo4j\javalib\geronimo-jta_1.1_spec-1.1.1.jar;C:\Python27\Lib\site-packages\neo4j\javalib\gremlin-1.2.jar;C:\Python27\Lib\site-packages\neo4j\javalib\groovy-1.8.1.jar;C:\Python27\Lib\site-packages\neo4j\javalib\jansi-1.5.jar;C:\Python27\Lib\site-packages\neo4j\javalib\jettison-1.3.jar;C:\Python27\Lib\site-packages\neo4j\javalib\lucene-core-3.1.0.jar;C:\Python27\Lib\site-packages\neo4j\javalib\neo4j-1.5-SNAPSHOT.jar;C:\Python27\Lib\site-packages\neo4j\javalib\neo4j-cypher-1.5-SNAPSHOT.jar;C:\Python27\Lib\site-packages\
ne
o4j\javalib\neo4j-graph-algo-1.5-SNAPSHOT.jar;C:\Python27\Lib\site-packages\neo4j\javalib\neo4j-graph-matching-1.5-SNAPSHOT.jar;C:\Python27\Lib\site-packages\neo4j\javalib\neo4j-jmx-1.5-SNAPSHOT.jar;C:\Python27\Lib\site-packages\neo4j\javalib\neo4j-kernel-1.5-SNAPSHOT.jar;C:\Python27\Lib\site-packages\neo4j\javalib\neo4j-lucene-index-1.5-SNAPSHOT.jar;C:\Python27\Lib\site-packages\neo4j\javalib\neo4j-udc-1.5-SNAPSHOT.jar;C:\Python27\Lib\site-packages\neo4j\javalib\pipes-0.7.jar;C:\Python27\Lib\site-packages\neo4j\javalib\scala-library-2.9.0-1.jar;C:\Python27\Lib\site-packages\neo4j\javalib\stax-api-1.0.1.jar;C:\Python27\Lib\site-packages\neo4j\javalib\__init__.py;C:\Python27\Lib\site-packages\neo4j\javalib\__init__.pyc;C:\Python27\Lib\site-packages\neo4j\javalib\__init__.pyo
Now it complains about DLL's:
import neo4j
Traceback (most recent call last):
File pyshell#10, line 1, in module
import neo4j
File C:\Python27\lib\site-packages\neo4j\__init__.py, line 29, in
module
from neo4j.core import GraphDatabase, Direction, NotFoundException, BOTH,
ANY, INCOMING, OUTGOING
File C:\Python27\lib\site-packages\neo4j\core.py, line 19, in module
from _backend import *
File C:\Python27\lib\site-packages\neo4j\_backend.py, line 113, in
module
raise Exception(Unable to start JVM, even though I found the JVM path.
If you are using windows, this may be due to missing system DLL files,
please see the windows installation instructions in the neo4j
documentation.,e)
Exception: ('Unable to start JVM, even though I found the JVM path. If you
are using windows, this may be due to missing system DLL files, please see
the windows installation instructions in the neo4j documentation.',
RuntimeError('Unable to start JVM at src/native/common/jp_env.cpp:54
Re: [Neo4j] neo4j python import error
Great :) If you run into any other problems, or have ideas for improvements, please feel free to post them here on the mailing list, or to create issues with the project bug tracker: https://github.com/neo4j/python-embedded/issues /Jake On Fri, Sep 23, 2011 at 10:57 AM, bmeagle dewald.piete...@gmail.com wrote: I've sorted the DLL issues, now I can import neo4j along with my os.environ path additions. On Fri, Sep 23, 2011 at 10:55 AM, Jacob Hansson-2 [via Neo4j Community Discussions] ml-node+s438527n3361314...@n3.nabble.com wrote: For the DLL issues, see the windows section of the installation instructions, here: http://docs.neo4j.org/chunked/1.5-SNAPSHOT/python-embedded-installation.html Keep me posted! Jake On Fri, Sep 23, 2011 at 10:37 AM, bmeagle [hidden email] http://user/SendEmail.jtp?type=nodenode=3361314i=0 wrote: On Thu, Sep 22, 2011 at 3:37 PM, Jacob Hansson-2 [via Neo4j Community Discussions] [hidden email] http://user/SendEmail.jtp?type=nodenode=3361314i=1 wrote: Update on this: I just pushed this to master, you will be able to follow the build via our build control panel. When the deploy Embedded Python project has run successfully, the updated version should be available on pypi for download. There is currently almost 200 builds queued up, so it may take a while.. http://builder.neo4j.org/project.html?projectId=project34tab=projectOverview (just click login as guest) Thanks for reporting this, it turned out to be a proper bug (using : instead of ; as classpath separators on windows). Let me know if the fix worked! /Jake I downloaded and installed the newest version, but I still got the same error, classpath was still incorrect. print os.environ['CLASSPATH'] .;C:\Program Files\Java\jre1.6.0_07\lib\ext\QTJava.zip;C:\Python27\Lib\site-packages\neo4j\javalib\ import neo4j Traceback (most recent call last): File pyshell#3, line 1, in module import neo4j File C:\Python27\lib\site-packages\neo4j\__init__.py, line 29, in module from neo4j.core import GraphDatabase, Direction, NotFoundException, BOTH, ANY, INCOMING, OUTGOING File C:\Python27\lib\site-packages\neo4j\core.py, line 19, in module from _backend import * File C:\Python27\lib\site-packages\neo4j\_backend.py, line 158, in module raise ImportError(Cannot find Neo4j java classes, used classpath: %s % classpath) ImportError: Cannot find Neo4j java classes, used classpath: .;C:\Program Files\Java\jre1.6.0_07\lib\ext\QTJava.zip;C:\Python27\Lib\site-packages\neo4j\javalib\ I then updated the classpath with a modified version of your code: libpath = 'C:\\Python27\\Lib\\site-packages\\neo4j\\javalib\\' os.environ['CLASSPATH']=';'.join(libpath+p for p in os.listdir(libpath)) print os.environ['CLASSPATH'] C:\Python27\Lib\site-packages\neo4j\javalib\antlr-2.7.7.jar;C:\Python27\Lib\site-packages\neo4j\javalib\asm-3.2.jar;C:\Python27\Lib\site-packages\neo4j\javalib\asm-analysis-3.2.jar;C:\Python27\Lib\site-packages\neo4j\javalib\asm-commons-3.2.jar;C:\Python27\Lib\site-packages\neo4j\javalib\asm-tree-3.2.jar;C:\Python27\Lib\site-packages\neo4j\javalib\asm-util-3.2.jar;C:\Python27\Lib\site-packages\neo4j\javalib\blueprints-core-0.9.jar;C:\Python27\Lib\site-packages\neo4j\javalib\geronimo-jta_1.1_spec-1.1.1.jar;C:\Python27\Lib\site-packages\neo4j\javalib\gremlin-1.2.jar;C:\Python27\Lib\site-packages\neo4j\javalib\groovy-1.8.1.jar;C:\Python27\Lib\site-packages\neo4j\javalib\jansi-1.5.jar;C:\Python27\Lib\site-packages\neo4j\javalib\jettison-1.3.jar;C:\Python27\Lib\site-packages\neo4j\javalib\lucene-core-3.1.0.jar;C:\Python27\Lib\site-packages\neo4j\javalib\neo4j-1.5-SNAPSHOT.jar;C:\Python27\Lib\site-packages\neo4j\javalib\neo4j-cypher-1.5-SNAPSHOT.jar;C:\Python27\Lib\site-packages\ ne o4j\javalib\neo4j-graph-algo-1.5-SNAPSHOT.jar;C:\Python27\Lib\site-packages\neo4j\javalib\neo4j-graph-matching-1.5-SNAPSHOT.jar;C:\Python27\Lib\site-packages\neo4j\javalib\neo4j-jmx-1.5-SNAPSHOT.jar;C:\Python27\Lib\site-packages\neo4j\javalib\neo4j-kernel-1.5-SNAPSHOT.jar;C:\Python27\Lib\site-packages\neo4j\javalib\neo4j-lucene-index-1.5-SNAPSHOT.jar;C:\Python27\Lib\site-packages\neo4j\javalib\neo4j-udc-1.5-SNAPSHOT.jar;C:\Python27\Lib\site-packages\neo4j\javalib\pipes-0.7.jar;C:\Python27\Lib\site-packages\neo4j\javalib\scala-library-2.9.0-1.jar;C:\Python27\Lib\site-packages\neo4j\javalib\stax-api-1.0.1.jar;C:\Python27\Lib\site-packages\neo4j\javalib\__init__.py;C:\Python27\Lib\site-packages\neo4j\javalib\__init__.pyc;C:\Python27\Lib\site-packages\neo4j\javalib\__init__.pyo Now it complains about DLL's: import neo4j Traceback (most recent call last): File pyshell#10, line 1, in module import neo4j File C:\Python27\lib\site-packages\neo4j
Re: [Neo4j] neo4j python import error
Hmm, I think the error you are getting is the result of neo4j-embedded not
finding java jars that it needs (I'll add it to the backlog to add an
explicit check for that, rather than failing like this).
Normally, neo4j-embedded will set its classpath itself, pointing to the java
jars it comes bundled with. However, it is possible to override this, by
setting the environment variable CLASSPATH to some value. Could you check
if this is the case?
If not, please let me know what platform you are running, and what version
of neo4j-embedded you have installed, and I'll dig deeper into it.
Oh, and I'll make a note that the environment var needs to be changed to
something more specific.
/Jake
On Thu, Sep 22, 2011 at 1:46 PM, bmeagle dewald.piete...@gmail.com wrote:
Hi all,
Decided to give neo4j via python a try but alas:
from neo4j import GraphDatabase
Traceback (most recent call last):
File lt;pyshell#1gt;, line 1, in module
from neo4j import GraphDatabase
File C:\Python27\lib\site-packages\neo4j\__init__.py, line 29, in
module
from neo4j.core import GraphDatabase, Direction, NotFoundException,
BOTH, ANY, INCOMING, OUTGOING
File C:\Python27\lib\site-packages\neo4j\core.py, line 27, in module
GraphDatabase = extends(GraphDatabaseService)
File C:\Python27\lib\site-packages\neo4j\_backend.py, line 43, in
extends
return ParentClass(getattr(CLASS,'__name__','Class'),(),{})
File C:\Python27\lib\site-packages\neo4j\_backend.py, line 34, in
__new__
return type.__new__(Class, name, (object,), body)
TypeError: type() argument 1 must be string, not JPackage
I am using winxp I installed neo4j-embedded-1.5.dev104.win32.exe and I am
using python 2.7
Regards
bmeagle
--
View this message in context:
http://neo4j-community-discussions.438527.n3.nabble.com/neo4j-python-import-error-tp3358482p3358482.html
Sent from the Neo4j Community Discussions mailing list archive at
Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Jacob Hansson
Phone: +46 (0) 763503395
Twitter: @jakewins
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] neo4j python import error
No, the sys.path thing won't work, JPype doesn't pull it's java libs off of
sys.path (unfortunately). Instead, it needs an explicit java classpath
passed to it when it gets instantiated.
Neo4j-embedded sets that classpath either on it's own (which may be broken
on windows, I'm investigating that now), or using the value of the CLASSPATH
environment variable.
You could try setting the CLASSPATH environment var before you import neo4j:
import os
libpath = 'path/to/neo4j/javalib'
os.environ['CLASSPATH'] = ';'.join( [ os.path.abspath(p) for p in
os.listdir(libpath)])
from neo4j import GraphDatabase
/Jake
On Thu, Sep 22, 2011 at 2:45 PM, bmeagle dewald.piete...@gmail.com wrote:
I tried adding C:\Python27\Lib\site-packages\neo4j\javalib to sys.path but
that didn't work, see below.
I am running python 2.7 on win xp I installed
neo4j-embedded-1.5.dev104.win32.exe
import neo4j
Traceback (most recent call last):
File lt;pyshell#0gt;, line 1, in module
import neo4j
File C:\Python27\lib\site-packages\neo4j\__init__.py, line 29, in
module
from neo4j.core import GraphDatabase, Direction, NotFoundException,
BOTH, ANY, INCOMING, OUTGOING
File C:\Python27\lib\site-packages\neo4j\core.py, line 27, in module
GraphDatabase = extends(GraphDatabaseService)
File C:\Python27\lib\site-packages\neo4j\_backend.py, line 43, in
extends
return ParentClass(getattr(CLASS,'__name__','Class'),(),{})
File C:\Python27\lib\site-packages\neo4j\_backend.py, line 34, in
__new__
return type.__new__(Class, name, (object,), body)
TypeError: type() argument 1 must be string, not JPackage
import sys
print sys.path
['C:\\Python27\\Lib\\idlelib',
'C:\\Python27\\lib\\site-packages\\zodb3-3.10.3-py2.7-win32.egg',
'C:\\Python27\\lib\\site-packages\\zope.interface-3.8.0-py2.7-win32.egg',
'C:\\Python27\\lib\\site-packages\\zope.event-3.5.1-py2.7.egg',
'C:\\Python27\\lib\\site-packages\\zdaemon-2.0.4-py2.7.egg',
'C:\\Python27\\lib\\site-packages\\zconfig-2.9.0-py2.7.egg',
'C:\\Python27\\lib\\site-packages\\zc.lockfile-1.0.0-py2.7.egg',
'C:\\Python27\\lib\\site-packages\\transaction-1.1.1-py2.7.egg',
'C:\\WINDOWS\\system32\\python27.zip', 'C:\\Python27\\DLLs',
'C:\\Python27\\lib', 'C:\\Python27\\lib\\plat-win',
'C:\\Python27\\lib\\lib-tk', 'C:\\Python27',
'C:\\Python27\\lib\\site-packages',
'C:\\Python27\\lib\\site-packages\\PIL',
'C:\\Python27\\lib\\site-packages\\win32',
'C:\\Python27\\lib\\site-packages\\win32\\lib',
'C:\\Python27\\lib\\site-packages\\Pythonwin']
sys.path.append('C:\Python27\Lib\site-packages\neo4j\javalib')
print sys.path
['C:\\Python27\\Lib\\idlelib',
'C:\\Python27\\lib\\site-packages\\zodb3-3.10.3-py2.7-win32.egg',
'C:\\Python27\\lib\\site-packages\\zope.interface-3.8.0-py2.7-win32.egg',
'C:\\Python27\\lib\\site-packages\\zope.event-3.5.1-py2.7.egg',
'C:\\Python27\\lib\\site-packages\\zdaemon-2.0.4-py2.7.egg',
'C:\\Python27\\lib\\site-packages\\zconfig-2.9.0-py2.7.egg',
'C:\\Python27\\lib\\site-packages\\zc.lockfile-1.0.0-py2.7.egg',
'C:\\Python27\\lib\\site-packages\\transaction-1.1.1-py2.7.egg',
'C:\\WINDOWS\\system32\\python27.zip', 'C:\\Python27\\DLLs',
'C:\\Python27\\lib', 'C:\\Python27\\lib\\plat-win',
'C:\\Python27\\lib\\lib-tk', 'C:\\Python27',
'C:\\Python27\\lib\\site-packages',
'C:\\Python27\\lib\\site-packages\\PIL',
'C:\\Python27\\lib\\site-packages\\win32',
'C:\\Python27\\lib\\site-packages\\win32\\lib',
'C:\\Python27\\lib\\site-packages\\Pythonwin',
'C:\\Python27\\Lib\\site-packages\\neo4j\\javalib']
sys.path.append('C:\\Python27\\Lib\\site-packages\\neo4j\\javalib')
import neo4j
Traceback (most recent call last):
File lt;pyshell#8gt;, line 1, in module
import neo4j
File C:\Python27\lib\site-packages\neo4j\__init__.py, line 29, in
module
from neo4j.core import GraphDatabase, Direction, NotFoundException,
BOTH, ANY, INCOMING, OUTGOING
File C:\Python27\lib\site-packages\neo4j\core.py, line 27, in module
GraphDatabase = extends(GraphDatabaseService)
File C:\Python27\lib\site-packages\neo4j\_backend.py, line 43, in
extends
return ParentClass(getattr(CLASS,'__name__','Class'),(),{})
File C:\Python27\lib\site-packages\neo4j\_backend.py, line 34, in
__new__
return type.__new__(Class, name, (object,), body)
TypeError: type() argument 1 must be string, not JPackage
Jacob Hansson-2 wrote:
Hmm, I think the error you are getting is the result of neo4j-embedded
not
finding java jars that it needs (I'll add it to the backlog to add an
explicit check for that, rather than failing like this).
Normally, neo4j-embedded will set its classpath itself, pointing to the
java
jars it comes bundled with. However, it is possible to override this, by
setting the environment variable CLASSPATH to some value. Could you
check
if this is the case?
If not, please let me know what platform you are running, and what
version
of neo4j-embedded you have installed, and I'll dig
Re: [Neo4j] neo4j python import error
On Thu, Sep 22, 2011 at 3:16 PM, bmeagle dewald.piete...@gmail.com wrote:
I manually added the path to the CLASSPATH windows environment variable via
system settings.
print os.environ['CLASSPATH']
.;C:\Program
Files\Java\jre1.6.0_07\lib\ext\QTJava.zip;C:\Python27\Lib\site-packages\neo4j\javalib\
checked it:
libpath = 'C:\\Python27\\Lib\\site-packages\\neo4j\\javalib\\'
print os.listdir(libpath)
['geronimo-jta_1.1_spec-1.1.1.jar', 'lucene-core-3.1.0.jar',
'neo4j-1.5-SNAPSHOT.jar', 'neo4j-cypher-1.5-SNAPSHOT.jar',
'neo4j-graph-algo-1.5-SNAPSHOT.jar',
'neo4j-graph-matching-1.5-SNAPSHOT.jar', 'neo4j-jmx-1.5-SNAPSHOT.jar',
'neo4j-kernel-1.5-SNAPSHOT.jar', 'neo4j-lucene-index-1.5-SNAPSHOT.jar',
'neo4j-udc-1.5-SNAPSHOT.jar', 'scala-library-2.9.0-1.jar', '__init__.py',
'__init__.pyc', '__init__.pyo']
but still no joy
import neo4j
Traceback (most recent call last):
File lt;pyshell#15gt;, line 1, in module
import neo4j
File C:\Python27\lib\site-packages\neo4j\__init__.py, line 29, in
module
from neo4j.core import GraphDatabase, Direction, NotFoundException,
BOTH, ANY, INCOMING, OUTGOING
File C:\Python27\lib\site-packages\neo4j\core.py, line 27, in module
GraphDatabase = extends(GraphDatabaseService)
File C:\Python27\lib\site-packages\neo4j\_backend.py, line 43, in
extends
return ParentClass(getattr(CLASS,'__name__','Class'),(),{})
File C:\Python27\lib\site-packages\neo4j\_backend.py, line 34, in
__new__
return type.__new__(Class, name, (object,), body)
TypeError: type() argument 1 must be string, not JPackage
Couldn't this be something else?
Well, yeah, but I think this is it. I was able to replicate your problem on
a local xp machine (and have a fix that seems to work, running the test
suite on it right now). It should be available for download in two hours or
so, depending on how our build servers are feeling today.
Until then, the problem with the classpath you sent above is that it just
points to the folder with the jars. The classpath needs to explicitly list
the path to each jar, separated by semicolons. The code I sent in my
previous mail should do that automatically for you, if you just give it the
correct javalib folder path.
bmeagle
--
View this message in context:
http://neo4j-community-discussions.438527.n3.nabble.com/neo4j-python-import-error-tp3358482p3358708.html
Sent from the Neo4j Community Discussions mailing list archive at
Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Jacob Hansson
Phone: +46 (0) 763503395
Twitter: @jakewins
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] neo4j python import error
Update on this:
I just pushed this to master, you will be able to follow the build via our
build control panel. When the deploy Embedded Python project has run
successfully, the updated version should be available on pypi for download.
There is currently almost 200 builds queued up, so it may take a while..
http://builder.neo4j.org/project.html?projectId=project34tab=projectOverview
(just click login as guest)
Thanks for reporting this, it turned out to be a proper bug (using :
instead of ; as classpath separators on windows). Let me know if the fix
worked!
/Jake
On Thu, Sep 22, 2011 at 3:20 PM, Jacob Hansson
jacob.hans...@neotechnology.com wrote:
On Thu, Sep 22, 2011 at 3:16 PM, bmeagle dewald.piete...@gmail.comwrote:
I manually added the path to the CLASSPATH windows environment variable
via
system settings.
print os.environ['CLASSPATH']
.;C:\Program
Files\Java\jre1.6.0_07\lib\ext\QTJava.zip;C:\Python27\Lib\site-packages\neo4j\javalib\
checked it:
libpath = 'C:\\Python27\\Lib\\site-packages\\neo4j\\javalib\\'
print os.listdir(libpath)
['geronimo-jta_1.1_spec-1.1.1.jar', 'lucene-core-3.1.0.jar',
'neo4j-1.5-SNAPSHOT.jar', 'neo4j-cypher-1.5-SNAPSHOT.jar',
'neo4j-graph-algo-1.5-SNAPSHOT.jar',
'neo4j-graph-matching-1.5-SNAPSHOT.jar', 'neo4j-jmx-1.5-SNAPSHOT.jar',
'neo4j-kernel-1.5-SNAPSHOT.jar', 'neo4j-lucene-index-1.5-SNAPSHOT.jar',
'neo4j-udc-1.5-SNAPSHOT.jar', 'scala-library-2.9.0-1.jar', '__init__.py',
'__init__.pyc', '__init__.pyo']
but still no joy
import neo4j
Traceback (most recent call last):
File lt;pyshell#15gt;, line 1, in module
import neo4j
File C:\Python27\lib\site-packages\neo4j\__init__.py, line 29, in
module
from neo4j.core import GraphDatabase, Direction, NotFoundException,
BOTH, ANY, INCOMING, OUTGOING
File C:\Python27\lib\site-packages\neo4j\core.py, line 27, in module
GraphDatabase = extends(GraphDatabaseService)
File C:\Python27\lib\site-packages\neo4j\_backend.py, line 43, in
extends
return ParentClass(getattr(CLASS,'__name__','Class'),(),{})
File C:\Python27\lib\site-packages\neo4j\_backend.py, line 34, in
__new__
return type.__new__(Class, name, (object,), body)
TypeError: type() argument 1 must be string, not JPackage
Couldn't this be something else?
Well, yeah, but I think this is it. I was able to replicate your problem on
a local xp machine (and have a fix that seems to work, running the test
suite on it right now). It should be available for download in two hours or
so, depending on how our build servers are feeling today.
Until then, the problem with the classpath you sent above is that it just
points to the folder with the jars. The classpath needs to explicitly list
the path to each jar, separated by semicolons. The code I sent in my
previous mail should do that automatically for you, if you just give it the
correct javalib folder path.
bmeagle
--
View this message in context:
http://neo4j-community-discussions.438527.n3.nabble.com/neo4j-python-import-error-tp3358482p3358708.html
Sent from the Neo4j Community Discussions mailing list archive at
Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Jacob Hansson
Phone: +46 (0) 763503395
Twitter: @jakewins
--
Jacob Hansson
Phone: +46 (0) 763503395
Twitter: @jakewins
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4jPHP batch insert benchmarks
On Fri, Aug 26, 2011 at 2:46 PM, jadell josh.ad...@gmail.com wrote: Jim, Fair enough. For now, I'll just know not to try and make batches that big :-) My own use case is for the transaction safety rather than trying to create thousands of entities at once, so it doesn't effect me that much. I just wanted to have something more concrete to tell other users who might try. Thanks to all for helping me investigate! Update on this: I did a bit of hacking on this just now, and was able to improve the memory usage quite a bit. The major culprits was jacksons deserializer (or rather, that we use their mapping serializer), and using a list of strings to store and eventually aggregate results rather than StringBuilder (which is my fault, since it's my code, and I have no clue why I did it like that). I just pushed this to master. My benchmarks look like this, showing number of inserts, each insert being a node and a property: Old: --- 1000 : 1.340s 5000 : 3.013s 1 : 4.304s 5 : 18.120s 10 : OutOfMemory 50 : OutOfMemory New: --- 1000 : 1.546s 5000 : 3.116s 1 : 3.702s 5 : 17.183s 10 : 35.427s 50 : OutOfMemory This is on a JVM with 1GB heap. The OutOfMemory that we see in the new setup is because I didn't go all the way and implement streaming output, so that would be cool to try as well. /Jake -- Josh -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4jPHP-batch-insert-benchmarks-tp3282984p3286721.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST batch completes with error code
(StringUtils.java:3088) at org.apache.commons.lang.StringUtils.join(StringUtils.java:3133) at org.neo4j.server.rest.repr.BatchOperationResults.toJSON(BatchOperationResults.java:99) at org.neo4j.server.rest.web.BatchOperationService.performBatchOperations(BatchOperationService.java:88) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:184) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:67) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:276) at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:83) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:133) at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:71) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1171) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1103) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1053) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1043) at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:406) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:477) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:662) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) -- Josh Adell -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/REST-batch-completes-with-error-code-tp3303247p3303247.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Websocket Server instead of REST for Neo4J for access a DB
/response, so there's not a lot of benefit to switching to websockets without a lot of rework I think thats a real challenge. Since locking and concurrent writes come into play - Except you would allow only one websocket connection - The real benefit will be when you Javascript and other web clients can use a pure binary protocol (not possible today) I pretty sure it is in the latest protocol definition. But i think no Browser at this time of writing has implemented the binary parts. regards Tom Rick -Original Message- From: user-boun...@lists.neo4j.org [mailto: user-boun...@lists.neo4j.org] On Behalf Of Thomas Fritz Sent: Friday, August 26, 2011 9:52 AM To: Neo4j user discussions Subject: [Neo4j] Websocket Server instead of REST for Neo4J for access a DB Hi all! Have anyone thought, or is it possible to implement a Websocket Endpoint in Neo4J Server so one can use this instead of the RESTful API. Which would be more performant and scalable than the RESTful HTTP API. It is possible to use such a Websocket through Java, Node and any other Server Side Language and even JavaScript on the client side. What do you think? Kind regards --- Thomas FRITZ web http://fritzthomas.com twitter http://twitter.com/thomasf ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Websocket Server instead of REST for Neo4J for access a DB
On Mon, Aug 29, 2011 at 4:35 PM, Thomas Fritz fritz...@gmail.com wrote: Hi! Keep in mind. Websocket is not only for the browser! Java: http://code.google.com/p/weberknecht/ Python: http://pypi.python.org/pypi/websocket-client/0.4 .net: https://github.com/sta/websocket-sharp and JavaScript / Node: socket.io for instance Absolutely, my point is that there are many other higher level transport protocols that would be a better fit for out-of-browser environments. The only benefit, that I can see, to using websockets is easy use from the browser (I may be wrong though, I'm not super-familiar with websocket details). Since the vast majority of use case environments are outside of the browser, it feels important to choose a transport protocol that considers those as first class citizens. Don't get me wrong, I'm a huge proponent for top-notch web support as well. That's what I meant by saying we might be able to design a binary protocol that is flexible enough to fit through many different transport protocols. /Jake --- Thomas FRITZ web http://fritzthomas.com twitter http://twitter.com/thomasf 2011/8/29 Jacob Hansson jacob.hans...@neotechnology.com: In my mind websockets is not necessarily something we want to focus on initially for a binary protocol. It would only benefit in-browser javascript access to the server, while setting back every other programming language, forcing them to add extra layers there to conform to the websocket protocol. It would be interesting to take this into consideration when designing the binary protocol, however. Perhaps we can put something together that would be able to use websockets as a transport protocol at a later point in time. It would be cool to, once a certain level of authorization/authentication logic has been implemented, build high performance in-browser clients that talk directly to the server.. /Jake On Mon, Aug 29, 2011 at 12:06 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Thomas, a websocket interface would be a great addition! We haven't come around to do a proper binary protocol to Neo4j yet (pushed to Neo4j 1.6), but as Rick mentions, a websocket interface would also mean to have streaming semantics on the client. If you wanna prototype one with some trivial serialization that can be changed later, that would rock! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Mon, Aug 29, 2011 at 12:01 PM, Thomas Fritz fritz...@gmail.com wrote: Hi! I hope you had some good bear friday! ;) Do you think a (web)socket interface to neo4j could be done and integrated into the core? I do not like the idea of using a high performance graph database with fast indices (like neo4j is) and than the only interface to it is a HTTP RESTful API to it. Or are my assumptions and fears wrong? kind regards --- Thomas FRITZ web http://fritzthomas.com twitter http://twitter.com/thomasf 2011/8/26 Rick Bullotta rick.bullo...@thingworx.com: We're doing some similar things using XMPP pub-sub and BOSH... -Original Message- From: user-boun...@lists.neo4j.org [mailto: user-boun...@lists.neo4j.org] On Behalf Of Thomas Fritz Sent: Friday, August 26, 2011 10:49 AM To: Neo4j user discussions Subject: Re: [Neo4j] Websocket Server instead of REST for Neo4J for access a DB If you like partial results take a look at this paper: http://research.microsoft.com/apps/pubs/default.aspx?id=131524 and this cast: http://channel9.msdn.com/Events/MIX/MIX11/RES04 It uses rabin fingerprinting to detect changed chunks and only transfers these changed chunks (like rsync does). In this case the connecting side - the client - would also need some logic baked in because the client, after connecting to the server, has to sent the hashes of already retrieved chunks. So the server side can compute and sent just those chunks which has updated. Maybe it makes sense for some really big datasets. Kind regards and cheers from Vienna --- Thomas FRITZ web http://fritzthomas.com twitter http://twitter.com/thomasf 2011/8/26 Peter Neubauer peter.neuba...@neotechnology.com: I like that! Now some beer. Free. Friday. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http
Re: [Neo4j] Neo4jPHP batch insert benchmarks
The heap space stuff would make sense I think, because we currently deserialize and serialize in-place, keeping the whole thing in memory. Would be interesting to see if we could implement a setup that can stream the deserialization/serialization, getting rid of the memory overhead.. You said you are using out-of-the-box settings for the server, I don't remember off the top of my head what the default heap size is, but you might want to try giving it more RAM. I'm gonna guess that's where performance dies.. I'll have to look at what is proper HTTP behavior, but there should be a way we could start streaming back the response as it is being calculated, as long as we can come up with a good way of aborting if something fails.. Doing that would mean we don't have to keep a hundred thousand requests and responses in memory, which would completely change the performance situation. Big thanks for taking the time to put this together! /jake On Thu, Aug 25, 2011 at 3:52 PM, jadell josh.ad...@gmail.com wrote: Hey Peter, I don't have any way of verifying on the server side, other than measuring the time it takes for curl_exec to return a response. On the client side I can see that PHP's json_encode/json_decode functions are taking less than .5% of the total run time, even with a batch size of 1. During one of my 10 node attempts, I printed out the server response of the 500 Heap space error. It seemed like the last method in the stack trace was dealing with a Deserializer class or method. I will try again and capture the stack trace output to post here. Thanks, -- Josh Adell -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4jPHP-batch-insert-benchmarks-tp3282984p3283926.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] neo4j.py install problems
Hey Eddy, sorry about the late reply, I'm on vacation, so I've been checking email rather sporadically. I'm not sure exactly what causes your problem, since we seem to have lost the nested exception at some point (presumably a java/jpype/python thing), but generally, what you are seeing is the under-the-hood database saying I can't create a database at that location. That could be because it does not have write access to the folder you give it, or that the folder does not exist. It could also be something more complicated, but make sure that the process this app runs in has write access, and that the folder exists, first. On the subject of the new neo4j.py bindings, I am currently assigned to them, and will be working on nothing but them until we get a release of them out. I on vacation this week, and will be travelling and moving next week (I will try to get some work done though). After the dust settles, I'll push hard to get a new version out. /Jacob On Sun, Jul 24, 2011 at 4:05 AM, Eddy eddy.respon...@gmail.com wrote: Anyone? Also whats the current status of neo4j.py? I read somewhere it was being updated some time ago. -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/neo4j-py-install-problems-tp3191386p3194919.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST Batch: failed to insert empty array as a node property
On Wed, Jul 6, 2011 at 7:55 AM, Igor Dovgiy ivd.cu...@gmail.com wrote:
When trying to process POSTing to batch-path of something like...
[{id:1,
method:POST,
to:/node,
body:{user_properties:[]}
}]
...server fails with...
exception : java.lang.RuntimeException,
stacktrace : [
org.neo4j.server.rest.web.BatchOperationService.performJob(BatchOperationService.java:137),
org.neo4j.server.rest.web.BatchOperationService.performBatchOperations(BatchOperationService.java:83),
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method),
sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source),
sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source),
java.lang.reflect.Method.invoke(Unknown Source) etc.
A bug, or have I missed something in Neo4j docs?
This is actually expected behaviour. The server cannot use an empty array to
set a property, because it doesn't know what type of array it is (and the
database requires typed arrays).
It does uncover two issues though - one is that the error reporting in the
batch API does not relay the underlying errors very well at all, I'll fix
that.
The other is that I think the expected behavior here should change. Empty
arrays shouldn't be a special case, in my opinion, and it would be easy for
us to just default to an empty boolean array or something.
/jake
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Jacob Hansson
Phone: +46 (0) 763503395
Twitter: @jakewins
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST Batch: failed to insert empty array as a node property
On Mon, Jul 18, 2011 at 5:39 PM, Mattias Persson
matt...@neotechnology.comwrote:
2011/7/18 Jacob Hansson ja...@voltvoodoo.com
On Wed, Jul 6, 2011 at 7:55 AM, Igor Dovgiy ivd.cu...@gmail.com wrote:
When trying to process POSTing to batch-path of something like...
[{id:1,
method:POST,
to:/node,
body:{user_properties:[]}
}]
...server fails with...
exception : java.lang.RuntimeException,
stacktrace : [
org.neo4j.server.rest.web.BatchOperationService.performJob(BatchOperationService.java:137),
org.neo4j.server.rest.web.BatchOperationService.performBatchOperations(BatchOperationService.java:83),
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method),
sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source),
sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source),
java.lang.reflect.Method.invoke(Unknown Source) etc.
A bug, or have I missed something in Neo4j docs?
This is actually expected behaviour. The server cannot use an empty array
to
set a property, because it doesn't know what type of array it is (and the
database requires typed arrays).
It does uncover two issues though - one is that the error reporting in
the
batch API does not relay the underlying errors very well at all, I'll fix
that.
The other is that I think the expected behavior here should change. Empty
arrays shouldn't be a special case, in my opinion, and it would be easy
for
us to just default to an empty boolean array or something.
You mean storing an empty boolean[] on the entity? It could possibly work
in a REST environment, but not embedded where if you added an empty int[]
you'd get back a boolean[] all of a sudden.
Oh snap, my head was all full of REST, you are absolutely right.
/jake
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Jacob Hansson
Phone: +46 (0) 763503395
Twitter: @jakewins
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Jacob Hansson
Phone: +46 (0) 763503395
Twitter: @jakewins
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Example of using WrappingNeoServerBootstrapper
The only dependency for that is the neo4j-server maven package. I am not sure, however, if that is a public maven artifact. If you can't find it, you'll have to build it locally for now. For a usage example, see this test: https://github.com/neo4j/community/blob/master/server/src/functionaltest/java/org/neo4j/server/WrappingNeoServerBootstrapperTest.java#L57 On Wed, Jul 13, 2011 at 9:42 AM, Michael Kilgore mfkilg...@infoclearonline.com wrote: In the documentation there is a short section about using WrappingNeoServerBootstrapper to provide server services (rest, stats etc) to an embedded database. Is there any example code where this is being used? Which jars files (maven dependencies) are required. I have an application (maven built) with an embedded database in place and would like to expose it to the server tools. Thanks in advance, Michael Kilgore ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] 1.4 Console Cypher dose not work, but Cypher Plugin works fine!
On Wed, Jul 13, 2011 at 9:35 AM, Zhiguo Zhang mikewolfx...@gmail.comwrote: OK, that is my fault. I read the tiny note again, and find you are totally right! :) Sorry about that, it's in the pipeline to make it more obvious, but I haven't gotten around to it yet. On Wed, Jul 13, 2011 at 4:31 PM, Michael Hunger michael.hun...@neotechnology.com wrote: Mike, how do you constitute that the cypher console does not work? Please note that you have to hit return twice as this console accepts multiline input. Cheers Michael Am 13.07.2011 um 16:22 schrieb Zhiguo Zhang: hi, maybe there is only few people use the console with the cypher language, but it does not work. So I install the cypher plugin for the rest api, with the rest api, the cypher works fine. funy, is it not? MfG Mike ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Constraints on index key names?
I'm not sure if there is anything keeping you from using them, but the names 'node_auto_index' and 'relationship_auto_index' are used by the auto indexing subsystem. On Wed, Jul 13, 2011 at 8:35 AM, Mattias Persson matt...@neotechnology.comwrote: Yes, the neo4j integration doesn't impose any additional restrictions other than those (if any) of Lucene. 2011/7/12 Rick Bullotta rick.bullo...@thingworx.com That would conflict with lucene's query syntax, I'd think. From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Aseem Kishore [aseem.kish...@gmail.com] Sent: Monday, July 11, 2011 7:41 PM To: Neo4j user discussions Subject: [Neo4j] Constraints on index key names? Is there any constraint on what an index key is allowed to be named? E.g. can it have a '+' in its name? Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Get all nodes and relationships in REST
Here is an example of how such a plugin would look: http://components.neo4j.org/neo4j-server-examples/snapshot/xref/org/neo4j/examples/server/plugins/GetAll.html On Tue, Jul 12, 2011 at 3:01 PM, Jim Webber j...@neotechnology.com wrote: Hey Javier, And this should work even if my graph has several unconnected componentes? Oh, no it won't. Then I'd suggest writing a managed/unmanaged plugin that calls getAllNodes() on the server side. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] auto indexes in REST in 1.4?
It is currently possible to configure auto indexing for the server using the neo4j.properties configuration file. Something like this should work: node_auto_indexing=true node_keys_indexable=name, age relationship_auto_indexing=true relationship_keys_indexable=strength, someotherkey Programatically setting up auto indexing via the REST API is currently not supported, because the modifying auto index settings during runtime does currently not persist across database restarts. Using the auto indexes programatically via REST would thus mean that clients would have to monitor when the server shuts down, and recreate runtime settings upon startup. Once we have programmatic auto index settings set up to where they persist across database shutdowns, these features will be added to the REST API. I am just now finishing support for querying auto indexes via REST, and it will, if it passes QA, be available in the next milestone release. It will also most likely be available in the latest SNAPSHOT release by tonight. Until such a version is available, it is possible to query auto indexes by name. Please note, however, that this is *not* an official API feature, we do not make any guarantees that this will be supported in subsequent versions. To query the node auto index, simply issue a normal index query to the node index named node_auto_index or, for relationships, to the relationship index named relationship_auto_index. /Jake On Tue, Jul 12, 2011 at 3:32 PM, Matt Luongo m...@scholr.ly wrote: +1, we could use configuration through REST as well. -- Matt Luongo Co-Founder, Scholr.ly On Tue, Jul 12, 2011 at 6:45 AM, mike_t t.pr...@gmx.net wrote: Please tell me, how can I do that? I couldn´t find anything about it in the api doc (http://components.neo4j.org/neo4j-server/snapshot/rest.html) -- View this message in context: http://neo4j-user-list.438527.n3.nabble.com/Neo4j-auto-indexes-in-REST-in-1-4-tp3159205p3162053.html Sent from the Neo4J User List mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] The URL for batching operations through REST
Hey, that was a miss on my part, thanks for letting us know. I just pushed a fix for this :) /Jake On Mon, Jul 11, 2011 at 9:21 AM, Javier de la Rosa ver...@gmail.com wrote: I'm just wondering why the URL http://localhost:7474/db/data/batch; is not returned with the initial request on root http://localhost:7474/db/data; like the rest of the URL's. Regards. -- Javier de la Rosa http://versae.es ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Webadmin browser dependencies?
Excellent. I've got a fix for the problem with the visualization, just need to slot in the time to write and test it :) On Sun, Jun 26, 2011 at 6:48 AM, Tatham Oddie tat...@oddie.com.au wrote: Sorry, I should have clarified that it's just the visualization that doesn't work. Everything else is perfect. -- Tatham -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Jacob Hansson Sent: Saturday, 25 June 2011 3:25 AM To: Neo4j user discussions Subject: Re: [Neo4j] Webadmin browser dependencies? Is that the whole data browser, or is it just the visualization? In IE8, everything appears to work except two things, history support (clicking back doesn't change the url-hash), and the visualization. I'm gonna get a hold of a Windows 7-copy as well so that I can test this in IE9. /Jake On Thu, Jun 23, 2011 at 9:35 PM, Tatham Oddie tat...@oddie.com.au wrote: Everything except data browser works fine for me in IE9. -- Tatham -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Paul Bandler Sent: Friday, 24 June 2011 1:03 AM To: Neo4j user discussions Cc: Neo4j user discussions Subject: Re: [Neo4j] Webadmin browser dependencies? Thanks but none of those browsers are allowed to be downloaded within my clients corporate network... Again, does anyone know what IE level works? Sent from my iPhone On 23 Jun 2011, at 13:53, Tatham Oddie tat...@oddie.com.au wrote: Data browser works in Chome / Firefox / Safari. It should give you a message to this effect on unsupported browsers. -- Tatham -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Paul Bandler Sent: Thursday, 23 June 2011 10:30 PM To: Neo4j user discussions Subject: [Neo4j] Webadmin browser dependencies? Unable to access the data tab using IE version 7. Tried using the Eclipse built in browser and it momentarily switches to the data tab but then it disappears again. What are the supported browser platforms? Sent from my iPhone ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Webadmin browser dependencies?
Is that the whole data browser, or is it just the visualization? In IE8, everything appears to work except two things, history support (clicking back doesn't change the url-hash), and the visualization. I'm gonna get a hold of a Windows 7-copy as well so that I can test this in IE9. /Jake On Thu, Jun 23, 2011 at 9:35 PM, Tatham Oddie tat...@oddie.com.au wrote: Everything except data browser works fine for me in IE9. -- Tatham -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Paul Bandler Sent: Friday, 24 June 2011 1:03 AM To: Neo4j user discussions Cc: Neo4j user discussions Subject: Re: [Neo4j] Webadmin browser dependencies? Thanks but none of those browsers are allowed to be downloaded within my clients corporate network... Again, does anyone know what IE level works? Sent from my iPhone On 23 Jun 2011, at 13:53, Tatham Oddie tat...@oddie.com.au wrote: Data browser works in Chome / Firefox / Safari. It should give you a message to this effect on unsupported browsers. -- Tatham -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Paul Bandler Sent: Thursday, 23 June 2011 10:30 PM To: Neo4j user discussions Subject: [Neo4j] Webadmin browser dependencies? Unable to access the data tab using IE version 7. Tried using the Eclipse built in browser and it momentarily switches to the data tab but then it disappears again. What are the supported browser platforms? Sent from my iPhone ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] TraversalDescription order
The order method indeed does decide what order the graph is traversed, you can either use it with the predefined traversal orders like so: td.order(CommonBranchOrdering.PREORDER_DEPTH_FIRST) or you can pass it your own branch selector implementation, see https://github.com/neo4j/community/blob/master/kernel/src/main/java/org/neo4j/kernel/PreorderDepthFirstSelector.javafor an example. An easier way to include weights would be to make the traversal only follow LIKES relationships that are over a certain weight threshold. You can also process the weights outside of the traversal entirely. For instance, you can calculate the average weight for each path returned by the traversal, sort the result by that, and only return the top results. /jake On Thu, Jun 23, 2011 at 8:30 AM, Igor Dovgiy ivd.cu...@gmail.com wrote: Hi all, I'm trying to implement a recommendations system app with Neo4j (1.4). Basically, I intend to have some 'User' nodes and some 'Interest' nodes in my graph. Then, to find users with similar interests, traverse it by something like... TraversalDescription td = new Traversal.description() .relationships(RelTypes.LIKES) .evaluator(Evaluators.atDepth(2)); The real question is, though, whether I'll be able to choose what Interest nodes to walk through first, depending on their weights (i.e., number of users linked to them)? As I understand, the order() method is designed right for this case, but sadly I was unable to find examples of its usage... Any help will be greatly appreciated! -- iD ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Webadmin browser dependencies?
Hey all, Our automatic tests do currently not run inside Internet Explorer (due to issues getting the webdriver IE driver to deliver repeatable results). It is, however, our goal to ensure support for all features in IE8 and upwards (currently, data visualization should be the only feature that is not available) and testing manually in that browser is part of our release process. We may have missed testing the data browser for the version of neo4j you are using (which version are you using?), or there may be issues that show up in IE7 that do not show up in IE8. I've recently reinstalled my laptop, but I'll be installing a windows VM today to see if I can hunt this down, sorry for the inconvenience. /jake On Thu, Jun 23, 2011 at 12:34 PM, Michael Hunger michael.hun...@neotechnology.com wrote: Which parts of webadmin are absolutely critical? What about neoclipse readonly? Sent from my iBrick4 Am 23.06.2011 um 19:00 schrieb Rick Bullotta rick.bullo...@thingworx.com : Welcome to my world, Paul... Web applications that don't recognize the realities of the enterprise desktop won't ultimately be able to leverage a given technology if it doesn't fit with their corporate standards. As painful as it may be at times, broad-based browser support is a must-have for applications that want to make a big push into the enterprise (behind the firewall) world. My experience, however, is that the corporate standards can usually be bypassed by developers/admins but not by the general user. Exceptions tend to be in defense, public safety/security or other highly sensitive industries. Perhaps your client would agree to violate the standard for admins? -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Paul Bandler Sent: Thursday, June 23, 2011 11:03 AM To: Neo4j user discussions Cc: Neo4j user discussions Subject: Re: [Neo4j] Webadmin browser dependencies? Thanks but none of those browsers are allowed to be downloaded within my clients corporate network... Again, does anyone know what IE level works? Sent from my iPhone On 23 Jun 2011, at 13:53, Tatham Oddie tat...@oddie.com.au wrote: Data browser works in Chome / Firefox / Safari. It should give you a message to this effect on unsupported browsers. -- Tatham -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Paul Bandler Sent: Thursday, 23 June 2011 10:30 PM To: Neo4j user discussions Subject: [Neo4j] Webadmin browser dependencies? Unable to access the data tab using IE version 7. Tried using the Eclipse built in browser and it momentarily switches to the data tab but then it disappears again. What are the supported browser platforms? Sent from my iPhone ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cannot execute Cypher
On Tue, Jun 14, 2011 at 9:11 AM, Andres Taylor andres.tay...@neotechnology.com wrote: Hi Pierre, On Tue, Jun 14, 2011 at 1:48 PM, Pierre De Wilde pierredewi...@gmail.com wrote: start n=(1) return n Nothing is returned... Could you look in the logs for any clues? They are in neo4j install dir/data/log This will most likely reveal the source of the problem. You could also install the firebug plugin for firefox, and use it to see what the raw response from the server is. Andrés ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Embedded with webadmin
Adriano: The fact that this code is not QA'd and isn't officially in the M03 release will mean that we do not guarantee the API won't change, and that there may be undiscovered problems with it. That said, if you want to use it, take a look at: https://github.com/neo4j/community/blob/master/server/src/main/java/org/neo4j/server/WrappingNeoServerBootstrapper.java Which should be available in the M03 release due later today. /Jake On Fri, May 27, 2011 at 9:55 AM, Jim Webber j...@neotechnology.com wrote: If you use it in the M03 release, you're on your own :-) Jim On 27 May 2011, at 00:04, Adriano Henrique de Almeida wrote: Wow... very good to know that the code exists! 2011/5/26 Jim Webber j...@neotechnology.com This code exists, but isn't yet QA'd or documented. It will appear in the M03 release, but won't be QA'd and documented until the M04 release. Jim On 26 May 2011, at 18:51, Peter Neubauer wrote: Yeah. just saw today some really cool stuff from Jacob in this direction. Stay tuned - it's lab day tomorrow :) Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, May 25, 2011 at 9:32 AM, Craig Taverner cr...@amanzi.com wrote: While HA is one option, with two processes 'sharing' a database, one being the server and the other the embedded app, there is another option, and that is to integrate the two apps. If your app is a web-app and also needs to exist in something like jetty or winstone, perhaps you could run both the server and your app together in the same process? One obvious way of doing this is to write your app as a server extension within the neo4j-server extensions API. I suspect there are other ways to do this where your app is in control and simply accesses (and starts) the relevant code from neo4j-server, but I don't know how to do that. Could be interesting to find out. On Tue, May 24, 2011 at 11:39 PM, Adriano Henrique de Almeida adrianoalmei...@gmail.com wrote: Yep, the neo4jserver is just a rest api over neo4j database, so it's still stored in at the disk. So, all you need to do, is to point your java application to the neo4j db directory. Remember, that you'll be unable to start both you app and the neo4j server at the same time, at the same database. For this situation, you'll need Neo4j HA. Regards 2011/5/24 Chris Baranowski pharcos...@gmail.com Hi all, I searched this mailing list some but couldn't find a definitive answer: is it possible to use the web admin with an embedded neo4j database? I'd like to run embedded in my project and also be able to administrate online. Thanks! Chris ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Adriano Almeida Caelum | Ensino e Inovação www.caelum.com.br ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Adriano Almeida Caelum | Ensino e Inovação www.caelum.com.br ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST results pagination
: 6dQBm9oYA1mxRSH0e96l5W
Last-Modified: Wed, 10 Mar 2010 18:11:41 GMT
{foo:bar}
--RJKFlAs9PrdBNfd74HANycvbA8C--
--JZi8W8pB0Z3nO3odw11GUB4LQCN--
* Connection #0 to host 127.0.0.1 left intact
* Closing connection #0
Riak - MapReduce:
Optional query parameters:
* chunked – when set to true, results will be returned one at a time in
multipart/mixed format using chunked-encoding.
Important headers:
* Content-Type – application/json when chunked is not true,
otherwise multipart/mixed with application/json parts
Other interesting endpoints: /ping, /stats
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Jacob Hansson
Phone: +46 (0) 763503395
Twitter: @jakewins
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST results pagination
On Tue, Apr 19, 2011 at 10:17 PM, Michael DeHaan michael.deh...@gmail.comwrote: On Tue, Apr 19, 2011 at 10:58 AM, Jim Webber j...@neotechnology.com wrote: I'd like to propose that we put this functionality into the plugin ( https://github.com/skanjila/gremlin-translation-plugin) that Peter and I are currently working on, thoughts? I'm thinking that, if we do it, it should be handled through content negotiation. That is if you ask for application/atom then you get paged lists of results. I don't necessarily think that's a plugin, it's more likely part of the representation logic in server itself. This is something I've been wondering about as I may have the need to feed very large graphs into the system and am wondering how the REST API will hold up compared to the native interface. What happens if the result of an index query (or traversal, whatever) legitimately needs to return 100k results? Wouldn't that be a bit large for one request? If anything, it's a lot of JSON to decode at once. Yeah, we can't do this right now, and implementing it is harder than it seems at first glance, since we first need to implement sorting of results, otherwise the paged result will be useless. Like Jim said though, this is another one of those *must be done* features. Feeds make sense for things that are feed-like, but do atom feeds really make sense for results of very dynamic queries that don't get subscribed to? Or, related question, is there a point where the result sets of operations get so large that things start to break down? What do people find this to generally be? I'm sure there are some awesome content types out there that we can look at that will fit our uses, I don't feel confident to say if Atom is a good choice, I've never worked with it.. The point where this breaks down I'm gonna guess is in server-side serialization, because we currently don't stream the serialized data, but build it up in memory and ship it off when it's done. I'd say you'll run out of memory after 1 nodes or so on a small server, which I think underlines how important this is to fix. Maybe it's not an issue, but pointers to any problems REST API usage has with large data sets (and solutions?) would be welcome. Not aware of anyone bumping into these limits yet, but I'm sure we'll start hearing about it.. The only current solution I can think of is a server plugin that emulates this, but it would have to sort the result, and I'm afraid that it will be hard (probably not impossible, but hard) to implement that in a memory-efficient way that far away from the kernel. You may just end up moving the OutOfMemeoryExceptions' to the plugin instead of the serialization system. --Michael ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST API thoughts/questions/feedback
trying to make sure graph elements are unique for some criteria, and
this requires that I make more API calls than normal, and have to
implement this in my library and not in the server -- which could be
fragile and certaintly isn't atomic.
I've also noticed some minor things, which were slight stumbling blocks:
* It seems that while the application says it takes JSON, it will
actually accept things as a key/value pair form submission, and may
prefer it that way. This could be my code though and I need to debug
this further.
Yeah, some of the rest resources speak form submissions as well as JSON. I
think some of them also return HTML if you send a properly formatted
request. It was meant to allow having a HTML interface to the data. I'm not
sure if the HTML interface is still a supported output format..
* At one point in the API docs it suggests POSTing a hash as { key,
value }. In JSON, this should be { key : value }.
Could you point to where in the docs this was? I'll fix it :)
* Some API documentation online refers to the default port being
and didn't mention the /db/... prefix to the URLs.
Same as above :)
* While I understand proper REST is politically correct, I'd be
really happy with simple endpoints that always could take POSTs, or
the ability to always do a post. Makes calling code simpler.
Do you mean that in places where we do PUT to update some property, we
should also allow POST? If so, I'm afraid I disagree. I've never experienced
any trouble switching between HTTP verbs in client code, and I think it's a
really good thing to keep a consistent model for how the API eats HTTP
verbs.
* In the documentation, it was unclear whether my_nodes really
needed to be my_nodes or was some sort of namespace that I could or
should use. Is there a way to keep graphs in different namespaces?
my_nodes is just a name for an index, or namespace if you will.
Think of the indexes as being completely separate from the neo4j database.
The database stores a graph structure and is able to to complex traversals
in that. The indexes (as many as you have created) live outside the
database, but keep pointers to nodes and relationships inside the
database.
When you index a node, you basically tell the index that when someone
searches for this, hand me back this node.
In all, it's actually looking pretty good, though knowing what this
object key is in advance, and having a way to avoid duplicate objects
would help tremendously. I like the idea that the URLs come back
when adding objects, in particular, as it helps make the REST APIs to
call about a particular node more self documenting.
I'd be happy to try to explain further if any of that didn't make
sense -- particularly I'd be very interested in how to specify a key
for an element in advance, so I didn't have to rely on lookups each
time I need the node ID. Since the lookup can return a list, it
doesn't guarantee I can get back a specific node.
It would be interesting to hear what some of the more in-depth developers
have to say about this. I can see that there is a benefit to somehow adding
an indexed based key/value-style abstraction that goes beyond what we have
today, but everything comes with a cost. I can see it being very confusing
to newcomers if we have all these different ways to index and fetch nodes.
Thanks!
--Michael DeHaan
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Jacob Hansson
Phone: +46 (0) 763503395
Twitter: @jakewins
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] [neo4j] Exposing Webadmin
Georg, Like you noticed and like Jim mentioned, the server will pick up on the domain you are using, but the actual URI structure is fixed. You are supposed to be able to change the mount points for the various server components, by setting these properties in your server config: org.neo4j.server.webadmin.data.uri=/db/data/ org.neo4j.server.webadmin.management.uri=/db/manage/ But I just tried it out, and there seems to be some sort of problem with it. I am investigating it, will commit a patch to fix whatever might be wrong. That said, I would recommend you not to mix the URIs of multiple applications this way. Although it is for sure possible and not necessarily a bad idea, unless there is some special reason for doing it, it is a lot less painful to use subdomains. That way you separate your infrastructure, making it a breeze to do things like fanning out your applications onto multiple servers, or adding various supporting HTTP layers and so on in the future. Cheers! Jacob On Thu, Apr 14, 2011 at 4:06 PM, Georg Summer georg.sum...@gmail.comwrote: so in theory if I changed the proxy settings to: ProxyPass / http://ip:7474 ProxyPassReverse / http://ip:7474 and reverse the neo4j-server.properties to the original state should work out. (To answer that myself: yes it does) Any idea how I could configure these proxy settings in a nicer way so that I am still able to access the other sides on the apache? e.g. phpmyadmin On a side node: I am doing this dance because I like the Browser and it would be perfect for a specific part of my concept. I do not know the complete structure, but in theory it should be possible to rib out the html/java script of the browser and just dump them in /var/www/neo_browser? Georg On 14 April 2011 14:56, Jim Webber j...@neotechnology.com wrote: Hi guys, Jim, what is your RESTy expertise on this? Our JAX-RS plumbing will dispatch on very specific parts of the URI (that's just the way JAX-RS works). So if ultimately you send the server something that doesn't match, it's either going to cause a 404 or similar, or where we think it's sensible it'll cause a redirect to webadmin. So the only solution is to preserve the URI structure on the way into the Noe4j server, and (potentially) have it mapped to something else on the network where your client lives. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] [neo4j] Exposing Webadmin
Looked over the configuration properties I mentioned, and it seems I was incorrect, am unable to reproduce the problems I saw before. Doing something like: org.neo4j.server.webadmin.management.uri=/asd/ Will make the server move management stuff to that URI, and webadmin will follow suit. So for your case, you should be able to do: org.neo4j.server.webadmin.data.uri=/neo4jdb/db/data/ org.neo4j.server.webadmin.management.uri=/neo4jdb/db/manage/ I still do not recommend doing this :) There are plans for features in the next release (1.4) that would mean we need to make backwards-incompatible changes to these configuration directives that will break this setup. :( /Jacob On Mon, Apr 18, 2011 at 11:24 AM, Jacob Hansson ja...@voltvoodoo.comwrote: Georg, Like you noticed and like Jim mentioned, the server will pick up on the domain you are using, but the actual URI structure is fixed. You are supposed to be able to change the mount points for the various server components, by setting these properties in your server config: org.neo4j.server.webadmin.data.uri=/db/data/ org.neo4j.server.webadmin.management.uri=/db/manage/ But I just tried it out, and there seems to be some sort of problem with it. I am investigating it, will commit a patch to fix whatever might be wrong. That said, I would recommend you not to mix the URIs of multiple applications this way. Although it is for sure possible and not necessarily a bad idea, unless there is some special reason for doing it, it is a lot less painful to use subdomains. That way you separate your infrastructure, making it a breeze to do things like fanning out your applications onto multiple servers, or adding various supporting HTTP layers and so on in the future. Cheers! Jacob On Thu, Apr 14, 2011 at 4:06 PM, Georg Summer georg.sum...@gmail.comwrote: so in theory if I changed the proxy settings to: ProxyPass / http://ip:7474 ProxyPassReverse / http://ip:7474 and reverse the neo4j-server.properties to the original state should work out. (To answer that myself: yes it does) Any idea how I could configure these proxy settings in a nicer way so that I am still able to access the other sides on the apache? e.g. phpmyadmin On a side node: I am doing this dance because I like the Browser and it would be perfect for a specific part of my concept. I do not know the complete structure, but in theory it should be possible to rib out the html/java script of the browser and just dump them in /var/www/neo_browser? Georg On 14 April 2011 14:56, Jim Webber j...@neotechnology.com wrote: Hi guys, Jim, what is your RESTy expertise on this? Our JAX-RS plumbing will dispatch on very specific parts of the URI (that's just the way JAX-RS works). So if ultimately you send the server something that doesn't match, it's either going to cause a 404 or similar, or where we think it's sensible it'll cause a redirect to webadmin. So the only solution is to preserve the URI structure on the way into the Noe4j server, and (potentially) have it mapped to something else on the network where your client lives. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Graph visualization in the web admin tool
We opted to change from graphdracula to arbor.js though (today, actually), because the dracula lib was buggy. My initial reaction to arbor.js is very positive, although it wasn't quite as straight-forward as dracula to use. I'd recommend downloading arbor.js and tweaking their example apps to get a feel for how the lib works. It's been fairly unpainful to work this into the webadmin app, together with neo4js you can rather quickly get up to speed visualizing data from a neo4j server on the web :) /Jacob On Mon, Mar 28, 2011 at 3:50 PM, Clement Honore honor...@gmail.com wrote: Thanks for your answer Peter. I will take a look at dracula. I've already check this link http://wiki.neo4j.org/content/Visualization_options_for_graphs# (this is the on this page on my first post but I forget the link... ^^) and there is so many solutions that I don't know which to choose... 2011/3/28 Peter Neubauer peter.neuba...@neotechnology.com Clement, the current iteration is building on http://www.graphdracula.net/ which is built on top of RaphaelJS I think. There are many others, look at http://wiki.neo4j.org/content/Visualization_options_for_graphs# and pick your choice. Let us know your experiences and feel free to contribute to the webadmin! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Mon, Mar 28, 2011 at 1:54 PM, Clement Honore honor...@gmail.com wrote: Hi! Since the 1.0 M5 release, it's possible to get a visual representation of the graph within the web administration tool. I've almost the same kind of representation to do in a web application and I wonder which library did you use. Is it something referenced on this page or something completely written by yourselves ? Thanks for the tips! Clément. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Webadmin improvements
Hey all! Just wanted to update you on some changes being made to the web admin tool. The last two weeks or so, webadmin development has lived in a nice, quiet branch, but it is now merged back into the server master. *First, a TL;DR:* I've made a bunch of big changes to webadmin, it is now more awesome, but there are probably lots of bugs, please give feedback :) *And then, giant detailed email:* *The merge means some good technical things have happened:* - We now use Backbone.js, which has lead to much better code structure ( http://documentcloud.github.com/backbone/) - All webadmin code is written in Coffeescript, which looks like a blend of Python, Ruby and JS. This has really sped up development, and I hope this will make it easier for non-JS fans to interact with the webadmin code base. (http://jashkenas.github.com/coffee-script/) - We use require.js to define dependencies between coffeescript files. This gets rid off a ton of stuff in the server POM, makes debugging easier, and allows us to use require.js tools to aggregate and optimize both JS and CSS for production packaging. This will also make it easy to dynamically load JS plugins into webadmin if we want that functionality later on. - All templates are written in HAML, which is a major improvement to the JST templates we used before. - Based on Andreas work on the web-visualization project, the webadmin integration tests can now be triggered to run in both IE, FF and Chrome. They pass in FF, there seems to be bugs/inconsistencies in how webdriver handles keyboard emulation in chrome, and I haven't had time to run the tests in IE (they won't pass, see further down). - All webdriver tests are written as Cucumber tests, allowing us to write easy-to-read tests that can be direct translations of features and related acceptance criteria from our user stories. See https://github.com/neo4j/server/blob/master/server/features/webadmin/databrowser-node.featurefor an example. * It also means some new features are now in place:* - A new and more polished UI - A complete rewrite of the data browser, hopefully making it easier, more straight forward and faster to use - The new data browser features both a tabular and a visualized view of the data. The visualized view is so far only a two-day hack to see that it works, but it will get lots of caressing and love before the 1.3 release. - Our first keyboard shortcut, and a simple framework to cleanly put more of them in place. Press s to get to the databrowser search bar from any page. *And there are a few things I need to work on before the 1.3 release*: - The webdriver tests are not currently run by teamcity. Andrés/Thomas: To have it run them, change the mvn call for the webdriver step to be mvn integration-test -Dtests=web. Note that b/c of the cucumber stuff, running them has the same requirements as running the server-QA branch. - IE currently does not work. I've resolved some issues, but have a few related to plotting and searching in the data browser left. Oh, and it looks like absolute bollocks and vomits exceptions. Will adress ASAP. - The build pipeline for webadmin is not included in the server pom. Yeah. Don't panic, doing mvn package will produce a server artifact with a working webadmin. What I mean is that the build system does not currently propagate changes in the coffeescript or HAML files (under src/main/coffeescript) into compiled js files in the webadmin-html directory. I wrote a mashup maven plugin that does exactly that ( https://github.com/jakewins/brew), but because I got sidetracked and because the general consensus appears to be maven will be replaced soon, I never put the effort in to get the plugin to maven central. Thus, currently and reluctantly, the toolchain for working with webadmin is disconnected from the normal build system. I'm really sorry about that, this will be my first priority to fix. If anyone is wanting to do any work that would require this toolchain before this is resolved, drop me an email and I'll explain how to set it up. Phew. Sorry about the giant email. I'm off for a week in Austria in a few hours, before the last snow melts, will be back to coding Monday evening the 28'th and will make the issues listed above, together with visualization, my top priorities. I would really love to hear your comments and see bug reports, so that we can deliver a seriously awesome, beautiful and stable web admin tool in the 1.3 release. Cheers! Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] [Neo tickets] #323: Dashboard charts show wrong number of primitives after 1.3 migration
Holy crap :) Was able to reproduce this. The problem appears to be that we've upgraded the version of the round robin database that we use to store server statistics. The new RRDB finds the old 1.2 data file (it's [server]/data/graph.db/rrd) and happily parses it, but interprets it in crazy ways. As a direct fix for this, turn off your server and then backup and delete the rrd file. The server will create a new one upon startup with the correct format. I'll have to look into what we can do to mitigate this - we'll make sure to document it at the least, but ultimately this upgrade should be possible without having to lose all previous monitoring data.. Thank you for taking the time to bring it to our attention! /Jake On Wed, Mar 16, 2011 at 5:13 PM, neo4j.org nore...@neo4j.org wrote: #323: Dashboard charts show wrong number of primitives after 1.3 migration -+-- Reporter: bnor...@drw.com | Owner: peter Type: bug report | Status: new Priority: major| Milestone: Component: server |Keywords: 1.3.M04 -+-- I upgraded a 1.2 graph to 1.3.M04 using the special upgrade instructions at http://docs.neo4j.org/chunked/milestone/deployment-upgrading.html . I have 4858 nodes in this graph, but the chart is using a scale of 10^307 for the y-axis. The attached screenshot will illustrate the entertainment value therein. ;-) -- Ticket URL: https://trac.neo4j.org/ticket/323 neo4j.org http://trac.neo4j.org/ The Neo4J.org Issue Tracker ___ Tickets mailing list tick...@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/tickets -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Using Data tab of Web-Admin to Delete Relationships
Hey Neal, I just looked into this, and you are absolutely right. The data browser uses an older method of removing relationships that is no longer supported by the underlying javascript neo4j client we use, and there was no test in place to ensure this feature works. I've fixed this, and written a test to make sure it doesn't happen again. I'll push this in to be available in the 1.3 M04 release that will be released later today. Thank you for bringing it to my attention! /Jacob On Thu, Mar 10, 2011 at 5:57 AM, Neal Walters nealwalt...@nealwalters.comwrote: I'm using the the Data tab of the Web-Admin (V1.3.M03) to model my data structure and learn Neo4j. I accidentally created a wrong relationship. I've selected that relationship (from both the parent side and child side/direction), and tried to remove it. No error appears, but it doesn't seem to remove it. (I press the back key and refresh, and it's still there.) Is there something I'm missing? Thanks, Neal Walters ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] web admin module in embedded graph
John, One approach that would be possible is to wait (or to write a patch) for the server to support a read-only database. That way, you could start the server, pointing at the same data directory as your app uses, and get webadmin for free (plus, you'd get the REST API for free). The drawback would be that only read-operations would work - webadmin would not be able to modify any data. If you need write support, webadmin will need the entire REST API, including the ability to discover where those APIs are by a HTTP GET request to the root of the server that serves webadmin. You might be able to add a dependency to the server component, bypass its normal startup, and instead inject your own database into it, along with configuration and other things normally provided by the happy events originating in org.neo4j.server.BootStrapper. You would then mount the servers jersey services somewhere in your web container, there are three packages of them that need to be mounted at different places: At / (the only url that is hard-coded) you need: org.neo4j.server.rest.discovery.* Since you are already serving web content from there, you could provide the behaviour of this in some other manner. Basically, a HTTP GET to /, with an Accept-header of application/json needs in response a JSON-object that explains where the data and the management APIs are mounted. Please refer to the discovery service and it's related response representation for how this should be formatted. The other two can then be mounted wherever you like, as long as you put the urls to them in the GET / response. They are as follows: Mounted at /db/data in server: org.neo4j.server.rest.web.* Mounted at /db/manage in server: org.neo4j.server.webadmin.rest.* When these are in place you need to serve the static webadmin stuff from somewhere, somewhere being anywhere with the same base url as the services you just mounted. These static files are available as part of the server component, in a JAR artifact with the classifier static-web. /Jacob On Wed, Feb 9, 2011 at 8:33 AM, John Howard johnyho...@gmail.com wrote: Thanks Jacob. What we have currently is a web application which has embedded neo graph and deployed on an app server. The embedded neo approach fits nicely in our application. We are open to any development effort it may involve. It would be highly helpful, if you can point to starting steps in getting web admin in our set up. On Sun, Feb 6, 2011 at 8:12 AM, Jacob Hansson ja...@voltvoodoo.com wrote: The web admin client is a javascript file, some html, some css and some images. If you serve those files from a server that also implements the neo4j REST API, and the neo4j server management REST API, webadmin would work. Needless to say, implementing those APIs (or using the neo4j server implementation of them in some way) would be very cumbersome. Perhaps an easier solution would be to wait for (or start working on :) ) a java client for the server that implements the GraphDatabaseService interface. As long as your app uses that, the switch from embedded to server should be fairly painless.. /Jacob On Fri, Feb 4, 2011 at 10:12 PM, John Howard johnyho...@gmail.com wrote: Hello, I like the web admin module in the neo restful server. Do we (or can we) have the same as some sort of war package or files which can be embedded in our embedded application? I do not mean just jmx but also graph specific info present in web admin page in the neo restful server. Thank you in advance. Honnur ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 +46763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] web admin module in embedded graph
The web admin client is a javascript file, some html, some css and some images. If you serve those files from a server that also implements the neo4j REST API, and the neo4j server management REST API, webadmin would work. Needless to say, implementing those APIs (or using the neo4j server implementation of them in some way) would be very cumbersome. Perhaps an easier solution would be to wait for (or start working on :) ) a java client for the server that implements the GraphDatabaseService interface. As long as your app uses that, the switch from embedded to server should be fairly painless.. /Jacob On Fri, Feb 4, 2011 at 10:12 PM, John Howard johnyho...@gmail.com wrote: Hello, I like the web admin module in the neo restful server. Do we (or can we) have the same as some sort of war package or files which can be embedded in our embedded application? I do not mean just jmx but also graph specific info present in web admin page in the neo restful server. Thank you in advance. Honnur ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j web admin connection error
___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] HTML5 graph viz library
Sorry Peter, misread you. What I was thinking was to render non-spatial graphs with neo4j spatial. A layout algorithm would calculate the coordinates of each node we want to visualize. That way we can view really big graphs in the browser, since the layout-work is already done by the server. Den 15 jan 2011 14.28 skrev Jacob Hansson ja...@voltvoodoo.com: Yeah, you couldn't do very much with the markers. But you could pre-render millions of nodes on the server, and serve it as a spatial layer as if it was a map. Geoserver would slice it up, allowing zooming and panning just like a map. Then you would add interactivity on top of the rendered image like google does with google maps. It would be difficult to allow moving nodes and so on, but clicking on them and adding relations etc would not be that difficult.. Especially cool if coupled with the styling available with the current geo stack. Imagine for instance visualizing the corporation ownership and board members dataset, enabling cheap and super-easy access to the entire visualized graph in any browser.. :) ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] HTML5 graph viz library
Craig: the example domain i mentioned is a dataset of all Corporations in Sweden, with ownerships and board members. I only ever used a subset, so i don't know the primitives count of it, perhaps few million in total, probably more a bit more. Rick: The open layers project uses tiled png images, with content pre-rendered on the server. Doing that makes it harder to check for mouse actions on nodes like you say, but there is virtually no limit to how much data we can show without regard for client hardware. I think they use svg for rendering dynamic items on top if the png layer(s). Den 15 jan 2011 17.44 skrev Rick Bullotta rick.bullo...@burningskysoftware.com: Out of curiosity, has anyone looked at re-doing the rendering engine of this in SVG? Now that SVG will be officially supported in IE9, its practicality for web apps is much more substantial. Issues like hit testing/item addressability/animation, conditional display based on zoom level, and other characteristics of smart graphs such as these are generally MUCH easier to do in SVG than by trying to write a ton of JavaScript code to render onto a canvas. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Mark Sent: Saturday, January 15, 2011 11:04 AM To: Neo4j user discussions Subject: Re: [Neo4j] HTML5 graph viz library Yep. I slippy graph like google maps - with details an different zoom levels! Awesome! On 15 January 2011 15:58, Peter Neubauer peter.neuba...@neotechnology.com wrote: Mhh, interesting! I wonder if there is any support for using e.g. GeoTools to render arbitrary layouts apart from spatial. Would be worth to investigate. The nice thing is that these algos support, as you mention, zomming into a static structure, and espose more and more detail on every layer. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Sat, Jan 15, 2011 at 2:34 PM, Jacob Hansson ja...@voltvoodoo.com wrote: Sorry Peter, misread you. What I was thinking was to render non-spatial graphs with neo4j spatial. A layout algorithm would calculate the coordinates of each node we want to visualize. That way we can view really big graphs in the browser, since the layout-work is already done by the server. Den 15 jan 2011 14.28 skrev Jacob Hansson ja...@voltvoodoo.com: Yeah, you couldn't do very much with the markers. But you could pre-render millions of nodes on the server, and serve it as a spatial layer as if it was a map. Geoserver would slice it up, allowing zooming and panning just like a map. Then you would add interactivity on top of the rendered image like google does with google maps. It would be difficult to allow moving nodes and so on, but clicking on them and adding relations etc would not be that difficult.. Especially cool if coupled with the styling available with the current geo stack. Imagine for instance visualizing the corporation ownership and board members dataset, enabling cheap and super-easy access to the entire visualized graph in any browser.. :) ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] HTML5 graph viz library
Using a pre-made layout solution would indeed be optimal, no reason to re-invent wheels. Apart from the layout, we'd get the rest for free from neo4j spatial and surrounding technologies. This would be a separate visualization tool, with an ability to let clients explore massive graphs via a browser that, afaik, is not currently available. There is a reason to let, for instance, neo4j webadmin ship with a visualization tool. Just as the table format is a natural way to show relational database data in relational database management tools, a visualized graph is a natural way to show neo4j data. The problem we've wrestled with is that the web based graph visualization solutions available have trouble rendering large graphs. I'm not sure this would be a good solution to that particular problem, but it is a stab at solving the problem of showing huge, dynamically rendered graphs on the web. Den 15 jan 2011 20.38 skrev Tim McNamara paperl...@timmcnamara.co.nz: On 16 January 2011 05:34, Peter Neubauer peter.neuba...@neotechnology.comwrote: In this case, the graph layout could be computed server side by something like graphviz or so, and then sliced into zoom level information. Then interactivity can be added via open layers . However, of course the question is how long it takes to calculate the layout for say 100.000 nodes. After that, the basic layout info is more or less static. Still, very interesting to think of this kind of mixed approach for big visualizations, switching to dynamic solutions under a certain threshold, like 500 nodes or so. WDYT? /peter Is there any need for database software to provide visualisations? There are graph visualisation packages that have Neo4j backends that provide this functionality amazingly well. If you have four minutes today, I strongly recommend watching a video on Gephi [1]. You'll be amazed by the beauty, speed and utility that can be achieved when built for purpose tools are used. Tim McNamara @timClicks http://timmcnamara.co.nz [1] http://vimeo.com/14899695 ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] More PHP and REST goodness on the way
Like Peter said, the credit for the PHP client should be on Todd and on Al that have built the bulk of it. I'm über-up for continuing work on the refactoring, but am crazy short on time in the next few months. Todd - I just talked to Mattias, and the changes you wrote are already available in the latest version of neo4j-rest :) See http://components.neo4j.org/neo4j-rest/#Finding_a_path_between_two_nodes I got a bit ahead of myself in my fork of the php client, and went ahead and changed the name to neo4j.php. I'd like to formally propose that as a name for the project. The current name, Neo4J-REST-PHP-API-client, doesn't quite roll of the tongue. That would go with the names of other clients like neo4j.rb and neo4j.py. Come to think of it, there might be confusion there though. Both the other projects are wrappers around the embedded graph database, not REST clients. Perhaps neo4j-client.php? /Jake On Tue, Oct 19, 2010 at 12:51 PM, Todd Chaffee t...@mikamai.com wrote: Hi Peter, I'll see if I can get me, Jake, and Olle all on board to finish the refactoring and merge back with the original branch. Functionally the PHP client works and is ready. We'd like to add unit tests and better organize the code. How about the REST api? Any word on whether the path algos will be included in an upcoming release? That and traversals are probably the most useful parts of the PHP client. Thanks, Todd -- MIKAMAI | Making Media Social http://mikamai.com +447868260229 ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Minor change to neo4j-rest for major advantage
Hey all, I'm looking into modifying the DatabaseLocator class in neo4j-rest to allow connecting to remote databases (ie. letting neo4j-rest expose RemoteGraphDatabases via REST, enabling monitoring and management via webadmin of completely external projects). The change would basically just check if DB_PATH is a local file or an rmi:// URI, and use RemoteGraphDatabase if it is the latter. After the change, you could expose your internal database over RMI as normal ( http://components.neo4j.org/neo4j-remote-graphdb/), and then specify the RMI url as your db path to neo4j-rest (how you do that depends on how you run the project). There would have to be some checks put into place also, since certain things won't be possible when not running an EmbeddedGraphDatabase. Some things may also need to be done a bit differently and might have to wait, like indexing. I would also like to deprecate this in the current API: DatabaseLocator.getXXX(URI baseUri) And change it for: DatabaseLocator.getXXX() Unless I am not understanding the reason for this syntax. Currently nothing is done with baseUri, it is simply ignored throughout DatabaseLocator. I assume it was meant to define the database path, but was replaced by a system setting instead as it is now? Neither is really a great solution, but it's confusing that there are two apparent ways to do it, and only one that works. Does anyone have any thoughts about this or does it all sound ok? -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Using the REST neo4j
On Thu, Sep 16, 2010 at 9:25 AM, Mattias Persson matt...@neotechnology.comwrote: 2010/9/15 Jacob Hansson ja...@voltvoodoo.com On Wed, Sep 15, 2010 at 5:37 PM, Jacob Hansson ja...@voltvoodoo.com wrote: Hi Alex, sorry this response took me so long, see responses inline! 2010/9/10 Alexandru Popescu ☀ the.mindstorm.mailingl...@gmail.com On Tuesday, September 7, 2010, Jacob Hansson ja...@voltvoodoo.com wrote: 2010/9/6 Alexandru Popescu ☀ the.mindstorm.mailingl...@gmail.com On Monday, September 6, 2010, Jim Webber j...@webber.name wrote: Hi Alex, While I still can achieve all these with the current packaging, it feels more hacky: I need to create a new Jetty6BasedWebServer or modify the existing one to enhance it with my own stuff. Each change would require compiling and repackaging the whole neo4j-rest. Definitely not as easy as dropping in my own jar and a new web.xml. That's an interesting point. In a sense, the neo-rest package is Neo's REST package. Interesting... My main question is: what exactly is this package offering to the end user in the current form? IMO it cannot be an off-the-shelf product as there is no security. It is not a library either, as extending it is not so easy. Basically, and without any intention to harm any feelings, it looks like one of those dummy web UI interface to X. And I'd say it has much more potential than that! Jacob, I must confess I'm totally confused by your comments below. I've always seen it as the beginnings of a proper stand-alone neo4j server. If it is the beginning, then what comes next? And more importantly from whom? Basically my proposal was meant to make things easier for people to built on top of it, so I'm not really sure how you see the continuation of it. There is lots of cool things that can be done next. Both continuing to extend the functionality of the REST server, but also (and more importantly) to look at and help out with the work being done on clients for the server in various languages. For instance I'd love to see simple-to-use ORMs on top of the python and php clients, enabling web developers to start building stuff with Neo4j as their database tier. Just to clarify (trying to keep my answers less confusing :) ): To answer the last portion of your question, I think making neo4j REST more extensible is great (see my last answer). However, I read your initial proposal as an argument for viewing neo4j-rest as a library and for switching to a WAR packaging model. Both of which I strongly oppose *if* it is done at the expense of the stand alone application. I don't think it's about making the standalone REST package into a WAR, but to add a packaging as a war as another option to use the REST service. There's really no reason not to have both packagings, or is there? Yeah, I think I very much misunderstood the scope of Alexandrus initial proposal - adding more packaging options, as well as making neo4j-rest easier to extend is a great idea :) A REST/JSON API to Neo4j opens up for remote clients in any language, and would be an important part in matching offerings from other database vendors. While extendability is a great thing, building it as a library and/or packaging it as a WAR makes it very java-centric. Currently the neo4j-rest is distributed as a java application. So it is java-centric. What makes it attractive is that it allows using the HTTP protocol. Providing neo4j-rest as both a self contained app and as a web app will give you exactly the same benefits, with additional freedom on choosing how to use it, what servers to deploy it too, etc. True, the neo4j-rest project is, but I see the biggest potential of the REST project in it's stand-alone version, neo4j-rest-standalone. The reason for that is precisely what I mentioned before, the fact that it is not java centric. Distributing both as a WAR and as a stand-alone application sounds like a great solution! Like you say, there is no security, and I agree it is currently the main culprit stopping neo4j REST from production use. This can of course be offset with firewalling etc, but I couldn't agree more of the importance of a proper security layer. Security was used as a basic example of things that could be much easier to be added on top of the neo4j-rest if provided in a simpler format. As you probably know already firewalls will give you at most a very basic sort of authentication, but nothing else. As far as UI interface to X goes, the area to focus on I think is the JSON part of the API. With that, a UI can be built in any language. Take a look at http://github.com/neo4j/webadmin for a more
Re: [Neo4j] Using the REST neo4j
Hi Alex, sorry this response took me so long, see responses inline! 2010/9/10 Alexandru Popescu ☀ the.mindstorm.mailingl...@gmail.com On Tuesday, September 7, 2010, Jacob Hansson ja...@voltvoodoo.com wrote: 2010/9/6 Alexandru Popescu ☀ the.mindstorm.mailingl...@gmail.com On Monday, September 6, 2010, Jim Webber j...@webber.name wrote: Hi Alex, While I still can achieve all these with the current packaging, it feels more hacky: I need to create a new Jetty6BasedWebServer or modify the existing one to enhance it with my own stuff. Each change would require compiling and repackaging the whole neo4j-rest. Definitely not as easy as dropping in my own jar and a new web.xml. That's an interesting point. In a sense, the neo-rest package is Neo's REST package. Interesting... My main question is: what exactly is this package offering to the end user in the current form? IMO it cannot be an off-the-shelf product as there is no security. It is not a library either, as extending it is not so easy. Basically, and without any intention to harm any feelings, it looks like one of those dummy web UI interface to X. And I'd say it has much more potential than that! Jacob, I must confess I'm totally confused by your comments below. I've always seen it as the beginnings of a proper stand-alone neo4j server. If it is the beginning, then what comes next? And more importantly from whom? Basically my proposal was meant to make things easier for people to built on top of it, so I'm not really sure how you see the continuation of it. There is lots of cool things that can be done next. Both continuing to extend the functionality of the REST server, but also (and more importantly) to look at and help out with the work being done on clients for the server in various languages. For instance I'd love to see simple-to-use ORMs on top of the python and php clients, enabling web developers to start building stuff with Neo4j as their database tier. A REST/JSON API to Neo4j opens up for remote clients in any language, and would be an important part in matching offerings from other database vendors. While extendability is a great thing, building it as a library and/or packaging it as a WAR makes it very java-centric. Currently the neo4j-rest is distributed as a java application. So it is java-centric. What makes it attractive is that it allows using the HTTP protocol. Providing neo4j-rest as both a self contained app and as a web app will give you exactly the same benefits, with additional freedom on choosing how to use it, what servers to deploy it too, etc. True, the neo4j-rest project is, but I see the biggest potential of the REST project in it's stand-alone version, neo4j-rest-standalone. The reason for that is precisely what I mentioned before, the fact that it is not java centric. Distributing both as a WAR and as a stand-alone application sounds like a great solution! Like you say, there is no security, and I agree it is currently the main culprit stopping neo4j REST from production use. This can of course be offset with firewalling etc, but I couldn't agree more of the importance of a proper security layer. Security was used as a basic example of things that could be much easier to be added on top of the neo4j-rest if provided in a simpler format. As you probably know already firewalls will give you at most a very basic sort of authentication, but nothing else. As far as UI interface to X goes, the area to focus on I think is the JSON part of the API. With that, a UI can be built in any language. Take a look at http://github.com/neo4j/webadmin for a more powerful browsing UI for neo4j REST. I think you mis-read my post. I'm not looking for a nice UI, but rather for a basis to further build REST services on top of a neo4j db. As Jim mentioned in his posts, currently neo4j-rest is just exposing the basics of a neo4j db. It's true that the functionality exposed by neo4j-rest so far is fairly basic. There are several important parts of the neo4j API that should be exposed via REST (like transactions). I don't see that as an argument to make neo4j-rest more extendable though, as I feel these core items should be added the same way the data browsing, index and traversal APIs have been added. That said - extendability would be a great thing, and there are ways to make neo4j-rest much more accessible than it is today. I know Andreas is looking into the possibility of making neo4j-rest use OSGi-magic, which if implemented would make it possible to hot-deploy extensions into neo4j-rest as well as package extensions with it. I think the main reason of our disagreement (and my confusing answers :) ) is that we view neo4j-rest from two sides. I see it through the eyes of a web-developer. I'm used to having my database at some given port and a client in my web tier that throws work at the database. You see
Re: [Neo4j] Using the REST neo4j
On Wed, Sep 15, 2010 at 5:37 PM, Jacob Hansson ja...@voltvoodoo.com wrote: Hi Alex, sorry this response took me so long, see responses inline! 2010/9/10 Alexandru Popescu ☀ the.mindstorm.mailingl...@gmail.com On Tuesday, September 7, 2010, Jacob Hansson ja...@voltvoodoo.com wrote: 2010/9/6 Alexandru Popescu ☀ the.mindstorm.mailingl...@gmail.com On Monday, September 6, 2010, Jim Webber j...@webber.name wrote: Hi Alex, While I still can achieve all these with the current packaging, it feels more hacky: I need to create a new Jetty6BasedWebServer or modify the existing one to enhance it with my own stuff. Each change would require compiling and repackaging the whole neo4j-rest. Definitely not as easy as dropping in my own jar and a new web.xml. That's an interesting point. In a sense, the neo-rest package is Neo's REST package. Interesting... My main question is: what exactly is this package offering to the end user in the current form? IMO it cannot be an off-the-shelf product as there is no security. It is not a library either, as extending it is not so easy. Basically, and without any intention to harm any feelings, it looks like one of those dummy web UI interface to X. And I'd say it has much more potential than that! Jacob, I must confess I'm totally confused by your comments below. I've always seen it as the beginnings of a proper stand-alone neo4j server. If it is the beginning, then what comes next? And more importantly from whom? Basically my proposal was meant to make things easier for people to built on top of it, so I'm not really sure how you see the continuation of it. There is lots of cool things that can be done next. Both continuing to extend the functionality of the REST server, but also (and more importantly) to look at and help out with the work being done on clients for the server in various languages. For instance I'd love to see simple-to-use ORMs on top of the python and php clients, enabling web developers to start building stuff with Neo4j as their database tier. Just to clarify (trying to keep my answers less confusing :) ): To answer the last portion of your question, I think making neo4j REST more extensible is great (see my last answer). However, I read your initial proposal as an argument for viewing neo4j-rest as a library and for switching to a WAR packaging model. Both of which I strongly oppose *if* it is done at the expense of the stand alone application. A REST/JSON API to Neo4j opens up for remote clients in any language, and would be an important part in matching offerings from other database vendors. While extendability is a great thing, building it as a library and/or packaging it as a WAR makes it very java-centric. Currently the neo4j-rest is distributed as a java application. So it is java-centric. What makes it attractive is that it allows using the HTTP protocol. Providing neo4j-rest as both a self contained app and as a web app will give you exactly the same benefits, with additional freedom on choosing how to use it, what servers to deploy it too, etc. True, the neo4j-rest project is, but I see the biggest potential of the REST project in it's stand-alone version, neo4j-rest-standalone. The reason for that is precisely what I mentioned before, the fact that it is not java centric. Distributing both as a WAR and as a stand-alone application sounds like a great solution! Like you say, there is no security, and I agree it is currently the main culprit stopping neo4j REST from production use. This can of course be offset with firewalling etc, but I couldn't agree more of the importance of a proper security layer. Security was used as a basic example of things that could be much easier to be added on top of the neo4j-rest if provided in a simpler format. As you probably know already firewalls will give you at most a very basic sort of authentication, but nothing else. As far as UI interface to X goes, the area to focus on I think is the JSON part of the API. With that, a UI can be built in any language. Take a look at http://github.com/neo4j/webadmin for a more powerful browsing UI for neo4j REST. I think you mis-read my post. I'm not looking for a nice UI, but rather for a basis to further build REST services on top of a neo4j db. As Jim mentioned in his posts, currently neo4j-rest is just exposing the basics of a neo4j db. It's true that the functionality exposed by neo4j-rest so far is fairly basic. There are several important parts of the neo4j API that should be exposed via REST (like transactions). I don't see that as an argument to make neo4j-rest more extendable though, as I feel these core items should be added the same way the data browsing, index and traversal APIs have been added. That said - extendability would be a great thing, and there are ways to make neo4j-rest much more accessible than
Re: [Neo4j] Using the REST neo4j
2010/9/6 Alexandru Popescu ☀ the.mindstorm.mailingl...@gmail.com On Monday, September 6, 2010, Jim Webber j...@webber.name wrote: Hi Alex, While I still can achieve all these with the current packaging, it feels more hacky: I need to create a new Jetty6BasedWebServer or modify the existing one to enhance it with my own stuff. Each change would require compiling and repackaging the whole neo4j-rest. Definitely not as easy as dropping in my own jar and a new web.xml. That's an interesting point. In a sense, the neo-rest package is Neo's REST package. Interesting... My main question is: what exactly is this package offering to the end user in the current form? IMO it cannot be an off-the-shelf product as there is no security. It is not a library either, as extending it is not so easy. Basically, and without any intention to harm any feelings, it looks like one of those dummy web UI interface to X. And I'd say it has much more potential than that! I've always seen it as the beginnings of a proper stand-alone neo4j server. A REST/JSON API to Neo4j opens up for remote clients in any language, and would be an important part in matching offerings from other database vendors. While extendability is a great thing, building it as a library and/or packaging it as a WAR makes it very java-centric. Like you say, there is no security, and I agree it is currently the main culprit stopping neo4j REST from production use. This can of course be offset with firewalling etc, but I couldn't agree more of the importance of a proper security layer. As far as UI interface to X goes, the area to focus on I think is the JSON part of the API. With that, a UI can be built in any language. Take a look at http://github.com/neo4j/webadmin for a more powerful browsing UI for neo4j REST. However the notion of just letting end users write their own code hadn't occurred at least to me. I guess I always assumed that if users really wanted a domain specific API then they'd write their own. But the notion of user-registered filters (at least) is pretty sensible. I think your initial assumption makes a lot of sense. But why would one have to duplicate all the work when this could provide him not only with a good example, but a common basis for a complete solution. I'm looking at it from the perspective of a DB vizualization tool: what's in there offers you the default view. Next you could build your own views, etc. You could even build your complete application using it. Best thing is that I don't even think it is difficult to get it being more a matter of packaging than anything else. All would be needed: - a web.xml file with some configuration options in it (db location) - providing better access to the GraphDatabaseService (see my previous suggestion) and other common shared resources (IndexService, etc) - a different final package in form of a war - done Does it make sense to you? :- alex Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j Java Rest Client
As far as I know, there is currently no Java implementation of a client to the REST server available. Until there is one you can, however, talk to it through Java just like any other REST service using, for instance, the jersey REST client. It won't be as high-level as the EmbeddedGraphDatabase or the GraphDatabaseService, but it will work. Here's a quick guide on using the Jersey Client: http://blogs.sun.com/enterprisetechtips/entry/consuming_restful_web_services_with /Jacob On Mon, Aug 23, 2010 at 10:51 AM, Amir Hossein Jadidinejad amir.jad...@yahoo.com wrote: Hi, We have a graph database and a bunch of applications that use it (each application run a special graph algorithm on this database-a framework for a special graph).We want to provide a REST server and change the application's code in order to interact with the REST server instead of the same embedded database. There are some pages in the wiki that describes installing and running neo4j-rest and get/post methods in JSON. I'm not familiar with accessing REST service through a Java client. Would you please help me? An example that compare EmbeddedGraphDatabase with REST service is great. Kind regards ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j Java Rest Client
As far as I know, there is currently no Java implementation of a client to the REST server available. Until there is one you can, however, talk to it through Java just like any other REST service using, for instance, the jersey REST client. It won't be as high-level as the EmbeddedGraphDatabase or the GraphDatabaseService, but it will work. Here's a quick guide on using the Jersey Client: http://blogs.sun.com/enterprisetechtips/entry/consuming_restful_web_services_with /Jacob On Mon, Aug 23, 2010 at 10:51 AM, Amir Hossein Jadidinejad amir.jad...@yahoo.com wrote: Hi, We have a graph database and a bunch of applications that use it (each application run a special graph algorithm on this database-a framework for a special graph).We want to provide a REST server and change the application's code in order to interact with the REST server instead of the same embedded database. There are some pages in the wiki that describes installing and running neo4j-rest and get/post methods in JSON. I'm not familiar with accessing REST service through a Java client. Would you please help me? An example that compare EmbeddedGraphDatabase with REST service is great. Kind regards ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] neo4j REST server configuration
So the current status is that David has got neo4j REST running on Jetty with all tests passing. We've also searched through the code, and found that there are no interrupt() calls in the jersey source, while there are a few on the grizzly side. There is one in particular that we have been looking at, related to keep-alive timeouts, that may be the culprit. If that was the problem, we've got a fix for it. We have, however, been unable to recreate the problem so far, so we can't tell if we've solved it or not :) Brock: could you give us an idea of what types of requests you were throwing at the server, and a rough estimate of how many? /Jacob On Wed, Aug 11, 2010 at 2:35 PM, Jacob Hansson ja...@voltvoodoo.com wrote: Hi all! Johan took a look at the stack trace, and explained the problem. What happens is that something, either the Grizzly server or the jersey wrapper calls Thread.interrupt() on one of the neo4j threads (which should be considered a bug in whichever one of them does that). This triggers an IOError deep down in neo4j, which in turn causes the rest of the problems. I'm working on recreating the situation, and David is working on switching the REST system over to run on Jetty instead of Grizzly. We'll keep you posted on the progress. /Jacob On Wed, Aug 11, 2010 at 1:51 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Nice, will try that out Jim! Grinder seems cool. Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, Aug 11, 2010 at 12:52 PM, Jim Webber j...@webber.name wrote: Perhaps something as simple as a Grinder script might help? Jim On 11 Aug 2010, at 17:57, Brock Rousseau wrote: Thanks Peter. Let us know if there is anything else we can provide in the way of logs or diagnosis from our server. -Brock On Tue, Aug 10, 2010 at 11:51 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Mmh, seems we should stresstest the server and Grizzly with e.g. http://www.soapui.org and see if we can reproduce the scenario, if there is no obvious hint to this. Will try to set it up ... Cheers, /peter neubauer COO and Sales, Neo Technology GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, Aug 11, 2010 at 4:14 AM, Brock Rousseau bro...@gmail.com wrote: The patch worked perfectly for increasing the concurrent transaction cap, but unfortunately exposed another issue. After increasing the load hitting our rest server, it performs smoothly for 10-15 minutes then begins issuing 500 responses on all transactions. When it happens, the number of open transactions freezes in JMX and the heap size essentially remains static. Below are the two stack traces we see in the wrapper.log. Here are what i think to be the relevant configuration lines: wrapper.conf: wrapper.java.additional.1=-d64 wrapper.java.additional.2=-server wrapper.java.additional.4=-Xmx8192m wrapper.java.additional.3=-XX:+UseConcMarkSweepGC wrapper.java.additional.4=-Dcom.sun.management.jmxremote neo4j.properties: rest_min_grizzly_threads=4 rest_max_grizzly_threads=128 neostore.nodestore.db.mapped_memory=4000M neostore.relationshipstore.db.mapped_memory=4M neostore.propertystore.db.mapped_memory=1800M neostore.propertystore.db.index.mapped_memory=100M neostore.propertystore.db.index.keys.mapped_memory=100M neostore.propertystore.db.strings.mapped_memory=3G neostore.propertystore.db.arrays.mapped_memory=0M The server has 64Gb of total RAM so there should be a little over 6 left for the system. At the initial time of failure there are several of this error: INFO | jvm 1| 2010/08/10 13:00:33 | Aug 10, 2010 1:00:33 PM com.sun.grizzly.http.KeepAliveThreadAttachment timedOut INFO | jvm 1| 2010/08/10 13:00:33 | WARNING: Interrupting idle Thread: Grizzly-9555-WorkerThread(1) INFO | jvm 1| 2010/08/10 13:00:33 | Aug 10, 2010 1:00:33 PM com.sun.jersey.spi.container.ContainerResponse mapMappableContainerException INFO | jvm 1| 2010/08/10 13:00:33 | SEVERE: The RuntimeException could not be mapped to a response, re-throwing to the HTTP container INFO | jvm 1| 2010/08/10 13:00:33 | org.neo4j.kernel.impl.nioneo.store.UnderlyingStorageException: Unable to load position[7280476
Re: [Neo4j] neo4j REST server configuration
Brock, I've written a patch that allows setting the max threads via the neo4j configuration file. Applying it fixes the problem you mention, but leads to some kind of resource starvation higher up. This is not as critical, at 400 concurrent requests it will make the server alternate between 12ms read response times (which should be the norm) and 300ms. It's still bad, so before this patch is applied I'd like to dig into where that comes from. If you want, though, you can apply the patch yourself. To do that, you would download the REST component and the REST standalone project, avaliable here: svn co https://svn.neo4j.org/laboratory/components/rest/ svn co https://svn.neo4j.org/assemblies/rest-standalone/trunk/ Apply the patch (attached to this email) to the REST component. Run mvn install on the REST component, and then mvn package on the standalone project. This will give you a patched version with a default max thread count of 128 (compared to 5 before). To increase it more, go to the neo4j-rest-db-folder in the deployed system, open (or create) the file neo4j.properties, and use the following two settings to modify the thread count: rest_min_grizzly_threads=16 rest_max_grizzly_threads=48 I can't give you a perfect number, but it makes a big difference in response times, in both directions, to change these settings. /Jacob On Mon, Aug 9, 2010 at 8:51 PM, Brock Rousseau bro...@gmail.com wrote: Hey Jacob, Thanks for the quick response! We saw your post on the grizzly mailing list about the transaction limit fix. Is that something we'd be able to implement on our end today? We've had to throttle back the traffic significantly and are eager to see it in action at full volume. -Brock ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] neo4j REST server configuration
Apparently the email list deletes attachments it does not recognize, I'll send it directly to your email. As far as the test failures go, I don't know what causes that. I ran and passed all tests with the patch, but the trunk version should do that too. It might be something platform specific.. what platform are you running on? /Jacob On Tue, Aug 10, 2010 at 10:06 AM, Brock Rousseau bro...@gmail.com wrote: Hey Jacob, The patch sounds perfect, but I didn't see an attachement. Did i miss it somehow? The standalone project built fine for me, but we're getting these test failures for mvn install on the rest component: Failed tests: shouldBeAbleToDescribeTraverser(org.neo4j.rest.web.JsonWebServiceTest) Tests in error: shouldBeAbleToTraverseEverything(org.neo4j.rest.domain.StorageActionsTest) shouldBeAbleToUseCustomReturnFilter(org.neo4j.rest.domain.StorageActionsTest) shouldBeAbleToTraverseWithMaxDepthAndPruneEvaluatorCombined(org.neo4j.rest.domain.StorageActionsTest) shouldGetExpectedHitsWhenTraversingWithDescription(org.neo4j.rest.functional.TraverserFunctionalTest) Should we be running the install by ignoring tests or should those be passing? Thanks, Brock On Tue, Aug 10, 2010 at 12:33 AM, Jacob Hansson ja...@voltvoodoo.com wrote: Brock, I've written a patch that allows setting the max threads via the neo4j configuration file. Applying it fixes the problem you mention, but leads to some kind of resource starvation higher up. This is not as critical, at 400 concurrent requests it will make the server alternate between 12ms read response times (which should be the norm) and 300ms. It's still bad, so before this patch is applied I'd like to dig into where that comes from. If you want, though, you can apply the patch yourself. To do that, you would download the REST component and the REST standalone project, avaliable here: svn co https://svn.neo4j.org/laboratory/components/rest/ svn co https://svn.neo4j.org/assemblies/rest-standalone/trunk/ Apply the patch (attached to this email) to the REST component. Run mvn install on the REST component, and then mvn package on the standalone project. This will give you a patched version with a default max thread count of 128 (compared to 5 before). To increase it more, go to the neo4j-rest-db-folder in the deployed system, open (or create) the file neo4j.properties, and use the following two settings to modify the thread count: rest_min_grizzly_threads=16 rest_max_grizzly_threads=48 I can't give you a perfect number, but it makes a big difference in response times, in both directions, to change these settings. /Jacob On Mon, Aug 9, 2010 at 8:51 PM, Brock Rousseau bro...@gmail.com wrote: Hey Jacob, Thanks for the quick response! We saw your post on the grizzly mailing list about the transaction limit fix. Is that something we'd be able to implement on our end today? We've had to throttle back the traffic significantly and are eager to see it in action at full volume. -Brock ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] neo4j REST server configuration
The problem with the resource starvation turned out to be fairly simple, and was due to a missing connection:close header in the neo4j REST client I was using. It would be interesting to look into the connection:keep-alive behaviour of Grizzly, using HTTP 1.1 pipelining would be an excellent way to increase performance in the clients, but the few things I've read so far suggests Grizzly does not yet support that. /Jacob On Tue, Aug 10, 2010 at 11:19 AM, Brock Rousseau bro...@gmail.com wrote: The patched trunk code is working fine after dropping it over our existing deployment. I'm going to wait until we have the support of our site-speed engineers in the morning before testing the transaction limit under full production load, but I'll post the results as soon as we have them. Thanks, Brock On Tue, Aug 10, 2010 at 1:47 AM, Brock Rousseau bro...@gmail.com wrote: I got the patch Jacob, thanks! We're running Centos 5.5, Java 1.6.0, and Maven 2.2.1 If it's fine for you guys then as you said, likely some configuration difference with our system, so i'll just run it with-Dmaven.test.skip=true Applying the patch now, i'll let you know how it goes. Thanks again for all the support. -Brock On Tue, Aug 10, 2010 at 1:37 AM, Anders Nawroth and...@neotechnology.com wrote: hi! On 08/10/2010 10:06 AM, Brock Rousseau wrote: The standalone project built fine for me, but we're getting these test failures for mvn install on the rest component: Try using Java 6 or skip the tests. From the pom.xml: !-- NOTICE: Tests will not run on standard Java 5! This is due to the fact that there is no easily available implementation of the javax.script API for Java 5 in any maven repositories. For building on Java 5, either skip running the tests, or include the javax.script js reference implementation from http://jcp.org/en/jsr/detail?id=223 in the classpath of the JVM. -- /anders ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] neo4j REST server configuration
Brock, I've been digging into this, and was able to replicate your problem with a concurrency cap at 5 transactions. It appears it is the Grizzly server running REST that imposes this limit by not expanding its thread pool the way it's supposed to. Increasing the initial number of threads (default is 5) leads to a proportional increase in the peak number of concurrent transactions reported. If I bring the minimum number of threads up to 128, I get 128 concurrent transactions too, so the culprit definitely is the web tier. By hammering the server really hard I can get grizzly to bring 13 threads online, still far from the several hundred that is the default maximum. I don't know yet why the thread pool does not grow as expected, I'm not very familiar with grizzly, but I've posted a message to the grizzly mailing list to see if anyone there has any input. /Jacob On Mon, Aug 9, 2010 at 11:03 AM, Anders Nawroth and...@neotechnology.comwrote: Hi! We were able to increase the heap size by removing the wrapper.java.initmemory and wrapper.java.maxmemory settings from wrapper.conf and instead used this: The Java Service Wrapper version used has a known bug not allowing more than four digits in the maxmemory setting. Thanks for reporting the issue! /anders wrapper.java.additional.1=-d64 wrapper.java.additional.2=-server wrapper.java.additional.3=-Dcom.sun.management.jmxremote wrapper.java.additional.4=-Xmx16144m wrapper.java.additional.5=-Xms16144m The rest server still seems to be capping itself at 5 concurrent transactions though despite the larger heap. When hit with the full load of the client servers we observed that the both the PeakNumberOfConcurrentTransactions and NumberOfOpenTransactions attributes in the Transactions section of JMX remain exactly at 5. Under lesser load, the client servers rarely time out and the NumberOfOpenTransactions hovers between 0 and 2. Is there a way to explicitly increase the maximum number of concurrent transactions? As a side note, we initially explored the heap size after reading this: “Having a larger heap space will mean that Neo4j can handle larger transactions and more concurrent transactions” on the Neo4j wiki ( http://wiki.neo4j.org/content/Configuration_Settings) Thanks, Brock ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] neo4j REST server standalone
Hi Mohit, the admin system is meant to eventually be able to provide support for live backups, although there is quite a bit of work left on other components first. The admin interface currently depends on the REST package which, afaik, does not turn on the command-line shell by default. There will be an integrated gremlin web console though. It would probably not be that difficult to add a setting for enabling/disabling command-line shell access though. I added this as a ticket to the github project - http://github.com/neo4j/webadmin/issues/issue/8. /Jacob On Thu, Jul 1, 2010 at 11:34 PM, Mohit Vazirani mohi...@yahoo.com wrote: - Original Message From: Mattias Persson matt...@neotechnology.com To: Neo4j user discussions user@lists.neo4j.org Sent: Thu, July 1, 2010 6:53:48 AM Subject: Re: [Neo4j] neo4j REST server standalone Hi, I'm guessing you would like to specify a configuration file for the database, right? Yes. That can't be done at the moment, but work is put into creating an admin interface to neo4j, initially quite tied to the rest API. I can see if I can whip something up which makes this possible as it is right now. That will be pretty useful. Also, will the admin interface allow me to connect through the shell in R/W mode while the REST server is running? By the way, memory mapping settings are auto-configured if no specific config is given for that. It looks at how much heap you've given it and how big your graph store files are (this is done at startup time, so if the store changes a lot then a restart would be required to have it optimally configured for the new store size). Interesting. Also, I'm trying to perform online backups with this rest server. Are logical logs enabled by default when using the REST server? I cannot find any files matching the pattern *log.v*, all I see is the file nioneo_logical.log.1And if so, is there a way to perform online backups while the REST server is running? 2010/6/30 Mohit Vazirani mohi...@yahoo.com Hi, I'm a beginner to neo4j. Recently set up the standalone rest server in linux (started using neo4j-rest start). I was reviewing http://wiki.neo4j.org/content/Configuration_Settings and was wondering how to specify memory-mapped I/O settings other backup-related settings to the REST server. I couldn't find any section in wrapper.conf that allowed me to do this. Thanks, ~Mohit ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Feedback on admin interface
Hi all! I'm creating a web-based admin interface for Neo4j. Something that can handle configuration, data browsing, backups and so on. I've been hacking on some Neo4j things related to the web previously, for instance the new UI at try.neo4j.org http://try.neo4j.org/http://try.neo4j.org and the simple html-browser that comes with the REST-interface for Neo4j. Since this is a much bigger thing, that will hopefully be useful to a lot people, I hope to get feedback and ideas from you guys. The initial idea is to build a one-stop-shop for setting up a Neo4j stand-alone database. It will essentially be an extension of the REST-standalone package, adding a web admin interface. In the future there are lots of possibilities for allowing administration of several Neo4j databases, both REST and embedded, but I want to try to keep it simple initially to get it up and running quickly. The architecture will be component-based to allow easy extensions as the need arises in the future. What are some tasks that you would like to be able to do with an admin interface like this? What are the most important parameters to get a quick grasp of the health of a running Neo4j db? Oh - and the project (which is bare-bones maven stuff right now) is available at http://github.com/neo4j/webadmin Cheers! -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
