Re: [Neo4j] error indexing node: status 405 returned
I changed the program as
public static void indexNode2() {
String uri = http://localhost:7474/db/data/index/node/myindex;;
String post = {\value\ : \phoenix\,\uri\ :
\http://localhost:7474/db/data/node/1\,\key\; : \name\};
WebResource resource = Client.create().resource(uri);
ClientResponse response =
resource.accept(MediaType.APPLICATION_JSON).type(MediaType.APPLICATION_JSON).entity(post).put(ClientResponse.class);
System.out.println(response + response.getEntity(String.class));
System.out.println(status + response.getStatus());
}
This should have worked on 1.5.M02 but it still returns 405 status
However, following command works on HTTP console with status 201 Created
POST /db/data/index/node/myindex
{key:name,value:phoenix,uri:http://localhost:7474/db/data/node/1}
What is the problem???
--
View this message in context:
http://neo4j-community-discussions.438527.n3.nabble.com/error-indexing-node-status-405-returned-tp3469788p3469959.html
Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j performance with 400million nodes
hello david, thank you for the quick reply! appreciate it very much. Am 01.11.2011 01:01, schrieb David Montag: Hi Alican, On Mon, Oct 31, 2011 at 6:26 AM, algecyaalican.gecya...@openconcept.chwrote: Hello everyone, We are relatively new to neo4j and are evaluating some test scenarios in order to decide to use neo4j in productive systems. We used the latest stable release 1.4.2. I wrote an import script and generated some random data with the given tree structure: http://neo4j-community-discussions.438527.n3.nabble.com/file/n3467806/neo4j_nodes.png Nodes Summary: Nodes with Type A: 1 Nodes with Type B: 100 Nodes with Type C: 50'000 (100x500) Nodes with Type D: 500'000 (50'000x10) Nodes with Type E: 25'000'000 (500'000x50) Nodes with Type F: 375'000'000 (25'000'000x15) This all worked quite OK, the import took approx. 30hours using the batchimport. We have multiple indexes, but we also have one index where all nodes are indexed. My first question would be, does it make sense to index all nodes with the same index? It depends on how you intend you access the data. If you always know the type, then it would be beneficial to use different indices. Otherwise you might want to put it all in a single index. Do remember that the index will consume some disk space as well. ok, we decided to create a type node for each type and let the nodes relate to it. (Instead of having the type as an attribute at each node) I guess I was thinking too much in relational database schemes. therefore we will have an index per type. If I would like to list all nodes with property type:type E it is quite slow the first time ~270s Second time it is fast ~1/2s. I know this is normal and mostlikely fixed in the current milestone version. But I am not sure how long the query will be cached in memory. Are there any configurations I should be concerned about? The difference there is all about disk access time. Will give me all 25 million E's be a common operation? We will need to find nodes with common attributes of type E , which may return approx. 1million results. But there will always be a search for different values. E.g., nodes with type E have an attribute date created and an attribute name. I will need to find all attributes created at the given date(say year 2011) and the given name (abc). The second search will be date (2011) and name (def). If certain time passes and memory is being used for other searches, I am afraid my first search (2011,abc) will be kicked out of memory and the search will take long again the next time I query for it. We also took the hardware sizing calculator. See the result here: http://neo4j-community-discussions.438527.n3.nabble.com/file/n3467806/neo4j_hardware.png Are these realistic result values? I guess 128GB RAM and 12TB SSD harddrives might be a bit cost intense. The reason that the disk usage is 12TB is because you specified that each node on average has 10kB of data, and each relationship on average has 1kB of data. What kind of data are you storing on the nodes and relationships? These are pretty rough estimates not taking into account the number of properties nor the type of them. Also, if you decrease the property data by a factor 100 (100B/node, 10B/rel), then your database will only consume ~150-200GB. Ok I see your point. I think I am getting the hang of graph-based databases now. I.e., I might not want to put all my data into attributes but create nodes instead... My rough guess was to increase the amount of nodes to a 1'000'000'000 and decrease the bytes consumed to 100B/node and 10/rel. The result is to have approx. 400GB (no problem at all). But I am still a bit concerned about the 128GB RAM.. Are there any reference applications with these amount of nodes and relations? We are in the process of adding case studies. Please get in touch with sales for more info at this time. Thank you, will do so. Also Neoclipse won't start/connect to the database anymore with these amount of data. Am I missing some configurations for neoclipse? Are you getting an error message? No error messages. Is there an option to enable logging? I let neoclipse run for almost an hour and suddenly the graph appeared. But I can not navigate(its like frozen, but there are calculations going on..) Not so sure why it takes so long though, the initial traversal depth is 1, there are 16 nodes and 15 relations. I also decreased the amount of nodes to be displayed to 50. I thought It would load data lazily? Best regards alican Best, David Best regards -- alican -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-performance-with-400million-nodes-tp3467806p3467806.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org
Re: [Neo4j] Cypher GEOFF
I'm not sure cypher is the right name for this. It could be confusing... On Mon, Oct 31, 2011 at 6:03 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Very cool, good work! I would like to have this packaged as a brew formula on OSX, do one cold do brew install cypher and be done. WDYT? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/- Öresund - Innovation happens HERE. On Mon, Oct 31, 2011 at 1:46 AM, Nigel Small ni...@nigelsmall.name wrote: Hi Michael Currently these bits only exist in the GitHub code. So you'll need to pull it from there until the next release. New output format is straightforward, could you add a ticket to the project? Cheers Nige On 31 Oct 2011 08:41, Michael Hunger michael.hun...@neotechnology.com wrote: Nigel, cool stuff. What is the easiest way to install py2neo currently? Could you add a simple tabular output as well (like in the webadmin-cypher-console) ? Michael Am 31.10.2011 um 09:24 schrieb Nigel Small: Hi all A quick note on the new command line additions to py2neo (which currently aren't completely finialised but shouldn't change too much from here on out). --- usage: cypher.py [-h] [-u DATABASE_URI] [-d] [-j] [-g] query Execute Cypher queries against a Neo4j database server and output the results. positional arguments: querythe Cypher query to execute optional arguments: -h, --help show this help message and exit -u DATABASE_URI the URI of the source Neo4j database server -d output all values in delimited format (default) -j output all values as a single JSON array -g output nodes and relationships in GEOFF format --- usage: geoff.py [-h] [-u U] [-f F] Import graph data from a GEOFF file into a Neo4j database. optional arguments: -h, --help show this help message and exit -u Uthe URI of the destination Neo4j database server -f Fthe GEOFF file to load --- The latter script can also read from stdin which allows pipes to be built to funnel data out of a db and back in again: ./cypher.py -g START n=node(23) match (n)-[r]-(x)--(y) return n, x.name , n, r, x, x.name, x.\`birth.date\`, y, y.name, r.\`marriage.date\`? | ./geoff.py Of course this example puts data back into the the same DB it reads it from so isn't much use! Putting together more docs this week so will update soon. Nige ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cypher GEOFF
The tools are heavily integrated into py2neo so you would probably need to call it py2neo or py2neo-cli. Nige On 1 November 2011 07:20, Andres Taylor andres.tay...@neotechnology.comwrote: I'm not sure cypher is the right name for this. It could be confusing... On Mon, Oct 31, 2011 at 6:03 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Very cool, good work! I would like to have this packaged as a brew formula on OSX, do one cold do brew install cypher and be done. WDYT? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/- Öresund - Innovation happens HERE. On Mon, Oct 31, 2011 at 1:46 AM, Nigel Small ni...@nigelsmall.name wrote: Hi Michael Currently these bits only exist in the GitHub code. So you'll need to pull it from there until the next release. New output format is straightforward, could you add a ticket to the project? Cheers Nige On 31 Oct 2011 08:41, Michael Hunger michael.hun...@neotechnology.com wrote: Nigel, cool stuff. What is the easiest way to install py2neo currently? Could you add a simple tabular output as well (like in the webadmin-cypher-console) ? Michael Am 31.10.2011 um 09:24 schrieb Nigel Small: Hi all A quick note on the new command line additions to py2neo (which currently aren't completely finialised but shouldn't change too much from here on out). --- usage: cypher.py [-h] [-u DATABASE_URI] [-d] [-j] [-g] query Execute Cypher queries against a Neo4j database server and output the results. positional arguments: querythe Cypher query to execute optional arguments: -h, --help show this help message and exit -u DATABASE_URI the URI of the source Neo4j database server -d output all values in delimited format (default) -j output all values as a single JSON array -g output nodes and relationships in GEOFF format --- usage: geoff.py [-h] [-u U] [-f F] Import graph data from a GEOFF file into a Neo4j database. optional arguments: -h, --help show this help message and exit -u Uthe URI of the destination Neo4j database server -f Fthe GEOFF file to load --- The latter script can also read from stdin which allows pipes to be built to funnel data out of a db and back in again: ./cypher.py -g START n=node(23) match (n)-[r]-(x)--(y) return n, x.name , n, r, x, x.name, x.\`birth.date\`, y, y.name, r.\`marriage.date\`? | ./geoff.py Of course this example puts data back into the the same DB it reads it from so isn't much use! Putting together more docs this week so will update soon. Nige ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] error indexing node: status 405 returned
When I changed the PUT to POST, It worked fine. -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/error-indexing-node-status-405-returned-tp3469788p3470055.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j performance with 400million nodes
Hi! Also Neoclipse won't start/connect to the database anymore with these amount of data. Am I missing some configurations for neoclipse? Are you getting an error message? No error messages. Is there an option to enable logging? I let neoclipse run for almost an hour and suddenly the graph appeared. But I can not navigate(its like frozen, but there are calculations going on..) Not so sure why it takes so long though, the initial traversal depth is 1, there are 16 nodes and 15 relations. I also decreased the amount of nodes to be displayed to 50. I thought It would load data lazily? If you start Neoclipse from the command line you may see some extra output there. Also, inside the Neoclipse directory there's a workspace directory, and inside that you'll find .metadata/.log Just a thought: how many relationship types are there? /anders ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] error indexing node: status 405 returned
Hi, When I changed the PUT to POST, It worked fine. That makes sense: adding to an index is a POST operation to make it explicit that you're annotating an existing resource (as per RFC2616). I just double checked the server functional tests and it's visible there, and it generates some documentation here: http://docs.neo4j.org/chunked/snapshot/rest-api-indexes.html#rest-api-add-node-to-index Hope that helps, Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j performance with 400million nodes
Hi Alican, But I am still a bit concerned about the 128GB RAM.. You can run it on less of course. You could run it on your laptop and it would still work. However Neo4j is clever in its use of RAM. The more RAM you can allocate to Neo4j, the more chance that database reads can come straight from memory rather than spending potentially milliseconds going to mechanical disk, yielding thousands of traversals per second rather than millions. So more RAM = less disk hits (statistically) which is where you'll get huge read performance benefits. Less RAM means more likelihood of going to disk. All things being equal, with 128GB RAM you can cache a lot of your dataset in main memory. Perhaps even all your *active* dataset in fact (since it's about a quarter the size of your full dataset). That's going to give you blistering performance. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j performance with 400million nodes
hey anders! Thanks for the pointers. Am 01.11.2011 09:49, schrieb Anders Nawroth: Hi! Also Neoclipse won't start/connect to the database anymore with these amount of data. Am I missing some configurations for neoclipse? Are you getting an error message? No error messages. Is there an option to enable logging? I let neoclipse run for almost an hour and suddenly the graph appeared. But I can not navigate(its like frozen, but there are calculations going on..) Not so sure why it takes so long though, the initial traversal depth is 1, there are 16 nodes and 15 relations. I also decreased the amount of nodes to be displayed to 50. I thought It would load data lazily? If you start Neoclipse from the command line you may see some extra output there. Also, inside the Neoclipse directory there's a workspace directory, and inside that you'll find .metadata/.log There is no such directory (I am using neo4j 1.4.2) (I am aware that it is supposed to be a hidden directory) There was no extra output on the shell, but after another hour I got a java out of mem heap size exception. Which was my first guess anyways. But I just dont see why, since it is supposed to load only the first 16 nodes. All Relation types are in the graph already. - 6 Relation Types all in all. Just a thought: how many relationship types are there? /anders ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- alican ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j performance with 400million nodes
hey jim! Thanks for the thoughts. I know I could run it on less RAM, it's not a matter of can or cannot. I am also aware that the more RAM the better the performance. But my question is more: how will it perform with less RAM, say 32GB. Every system is quite fast with this amount of RAM. I am not sure if we can convince our customer to invest into 3x128GB RAM. (productive system, staging system, test system) Especially since there is not yet any reference application, which would guarantee an acceptable performance with this kind of system. Am 01.11.2011 10:27, schrieb Jim Webber: Hi Alican, But I am still a bit concerned about the 128GB RAM.. You can run it on less of course. You could run it on your laptop and it would still work. However Neo4j is clever in its use of RAM. The more RAM you can allocate to Neo4j, the more chance that database reads can come straight from memory rather than spending potentially milliseconds going to mechanical disk, yielding thousands of traversals per second rather than millions. So more RAM = less disk hits (statistically) which is where you'll get huge read performance benefits. Less RAM means more likelihood of going to disk. All things being equal, with 128GB RAM you can cache a lot of your dataset in main memory. Perhaps even all your *active* dataset in fact (since it's about a quarter the size of your full dataset). That's going to give you blistering performance. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- alican ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j performance with 400million nodes
Hi! If you start Neoclipse from the command line you may see some extra output there. Also, inside the Neoclipse directory there's a workspace directory, and inside that you'll find .metadata/.log There is no such directory (I am using neo4j 1.4.2) (I am aware that it is supposed to be a hidden directory) Ok, then the directory was created in the current directory when you started Neoclipse the first time. It's probably named just workspace. Later versions will always put it in the neoclipse dir (fixed after the 1.4.x cycle, apparently). There was no extra output on the shell, but after another hour I got a java out of mem heap size exception. Which was my first guess anyways. But I just dont see why, since it is supposed to load only the first 16 nodes. All Relation types are in the graph already. - 6 Relation Types all in all. Seems like you hit a bug then. If you send me code to generate a graph like yours, I'll try it out. /anders Just a thought: how many relationship types are there? /anders ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j performance with 400million nodes
Alican, we have other customers with that RAM sizes. It is always about the size of your hot data-set (i.e. cached) the better you understand your use-cases, the better you can estimate the # of nodes and rels (and their properties) that have to be cached. Michael Am 01.11.2011 um 10:53 schrieb Alican Gecyasar: hey jim! Thanks for the thoughts. I know I could run it on less RAM, it's not a matter of can or cannot. I am also aware that the more RAM the better the performance. But my question is more: how will it perform with less RAM, say 32GB. Every system is quite fast with this amount of RAM. I am not sure if we can convince our customer to invest into 3x128GB RAM. (productive system, staging system, test system) Especially since there is not yet any reference application, which would guarantee an acceptable performance with this kind of system. Am 01.11.2011 10:27, schrieb Jim Webber: Hi Alican, But I am still a bit concerned about the 128GB RAM.. You can run it on less of course. You could run it on your laptop and it would still work. However Neo4j is clever in its use of RAM. The more RAM you can allocate to Neo4j, the more chance that database reads can come straight from memory rather than spending potentially milliseconds going to mechanical disk, yielding thousands of traversals per second rather than millions. So more RAM = less disk hits (statistically) which is where you'll get huge read performance benefits. Less RAM means more likelihood of going to disk. All things being equal, with 128GB RAM you can cache a lot of your dataset in main memory. Perhaps even all your *active* dataset in fact (since it's about a quarter the size of your full dataset). That's going to give you blistering performance. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- alican ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Spring Data Graph / Neo4j – Problems with OO Inheritance Polymorphism relation semantics
2011/10/29 Agelos Pikoulas agelos.pikou...@gmail.com
Thank you Jean-Pierre/James for your answer - what you propose is certainly
one valid way of doing it, but it is still doesnot feel like a 'natural'
way
: It doesn't allow the dynamic 'navigation' of bean properties that are
them
selves Collections that each entry might have other Collections etc.
For instance I might have a User which has a getFollowers() or
getLikedObjects() etc and I want to display some of their stuff, following
some display logic etc :-) Perhaps this wouldn't be the most elegant way
MVC
wise, but I am not a spring/jsp/mvc expert anyway.
I just remembered that there is an iterator() method on collections in some
newer servlet implementations (maybe a servlet 3 spec?). So this is
possible:
c:forEach items=${device.measurements.iterator() } var=measurement
c:out value=${measurement.collector.name} /
/c:forEach
We are using tomcat 7 (inside Virgo 3) and it works.
Best regards,
James
Surelly this isn't SDN's fault, but JSP's. It would though be very helpful
(and perhaps its a twoliner) if we could annotate @Query on more (immutable
or not) Collection/iteration interfaces, say Iterator for a start
Regards
On Thu, Oct 27, 2011 at 10:01 PM, Jean-Pierre Bergamin
jpberga...@gmail.com
wrote:
Hi Agelos
Regarding your 2nd point:
2) 2) The minor issue I have regards (the otherwise brilliant)
@Query,
due to its constraint of annotating (mainly) Iterable and NOT allowing
Set,
List etc (a runtime exception is thrown
org.springframework.data.neo4j.conversion.QueryResultBuilder$1 cannot
be
cast to java.util.List).
This wouldn’t be a huge problem, but the JSP/JSLT forEach tag DOESNOT
iterate Iterable (!!!), nor you can directly call .iterator() from
within
JSP, making life hard on both ends.
From the controller, just pass the iterator of the iterable to your JSP.
With Spring MVC, this would look like:
@RequestMapping(method = RequestMethod.GET)
public String list(Model uiModel) {
uiModel.addAttribute(kpis, kpiRepository.findAll().iterator());
return secure/kpis/list;
}
In your JSP, you then can use this iterator as expected:
c:forEach items=${kpis} var=kpi
tr
tdc:out value=${kpi.uuid} //td
tdc:out value=${kpi.name} //td
/tr
/c:forEach
Best regards,
James
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST traverse deprecation
It would be interesting to rephrase the question like this: Are there any use cases that the REST traversal API covers, that we currently cannot replicate in Cypher or Gremlin? On Sun, Oct 30, 2011 at 2:45 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Well, Andres has been putting the first algo into cypher and we are thinking of exposing them there so thru can be used as path functions in an expression. However, that will be step 2. WDYT? On Oct 29, 2011 6:47 PM, maxdemarzi maxdema...@gmail.com wrote: I think the traversal rest api is fine for my purposes, but if cypher is where we want to go long term, then we need to deprecate it in 1.6 and drop it in 1.7. This gives us some time to update our libraries and for cypher to settle down. Are you guys thinking of getting rid of the rest built in graph algorithms or are those there to stay? -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-REST-traverse-deprecation-tp3460396p3464979.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Java segfault using paste and Python bindings
I solved my problem by ensuring that I don't 'import neo4j' until I'm in a
dedicated thread. So long as all neo4j interaction is handled within that
thread, I don't crash.
The Python neo4j bindings use JPype; I noticed this note from the JPype
docs (http://jpype.sourceforge.net/doc/user-guide/userguide.html#threading):
For the most part, python threads based on OS level threads (i.e posix
threads), will work without problem. The only thing to remember is to
call jpype.attachThreadToJVM() in the thread body to make the JVM usable
from that thread. For threads that you do not start yourself, you can
call isThreadAttachedToJVM() to check.
I couldn't find any evidence for this, but I think that some of the Java
code in the neo4j bindings may call attachThreadToJVM at import time.
Unfortunately, this means that even though neo4j is thread-safe, it must be
constrained to a single thread when used from Python. But that's not too
disappointing, considering.
On Sat, Oct 29, 2011 at 4:25 PM, Michael Rene Armida m...@marmida.com wrote:
I am attempting to use Paste to serve a small web app that uses neo4j via
the Python bindings. I get a segfault whenever I try to create a new
database via 'neo4j.GraphDatabase'. I have narrowed this down to the
following minimal sample:
http://pastebin.com/MkNRvLR6
The top of the stack trace looks like this:
Stack: [0xb42e7000,0xb4ae8000], sp=0xb4ae44f0, free space=8181k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
code)
C [_jpype.so+0x26497] JPJavaEnv::NewObjectA(_jclass*, _jmethodID*,
jvalue*)+0x37
C [_jpype.so+0x3c0e8] JPMethodOverload::invokeConstructor(_jclass*,
std::vectorHostRef*, std::allocatorHostRef* )+0x178
C [_jpype.so+0x3a417] JPMethod::invokeConstructor(std::vectorHostRef*,
std::allocatorHostRef* )+0x47
C [_jpype.so+0x1beba] JPClass::newInstance(std::vectorHostRef*,
std::allocatorHostRef* )+0x2a
C [_jpype.so+0x67b9c] PyJPClass::newClassInstance(_object*,
_object*)+0xfc
C [python+0x96822] PyEval_EvalFrameEx+0x4332
C [python+0x991e7] PyEval_EvalCodeEx+0x127
I believe that is JPype trying to create an EmbeddedGraphDatabase via
neo4j.GraphDatabase() in python.
I noticed this previous thread:
http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-CPython-Pylons-and-threading-td942435.html
That looks like it might be related, except this problem occurs whenever I
try to create a database, not on shutdown. The issue definitely has
something to do with using neo4j under Paste; here's a Python shell session
which shows neo4j working:
import webob
import app
debug_app = app.DebugApp()
response = debug_app(webob.Request.blank('/'))
response.body
'it worked'
Relevant version info:
- ubuntu 11.04 (same crash under 32- and 64-bit systems)
- Python 2.7.1-0ubuntu5
- neo4j_embedded-1.5.b2
- jpype 0.5.4.1-3
- paste 1.7.5.1
- OpenJDK 6b22-1.10.2 (same crash under the Sun JRE)
Thanks for your time and any clues you might have.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST traverse deprecation
Also, there is another possibility: We are exposing a ECMA script execution engine via the Traverse enpoints. Instead of trying to shoehorn the traversal API into REST calls and then anyway executing script snippets, Why not exposing a JavaScript plugin along exactly the same lines as the Groovy plugin, and provide examples on how to use the same usecases via that plugin, so we cover the traversal API that way? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. On Tue, Nov 1, 2011 at 9:00 AM, Jacob Hansson jacob.hans...@neotechnology.com wrote: It would be interesting to rephrase the question like this: Are there any use cases that the REST traversal API covers, that we currently cannot replicate in Cypher or Gremlin? On Sun, Oct 30, 2011 at 2:45 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Well, Andres has been putting the first algo into cypher and we are thinking of exposing them there so thru can be used as path functions in an expression. However, that will be step 2. WDYT? On Oct 29, 2011 6:47 PM, maxdemarzi maxdema...@gmail.com wrote: I think the traversal rest api is fine for my purposes, but if cypher is where we want to go long term, then we need to deprecate it in 1.6 and drop it in 1.7. This gives us some time to update our libraries and for cypher to settle down. Are you guys thinking of getting rid of the rest built in graph algorithms or are those there to stay? -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-REST-traverse-deprecation-tp3460396p3464979.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Java segfault using paste and Python bindings
Hey Michael, thanks for the investigative work you put into this, and sorry
for not replying earlier.
I was not aware of the thread limitations of JPype, that's quite
dissapointing.. Overall, it feels like the JPype solution is not a
manageable one in the long term. The embedded use case is cool, because
it's so freaking convenient, but using JPype to embed the JVM is painful.
There are a few other options for JVM-python integration that are starting
to look interesting, perhaps we can look around a little bit and see what
is available. It would be especially cool if we can use something that
works across CPython, Jython as well as PyPy. I know Tobias was thinking
about something that would make that possible.
In the short to medium term though, I'd like to focus on two things (please
note that this is *my opinion*, not a product roadmap):
*Getting cypher and gremlin into the embedded python API*
Eventually, I think we should deprecate the traversal API in the embedded
client in favor of Cypher. Traversals need callbacks and shared state and
so on between the two VMs, and its gonna hurt, I think, if we depend on
that further down the road.
Doing this will also make it simple to keep full 1-1 API compatibility
between the embedded client and REST clients, making it easy for end users
to switch between the two.
*Making the REST API fast enough to make that use case more appealing.*
JPype is really just a cheap way of implementing a form of remoting API.
It's currently worth the pain, since it is a ton faster than our REST API,
but it's a band-aid, not a proper fix. We should focus on speeding up the
REST API.
WDYT?
/Jake
On Tue, Nov 1, 2011 at 5:00 PM, Michael Rene Armida m...@marmida.com wrote:
I solved my problem by ensuring that I don't 'import neo4j' until I'm in a
dedicated thread. So long as all neo4j interaction is handled within that
thread, I don't crash.
The Python neo4j bindings use JPype; I noticed this note from the JPype
docs (http://jpype.sourceforge.net/doc/user-guide/userguide.html#threading
):
For the most part, python threads based on OS level threads (i.e posix
threads), will work without problem. The only thing to remember is to
call jpype.attachThreadToJVM() in the thread body to make the JVM usable
from that thread. For threads that you do not start yourself, you can
call isThreadAttachedToJVM() to check.
I couldn't find any evidence for this, but I think that some of the Java
code in the neo4j bindings may call attachThreadToJVM at import time.
Unfortunately, this means that even though neo4j is thread-safe, it must be
constrained to a single thread when used from Python. But that's not too
disappointing, considering.
On Sat, Oct 29, 2011 at 4:25 PM, Michael Rene Armida m...@marmida.com
wrote:
I am attempting to use Paste to serve a small web app that uses neo4j via
the Python bindings. I get a segfault whenever I try to create a new
database via 'neo4j.GraphDatabase'. I have narrowed this down to the
following minimal sample:
http://pastebin.com/MkNRvLR6
The top of the stack trace looks like this:
Stack: [0xb42e7000,0xb4ae8000], sp=0xb4ae44f0, free space=8181k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
code)
C [_jpype.so+0x26497] JPJavaEnv::NewObjectA(_jclass*, _jmethodID*,
jvalue*)+0x37
C [_jpype.so+0x3c0e8] JPMethodOverload::invokeConstructor(_jclass*,
std::vectorHostRef*, std::allocatorHostRef* )+0x178
C [_jpype.so+0x3a417] JPMethod::invokeConstructor(std::vectorHostRef*,
std::allocatorHostRef* )+0x47
C [_jpype.so+0x1beba] JPClass::newInstance(std::vectorHostRef*,
std::allocatorHostRef* )+0x2a
C [_jpype.so+0x67b9c] PyJPClass::newClassInstance(_object*,
_object*)+0xfc
C [python+0x96822] PyEval_EvalFrameEx+0x4332
C [python+0x991e7] PyEval_EvalCodeEx+0x127
I believe that is JPype trying to create an EmbeddedGraphDatabase via
neo4j.GraphDatabase() in python.
I noticed this previous thread:
http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-CPython-Pylons-and-threading-td942435.html
That looks like it might be related, except this problem occurs whenever
I
try to create a database, not on shutdown. The issue definitely has
something to do with using neo4j under Paste; here's a Python shell
session
which shows neo4j working:
import webob
import app
debug_app = app.DebugApp()
response = debug_app(webob.Request.blank('/'))
response.body
'it worked'
Relevant version info:
- ubuntu 11.04 (same crash under 32- and 64-bit systems)
- Python 2.7.1-0ubuntu5
- neo4j_embedded-1.5.b2
- jpype 0.5.4.1-3
- paste 1.7.5.1
- OpenJDK 6b22-1.10.2 (same crash under the Sun JRE)
Thanks for your time and any clues you might have.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Jacob Hansson
Phone:
Re: [Neo4j] REST traverse deprecation
You, something like Neo4js https://github.com/neo4j/neo4js? We've had that since forever. Or do you mean something different? Andrés On Tue, Nov 1, 2011 at 5:06 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Also, there is another possibility: We are exposing a ECMA script execution engine via the Traverse enpoints. Instead of trying to shoehorn the traversal API into REST calls and then anyway executing script snippets, Why not exposing a JavaScript plugin along exactly the same lines as the Groovy plugin, and provide examples on how to use the same usecases via that plugin, so we cover the traversal API that way? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/- Öresund - Innovation happens HERE. On Tue, Nov 1, 2011 at 9:00 AM, Jacob Hansson jacob.hans...@neotechnology.com wrote: It would be interesting to rephrase the question like this: Are there any use cases that the REST traversal API covers, that we currently cannot replicate in Cypher or Gremlin? On Sun, Oct 30, 2011 at 2:45 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Well, Andres has been putting the first algo into cypher and we are thinking of exposing them there so thru can be used as path functions in an expression. However, that will be step 2. WDYT? On Oct 29, 2011 6:47 PM, maxdemarzi maxdema...@gmail.com wrote: I think the traversal rest api is fine for my purposes, but if cypher is where we want to go long term, then we need to deprecate it in 1.6 and drop it in 1.7. This gives us some time to update our libraries and for cypher to settle down. Are you guys thinking of getting rid of the rest built in graph algorithms or are those there to stay? -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-REST-traverse-deprecation-tp3460396p3464979.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST traverse deprecation
Nope, I mean a server side script execution (which we already are doing in http://docs.neo4j.org/chunked/snapshot/rest-api-traverse.html) to execute traversal API code on the server. Neo4js is a client driver. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. On Tue, Nov 1, 2011 at 9:19 AM, Andres Taylor andres.tay...@neotechnology.com wrote: You, something like Neo4js https://github.com/neo4j/neo4js? We've had that since forever. Or do you mean something different? Andrés On Tue, Nov 1, 2011 at 5:06 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Also, there is another possibility: We are exposing a ECMA script execution engine via the Traverse enpoints. Instead of trying to shoehorn the traversal API into REST calls and then anyway executing script snippets, Why not exposing a JavaScript plugin along exactly the same lines as the Groovy plugin, and provide examples on how to use the same usecases via that plugin, so we cover the traversal API that way? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. On Tue, Nov 1, 2011 at 9:00 AM, Jacob Hansson jacob.hans...@neotechnology.com wrote: It would be interesting to rephrase the question like this: Are there any use cases that the REST traversal API covers, that we currently cannot replicate in Cypher or Gremlin? On Sun, Oct 30, 2011 at 2:45 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Well, Andres has been putting the first algo into cypher and we are thinking of exposing them there so thru can be used as path functions in an expression. However, that will be step 2. WDYT? On Oct 29, 2011 6:47 PM, maxdemarzi maxdema...@gmail.com wrote: I think the traversal rest api is fine for my purposes, but if cypher is where we want to go long term, then we need to deprecate it in 1.6 and drop it in 1.7. This gives us some time to update our libraries and for cypher to settle down. Are you guys thinking of getting rid of the rest built in graph algorithms or are those there to stay? -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-REST-traverse-deprecation-tp3460396p3464979.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] SDN w/ write and readonly Graph Database Service?
I'm using SDN to build a graph db I'm expecting to have quite a heavy volume of write and read activity at peak times. We're not ready to start using HA at this point. I was wondering, if using Neo4J using two instances of the Graph Database Service, one EmbeddedGraphDatabase (writes), and one or more EmbeddedReadOnlyGraphDatabase (reads) would make any difference? Are there any benefits for splitting the db access that way? Or would accessing the db through the writable instance for all operations be pretty much the same? -TPP ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] SDN w/ write and readonly Graph Database Service?
Probably the opposite (if you could even do it). You'd lose the LRU caching across the boundary. Is the data being written the same as the data being read, or is there a natural segmentation? If so you could implement a crude form of sharding/partioning to avoid hot spots (concurrency related) during these periods. As always, SSD + plenty of RAM. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Tero Paananen Sent: Tuesday, November 01, 2011 12:16 PM To: user@lists.neo4j.org Subject: [Neo4j] SDN w/ write and readonly Graph Database Service? I'm using SDN to build a graph db I'm expecting to have quite a heavy volume of write and read activity at peak times. We're not ready to start using HA at this point. I was wondering, if using Neo4J using two instances of the Graph Database Service, one EmbeddedGraphDatabase (writes), and one or more EmbeddedReadOnlyGraphDatabase (reads) would make any difference? Are there any benefits for splitting the db access that way? Or would accessing the db through the writable instance for all operations be pretty much the same? -TPP ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST traverse deprecation
On Tue, Nov 1, 2011 at 5:22 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Nope, I mean a server side script execution (which we already are doing in http://docs.neo4j.org/chunked/snapshot/rest-api-traverse.html) to execute traversal API code on the server. Neo4js is a client driver. Remember that adding yet another way to talk to the server comes at a cost of confusion and spreading engineering focus thin. I don't think we should do this unless there is a good answer to the question: *Is there anything* *that this solves that is not solved by Cypher or Gremlin, and if so, why don't we expand Cypher or Gremlin to incorporate that use case instead? *Collecting our efforts in fewer buckets means that improvements will benefit everyone, not just the group that uses the Groovy endpoint, or the Traversal API, or the JS endpoint and so on. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/- Öresund - Innovation happens HERE. On Tue, Nov 1, 2011 at 9:19 AM, Andres Taylor andres.tay...@neotechnology.com wrote: You, something like Neo4js https://github.com/neo4j/neo4js? We've had that since forever. Or do you mean something different? Andrés On Tue, Nov 1, 2011 at 5:06 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Also, there is another possibility: We are exposing a ECMA script execution engine via the Traverse enpoints. Instead of trying to shoehorn the traversal API into REST calls and then anyway executing script snippets, Why not exposing a JavaScript plugin along exactly the same lines as the Groovy plugin, and provide examples on how to use the same usecases via that plugin, so we cover the traversal API that way? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/- Öresund - Innovation happens HERE. On Tue, Nov 1, 2011 at 9:00 AM, Jacob Hansson jacob.hans...@neotechnology.com wrote: It would be interesting to rephrase the question like this: Are there any use cases that the REST traversal API covers, that we currently cannot replicate in Cypher or Gremlin? On Sun, Oct 30, 2011 at 2:45 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Well, Andres has been putting the first algo into cypher and we are thinking of exposing them there so thru can be used as path functions in an expression. However, that will be step 2. WDYT? On Oct 29, 2011 6:47 PM, maxdemarzi maxdema...@gmail.com wrote: I think the traversal rest api is fine for my purposes, but if cypher is where we want to go long term, then we need to deprecate it in 1.6 and drop it in 1.7. This gives us some time to update our libraries and for cypher to settle down. Are you guys thinking of getting rid of the rest built in graph algorithms or are those there to stay? -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-REST-traverse-deprecation-tp3460396p3464979.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST traverse deprecation
Well, I think that Cypher is not yes taking into account Uniqueness, or is it Andres? Otherwise, most of the normal traversals are covered. To keep things consistent and give people a way of doing fine grained traversals, I would then suggest to document all the existing examples using Cypher or Groovy code, so we can document that things are working as expected and not loose functionality. The JavaScript exposure then would totally go away from the REST API and possibly be factored out into a plugin like Groovy/Gremlin. Or dropped. WDYT? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. On Tue, Nov 1, 2011 at 9:31 AM, Jacob Hansson jacob.hans...@neotechnology.com wrote: On Tue, Nov 1, 2011 at 5:22 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Nope, I mean a server side script execution (which we already are doing in http://docs.neo4j.org/chunked/snapshot/rest-api-traverse.html) to execute traversal API code on the server. Neo4js is a client driver. Remember that adding yet another way to talk to the server comes at a cost of confusion and spreading engineering focus thin. I don't think we should do this unless there is a good answer to the question: *Is there anything* *that this solves that is not solved by Cypher or Gremlin, and if so, why don't we expand Cypher or Gremlin to incorporate that use case instead? *Collecting our efforts in fewer buckets means that improvements will benefit everyone, not just the group that uses the Groovy endpoint, or the Traversal API, or the JS endpoint and so on. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. On Tue, Nov 1, 2011 at 9:19 AM, Andres Taylor andres.tay...@neotechnology.com wrote: You, something like Neo4js https://github.com/neo4j/neo4js? We've had that since forever. Or do you mean something different? Andrés On Tue, Nov 1, 2011 at 5:06 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Also, there is another possibility: We are exposing a ECMA script execution engine via the Traverse enpoints. Instead of trying to shoehorn the traversal API into REST calls and then anyway executing script snippets, Why not exposing a JavaScript plugin along exactly the same lines as the Groovy plugin, and provide examples on how to use the same usecases via that plugin, so we cover the traversal API that way? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. On Tue, Nov 1, 2011 at 9:00 AM, Jacob Hansson jacob.hans...@neotechnology.com wrote: It would be interesting to rephrase the question like this: Are there any use cases that the REST traversal API covers, that we currently cannot replicate in Cypher or Gremlin? On Sun, Oct 30, 2011 at 2:45 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Well, Andres has been putting the first algo into cypher and we are thinking of exposing them there so thru can be used as path functions in an expression. However, that will be step 2. WDYT? On Oct 29, 2011 6:47 PM, maxdemarzi maxdema...@gmail.com wrote: I think the traversal rest api is fine for my purposes, but if cypher is where we want to go long term, then we need to deprecate it in 1.6 and drop it in 1.7. This gives us some time to update our libraries and for cypher to settle down. Are you guys thinking of getting rid of the rest built in graph algorithms or are those there to stay? -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Neo4j-REST-traverse-deprecation-tp3460396p3464979.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins
Re: [Neo4j] SDN w/ write and readonly Graph Database Service?
On Tue, Nov 1, 2011 at 12:26 PM, Rick Bullotta rick.bullo...@thingworx.com wrote: Probably the opposite (if you could even do it). You'd lose the LRU caching across the boundary. Is the data being written the same as the data being read, or is there a natural segmentation? If so you could implement a crude form of sharding/partioning to avoid hot spots (concurrency related) during these periods. It's largely going to be the same data. It's definitely the same type of data. Basically there will be discreet sets of same type of data (kinda like sub-graphs, but they can be connected) inserted in a batch-like manner. Those discreet sets of data will then immediately be consumed by processes that use it to calculate additional data and store the results to various data stores, incl. the same Neo4J db. After the initial consumption, the data sets will be consumed on-demand, but I'm not sure about how frequently at this point. It'll likely be something like once or twice a day or even less frequently, but I'll have to see how the usage patterns emerge after the solution goes live to be sure. Thanks for the answer Rick. I'll do some quick-and-dirty testing later this week to guide the decision making on this. I'll see if I can post what I find on this thread afterwards. -TPP ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] SDN w/ write and readonly Graph Database Service?
Cool. If you can control access to the data (e.g. no access until the entire subgraph/dataset has been imported) you can probably avoid a lot of the concurrency issues. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Tero Paananen Sent: Tuesday, November 01, 2011 12:49 PM To: Neo4j user discussions Subject: Re: [Neo4j] SDN w/ write and readonly Graph Database Service? On Tue, Nov 1, 2011 at 12:26 PM, Rick Bullotta rick.bullo...@thingworx.com wrote: Probably the opposite (if you could even do it). You'd lose the LRU caching across the boundary. Is the data being written the same as the data being read, or is there a natural segmentation? If so you could implement a crude form of sharding/partioning to avoid hot spots (concurrency related) during these periods. It's largely going to be the same data. It's definitely the same type of data. Basically there will be discreet sets of same type of data (kinda like sub-graphs, but they can be connected) inserted in a batch-like manner. Those discreet sets of data will then immediately be consumed by processes that use it to calculate additional data and store the results to various data stores, incl. the same Neo4J db. After the initial consumption, the data sets will be consumed on-demand, but I'm not sure about how frequently at this point. It'll likely be something like once or twice a day or even less frequently, but I'll have to see how the usage patterns emerge after the solution goes live to be sure. Thanks for the answer Rick. I'll do some quick-and-dirty testing later this week to guide the decision making on this. I'll see if I can post what I find on this thread afterwards. -TPP ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] [bug?] Unicode node property not returned correctly from bulk REST index search
Nuo,
this is recreated now, but I had no time to fix it yet, just FYI.
https://github.com/neo4j/community/issues/69
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - NOSQL for the Enterprise.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
On Fri, Oct 21, 2011 at 1:26 AM, Peter Neubauer
peter.neuba...@neotechnology.com wrote:
Thanks Nuo,
submitted as a bug at https://github.com/neo4j/community/issues/69
Feel free to track it - hope to get to it next week.
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - NOSQL for the Enterprise.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
On Fri, Oct 21, 2011 at 10:07 AM, Nuo Yan yan@gmail.com wrote:
Hey Peter, using the example of my original email, this is the corresponding
json body I post when I created the node:
{\uid\:\12345\,\name\:\\\u4f8b\\u5b50\}
A single GET query returns the name data (as raw json) as:
\name\ : \\xE4\xBE\x8B\xE5\xAD\x90\\n
which after JSON parse becomes the correct one:
name = 例子
When sending the same request as a POST to /batch, the server returns the
following raw json:
\name\ :
\\xEF\xBF\xA4\xEF\xBE\xBE\xEF\xBE\x8B\xEF\xBF\xA5\xEF\xBE\xAD\xEF\xBE\x90\\n
which after JSON parse becomes the busted result:
name=￤ﾾﾋ￥ﾭﾐ
On Thu, Oct 20, 2011 at 11:28 PM, Peter Neubauer
peter.neuba...@neotechnology.com wrote:
Guys,
Do you have a JSON string you would use to set properties on a node? Can
then update a test with it and check.
On Oct 21, 2011 7:27 AM, Nuo Yan yan@gmail.com wrote:
Yea, I'm pretty sure it's not a client parse issue. The data is correct
in
the database, and a single GET query returns the right data, only when
doing
the same request as a part of the bulk request, it returns busted data.
It can be reproduced using curl and as well as rest client. I'm using
1.4.2
stable.
Anyone from the neo4j team has any insight on this?
On Thu, Oct 20, 2011 at 7:59 PM, Rick Bullotta
rick.bullo...@thingworx.comwrote:
I doubt it, since a GET works fine. It's probably an encoding issue
somewhere in the batch processing pipeline.
-Original Message-
From: user-boun...@lists.neo4j.org [mailto:
user-boun...@lists.neo4j.org]
On Behalf Of Daniel Fitzpatrick
Sent: Thursday, October 20, 2011 10:37 PM
To: Neo4j user discussions
Subject: Re: [Neo4j] [bug?] Unicode node property not returned
correctly
from bulk REST index search
Possibly an issue with the client code not understanding unicode. Is
there
something you could use as a baseline to rule the database out eg maybe
the
web admin?
On Oct 20, 2011 7:48 PM, Nuo Yan yan@gmail.com wrote:
I have nodes with data properties with unicode (Chinese/Japanese)
characters
such as:
{uid = 12345, name = 例子}
I index such nodes with their id, so that by doing this (where
users_index
is the index, uid is the key, 12345 is the value):
GET to /index/node/users_index/uid/12345
I can get back the right result:
{indexed=
http://localhost:7474/db/data/index/node/users_node/uid/12345/6638;,
outgoing_relationships=
http://localhost:7474/db/data/node/6638/relationships/out;,
* data={uid=12345, name=例子}, *
traverse=
http://localhost:7474/db/data/node/6638/traverse/{returnType}
,
all_typed_relationships=
http://localhost:7474/db/data/node/6638/relationships/all/{-list||types}
,
property=http://localhost:7474/db/data/node/6638/properties/{key}
,
self=http://localhost:7474/db/data/node/6638;, properties=
http://localhost:7474/db/data/node/6638/properties;,
outgoing_typed_relationships=
http://localhost:7474/db/data/node/6638/relationships/out/{-list||types}
,
incoming_relationships=
http://localhost:7474/db/data/node/6638/relationships/in;,
extensions={},
create_relationship=
http://localhost:7474/db/data/node/6638/relationships;,
paged_traverse=
http://localhost:7474/db/data/node/6638/paged/traverse/{returnType}{?pageSize,leaseTime}
,
all_relationships=
http://localhost:7474/db/data/node/6638/relationships/all;,
incoming_typed_relationships=
http://localhost:7474/db/data/node/6638/relationships/in/{-list||types}
}
However, if I do the same search query as a part of a bulk REST
request:
POST to /batch:
[{method = GET,
to = /index/node/users_index/uid/12345,
body = {},
id = 0}]
Returns the node in the body, however, with bad characters in the data
field:
[{id=0,
Re: [Neo4j] [bug?] Unicode node property not returned correctly from bulk REST index search
Cool. Thanks.
On Tue, Nov 1, 2011 at 10:00 AM, Peter Neubauer
peter.neuba...@neotechnology.com wrote:
Nuo,
this is recreated now, but I had no time to fix it yet, just FYI.
https://github.com/neo4j/community/issues/69
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - NOSQL for the Enterprise.
http://startupbootcamp.org/- Öresund - Innovation happens HERE.
On Fri, Oct 21, 2011 at 1:26 AM, Peter Neubauer
peter.neuba...@neotechnology.com wrote:
Thanks Nuo,
submitted as a bug at https://github.com/neo4j/community/issues/69
Feel free to track it - hope to get to it next week.
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - NOSQL for the Enterprise.
http://startupbootcamp.org/- Öresund - Innovation happens HERE.
On Fri, Oct 21, 2011 at 10:07 AM, Nuo Yan yan@gmail.com wrote:
Hey Peter, using the example of my original email, this is the
corresponding
json body I post when I created the node:
{\uid\:\12345\,\name\:\\\u4f8b\\u5b50\}
A single GET query returns the name data (as raw json) as:
\name\ : \\xE4\xBE\x8B\xE5\xAD\x90\\n
which after JSON parse becomes the correct one:
name = 例子
When sending the same request as a POST to /batch, the server returns
the
following raw json:
\name\ :
\\xEF\xBF\xA4\xEF\xBE\xBE\xEF\xBE\x8B\xEF\xBF\xA5\xEF\xBE\xAD\xEF\xBE\x90\\n
which after JSON parse becomes the busted result:
name=￤ﾾﾋ￥ﾭﾐ
On Thu, Oct 20, 2011 at 11:28 PM, Peter Neubauer
peter.neuba...@neotechnology.com wrote:
Guys,
Do you have a JSON string you would use to set properties on a node?
Can
then update a test with it and check.
On Oct 21, 2011 7:27 AM, Nuo Yan yan@gmail.com wrote:
Yea, I'm pretty sure it's not a client parse issue. The data is
correct
in
the database, and a single GET query returns the right data, only
when
doing
the same request as a part of the bulk request, it returns busted
data.
It can be reproduced using curl and as well as rest client. I'm using
1.4.2
stable.
Anyone from the neo4j team has any insight on this?
On Thu, Oct 20, 2011 at 7:59 PM, Rick Bullotta
rick.bullo...@thingworx.comwrote:
I doubt it, since a GET works fine. It's probably an encoding
issue
somewhere in the batch processing pipeline.
-Original Message-
From: user-boun...@lists.neo4j.org [mailto:
user-boun...@lists.neo4j.org]
On Behalf Of Daniel Fitzpatrick
Sent: Thursday, October 20, 2011 10:37 PM
To: Neo4j user discussions
Subject: Re: [Neo4j] [bug?] Unicode node property not returned
correctly
from bulk REST index search
Possibly an issue with the client code not understanding unicode.
Is
there
something you could use as a baseline to rule the database out eg
maybe
the
web admin?
On Oct 20, 2011 7:48 PM, Nuo Yan yan@gmail.com wrote:
I have nodes with data properties with unicode (Chinese/Japanese)
characters
such as:
{uid = 12345, name = 例子}
I index such nodes with their id, so that by doing this (where
users_index
is the index, uid is the key, 12345 is the value):
GET to /index/node/users_index/uid/12345
I can get back the right result:
{indexed=
http://localhost:7474/db/data/index/node/users_node/uid/12345/6638
,
outgoing_relationships=
http://localhost:7474/db/data/node/6638/relationships/out;,
* data={uid=12345, name=例子}, *
traverse=
http://localhost:7474/db/data/node/6638/traverse/{returnType}
,
all_typed_relationships=
http://localhost:7474/db/data/node/6638/relationships/all/{-list||types}
,
property=
http://localhost:7474/db/data/node/6638/properties/{key}
,
self=http://localhost:7474/db/data/node/6638;, properties=
http://localhost:7474/db/data/node/6638/properties;,
outgoing_typed_relationships=
http://localhost:7474/db/data/node/6638/relationships/out/{-list||types}
,
incoming_relationships=
http://localhost:7474/db/data/node/6638/relationships/in;,
extensions={},
create_relationship=
http://localhost:7474/db/data/node/6638/relationships;,
paged_traverse=
http://localhost:7474/db/data/node/6638/paged/traverse/{returnType}{?pageSize,leaseTime}
,
all_relationships=
http://localhost:7474/db/data/node/6638/relationships/all;,
incoming_typed_relationships=
http://localhost:7474/db/data/node/6638/relationships/in/{-list||types}
}
