Re: [Neo4j] Cannot find a path
Josh,
the docs are now at
http://docs.neo4j.org/chunked/snapshot/rest-api-graph-algos.html
Let me know if you want to see more, I am planning to put in an
example of Dijkstra, too.
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Jun 15, 2011 at 4:50 AM, Josh Adell josh.ad...@gmail.com wrote:
Thank you, Peter, that was exactly it. Yeah, I didn't realize that the max
depth defaulted to 1. For now, I guess the library I'm writing will have to
support max depth and max_depth since I won't always know what version
it will be running against.
Thanks again,
Josh Adel
Message: 4
Date: Tue, 14 Jun 2011 16:35:07 +0200
From: Peter Neubauer peter.neuba...@neotechnology.com
Subject: Re: [Neo4j] Cannot find a path
To: Neo4j user discussions user@lists.neo4j.org
Message-ID: BANLkTi=7xngcpf1e21euivje0xrw6fb...@mail.gmail.com
Content-Type: text/plain; charset=ISO-8859-1
Josh,
you need to spcify the max depth parameter for the algo, which
defaults to 1 and thus is not gaining any results. I am pushing
documentation on this, and I am changing this to max_depth, but for
the time being, it is something like
{\to\: \http://localhost:7474/db/data/node/+data.get().get( him
).getId()+\ +
,\max depth\:2}
Does that work?
Cheers,
/peter neubauer
GTalk:? ? ? neubauer.peter
Skype? ? ?? peter.neubauer
Phone? ? ?? +46 704 106975
LinkedIn?? http://www.linkedin.com/in/neubauer
Twitter? ? ? http://twitter.com/peterneubauer
http://www.neo4j.org? ? ? ? ? ? ?? - Your high performance graph database.
http://startupbootcamp.org/ ? ?- ?resund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Tue, Jun 14, 2011 at 2:55 PM, Peter Neubauer
peter.neuba...@neotechnology.com wrote:
Josh,
working on it ...
Cheers,
/peter neubauer
GTalk:? ? ? neubauer.peter
Skype? ? ?? peter.neubauer
Phone? ? ?? +46 704 106975
LinkedIn?? http://www.linkedin.com/in/neubauer
Twitter? ? ? http://twitter.com/peterneubauer
http://www.neo4j.org? ? ? ? ? ? ?? - Your high performance graph
database.
http://startupbootcamp.org/ ? ?- ?resund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Tue, Jun 14, 2011 at 2:44 PM, Josh Adell josh.ad...@gmail.com
wrote:
Hello all, sorry for the re-post, but I am still stuck. ?Does anyone
have an
idea about what I may be doing wrong, or is there any more information I
can
provide to help diagnose?
Thanks!
-- Josh Adell
--
Message: 6
Date: Sun, 12 Jun 2011 14:17:17 -0400
From: Josh Adell josh.ad...@gmail.com
Subject: [Neo4j] Cannot find a path
To: user@lists.neo4j.org
Message-ID: BANLkTi=zoSX1gMbe=txbs53vjvh+tra...@mail.gmail.com
Content-Type: text/plain; charset=ISO-8859-1
Hello all,
I recently began exploring Neo4j REST interface, and I am having a
problem
finding paths between nodes.
I am using neo4j-community-1.4.M03
My nodes and relationships look like this (verified using the webadmin
interface):
(1: Node A) IN (3: Node C) IN (2: Node B)
Using curl, I can query from node A to C:
$ curl -i -HAccept:application/json -HContent-Type:application/json -X
POST
-d '{to:http://localhost:7474/db/data/node/3}'
http://localhost:7474/db/data/node/1/path
HTTP/1.1 200 OK
Content-Length: 292
Content-Encoding: UTF-8
Content-Type: application/json
Access-Control-Allow-Origin: *
Server: Jetty(6.1.25)
{
?start : http://localhost:7474/db/data/node/1;,
?nodes : [ http://localhost:7474/db/data/node/1;,
http://localhost:7474/db/data/node/3; ],
?length : 1,
?relationships : [ http://localhost:7474/db/data/relationship/5; ],
?end : http://localhost:7474/db/data/node/3;
}
I get similar results for querying for paths from B to C.
But when I try to get a path from A to B (which is through C), I get an
empty result set:
$ curl -HAccept:application/json -HContent-Type:application/json -i -X
POST
-d '{to:http://localhost:7474/db/data/node/2}'
http://localhost:7474/db/data/node/1/path
HTTP/1.1 404 Not Found
Access-Control-Allow-Origin: *
Content-Type: text/html; charset=iso-8859-1
Cache-Control: must-revalidate,no-cache,no-store
Content-Length: 1385
Server: Jetty(6.1.25)
html
head
meta http-equiv=Content-Type content=text/html;
charset=ISO-8859-1/
titleError 404 Not Found/title
/head
bodyh2HTTP ERROR 404/h2
Re: [Neo4j] Extent of Indexing
Aman, I think to start with, you probably want to set up indexing on only the interesting aspects of your CMS and documents, not everything. Normally, that leads to a much slimmer storage, and better speed in mutating operations. Given 50K users, the question is how much index-relevant content there will be, and what the peak load of that system is. Do you have any estimations for that? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Tue, Jun 14, 2011 at 7:24 PM, Aman aman.6...@gmail.com wrote: I've been working on a project (a CMS) that would require a lot of indexing as it would manage user generated content (mostly strings). Could I get some suggestions on what is the extent of indexing that can be useful in a typical CMS which could have a fair use by about 5 people? And how does this indexing matter if I have to scale it any further? Also, as I said above, the index would mostly deal with strings. Would it be any better if I put up another layer in front of my neo4j db that maps those strings to ints so that I could index those easily in neo4j? ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Query Number of Nodes with Property Filter
Tatham, you domain model sounds good. So you should do both things. First try to use connected graphs for the agencies and try to traverse from the agency node. If you need to do index lookups you should always use queries that add the agency key (which is in the session) to the lucene query (using an and query). For creating unique agencies you can also use a category node for the agencies (probably the root node) and check with a traversal that it doesn't contain the agency you want to create. Or you use an agencies index that is used to check for duplicates. What other dynamic language experience is on your team (besides c#) ? We're working on script-server extensions that could handle some of your domain logic on the server side and talk a domain level REST protocol to your .net client app. In general you would use indexes when you're executing context free global queries on the graph and traversals if you work on dependencies from the context of a start-node. Can you give some figures ? #of agencies, avg # of users per agency, avg #of cases per agency? One problem is that currently when you use the pure REST API it doesn't enforce your multi-tenancy concerns. So you have to encapsulate that in your application logic. There were talks about multi-database server variants and also some experiments with that, but it doesn't have that high a priority. This would solve your problem easily from a modelling perspective (i.e. have different contexts for the agencies). But a simpilistic approach would impact server performance as it would just run several embedded neo4j instances alongside each other without resource sharing/optimization. HTH Cheers Michael Am 15.06.2011 um 07:42 schrieb Tatham Oddie: ... and the context of the original question was to ensure that agency keys are unique when creating them. -- Tatham -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Tatham Oddie Sent: Wednesday, 15 June 2011 3:35 PM To: Neo4j user discussions Subject: Re: [Neo4j] Query Number of Nodes with Property Filter We're building a client and case management system for social work agencies. It will be a multitenant system, hosting multiple agencies. Agencies have a name Acme Agency Australia and a key acme that is used in places like URLs (http://thesystem.org/acme). Users are scoped to agencies. Usernames need to be unique within agencies. When a user logs in, they will complete three fields - agency key (acme), username (tatham), password. We need to find that user, within the scope of that agency to validate their login. Another group of users are scoped to the entire system - they are the system wide administrators. They only operate at a system level. We need to be *really* careful about separating each agency's data. I'm currently thinking along the lines that when a user logs in, and we resolve the agency node they belong to, we'd store a handle/reference to that agency node in their session/auth ticket/somewhere. We'd then architect all of our data access services to always commence their queries from this agency node. Basically promoting the agency nodes to reference nodes in the context of each session. Currently we're just dealing with agencies and users as our first features, then will start building out client management. (Architecturally, we're building in .NET via the REST API.) -- Tatham -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Michael Hunger Sent: Wednesday, 15 June 2011 2:54 PM To: Neo4j user discussions Subject: Re: [Neo4j] Query Number of Nodes with Property Filter just describe your domain and your use-cases for usage and the graph model you developed and we can discuss it. Michael Sent from my iBrick4 Am 15.06.2011 um 02:43 schrieb Tatham Oddie tat...@oddie.com.au: Hi Michael, (Warning: graph db n00b disclaimer still applies) Category nodes are an interesting approach. Would you mind validating the graph we've come up with to make sure we're on the right track? -- Tatham -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Michael Hunger Sent: Wednesday, 15 June 2011 1:42 AM To: Neo4j user discussions Subject: Re: [Neo4j] Query Number of Nodes with Property Filter you can use category nodes to connect nodes of a certain type to. then you can count the nodes using a traversal you dont have to go via an index to do things like that Sent from my iBrick4 Am 14.06.2011 um 17:10 schrieb Marko Rodriguez okramma...@gmail.com: Hi, Is it possible that when we create the Automatic Index, that we can filter it on specific nodes? No. Referring to the diagram below, we have a Node with a property _Type= Agency. I would like to
Re: [Neo4j] Which version of neo4j does spring data graph project use ?
Sorry, mixed up the threads on the iphone :( Will post an SDG update right ahead. Michael Am 15.06.2011 um 05:42 schrieb Vipul Gupta: Michael, I don't see anything released on maven repo yet? Please update. -VIpul On Fri, Jun 10, 2011 at 3:40 PM, Michael Hunger michael.hun...@neotechnology.com wrote: Wait for our spring data graph 1.1.0.M1 release on Monday :) it will be updated to Neo4j 1.4.M04 Cheers Michael Am 10.06.2011 um 11:55 schrieb Vipul Gupta: Please let me know.. I want to use the latest version of neo4j say Neo4j 1.4 M03 “Kiruna with spring data graph. WHat do I do ? ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] How to embed neo4j to OSGI env without spring?
Yes, that's what I managed to do : remaned master to a new branch then checkout upstream/master in local master. Thanks. 2011/6/15 Peter Neubauer peter.neuba...@neotechnology.com Nicolas, I think you could just make another branch, or rename the master branch to something else. Then, you could set up a new master and let it track the original master? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Tue, Jun 14, 2011 at 10:12 PM, Nicolas Jouanin nicolas.joua...@gmail.com wrote: Hi Peter (and others), May be you can help me with github... I'dl like to work on this superbundle using the last version of neo4j-community. The problem is that i've already forked this repo for my previous work and now I can't fork it anymore. Also, i've made my modifications on the master branch. Do you know a way of getting the last version of the source code without loosing my previous work ? I was thinking of checking out back to the time I've forked, then create a branch, then pull changes from the original repo. Thx. Le 14 juin 2011 à 09:03, Peter Neubauer a écrit : Nicolas, yes, after looking into the details involved, I think an official superbundle with the core Neo4j components bundled and exported woudl be the best way forward. Also, it would expose less granular bundles into an OSGi environment. Other IndexProviders etc could be then inserted as fragments into that bundle. It would be absolutely fantastic if you could work on that, maybe using the neo4j-osgi-examples as the demo project? We could then have a packaging project there or in a neo4j-osgi component that does the actual production of the superbundle. WDYT? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Mon, Jun 13, 2011 at 11:10 AM, Nicolas Jouanin nicolas.joua...@gmail.com wrote: Hi Peter, As shown below, exporting the implementation package is enough to make the IndexProvider registered as OSGi service (see first line of dump). Now an exception comes later when registering the index. I guess this come from the fact that you register a service of class IndexProvider whereas db.index().forNodes() returns an instance of Index. [Framework Event Dispatcher] INFO org.neo4j.kernel.impl.osgi.OSGiExtensionLoader - Bundle org.neo4j.lucene-index: Service org.neo4j.graphdb.index.IndexProvider registered with implementation org.neo4j.graphdb.index.IndexProvider[lucene] Kernel: attempting to load extensions of type org.neo4j.kernel.KernelExtension Kernel: attempting to load extensions of type org.neo4j.graphdb.index.IndexProvider Kernel: attempting to load extensions of type org.neo4j.kernel.Version [Start Level Event Dispatcher] DEBUG BuildByTinyBundlestinybundles904955895969775UID - ServiceEvent REGISTERED [Start Level Event Dispatcher] DEBUG BuildByTinyBundlestinybundles904955895969775UID - ServiceEvent UNREGISTERING [Framework Event Dispatcher] DEBUG BuildByTinyBundlestinybundles904955895969775UID - BundleEvent STOPPED [Framework Event Dispatcher] DEBUG BuildByTinyBundlestinybundles904955895969775UID - FrameworkEvent ERROR org.osgi.framework.BundleException: Exception in org.neo4j.examples.osgi.Neo4jActivator.start() of bundle BuildByTinyBundlestinybundles904955895969775UID. at org.eclipse.osgi.framework.internal.core.BundleContextImpl.startActivator(BundleContextImpl.java:806) at org.eclipse.osgi.framework.internal.core.BundleContextImpl.start(BundleContextImpl.java:755) at org.eclipse.osgi.framework.internal.core.BundleHost.startWorker(BundleHost.java:370) at org.eclipse.osgi.framework.internal.core.AbstractBundle.resume(AbstractBundle.java:374) at org.eclipse.osgi.framework.internal.core.Framework.resumeBundle(Framework.java:1067) at org.eclipse.osgi.framework.internal.core.StartLevelManager.resumeBundles(StartLevelManager.java:561) at org.eclipse.osgi.framework.internal.core.StartLevelManager.resumeBundles(StartLevelManager.java:546) at org.eclipse.osgi.framework.internal.core.StartLevelManager.incFWSL(StartLevelManager.java:459) at
[Neo4j] Most Efficient way to query in my use cases
Hello, I am using neo4j as database in my website. I am having problem in deciding best way to query the database. An user buys, lets say pen, from a shop. User and shop are the nodes and pen is also a node . Relationships are like User1-buysfrom-shop1 User1-uses-pen buysfrom and uses are relationships. Now I want to find all other users who buy pen from shop1. Some ways which I have thought to get the result. I start from user1 and move to shop1 node and from there I find all nodes with relationship buysfrom incoming to shop1. But it will give me all nodes who buys different stuff than pen. In that case I have to check all the hits if they uses pen by traversing from user to pen node if such relationship exist this user is my answer. In my use case if they use pen then they buy from shop1 thats for sure. Another way I can put an attribute pen in relationship buysfrom and index relationship using that attribute. But this will give me all the user who buys pen from all the shops. Again lots of checking have to de done again like for every hit I have to check if shop1 is end node of the hit If yes then start node is the answer I need. Now I need advice if above mentioned schemes are any good and doubts which I have regarding them are genuine or not? I am expecting minimum 40-50k users and lots of reads and writes in my system. I am sorry if this is a very big question at once but I am thinking a lot over these algos for past 1 month and need some advice how neo4j handles these things. -- Manav Goel ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] No index provider 'lucene' found
Ian, I used your fragments to rebuild your setup, please send a minimal functional application (it missed the entry-points and also the startup-html files) next time, that will help a lot in actually looking for the problem instead taking a lot of time setting it up. It turned out that your war/WEB-INF/lib didn't contain the libraries but the gwt (jetty) server requires them there for the service-location to work, other IDE's automatically setup the WEB-INF/lib directory with the libraries used. I don't know why eclipse and/or the gwt plugin don't do it. The simple fix was to provide all the libraries used in the war/WEB-INF/lib directory. So that issue had nothing to do with Neo4j's service lookup mechanism but just with the webapp library configuration of your project. Cheers Michael Am 12.06.2011 um 22:02 schrieb Ian Bussières: I was wondering if anybody had a workaround or fix to my issue. Otherwise, is there a new or other way to accomplish indexing, as that portion of work would take me very little time to recode. Thanks ! 2011/6/10 Ian Bussières ian.bussie...@gmail.com: Here you go, took out src, war, lib, test, test-classes and also cleaned some of the stuff in the war folder. Oh and took out all the .jar files there too, otherwise this was just way to huge to share. Hope this will shine some light on the problem :S On Fri, Jun 10, 2011 at 07:18, Michael Hunger michael.hun...@neotechnology.com wrote: Eclipse .project and .classpath files. In general if you copy the project and remove all *.java files, that would be sufficient. Thanks so much Michael Am 10.06.2011 um 13:11 schrieb Ian Bussières: GWT 2.3 and Eclipse Helios. I can't share too much information from the project itself, I suppose I could share the config stuff. What config are you referring to? -- Ian Bussières Étudiant en Génie des Technologies de l'Information École de technologie supérieure [514] 814-0136 On Fri, Jun 10, 2011 at 01:31, Michael Hunger michael.hun...@neotechnology.com wrote: Ian, can you share your project somehow at least config and the startup class? What are eclipse and gwt plugin versions? M Sent from my iBrick4 Am 10.06.2011 um 03:33 schrieb Ian Bussières ian.bussie...@gmail.com: The web server is slightly ambiguous, because it's a GWT eclipse plugin that is used to run. I do believe it to be a Tomcat server. I'm trying to find a source that confirms that. Nothing about OSGi, no... The messages.log looks ok, apart from a few non clean shutdowns but successful recoveries. Ian Bussières Étudiant en Génie des Technologies de l'Information École de technologie supérieure [514] 814-0136 On Thu, Jun 9, 2011 at 21:03, Michael Hunger michael.hun...@neotechnology.com wrote: Ian, is that web-server a stock tomcat or jetty or something running an OSGi environment? So do you see something about OSGi containers being started in your console output (felix, equinox)? Could you have a look at the messages.log file in your graphdb directory if it shows anything unusual? Thanks Michael Am 10.06.2011 um 02:49 schrieb Ian Bussières: New buildpath, same problem. C:\Users\Ian\Dev\PFE\pfe\src;C:\Users\Ian\Dev\PFE\pfe\lib;C:\Users\Ian\Dev\PFE\pfe\test-classes;C:\Users\Ian\Dev\PFE\pfe\war\WEB-INF\classes;C:\Program Files (x86)\Eclipse\plugins\com.google.gwt.eclipse.sdkbundle_2.3.0.r36v201104261928\gwt-2.3.0\gwt-user.jar;C:\Program Files (x86)\Eclipse\plugins\com.google.gwt.eclipse.sdkbundle_2.3.0.r36v201104261928\gwt-2.3.0\gwt-dev.jar;C:\Program Files (x86)\Eclipse\plugins\com.google.gwt.eclipse.sdkbundle_2.3.0.r36v201104261928\gwt-2.3.0\validation-api-1.0.0.GA-sources.jar;C:\Program Files (x86)\Eclipse\plugins\com.google.gwt.eclipse.sdkbundle_2.3.0.r36v201104261928\gwt-2.3.0\validation-api-1.0.0.GA.jar;C:\Users\Ian\Dev\PFE\pfe\lib\geronimo-jta_1.1_spec-1.1.1.jar;C:\Users\Ian\Dev\PFE\pfe\lib\neo4j-community-1.4.M03.jar;C:\Users\Ian\Dev\PFE\pfe\lib\neo4j-graph-algo-1.4.M03.jar;C:\Users\Ian\Dev\PFE\pfe\lib\neo4j-jmx-1.4.M03.jar;C:\Users\Ian\Dev\PFE\pfe\lib\neo4j-kernel-1.4.M03.jar;C:\Users\Ian\Dev\PFE\pfe\lib\neo4j-lucene-index-1.4.M03.jar;C:\Users\Ian\Dev\PFE\pfe\lib\lucene-core-3.1.0.jar;C:\Users\Ian\Dev\PFE\pfe\lib\org.apache.servicemix.bundles.jline-0.9.94_1.jar;C:\Users\Ian\Dev\PFE\pfe\lib\neo4j-shell-1.4.M03.jar;C:\Users\Ian\Dev\PFE\pfe\lib\neo4j-udc-1.4.M03.jar;C:\Users\Ian\Dev\PFE\pfe\lib\server-api-1.4.M03.jar On Thu, Jun 9, 2011 at 20:32, Michael Hunger michael.hun...@neotechnology.com wrote: Can you please try to remove the lucene-core-3.0.1. It is possible that mixing two versions of lucene at the classpath messes things up. Cheers Michael Am 10.06.2011 um 02:27 schrieb Ian Bussières: Definitely on server side code. I start up the application directly in eclipse with the built in web server.
[Neo4j] Spring Data Graph 1.1.0.M1 milestone released
Dear Graphistas,
After some quiet time that I spend on other important issues, we're back on our
regular release cycle/rhythm.
We discussed the future SDG roadmap with our CEO Emil and decided to work first
on the issues that were
reported by our users here on the mailing list, on the spring forums and on
twitter.
So primarily we updated the dependencies to Neo4j 1.4.M04 and AspectJ
(1.6.12.M1), the latter fixing some issues.
With the new Neo4j release we are also able to start working some important new
features.
The most interesting one is the adoption of the query language cypher which
is now accessible from NodeEntities (dynamic annotated fields, introduced
methods) and repositories (annotated repository interface methods).
(https://jira.springsource.org/browse/DATAGRAPH-91)
Example, NodeEntity-field:
@GraphQuery(start me=(%d) match
(me)-[:works_at]-(company)-[:works_at]-(colleague) return colleague)
IterablePerson colleagues;
Example, Repository:
interface PersonRepository extends GraphRepositoryPerson {
@GraphQuery(start person=(%d) match (person)-[:boss]-(boss) return boss)
Person findBoss(Person developer);
}
We also accommodate for the change of allowing self-relationships.
Another annoyance for SDG users - the need to specify an elementClass on
@RelatedTo annotations has also been removed this change will also be added to
the other annotations that we use.
We would like to get your feedback on those changes and also on Spring Data
Graph in general, so please check it out and get back to us.
And please spread the word:
http://twitter.com/#!/neo4j/statuses/80503228319547392
Cheers
Michael
Announcement: http://www.springsource.org/node/3152
JIRA: https://jira.springsource.org/browse/DATAGRAPH
Project Page: http://www.springsource.org/spring-data/neo4j
Forum: forum.springsource.org/forumdisplay.php?f=80
Github: http://github.com/springsource/spring-data-graph
Download:
http://www.springsource.com/download/community?project=Spring%20Data%20Neo4j
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Maintaining a subreference for relationships
2011/5/28 Chris Baranowski pharcos...@gmail.com Nice, yeah that's what I was thinking too. So then did you bother with subreferences for node types or just keep your approach consistent by using indexes for nodes as well? Also, iterating over the values of an index is a little more cumbersome than iterating over edges, namely because you have to call close() on the IndexHits object. I'd like to not force the users of my neo4j wrapper API to call close(), any advice? Then I think you'd have to gather the results up front, i.e. loop through the IndexHits instance (making it close itself) and return the entire result as a list. It may not be desirable to take that hit of looping through the result before even the first one can be returned though. Thanks! Chris ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Extent of Indexing
Peter, The project requires quick retrieval of data bases upon certain parameters, which, without indexing, would not be feasible (as the relations that hold the data are all same, so simple traversals won't work). As I said earlier, I would have to extract data based upon combination of some parameters, which I could easily do by indexing the same data on different parameters and taking advantage of the lucene's queryparser. Regarding those 50k users, they will be making much use of the C and R or the CRUD (Reads will be the most used though). I estimate that at any time, 30-50% of them would be using the project. Suppose a user generates 20 new nodes and 20 new relationships (not relationship types) per day. I would not index the data that they're posting, but the node number, so that I get to node with less memory usage. That seems efficient to me because I may make use of more number of nodes, but I get a smaller Index. To index a node X ( with some data in it), I can index node Y (empty) that has a direct relation with X. The nodes are getting exhausted, but atleast it gives me a smaller and faster index (and I have virtually unlimited number of nodes with Neo4j). (I know this scheme might seem a little vague because I am wasting nodes, and someday when the scalability factor kicks in, I might have to rectify this!) Any suggestions regarding the same? Also, my database would need store mostly strings. What about putting up another layer in front of my neo4j db that maps those strings to ints so that I could index those easily in neo4j? Example of this can be: This addictional layer can map an email address (which I need to index) to a unique user id which I can index using Neo4j ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST API LuceneTimeline
Matt, what URL are you using to create the timeline? I don't recall that being exposed via REST ... Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Mon, May 30, 2011 at 11:16 PM, Matt Luongo m...@scholr.ly wrote: Forgive me if this is a foolish question. I'm really excited about using the LuceneTimeline in our product. I see that I can create it via type:timeline in the REST interface. I'm not sure, though, how to add nodes to the timeline- it seems like you add things to a timeline by specifying just a timestamp, not key/value, but I can only find REST examples for indexing nodes by key/value. Anyone care to share a code snippet? -- Matt Luongo Co-Founder, Scholr.ly ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] BatchInserter exhausting heap...?
Could you perhaps fire up VisualVM http://visualvm.java.net/ and ask it to
sample the heap while running? It will tell you exactly what type of objects
are allocated the most and how much heap everything takes. Any profiler
would suffice, but I really like that one.
2011/6/2 Paul Bandler pband...@cseuk.co.uk
Contrary to what I indicated earlier, the program is exhausting the heap.
So far I have:-
Tried significantly reducing the default parameters as below:-
Physical mem: 1535MB, Heap size: 496MB
use_memory_mapped_buffers=false
neostore.propertystore.db.index.keys.mapped_memory=1M
neostore.propertystore.db.strings.mapped_memory=30M
neostore.propertystore.db.arrays.mapped_memory=5M
neo_store=c:\neo4j-advanced-1.3\data\graph.db\neostore
neostore.relationshipstore.db.mapped_memory=10M
neostore.propertystore.db.index.mapped_memory=1M
neostore.propertystore.db.mapped_memory=20M
dump_configuration=true
cache_type=weak
neostore.nodestore.db.mapped_memory=10M
Split my data reading code into small selects (thinking that maybe the JDBC
driver was accumulating memory)
Neither has made a noticeable difference – it runs out of memory after
loading about the same amount of data.
I’m trying to do a relatively simple thing – write a batch importer that
loads data one row at a time and writes one node at a time. I don’t want
anything cached. It would seem reasonable for the default configuration
when using the BatchInserter would facilitate this but so far it’s not been
possible.
Any further suggestions folks?
Regards,
Paul Bandler
Sent from my iPhone
On 2 Jun 2011, at 08:53, Paul Bandler pband...@cseuk.co.uk wrote:
I monitored the heap using jconsole and much to my surprise observed that
the heap stayed relatively stable while the overall memory occupancy of the
process grew steadily until it reached the ~500M . I'm now rather confused
as to what else can be consuming memory like that Any ideas folks?
Sent from my iPhone
On 1 Jun 2011, at 20:52, Michael Hunger
michael.hun...@neotechnology.com wrote:
props passed in to the batchinserter
look into messages log
you see the different gc behaviour
Michael
Sent from my iBrick4
Am 01.06.2011 um 20:44 schrieb Paul Bandler pband...@cseuk.co.uk:
Is that simply set as a system property or via the Map passed as the
second parameter to the BatchInserterImpl constructor? I've tried both and
doesn't seem to help. Is there some way I can verify that it's being used?
I'm using 1.3
On 1 Jun 2011, at 18:49, Michael Hunger wrote:
you could use cache_type=weak
in the db properties
you can easily introspect java programs (heap) using jmap jconsole or
visualvm
what version of neo4j are you using?
index.flush just sets a flag for immediate index querying
Sent from my iBrick4
Am 01.06.2011 um 19:18 schrieb Paul Bandler pband...@cseuk.co.uk:
I have a simple program that uses the BatchInserter to load rows from
a SQL database and am running it on a modestly configured Windows machine
with 2GB of RAM and setting the max heap to 500M.
Initially it was running out of memory quite soon so I introduced a
flush after every 5000 nodes and it appeared that all was well. But having
got further in the data load it appears to hop along nicely but the memory
allocated (simply visible using windows task manager) grows and grows until
I suspect it's reached its max heap size and it's written about 2M nodes
then abruptly stops making any further discernible progress. It doesn't
fail, just the logging I've put in to log every 5000 nodes has stopped and
the CPU is 100% used - garbage collecting I suspect.
Is there something I should be doing periodically in addition to the
index flush to stop the heap exhaustion? My code is really simple,
here's the method for loading nodes from each table:-
public long restoreCollection() {
resolveSql();
_log.debug(restore collection: + getCollectionName() + using:
+ _sql + and: + Arrays.deepToString(_columns));
final BatchInserterIndex _index = makeIndex();
final long collectionNode =
_inserter.createNode(MapUtil.map(name,
getCollectionName()));
_log.debug(Query db...);
getJdbcTemplate().query(_sql, new Object[] {},
new RowCallbackHandler() {
public void processRow(ResultSet row) throws
SQLException {
final MapString, Object properties =
extractproperties(row);
long node = _inserter.createNode(properties);
_inserter.createRelationship(node, collectionNode,
RdmRelationship.MEMBER_OF, null);
if (_index != null)
for (DbColumn col : _columns) {
if (col.isIndexed())
_index.add(node,
MapUtil.map(col.getName(),
properties.get(col.getName(;
Re: [Neo4j] In-graph Timeline index and Neo4j 1.4
2011/6/3 Rick Bullotta rick.bullo...@thingworx.com Alternatively, if we could have the composite index functionality I described in a few previous e-mails (mix timestamp, textual and other numeric key/values in the same index) - e.g. a Lucene timeline index with extra keys, that might work well also, would it not? That is possible using a lucene index as it is today afaik. Just get inspiration from LuceneTimeline for the timestamp parts and fire away! -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Niels Hoogeveen Sent: Friday, June 03, 2011 2:44 PM To: user@lists.neo4j.org Subject: [Neo4j] In-graph Timeline index and Neo4j 1.4 Today, I tried to migrate my application from Neo4j 1.3 to 1.4M03 and ran into problems with respect to the in-graph Timeline index in the legacy component Neo4j-index. For all Lucene related indexing, I have moved to greener pastures and use the new indexing framework, but for several indexing needs only an in-graph index is suitable. Examples:1) Most of the nodes in my application have versioning enabled. To do so, I maintain a in-graph Timeline index containing version nodes. The Timeline index is needed to maintain order and to register a timestamp for each version. 2) Most of the nodes in my application are related to a context. Every user or user group maintains two or more contexts. The relationship between node and context is again stored in the Timeline index, to make it possible to retrieve the most recent additions for a user or user group. Both scenarios can potentially create a huge number of indexes, most of them relatively small, but some become large enough that in-memory sorting is not an option. The in-graph Timeline index offers the right functionality for these scenarios and the Lucene index service is not a feasible replacement in these cases. The in-graph Timeline index is now fixed to version Neo4j 1.3, and given the legacy Lucene code in that component will not likely be upgraded to version 1.4. Using Neo4j-index 1.3-SNAPSHOT with Neo4j 1.4M03 is not possible without hacking the POM (which I have done, but don't feel too happy about). Neo4j-index 1.3-SNAPSHOT requires Lucene 3.0.1, while Neo4j 1.4M03 requires Lucene 3.1.0, leading to version conflicts in projects. Approximately a month ago, I made the suggestion (see: http://lists.neo4j.org/pipermail/user/2011-May/008461.html) to move the in-graph Btree index and its related classes (including Timeline) to a new component Neo4j-collections, while keeping the old Lucene index stuff in Neo4j-index, so it can eventually become deprecated. I hope my suggestion will be taken into consideration. Kind regards,Niels Hoogeveen ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Traversing nodes using Indexed relationship's attribute
Hello, I am using neo4j as database in my website. I am having problem in deciding best way to query the database. An user buys, lets say pen, from a shop. User and shop are the nodes and pen is also a node . Relationships are like User1-buysfrom-shop1 User1-uses-pen buysfrom and uses are relationships. Now I want to find all other users who buy pen from shop1. Some ways which I have thought to get the result. I start from user1 and move to shop1 node and from there I find all nodes with relationship buysfrom incoming to shop1. But it will give me all nodes who buys different stuff than pen. In that case I have to check all the hits if they uses pen by traversing from user to pen node if such relationship exist this user is my answer. In my use case if they use pen then they buy from shop1 thats for sure. Another way I can put an attribute pen in relationship buysfrom and index relationship using that attribute. But this will give me all the user who buys pen from all the shops. Again lots of checking have to de done again like for every hit I have to check if shop1 is end node of the hit If yes then start node is the answer I need. Now I need advice if above mentioned schemes are any good and doubts which I have regarding them are genuine or not? I am expecting minimum 40-50k users and lots of reads and writes in my system. I am sorry if this is a very big question at once but I am thinking a lot over these algos for past 1 month and need some advice how neo4j handles these things. -- Manav Goel ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] In-graph Timeline index and Neo4j 1.4
Done! - Reply message - From: Mattias Persson matt...@neotechnology.com Date: Wed, Jun 15, 2011 7:14 am Subject: [Neo4j] In-graph Timeline index and Neo4j 1.4 To: Neo4j user discussions user@lists.neo4j.org 2011/6/3 Rick Bullotta rick.bullo...@thingworx.com Alternatively, if we could have the composite index functionality I described in a few previous e-mails (mix timestamp, textual and other numeric key/values in the same index) - e.g. a Lucene timeline index with extra keys, that might work well also, would it not? That is possible using a lucene index as it is today afaik. Just get inspiration from LuceneTimeline for the timestamp parts and fire away! -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Niels Hoogeveen Sent: Friday, June 03, 2011 2:44 PM To: user@lists.neo4j.org Subject: [Neo4j] In-graph Timeline index and Neo4j 1.4 Today, I tried to migrate my application from Neo4j 1.3 to 1.4M03 and ran into problems with respect to the in-graph Timeline index in the legacy component Neo4j-index. For all Lucene related indexing, I have moved to greener pastures and use the new indexing framework, but for several indexing needs only an in-graph index is suitable. Examples:1) Most of the nodes in my application have versioning enabled. To do so, I maintain a in-graph Timeline index containing version nodes. The Timeline index is needed to maintain order and to register a timestamp for each version. 2) Most of the nodes in my application are related to a context. Every user or user group maintains two or more contexts. The relationship between node and context is again stored in the Timeline index, to make it possible to retrieve the most recent additions for a user or user group. Both scenarios can potentially create a huge number of indexes, most of them relatively small, but some become large enough that in-memory sorting is not an option. The in-graph Timeline index offers the right functionality for these scenarios and the Lucene index service is not a feasible replacement in these cases. The in-graph Timeline index is now fixed to version Neo4j 1.3, and given the legacy Lucene code in that component will not likely be upgraded to version 1.4. Using Neo4j-index 1.3-SNAPSHOT with Neo4j 1.4M03 is not possible without hacking the POM (which I have done, but don't feel too happy about). Neo4j-index 1.3-SNAPSHOT requires Lucene 3.0.1, while Neo4j 1.4M03 requires Lucene 3.1.0, leading to version conflicts in projects. Approximately a month ago, I made the suggestion (see: http://lists.neo4j.org/pipermail/user/2011-May/008461.html) to move the in-graph Btree index and its related classes (including Timeline) to a new component Neo4j-collections, while keeping the old Lucene index stuff in Neo4j-index, so it can eventually become deprecated. I hope my suggestion will be taken into consideration. Kind regards,Niels Hoogeveen ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.comhttp://www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Online Backup to local disk
Rick, there is always a network transfer going on, but of course in the localhost case this is over loopback. Any particular reason you only want file operations to be involved? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Mon, Jun 13, 2011 at 6:52 PM, Rick Bullotta rick.bullo...@thingworx.com wrote: How can we use online backup in Neo4J to do a hot backup to another local disk? It seems that the only way to instantiate the OnlineBackup class is something like: OnlineBackup.from(localhost), but that would seem to imply that it will be doing network communications to do the backup, vs just doing it via file system operations. Is there a better/alternative way to achieve this for a local hot backup? Thanks, Rick ThingWorx ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] How to combine both traversing and index queries?
McKinley, we are just in the process of adding index hits as starting points to Cypher, which will then do exactly what you are asking. No ETA yet, but that is high on the list and will show up in the docs :) Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Fri, Jun 10, 2011 at 9:25 PM, McKinley mckinley1...@gmail.com wrote: Mattias, Is this now possible as of today's milestone release? Thanks to you and the rest of the team for adding this! http://docs.neo4j.org/chunked/snapshot/query-start.html Every query describes a pattern, and in that pattern one can have *multiple bound points*. A bound point is a relationship or a node that form the *starting points* for a pattern match. You can either *bind points by id, or by index lookups*. Cheers, McKinley 2011/4/18 Mattias Persson mattias at neotechnology.com: * Traversing all friends within two steps from the reference node is** trivial, but I find it a bit inefficient to apply a return evaluator** in each of the nodes visited during traversal. Or is it so? How about** more complex criteria which may involve more than one property or even** more complex (Lucene) queries?*** The best solution IMHO (one that isn't available yet) would be to let a traversal have multiple starting points, that is have the index result as starting point. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] How to combine both traversing and index queries?
On Wed, Jun 15, 2011 at 1:51 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: McKinley, we are just in the process of adding index hits as starting points to Cypher, which will then do exactly what you are asking. No ETA yet, but that is high on the list and will show up in the docs :) The ETA is last week. :) You can already do this: http://docs.neo4j.org/chunked/1.4.M04/query-start.html It's only for exact index searches though. Do you mean free form lucene query? Andrés ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cannot find a path
Peter, Thanks, that is all very useful for me. For future reference, is it safe to assume that all multi-word data properties (default_cost, cost_property, etc.) will be _ separated? Or will some be separated, like max depth? Is there a plan to standardize? -- Josh Message: 5 Date: Wed, 15 Jun 2011 13:44:01 +0200 From: Peter Neubauer peter.neuba...@neotechnology.com Subject: Re: [Neo4j] Cannot find a path To: Neo4j user discussions user@lists.neo4j.org Message-ID: banlktikyt0ljqmp5raucjgtpsymw_qh...@mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 Josh, we documented a bit more of the functional test to be included in the docs, see http://docs.neo4j.org/chunked/snapshot/rest-api-graph-algos.html for examples of Dijkstra also. Cheers, /peter neubauer GTalk:? ? ? neubauer.peter Skype? ? ?? peter.neubauer Phone? ? ?? +46 704 106975 LinkedIn?? http://www.linkedin.com/in/neubauer Twitter? ? ? http://twitter.com/peterneubauer http://www.neo4j.org? ? ? ? ? ? ?? - Your high performance graph database. http://startupbootcamp.org/ ? ?- ?resund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, Jun 15, 2011 at 8:36 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Josh, the docs are now at http://docs.neo4j.org/chunked/snapshot/rest-api-graph-algos.html Let me know if you want to see more, I am planning to put in an example of Dijkstra, too. Cheers, /peter neubauer GTalk:? ? ? neubauer.peter Skype? ? ?? peter.neubauer Phone? ? ?? +46 704 106975 LinkedIn?? http://www.linkedin.com/in/neubauer Twitter? ? ? http://twitter.com/peterneubauer http://www.neo4j.org? ? ? ? ? ? ?? - Your high performance graph database. http://startupbootcamp.org/ ? ?- ?resund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Slow Traversals on Nodes with too many Relationships
I have a few Part nodes related with each via HASPART relationship/edges. (eg Part1---HASPART---Part2---HASPART---Part3 etc) . TraversalDescription works fine, following each Part's outgoing HASPART relationship. Then I add a large number (say 100.000) of Container Nodes, where each Container has a CONTAINS relation to almost *every* Part node. Hence each Part node now has a 100.000 incoming CONTAINS relationships from Container nodes, but only a few outgoing HASPART relationships to other Part nodes. Now my previous TraversalDescription run extremely slow (several seconds inside each IteratorPath.next() call) Note that I do define relationships(RT.HASPART, Direction.OUTGOING) on the TraversalDescription, but it seems its not used by neo4j as a hint. Note that on a subsequent run of the same Traversal, its very quick indeed. Is there any way to use Indexing on relationships for such a scenario, to boost things up ? Ideally, the Traversal framework could use automatic/declerative indexing on Node Relationship types and/or direction to perform such traversals quicker. Regards ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST API LuceneTimeline
Peter,
I believe it was something like
curl -X POST -H Accept:application/json -HContent-Type:application/json -d \
'{name:time, config:{type:timeline,provider:lucene}}' \
http://localhost:7474/db/data/index/node
curl -HContent-Type:application/json -X POST -d \
'http://localhost:7474/db/data/node/0;' \
http://localhost:7474/db/data/index/node/time/timestamp/25
which gives a 500.
--
Matt Luongo
Co-Founder, Scholr.ly
On Wed, Jun 15, 2011 at 8:55 AM, Rick Bullotta
rick.bullo...@thingworx.com wrote:
You really don't need a timeline index to do a timeline index, if that makes
sense. As mattias points out, it is just a convenient wrapper around a plain
lucene (neo) index. Just index the timestamp as a numeric field.
- Reply message -
From: Matt Luongo m...@scholr.ly
Date: Wed, Jun 15, 2011 8:48 am
Subject: [Neo4j] REST API LuceneTimeline
To: Neo4j user discussions user@lists.neo4j.org
Peter,
I've done a bit more research, and you're right- it isn't. However, the REST
api let's me create an index of type 'timeline'. The first time I try to
index a node, there's a 500 error (nullpointer) that stems from the index
not being created. I'd be happy to file a bug report.
Anyway, we're trying to put together a plugin to expose timeline indexing
via REST.
- Matt
On Jun 15, 2011 5:49 AM, Peter Neubauer peter.neuba...@neotechnology.com
wrote:
Matt,
what URL are you using to create the timeline? I don't recall that
being exposed via REST ...
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Mon, May 30, 2011 at 11:16 PM, Matt Luongo m...@scholr.ly wrote:
Forgive me if this is a foolish question. I'm really excited about using
the
LuceneTimeline in our product. I see that I can create it via
type:timeline in the REST interface. I'm not sure, though, how to add
nodes to the timeline- it seems like you add things to a timeline by
specifying just a timestamp, not key/value, but I can only find REST
examples for indexing nodes by key/value. Anyone care to share a code
snippet?
--
Matt Luongo
Co-Founder, Scholr.ly
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST API LuceneTimeline
Rick, Is there a sensible way to do that while accessing the index via REST? I'd prefer if I could create *and* access the index via the REST interface, but just querying and inserting into the index would be great. -- Matt Luongo Co-Founder, Scholr.ly On Wed, Jun 15, 2011 at 8:55 AM, Rick Bullotta rick.bullo...@thingworx.com wrote: You really don't need a timeline index to do a timeline index, if that makes sense. As mattias points out, it is just a convenient wrapper around a plain lucene (neo) index. Just index the timestamp as a numeric field. - Reply message - From: Matt Luongo m...@scholr.ly Date: Wed, Jun 15, 2011 8:48 am Subject: [Neo4j] REST API LuceneTimeline To: Neo4j user discussions user@lists.neo4j.org Peter, I've done a bit more research, and you're right- it isn't. However, the REST api let's me create an index of type 'timeline'. The first time I try to index a node, there's a 500 error (nullpointer) that stems from the index not being created. I'd be happy to file a bug report. Anyway, we're trying to put together a plugin to expose timeline indexing via REST. - Matt On Jun 15, 2011 5:49 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Matt, what URL are you using to create the timeline? I don't recall that being exposed via REST ... Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Mon, May 30, 2011 at 11:16 PM, Matt Luongo m...@scholr.ly wrote: Forgive me if this is a foolish question. I'm really excited about using the LuceneTimeline in our product. I see that I can create it via type:timeline in the REST interface. I'm not sure, though, how to add nodes to the timeline- it seems like you add things to a timeline by specifying just a timestamp, not key/value, but I can only find REST examples for indexing nodes by key/value. Anyone care to share a code snippet? -- Matt Luongo Co-Founder, Scholr.ly ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Possible bug/omission in LuceneTimeline?
Correct, there's nothing special at all with a timeline. It just works its magic with ValueContext/QueryContext for convenience! 2011/6/9 Rick Bullotta rick.bullo...@thingworx.com Doh. Forget it. Grokking the code a bit and I see how LuceneTimeline is just a wrapper around a regular Index. Nevermind. ;-) -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Rick Bullotta Sent: Thursday, June 09, 2011 2:41 PM To: Neo4j user discussions Subject: [Neo4j] Possible bug/omission in LuceneTimeline? Importance: High One big issue we've encountered (actually, a fairly critical one) is that there does not seem to be any way to *delete* a LuceneTimeline index. This raises a more general question: - Is LuceneTimeline part of the IndexFramework or is it an experimental piece of code - Is there a way to achieve the same functionality of the LuceneTimeline using the normal Neo4J index framework? Thanks in advance for any help! ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST API LuceneTimeline
I guess the other question is how I might accomplish range queries, using REST. -- Matt Luongo Co-Founder, Scholr.ly On Wed, Jun 15, 2011 at 9:05 AM, Matt Luongo m...@scholr.ly wrote: Rick, Is there a sensible way to do that while accessing the index via REST? I'd prefer if I could create *and* access the index via the REST interface, but just querying and inserting into the index would be great. -- Matt Luongo Co-Founder, Scholr.ly On Wed, Jun 15, 2011 at 8:55 AM, Rick Bullotta rick.bullo...@thingworx.com wrote: You really don't need a timeline index to do a timeline index, if that makes sense. As mattias points out, it is just a convenient wrapper around a plain lucene (neo) index. Just index the timestamp as a numeric field. - Reply message - From: Matt Luongo m...@scholr.ly Date: Wed, Jun 15, 2011 8:48 am Subject: [Neo4j] REST API LuceneTimeline To: Neo4j user discussions user@lists.neo4j.org Peter, I've done a bit more research, and you're right- it isn't. However, the REST api let's me create an index of type 'timeline'. The first time I try to index a node, there's a 500 error (nullpointer) that stems from the index not being created. I'd be happy to file a bug report. Anyway, we're trying to put together a plugin to expose timeline indexing via REST. - Matt On Jun 15, 2011 5:49 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Matt, what URL are you using to create the timeline? I don't recall that being exposed via REST ... Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Mon, May 30, 2011 at 11:16 PM, Matt Luongo m...@scholr.ly wrote: Forgive me if this is a foolish question. I'm really excited about using the LuceneTimeline in our product. I see that I can create it via type:timeline in the REST interface. I'm not sure, though, how to add nodes to the timeline- it seems like you add things to a timeline by specifying just a timestamp, not key/value, but I can only find REST examples for indexing nodes by key/value. Anyone care to share a code snippet? -- Matt Luongo Co-Founder, Scholr.ly ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cannot find a path
Josh, well, a while back we changed the whole REST API not to have whitespace in any keys, so it is safe to assume _ for whitespace, and report as bugs otherwise :) Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, Jun 15, 2011 at 2:45 PM, Josh Adell josh.ad...@gmail.com wrote: Peter, Thanks, that is all very useful for me. For future reference, is it safe to assume that all multi-word data properties (default_cost, cost_property, etc.) will be _ separated? Or will some be separated, like max depth? Is there a plan to standardize? -- Josh Message: 5 Date: Wed, 15 Jun 2011 13:44:01 +0200 From: Peter Neubauer peter.neuba...@neotechnology.com Subject: Re: [Neo4j] Cannot find a path To: Neo4j user discussions user@lists.neo4j.org Message-ID: banlktikyt0ljqmp5raucjgtpsymw_qh...@mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 Josh, we documented a bit more of the functional test to be included in the docs, see http://docs.neo4j.org/chunked/snapshot/rest-api-graph-algos.html for examples of Dijkstra also. Cheers, /peter neubauer GTalk:? ? ? neubauer.peter Skype? ? ?? peter.neubauer Phone? ? ?? +46 704 106975 LinkedIn?? http://www.linkedin.com/in/neubauer Twitter? ? ? http://twitter.com/peterneubauer http://www.neo4j.org? ? ? ? ? ? ?? - Your high performance graph database. http://startupbootcamp.org/ ? ?- ?resund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, Jun 15, 2011 at 8:36 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Josh, the docs are now at http://docs.neo4j.org/chunked/snapshot/rest-api-graph-algos.html Let me know if you want to see more, I am planning to put in an example of Dijkstra, too. Cheers, /peter neubauer GTalk:? ? ? neubauer.peter Skype? ? ?? peter.neubauer Phone? ? ?? +46 704 106975 LinkedIn?? http://www.linkedin.com/in/neubauer Twitter? ? ? http://twitter.com/peterneubauer http://www.neo4j.org? ? ? ? ? ? ?? - Your high performance graph database. http://startupbootcamp.org/ ? ?- ?resund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST API LuceneTimeline
Peter,
Kind of. I see how I could do that server-side, but not how to insert
elements into the index numerically (with
ValueContext(val).indexNumeric()) or query by range (
.query(QueryContext.numericRange() ) via REST. Is this something
that will require our writing a plugin to support, or is there
currently some way to index and query numerically via the REST API?
--
Matt Luongo
Co-Founder, Scholr.ly
On Wed, Jun 15, 2011 at 9:25 AM, Peter Neubauer
neubauer.pe...@gmail.com wrote:
Yeah,
just checked with Mattias. There is no such index configuration, so
this does not work. I think you should use the numeric field in a
normal index for this, just like Rick points out.
Does that make sense?
/peter
On Wed, Jun 15, 2011 at 3:03 PM, Matt Luongo m...@scholr.ly wrote:
Peter,
I believe it was something like
curl -X POST -H Accept:application/json -HContent-Type:application/json -d \
'{name:time, config:{type:timeline,provider:lucene}}' \
http://localhost:7474/db/data/index/node
curl -HContent-Type:application/json -X POST -d \
'http://localhost:7474/db/data/node/0;' \
http://localhost:7474/db/data/index/node/time/timestamp/25
which gives a 500.
--
Matt Luongo
Co-Founder, Scholr.ly
On Wed, Jun 15, 2011 at 8:55 AM, Rick Bullotta
rick.bullo...@thingworx.com wrote:
You really don't need a timeline index to do a timeline index, if that
makes sense. As mattias points out, it is just a convenient wrapper around
a plain lucene (neo) index. Just index the timestamp as a numeric field.
- Reply message -
From: Matt Luongo m...@scholr.ly
Date: Wed, Jun 15, 2011 8:48 am
Subject: [Neo4j] REST API LuceneTimeline
To: Neo4j user discussions user@lists.neo4j.org
Peter,
I've done a bit more research, and you're right- it isn't. However, the REST
api let's me create an index of type 'timeline'. The first time I try to
index a node, there's a 500 error (nullpointer) that stems from the index
not being created. I'd be happy to file a bug report.
Anyway, we're trying to put together a plugin to expose timeline indexing
via REST.
- Matt
On Jun 15, 2011 5:49 AM, Peter Neubauer peter.neuba...@neotechnology.com
wrote:
Matt,
what URL are you using to create the timeline? I don't recall that
being exposed via REST ...
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Mon, May 30, 2011 at 11:16 PM, Matt Luongo m...@scholr.ly wrote:
Forgive me if this is a foolish question. I'm really excited about using
the
LuceneTimeline in our product. I see that I can create it via
type:timeline in the REST interface. I'm not sure, though, how to add
nodes to the timeline- it seems like you add things to a timeline by
specifying just a timestamp, not key/value, but I can only find REST
examples for indexing nodes by key/value. Anyone care to share a code
snippet?
--
Matt Luongo
Co-Founder, Scholr.ly
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST API LuceneTimeline
Hi, Matt.
We don't use the REST API (we use Neo4J in an embedded mode), so I can't be of
much help there. If the REST API supports Lucene query syntax, you can use
something like:
timestamp : [1 to 2]
You'd need to do the Date - UTC milliseconds conversion on the front end, I'd
think.
Rick
-Original Message-
From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On
Behalf Of Matt Luongo
Sent: Wednesday, June 15, 2011 10:03 AM
To: Peter Neubauer
Cc: user@lists.neo4j.org
Subject: Re: [Neo4j] REST API LuceneTimeline
Peter,
Kind of. I see how I could do that server-side, but not how to insert
elements into the index numerically (with
ValueContext(val).indexNumeric()) or query by range (
.query(QueryContext.numericRange() ) via REST. Is this something
that will require our writing a plugin to support, or is there
currently some way to index and query numerically via the REST API?
--
Matt Luongo
Co-Founder, Scholr.ly
On Wed, Jun 15, 2011 at 9:25 AM, Peter Neubauer
neubauer.pe...@gmail.com wrote:
Yeah,
just checked with Mattias. There is no such index configuration, so
this does not work. I think you should use the numeric field in a
normal index for this, just like Rick points out.
Does that make sense?
/peter
On Wed, Jun 15, 2011 at 3:03 PM, Matt Luongo m...@scholr.ly wrote:
Peter,
I believe it was something like
curl -X POST -H Accept:application/json -HContent-Type:application/json -d \
'{name:time, config:{type:timeline,provider:lucene}}' \
http://localhost:7474/db/data/index/node
curl -HContent-Type:application/json -X POST -d \
'http://localhost:7474/db/data/node/0;' \
http://localhost:7474/db/data/index/node/time/timestamp/25
which gives a 500.
--
Matt Luongo
Co-Founder, Scholr.ly
On Wed, Jun 15, 2011 at 8:55 AM, Rick Bullotta
rick.bullo...@thingworx.com wrote:
You really don't need a timeline index to do a timeline index, if that
makes sense. As mattias points out, it is just a convenient wrapper around
a plain lucene (neo) index. Just index the timestamp as a numeric field.
- Reply message -
From: Matt Luongo m...@scholr.ly
Date: Wed, Jun 15, 2011 8:48 am
Subject: [Neo4j] REST API LuceneTimeline
To: Neo4j user discussions user@lists.neo4j.org
Peter,
I've done a bit more research, and you're right- it isn't. However, the REST
api let's me create an index of type 'timeline'. The first time I try to
index a node, there's a 500 error (nullpointer) that stems from the index
not being created. I'd be happy to file a bug report.
Anyway, we're trying to put together a plugin to expose timeline indexing
via REST.
- Matt
On Jun 15, 2011 5:49 AM, Peter Neubauer peter.neuba...@neotechnology.com
wrote:
Matt,
what URL are you using to create the timeline? I don't recall that
being exposed via REST ...
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Mon, May 30, 2011 at 11:16 PM, Matt Luongo m...@scholr.ly wrote:
Forgive me if this is a foolish question. I'm really excited about using
the
LuceneTimeline in our product. I see that I can create it via
type:timeline in the REST interface. I'm not sure, though, how to add
nodes to the timeline- it seems like you add things to a timeline by
specifying just a timestamp, not key/value, but I can only find REST
examples for indexing nodes by key/value. Anyone care to share a code
snippet?
--
Matt Luongo
Co-Founder, Scholr.ly
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Slow Traversals on Nodes with too many Relationships
That is rather a case of warming up your caches. Determining the traversal speed from the first run is not a good benchmark as it doesn't represent production usage :) The same (warming up) is true for all kinds of benchmarks (except for startup performance benchmarks). Cheers Michael Am 15.06.2011 um 14:48 schrieb Agelos Pikoulas: I have a few Part nodes related with each via HASPART relationship/edges. (eg Part1---HASPART---Part2---HASPART---Part3 etc) . TraversalDescription works fine, following each Part's outgoing HASPART relationship. Then I add a large number (say 100.000) of Container Nodes, where each Container has a CONTAINS relation to almost *every* Part node. Hence each Part node now has a 100.000 incoming CONTAINS relationships from Container nodes, but only a few outgoing HASPART relationships to other Part nodes. Now my previous TraversalDescription run extremely slow (several seconds inside each IteratorPath.next() call) Note that I do define relationships(RT.HASPART, Direction.OUTGOING) on the TraversalDescription, but it seems its not used by neo4j as a hint. Note that on a subsequent run of the same Traversal, its very quick indeed. Is there any way to use Indexing on relationships for such a scenario, to boost things up ? Ideally, the Traversal framework could use automatic/declerative indexing on Node Relationship types and/or direction to perform such traversals quicker. Regards ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Slow Traversals on Nodes with too many Relationships
I would respectfully disagree that it doesn't necessarily represent production usage, since in some cases, each query/traversal will be unique and isolated to a part of a subgraph, so in some cases, a cold query may be the norm -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Michael Hunger Sent: Wednesday, June 15, 2011 10:25 AM To: Neo4j user discussions Subject: Re: [Neo4j] Slow Traversals on Nodes with too many Relationships That is rather a case of warming up your caches. Determining the traversal speed from the first run is not a good benchmark as it doesn't represent production usage :) The same (warming up) is true for all kinds of benchmarks (except for startup performance benchmarks). Cheers Michael Am 15.06.2011 um 14:48 schrieb Agelos Pikoulas: I have a few Part nodes related with each via HASPART relationship/edges. (eg Part1---HASPART---Part2---HASPART---Part3 etc) . TraversalDescription works fine, following each Part's outgoing HASPART relationship. Then I add a large number (say 100.000) of Container Nodes, where each Container has a CONTAINS relation to almost *every* Part node. Hence each Part node now has a 100.000 incoming CONTAINS relationships from Container nodes, but only a few outgoing HASPART relationships to other Part nodes. Now my previous TraversalDescription run extremely slow (several seconds inside each IteratorPath.next() call) Note that I do define relationships(RT.HASPART, Direction.OUTGOING) on the TraversalDescription, but it seems its not used by neo4j as a hint. Note that on a subsequent run of the same Traversal, its very quick indeed. Is there any way to use Indexing on relationships for such a scenario, to boost things up ? Ideally, the Traversal framework could use automatic/declerative indexing on Node Relationship types and/or direction to perform such traversals quicker. Regards ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST API LuceneTimeline
Matt,
inserting by other than String in
http://docs.neo4j.org/chunked/snapshot/rest-api-indexes.html#rest-api-add-node-to-index
is not supported. A shortcoming. This is probably what your plugin
should be doing in that case. Will add that as a ticket to be tracked.
http://neo4jdb.lighthouseapp.com/projects/77609-neo4j-community/tickets/9-support-inserting-index-values-other-than-string-via-rest
-peter
On Wed, Jun 15, 2011 at 4:27 PM, Matt Luongo m...@scholr.ly wrote:
It does- awesome, I didn't realize the query syntax would work for
numeric fields, as well. Now I just need to figure out how to insert
into a numeric field via rest.
--
Matt Luongo
Co-Founder, Scholr.ly
On Wed, Jun 15, 2011 at 10:24 AM, Rick Bullotta
rick.bullo...@thingworx.com wrote:
Hi, Matt.
We don't use the REST API (we use Neo4J in an embedded mode), so I can't be
of much help there. If the REST API supports Lucene query syntax, you can
use something like:
timestamp : [1 to 2]
You'd need to do the Date - UTC milliseconds conversion on the front end,
I'd think.
Rick
-Original Message-
From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On
Behalf Of Matt Luongo
Sent: Wednesday, June 15, 2011 10:03 AM
To: Peter Neubauer
Cc: user@lists.neo4j.org
Subject: Re: [Neo4j] REST API LuceneTimeline
Peter,
Kind of. I see how I could do that server-side, but not how to insert
elements into the index numerically (with
ValueContext(val).indexNumeric()) or query by range (
.query(QueryContext.numericRange() ) via REST. Is this something
that will require our writing a plugin to support, or is there
currently some way to index and query numerically via the REST API?
--
Matt Luongo
Co-Founder, Scholr.ly
On Wed, Jun 15, 2011 at 9:25 AM, Peter Neubauer
neubauer.pe...@gmail.com wrote:
Yeah,
just checked with Mattias. There is no such index configuration, so
this does not work. I think you should use the numeric field in a
normal index for this, just like Rick points out.
Does that make sense?
/peter
On Wed, Jun 15, 2011 at 3:03 PM, Matt Luongo m...@scholr.ly wrote:
Peter,
I believe it was something like
curl -X POST -H Accept:application/json -HContent-Type:application/json -d
\
'{name:time, config:{type:timeline,provider:lucene}}' \
http://localhost:7474/db/data/index/node
curl -HContent-Type:application/json -X POST -d \
'http://localhost:7474/db/data/node/0;' \
http://localhost:7474/db/data/index/node/time/timestamp/25
which gives a 500.
--
Matt Luongo
Co-Founder, Scholr.ly
On Wed, Jun 15, 2011 at 8:55 AM, Rick Bullotta
rick.bullo...@thingworx.com wrote:
You really don't need a timeline index to do a timeline index, if that
makes sense. As mattias points out, it is just a convenient wrapper
around a plain lucene (neo) index. Just index the timestamp as a numeric
field.
- Reply message -
From: Matt Luongo m...@scholr.ly
Date: Wed, Jun 15, 2011 8:48 am
Subject: [Neo4j] REST API LuceneTimeline
To: Neo4j user discussions user@lists.neo4j.org
Peter,
I've done a bit more research, and you're right- it isn't. However, the
REST
api let's me create an index of type 'timeline'. The first time I try to
index a node, there's a 500 error (nullpointer) that stems from the index
not being created. I'd be happy to file a bug report.
Anyway, we're trying to put together a plugin to expose timeline indexing
via REST.
- Matt
On Jun 15, 2011 5:49 AM, Peter Neubauer
peter.neuba...@neotechnology.com
wrote:
Matt,
what URL are you using to create the timeline? I don't recall that
being exposed via REST ...
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph
database.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Mon, May 30, 2011 at 11:16 PM, Matt Luongo m...@scholr.ly wrote:
Forgive me if this is a foolish question. I'm really excited about
using
the
LuceneTimeline in our product. I see that I can create it via
type:timeline in the REST interface. I'm not sure, though, how to
add
nodes to the timeline- it seems like you add things to a timeline by
specifying just a timestamp, not key/value, but I can only find REST
examples for indexing nodes by key/value. Anyone care to share a code
snippet?
--
Matt Luongo
Co-Founder, Scholr.ly
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cannot execute Cypher
Could you check what the value of this is? scala.util.Properties.versionString Thanks, Andrés On Wed, Jun 15, 2011 at 11:41 AM, Pierre De Wilde pierredewi...@gmail.comwrote: HiAndrés, I also suspect that it's a scala version issue, but unfortunately only scala-library-2.8.1.jar is present on my machine: /Applications/neo4j-community-1.4.M04/lib/scala-library-2.8.1.jar Here is the complete log: NFO: Server started on [http://192.168.1.3:7474/] Jun 15, 2011 11:28:19 AM com.sun.jersey.spi.container.ContainerResponse mapMappableContainerException SEVERE: The exception contained within MappableContainerException could not be mapped to a response, re-throwing to the HTTP container java.lang.IncompatibleClassChangeError: Class scala.collection.mutable.HashMap$$anon$1 does not implement the requested interface scala.collection.generic.Addable at org.neo4j.cypher.SymbolTable.identifiers(SymbolTable.scala:130) at org.neo4j.cypher.SymbolTable.checkConnectednessOfPatternGraph(SymbolTable.scala:74) at org.neo4j.cypher.ExecutionEngine.execute(ExecutionEngine.scala:55) at org.neo4j.cypher.javacompat.ExecutionEngine.execute(ExecutionEngine.java:40) at org.neo4j.server.webadmin.console.CypherSession.evaluate(CypherSession.java:70) at org.neo4j.server.webadmin.rest.ConsoleService.exec(ConsoleService.java:110) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:184) at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:67) at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:276) at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:83) at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:133) at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:71) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1171) at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1103) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1053) at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1043) at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:406) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:477) at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:662) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:943) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:756) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) Jun 15, 2011 11:28:19 AM org.mortbay.log.Slf4jLog warn SEVERE: /db/manage/server/console/ java.lang.IncompatibleClassChangeError: Class scala.collection.mutable.HashMap$$anon$1 does not implement the requested interface scala.collection.generic.Addable at org.neo4j.cypher.SymbolTable.identifiers(SymbolTable.scala:130) at org.neo4j.cypher.SymbolTable.checkConnectednessOfPatternGraph(SymbolTable.scala:74) at org.neo4j.cypher.ExecutionEngine.execute(ExecutionEngine.scala:55) at org.neo4j.cypher.javacompat.ExecutionEngine.execute(ExecutionEngine.java:40) at org.neo4j.server.webadmin.console.CypherSession.evaluate(CypherSession.java:70) at org.neo4j.server.webadmin.rest.ConsoleService.exec(ConsoleService.java:110) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
Re: [Neo4j] No index provider 'lucene' found
Wow, how did I miss that... Well, thanks for your time. Ian On 2011-06-15, at 04:06, Michael Hunger michael.hun...@neotechnology.com wrote: Ian, I used your fragments to rebuild your setup, please send a minimal functional application (it missed the entry-points and also the startup-html files) next time, that will help a lot in actually looking for the problem instead taking a lot of time setting it up. It turned out that your war/WEB-INF/lib didn't contain the libraries but the gwt (jetty) server requires them there for the service-location to work, other IDE's automatically setup the WEB-INF/lib directory with the libraries used. I don't know why eclipse and/or the gwt plugin don't do it. The simple fix was to provide all the libraries used in the war/WEB-INF/lib directory. So that issue had nothing to do with Neo4j's service lookup mechanism but just with the webapp library configuration of your project. Cheers Michael Am 12.06.2011 um 22:02 schrieb Ian Bussières: I was wondering if anybody had a workaround or fix to my issue. Otherwise, is there a new or other way to accomplish indexing, as that portion of work would take me very little time to recode. Thanks ! 2011/6/10 Ian Bussières ian.bussie...@gmail.com: Here you go, took out src, war, lib, test, test-classes and also cleaned some of the stuff in the war folder. Oh and took out all the .jar files there too, otherwise this was just way to huge to share. Hope this will shine some light on the problem :S On Fri, Jun 10, 2011 at 07:18, Michael Hunger michael.hun...@neotechnology.com wrote: Eclipse .project and .classpath files. In general if you copy the project and remove all *.java files, that would be sufficient. Thanks so much Michael Am 10.06.2011 um 13:11 schrieb Ian Bussières: GWT 2.3 and Eclipse Helios. I can't share too much information from the project itself, I suppose I could share the config stuff. What config are you referring to? -- Ian Bussières Étudiant en Génie des Technologies de l'Information École de technologie supérieure [514] 814-0136 On Fri, Jun 10, 2011 at 01:31, Michael Hunger michael.hun...@neotechnology.com wrote: Ian, can you share your project somehow at least config and the startup class? What are eclipse and gwt plugin versions? M Sent from my iBrick4 Am 10.06.2011 um 03:33 schrieb Ian Bussières ian.bussie...@gmail.com: The web server is slightly ambiguous, because it's a GWT eclipse plugin that is used to run. I do believe it to be a Tomcat server. I'm trying to find a source that confirms that. Nothing about OSGi, no... The messages.log looks ok, apart from a few non clean shutdowns but successful recoveries. Ian Bussières Étudiant en Génie des Technologies de l'Information École de technologie supérieure [514] 814-0136 On Thu, Jun 9, 2011 at 21:03, Michael Hunger michael.hun...@neotechnology.com wrote: Ian, is that web-server a stock tomcat or jetty or something running an OSGi environment? So do you see something about OSGi containers being started in your console output (felix, equinox)? Could you have a look at the messages.log file in your graphdb directory if it shows anything unusual? Thanks Michael Am 10.06.2011 um 02:49 schrieb Ian Bussières: New buildpath, same problem. C:\Users\Ian\Dev\PFE\pfe\src;C:\Users\Ian\Dev\PFE\pfe\lib;C:\Users\Ian\Dev\PFE\pfe\test-classes;C:\Users\Ian\Dev\PFE\pfe\war\WEB-INF\classes;C:\Program Files (x86)\Eclipse\plugins\com.google.gwt.eclipse.sdkbundle_2.3.0.r36v201104261928\gwt-2.3.0\gwt-user.jar;C:\Program Files (x86)\Eclipse\plugins\com.google.gwt.eclipse.sdkbundle_2.3.0.r36v201104261928\gwt-2.3.0\gwt-dev.jar;C:\Program Files (x86)\Eclipse\plugins\com.google.gwt.eclipse.sdkbundle_2.3.0.r36v201104261928\gwt-2.3.0\validation-api-1.0.0.GA-sources.jar;C:\Program Files (x86)\Eclipse\plugins\com.google.gwt.eclipse.sdkbundle_2.3.0.r36v201104261928\gwt-2.3.0\validation-api-1.0.0.GA.jar;C:\Users\Ian\Dev\PFE\pfe\lib\geronimo-jta_1.1_spec-1.1.1.jar;C:\Users\Ian\Dev\PFE\pfe\lib\neo4j-community-1.4.M03.jar;C:\Users\Ian\Dev\PFE\pfe\lib\neo4j-graph-algo-1.4.M03.jar;C:\Users\Ian\Dev\PFE\pfe\lib\neo4j-jmx-1.4.M03.jar;C:\Users\Ian\Dev\PFE\pfe\lib\neo4j-kernel-1.4.M03.jar;C:\Users\Ian\Dev\PFE\pfe\lib\neo4j-lucene-index-1.4.M03.jar;C:\Users\Ian\Dev\PFE\pfe\lib\lucene-core-3.1.0.jar;C:\Users\Ian\Dev\PFE\pfe\lib\org.apache.servicemix.bundles.jline-0.9.94_1.jar;C:\Users\Ian\Dev\PFE\pfe\lib\neo4j-shell-1.4.M03.jar;C:\Users\Ian\Dev\PFE\pfe\lib\neo4j-udc-1.4.M03.jar;C:\Users\Ian\Dev\PFE\pfe\lib\server-api-1.4.M03.jar On Thu, Jun 9, 2011 at 20:32, Michael Hunger michael.hun...@neotechnology.com wrote: Can you please try to remove the lucene-core-3.0.1. It is possible that mixing two versions of lucene at the classpath messes things up. Cheers Michael Am 10.06.2011 um 02:27 schrieb Ian Bussières:
Re: [Neo4j] REST API LuceneTimeline
It does- awesome, I didn't realize the query syntax would work for
numeric fields, as well. Now I just need to figure out how to insert
into a numeric field via rest.
--
Matt Luongo
Co-Founder, Scholr.ly
On Wed, Jun 15, 2011 at 10:24 AM, Rick Bullotta
rick.bullo...@thingworx.com wrote:
Hi, Matt.
We don't use the REST API (we use Neo4J in an embedded mode), so I can't be
of much help there. If the REST API supports Lucene query syntax, you can
use something like:
timestamp : [1 to 2]
You'd need to do the Date - UTC milliseconds conversion on the front end,
I'd think.
Rick
-Original Message-
From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On
Behalf Of Matt Luongo
Sent: Wednesday, June 15, 2011 10:03 AM
To: Peter Neubauer
Cc: user@lists.neo4j.org
Subject: Re: [Neo4j] REST API LuceneTimeline
Peter,
Kind of. I see how I could do that server-side, but not how to insert
elements into the index numerically (with
ValueContext(val).indexNumeric()) or query by range (
.query(QueryContext.numericRange() ) via REST. Is this something
that will require our writing a plugin to support, or is there
currently some way to index and query numerically via the REST API?
--
Matt Luongo
Co-Founder, Scholr.ly
On Wed, Jun 15, 2011 at 9:25 AM, Peter Neubauer
neubauer.pe...@gmail.com wrote:
Yeah,
just checked with Mattias. There is no such index configuration, so
this does not work. I think you should use the numeric field in a
normal index for this, just like Rick points out.
Does that make sense?
/peter
On Wed, Jun 15, 2011 at 3:03 PM, Matt Luongo m...@scholr.ly wrote:
Peter,
I believe it was something like
curl -X POST -H Accept:application/json -HContent-Type:application/json -d \
'{name:time, config:{type:timeline,provider:lucene}}' \
http://localhost:7474/db/data/index/node
curl -HContent-Type:application/json -X POST -d \
'http://localhost:7474/db/data/node/0;' \
http://localhost:7474/db/data/index/node/time/timestamp/25
which gives a 500.
--
Matt Luongo
Co-Founder, Scholr.ly
On Wed, Jun 15, 2011 at 8:55 AM, Rick Bullotta
rick.bullo...@thingworx.com wrote:
You really don't need a timeline index to do a timeline index, if that
makes sense. As mattias points out, it is just a convenient wrapper
around a plain lucene (neo) index. Just index the timestamp as a numeric
field.
- Reply message -
From: Matt Luongo m...@scholr.ly
Date: Wed, Jun 15, 2011 8:48 am
Subject: [Neo4j] REST API LuceneTimeline
To: Neo4j user discussions user@lists.neo4j.org
Peter,
I've done a bit more research, and you're right- it isn't. However, the
REST
api let's me create an index of type 'timeline'. The first time I try to
index a node, there's a 500 error (nullpointer) that stems from the index
not being created. I'd be happy to file a bug report.
Anyway, we're trying to put together a plugin to expose timeline indexing
via REST.
- Matt
On Jun 15, 2011 5:49 AM, Peter Neubauer
peter.neuba...@neotechnology.com
wrote:
Matt,
what URL are you using to create the timeline? I don't recall that
being exposed via REST ...
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph
database.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Mon, May 30, 2011 at 11:16 PM, Matt Luongo m...@scholr.ly wrote:
Forgive me if this is a foolish question. I'm really excited about using
the
LuceneTimeline in our product. I see that I can create it via
type:timeline in the REST interface. I'm not sure, though, how to
add
nodes to the timeline- it seems like you add things to a timeline by
specifying just a timestamp, not key/value, but I can only find REST
examples for indexing nodes by key/value. Anyone care to share a code
snippet?
--
Matt Luongo
Co-Founder, Scholr.ly
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
[Neo4j] Plugin for REST server not appearing
Hi people ! So, I build a plugin just to ease a certain query that's really constant on my system, but it doesn't seams to work. I built it with maven, the .jar that it generates is this: http://www.4shared.com/file/W8_s4LSu/neo4j-customquery-plugin-14M03.html I put it in the plugins folder in the neo4j folder, restart the server, but the extensions parameter is still empty. Any ideas what am I doing wrong ? Thanks ! Milena Araujo ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST API LuceneTimeline
Depends I would think on how you are putting/posting the data. If via XML or
URL parameters, I can't see how Neo would know whether it was a string or a
number, but if via JSON you should be able to get it in there properly.
Perhaps the REST API should support passing content via either the URL or via a
POST'ed payload. That's what we do with our internally created REST API today
(support URL parameters, XML, or JSON for all calls, with the URI defining the
intent or the method and query parameters XML/JSON defining the
parameters).
I know the REST API has some soak time already, but I would seriously
consider redesigning parts of it to treat some of the parameters differently
when the Content-Type is JSON (e.g. pass them in the payload, not the URI, as
with the REST API that lets you create a node w/properties). Let's take
indexing for example. The approach I'm proposing would allow an atomic call to
index multiple fields, rather than one at a time, as with the current approach,
and it would allow more flexible data types. Indexing all the fields at once
is, to me, an essential capability to ensure the semantic integrity of the
graph.
An example is:
POST http://localhost:7474/db/data/index/node/favorites
POST Content :
{
fields : [
{
key : key1
value : 12345
},
{
key : key2
value : ABC
},
{
key : key3
value : [ Dogs, Cats, Sheep ]
}
]
}
*Accept: application/json
*Content-Type: application/json
-Original Message-
From: Peter Neubauer [mailto:neubauer.pe...@gmail.com]
Sent: Wednesday, June 15, 2011 10:30 AM
To: Matt Luongo
Cc: Neo4j user discussions; Rick Bullotta
Subject: Re: [Neo4j] REST API LuceneTimeline
Matt,
inserting by other than String in
http://docs.neo4j.org/chunked/snapshot/rest-api-indexes.html#rest-api-add-node-to-index
is not supported. A shortcoming. This is probably what your plugin
should be doing in that case. Will add that as a ticket to be tracked.
http://neo4jdb.lighthouseapp.com/projects/77609-neo4j-community/tickets/9-support-inserting-index-values-other-than-string-via-rest
-peter
On Wed, Jun 15, 2011 at 4:27 PM, Matt Luongo m...@scholr.ly wrote:
It does- awesome, I didn't realize the query syntax would work for
numeric fields, as well. Now I just need to figure out how to insert
into a numeric field via rest.
--
Matt Luongo
Co-Founder, Scholr.ly
On Wed, Jun 15, 2011 at 10:24 AM, Rick Bullotta
rick.bullo...@thingworx.com wrote:
Hi, Matt.
We don't use the REST API (we use Neo4J in an embedded mode), so I can't be
of much help there. If the REST API supports Lucene query syntax, you can
use something like:
timestamp : [1 to 2]
You'd need to do the Date - UTC milliseconds conversion on the front end,
I'd think.
Rick
-Original Message-
From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On
Behalf Of Matt Luongo
Sent: Wednesday, June 15, 2011 10:03 AM
To: Peter Neubauer
Cc: user@lists.neo4j.org
Subject: Re: [Neo4j] REST API LuceneTimeline
Peter,
Kind of. I see how I could do that server-side, but not how to insert
elements into the index numerically (with
ValueContext(val).indexNumeric()) or query by range (
.query(QueryContext.numericRange() ) via REST. Is this something
that will require our writing a plugin to support, or is there
currently some way to index and query numerically via the REST API?
--
Matt Luongo
Co-Founder, Scholr.ly
On Wed, Jun 15, 2011 at 9:25 AM, Peter Neubauer
neubauer.pe...@gmail.com wrote:
Yeah,
just checked with Mattias. There is no such index configuration, so
this does not work. I think you should use the numeric field in a
normal index for this, just like Rick points out.
Does that make sense?
/peter
On Wed, Jun 15, 2011 at 3:03 PM, Matt Luongo m...@scholr.ly wrote:
Peter,
I believe it was something like
curl -X POST -H Accept:application/json -HContent-Type:application/json -d
\
'{name:time, config:{type:timeline,provider:lucene}}' \
http://localhost:7474/db/data/index/node
curl -HContent-Type:application/json -X POST -d \
'http://localhost:7474/db/data/node/0;' \
http://localhost:7474/db/data/index/node/time/timestamp/25
which gives a 500.
--
Matt Luongo
Co-Founder, Scholr.ly
On Wed, Jun 15, 2011 at 8:55 AM, Rick Bullotta
rick.bullo...@thingworx.com wrote:
You really don't need a timeline index to do a timeline index, if that
makes sense. As mattias points out, it is just a convenient wrapper
around a plain lucene (neo) index. Just index the timestamp as a numeric
field.
- Reply message -
From: Matt Luongo m...@scholr.ly
Date: Wed, Jun 15, 2011 8:48 am
Subject: [Neo4j] REST API LuceneTimeline
To: Neo4j user discussions user@lists.neo4j.org
Peter,
I've done a bit more research, and you're right- it isn't. However, the
REST
api
Re: [Neo4j] Slow Traversals on Nodes with too many Relationships
Could this also be related to the possibility that in order to determine relationship type and direction, the relationships need to be loaded from disk? If so, then having a large number of relationships on the same node would decrease performance, if the number was large enough to affect the disk io caching. If this is the case, perhaps adding a proxy node for the incoming relationships would work-around the problem? Of course then you have doubled the number of part nodes (two for each part, one part and one containers proxy). On Wed, Jun 15, 2011 at 10:27 PM, Rick Bullotta rick.bullo...@thingworx.com wrote: I would respectfully disagree that it doesn't necessarily represent production usage, since in some cases, each query/traversal will be unique and isolated to a part of a subgraph, so in some cases, a cold query may be the norm -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Michael Hunger Sent: Wednesday, June 15, 2011 10:25 AM To: Neo4j user discussions Subject: Re: [Neo4j] Slow Traversals on Nodes with too many Relationships That is rather a case of warming up your caches. Determining the traversal speed from the first run is not a good benchmark as it doesn't represent production usage :) The same (warming up) is true for all kinds of benchmarks (except for startup performance benchmarks). Cheers Michael Am 15.06.2011 um 14:48 schrieb Agelos Pikoulas: I have a few Part nodes related with each via HASPART relationship/edges. (eg Part1---HASPART---Part2---HASPART---Part3 etc) . TraversalDescription works fine, following each Part's outgoing HASPART relationship. Then I add a large number (say 100.000) of Container Nodes, where each Container has a CONTAINS relation to almost *every* Part node. Hence each Part node now has a 100.000 incoming CONTAINS relationships from Container nodes, but only a few outgoing HASPART relationships to other Part nodes. Now my previous TraversalDescription run extremely slow (several seconds inside each IteratorPath.next() call) Note that I do define relationships(RT.HASPART, Direction.OUTGOING) on the TraversalDescription, but it seems its not used by neo4j as a hint. Note that on a subsequent run of the same Traversal, its very quick indeed. Is there any way to use Indexing on relationships for such a scenario, to boost things up ? Ideally, the Traversal framework could use automatic/declerative indexing on Node Relationship types and/or direction to perform such traversals quicker. Regards ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] unique indexes
Or you can use the write locks on nodes to enforce just that. Just grab a write lock prior to reading/writing to it (writing will grab a write lock automatically of course). There's no exposed way of grabbing write locks manually, but removing a non-existent property is a temporary solution. It even solves the indexing problem if you do the index checks/inserts after you've grabbed a write lock on a certain node/relationship that will act as the synchronizer (that way it'll would also work in HA). 2011/6/10 yougotroo...@gmail.com Right, I can leverage that. I could have a ConncurrentSkipListSetLong or something with the nodes unique Id in it. If I add() the nodes unique Id to the Set and it returns true, I know I can safely add children. If it returns false then I know another thread beat me to it, and I can requeue the thread On , Rick Bullotta rick.bullo...@thingworx.com wrote: Not sure it applies, but also remember that the node id itself is essentially a unique key across the entire set of nodes. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of yougotroo...@gmail.com Sent: Friday, June 10, 2011 3:10 PM To: Neo4j user discussions Subject: Re: [Neo4j] unique indexes The only way I know of is to synchronize writes to the DB. This shouldn't be a problem for me actually, even with a cluster, since what I really need is for each node with a particular relationship to have a unique name property, like how good parents would ensure their children all have unique names. I could use terracottta and a concurrent data structure to ensure there is only one thread writing to a particular node at any given time, then I'd be guaranteed that checking for existence would work. -Matt On , Michael Hunger michael.hun...@neotechnology.com wrote: Another idea is a merge after the fact, so either in a nightly batch or during a get operation duplicates are discovered they are merged. Cheers Michael Am 10.06.2011 um 18:55 schrieb Rick Bullotta: Good point. Same would apply to relationship-based indexing, I suppose. In our use case, all of the entities that must be unique are maintained in a domain model that is managed by an in-memory hash, so we can enforce it prior to persistence. I suppose that one brute force way would be to synchronize/pipeline writes that required a unique index so that the situation of uncommitted changes on another transaction could not occur. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Alexander Smirnov Sent: Friday, June 10, 2011 12:48 PM To: Neo4j user discussions Subject: Re: [Neo4j] unique indexes Check befor update could not help there, because Neo4j does not show uncommited changes to other transactions, so it's possible to store the same value in concurrent transactions. I also tried to figure out how to avoid duplicated values in database. Seems that it needed some locks on database objects. Can someone provide an example how to lock database objects properly ? On Fri, Jun 10, 2011 at 8:01 AM, Rick Bullotta rick.bullo...@thingworx.com wrote: If you need to, you could check the existence of a match prior to every attempted add/insert/update. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Matt C Sent: Friday, June 10, 2011 10:25 AM To: Neo4j user discussions Subject: Re: [Neo4j] unique indexes An index that would enforce every value added to it be unique. I don't even see support for that in Lucene, so, nevermind. Just something I'm used to SQL handling for me. On Fri, Jun 10, 2011 at 1:05 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Matt, what do you mean by unique indexes? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Fri, Jun 10, 2011 at 7:30 AM, Matt C yougotroo...@gmail.com wrote: Sorry if this is covered somewhere, I couldn't find it in the docs. I was wondering if its possible to create unique indexes with the LuceneIndexProvider? -Matt
Re: [Neo4j] Slow Traversals on Nodes with too many Relationships
Re: [Neo4j] Slow Traversals on Nodes with too many Relationships I have to respectfully agree with Rick Bullotta. I was suspecting the big-O is not linear for this case. To verify I added x4 Container nodes (400.000) and their appropriate Relationships, and it is now *unbelievably* slow : It does not take x4 more, but it takes more than 30-40 seconds for each next() Remind you 100K nodes = ~2secs for each next() !!! And only to make matters worse, the subsequent runs weren't fast either - they actually took more time than the first (1st TotalTraversalTime= 389936ms, 2nd TotalTraversalTime= 443948ms) The whole setup is running on Eclipse 3.6, with -Xmx512m on JavaVM, Windows2003 VMWare machine with 4GB, running on a fast 2nd gen SSD (OCZ Vertex 2). The neo4J data resides on this SSD. The 100.000 nodes data files were ~250MB, the 400.000 one is ~1GB. I wonder what would happen if the Container nodes were a few million (which will be my case) - it will run forever. Could you please looking into my suggestion - i.e Using a 'smart' behind the scenes Indexing on both *RelationshipType* and *Direction* that Traversals actually use to boost things up ? To another topic, how does one use this mailing list - I use it through gmail and I am utterly lost - is there a better client/UI to actually post/reply into threads ? ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Slow Traversals on Nodes with too many Relationships
Hi, when we started to evaluate neo4j we made some measurements and for us it seemed that 50.000 is a magical number: this many relationships and properties on one node seemed to be a limit, which once reached makes things slow. But we didn't actually need that much relationship/property in our case, so we could live with it, or could make workarounds (eg. storing things in properties and doing indexed lookups instead of using relationships) An automatic indexed lookup on relationship types and directions would be awsome, definitely. Regards, --- balazs On 6/15/11 5:40 PM, Agelos Pikoulas wrote: Re: [Neo4j] Slow Traversals on Nodes with too many Relationships I have to respectfully agree with Rick Bullotta. I was suspecting the big-O is not linear for this case. To verify I added x4 Container nodes (400.000) and their appropriate Relationships, and it is now *unbelievably* slow : It does not take x4 more, but it takes more than 30-40 seconds for each next() Remind you 100K nodes = ~2secs for each next() !!! And only to make matters worse, the subsequent runs weren't fast either - they actually took more time than the first (1st TotalTraversalTime= 389936ms, 2nd TotalTraversalTime= 443948ms) The whole setup is running on Eclipse 3.6, with -Xmx512m on JavaVM, Windows2003 VMWare machine with 4GB, running on a fast 2nd gen SSD (OCZ Vertex 2). The neo4J data resides on this SSD. The 100.000 nodes data files were ~250MB, the 400.000 one is ~1GB. I wonder what would happen if the Container nodes were a few million (which will be my case) - it will run forever. Could you please looking into my suggestion - i.e Using a 'smart' behind the scenes Indexing on both *RelationshipType* and *Direction* that Traversals actually use to boost things up ? To another topic, how does one use this mailing list - I use it through gmail and I am utterly lost - is there a better client/UI to actually post/reply into threads ? ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Slow Traversals on Nodes with too many Relationships
Agelos, sorry, didn't want to sound that way. 512M ram is not very much for larger graphs. Neo4j has to cache nodes, relationships in the heap as well as you own datastructures. The memory mapped files for the datastores are kept outside the heap. Normally with your 4G I'd suggest using about 1.5G for heap and 1.5G for the memory mapped files. http://wiki.neo4j.org/content/Configuration_Settings Do you have a small test-case available that creates your graph and runs your traversal? Then I could have a look at that and also do some profiling to determine the issues for this slowdown. The indexing doesn't help as it also has to hit caches or disk. The graph traversal is normally a very efficient operation that shouldn't experience this bad performance. Cheers Michael P.S. I just use my mail client for handling the mailing list and it works fine for me. Imho Gmail groups threads automatically. Am 15.06.2011 um 17:40 schrieb Agelos Pikoulas: Re: [Neo4j] Slow Traversals on Nodes with too many Relationships I have to respectfully agree with Rick Bullotta. I was suspecting the big-O is not linear for this case. To verify I added x4 Container nodes (400.000) and their appropriate Relationships, and it is now *unbelievably* slow : It does not take x4 more, but it takes more than 30-40 seconds for each next() Remind you 100K nodes = ~2secs for each next() !!! And only to make matters worse, the subsequent runs weren't fast either - they actually took more time than the first (1st TotalTraversalTime= 389936ms, 2nd TotalTraversalTime= 443948ms) The whole setup is running on Eclipse 3.6, with -Xmx512m on JavaVM, Windows2003 VMWare machine with 4GB, running on a fast 2nd gen SSD (OCZ Vertex 2). The neo4J data resides on this SSD. The 100.000 nodes data files were ~250MB, the 400.000 one is ~1GB. I wonder what would happen if the Container nodes were a few million (which will be my case) - it will run forever. Could you please looking into my suggestion - i.e Using a 'smart' behind the scenes Indexing on both *RelationshipType* and *Direction* that Traversals actually use to boost things up ? To another topic, how does one use this mailing list - I use it through gmail and I am utterly lost - is there a better client/UI to actually post/reply into threads ? ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Archiving in time / being able to quickly refer to a historical state
On Wed, Jun 15, 2011 at 2:31 AM, Patrik Sundberg patrik.sundb...@gmail.comwrote: On 14 Jun 2011, at 22:51, Michael Hunger michael.hun...@neotechnology.com wrote: Patrick, David Montag created a versioned graph approach on github: https://github.com/dmontag/neo4j-versioning Perhaps you can take some inspiration from that. I'll take a look, thanks. I had a look at this. It's a really cool idea. It's in principal a very good solution to what I'd like to accomplish, even more ambitious even. And for part of my graph it's ideal in practical terms as well - the parts that don't change so often and where number of relationships wont mushroom. However, I have a nagging suspicion that it may not be practical in the Book, Elements, Effects use case - may get too many relationships. If I have versioned relationships that keeps growing from Books to Elements then over time a Book will get a lot of direct relationships it's involved in. Let's say I have ~500 elements/book (at any point in time), and they'll change every day (not quite true but lets assume so for now), then a book will add on 125,000 relationships/year (assuming 250 business days). Over a few years you'll start having half a million direct relationships of a single Book. That's probably not an ideal deisgn, correct? What I was thinking is that I could have an ElementSet reference node layer sitting between a Book and Elements. An ElementSet has a timestamp property for when the snapshot was made or the relationship from Book to ElementSet has a valid range on it. Then an ElementSet node in turn has Elements. So in a year you'd add ~250 Book to ElementSet relationships, and each ElementSet would have ~500 relationships to it's Elements. I can still get the state of the book quite easily via a simple 2 level traversal. Another thought I'd be happy to get some input on: My Elements are essentially consisting of an Entity and a Quantity. The Effects as well. If we assume we'll have millions of Effects and 100s of thousands of Elements then I'm thinking that it's not a good idea to have actual relationships pointing to the Entity nodes from the Element and Effect nodes, correct? The Entity nodes would get millions of direct relationships. I could instead put in the identifier or the Entity in the Element and the Effect as properties and use index based queries for that step. Patrik Am 14.06.2011 um 21:14 schrieb Patrik Sundberg: I should probably put in some estimates of the sort of orders of magnitude of data involved: Books are in the 100s Effects are in the 10,000s / day Elements are in the 100s / Book On Tue, Jun 14, 2011 at 8:07 PM, Patrik Sundberg patrik.sundb...@gmail.comwrote: Hi, Design question: I have a Book. A book has Elements. To change what Elements are in a Book there are Effects. An Effect adds or deletes an element from a Book (and has other info). Everyday I archive the state of a Book so that I can easily get the state of the Book (it's Elements) at a historical point in time. The number of elements and which elements are in a Book vary from day to day depending on the Effects taking place in the time period. So it's a bit like Book contents archive of day X + 1 = Book contents archive of day X + the Effects between X and X + 1 There's no need to represent the daily archives in any way, it's the functionality of having historical snapshots that matter to me, so that one can easily get the state of a Book at any point in time by finding the closest archive point before the time and applying the Effects from that archive timestamp and the point of interest. Ideally want to keep a long time history of Book state and Effects. Any ideas for an effecient of organizing this in a graph friendly way? Thanks, Patrik ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Check if node exists in Neo4J index + possible bug in the remove function
Hey all, We've elevated this thread into a tracked discussion at: http://help.neo4j.org/discussions/questions/3-neo4j-check-if-node-exists-in-neo4j-index-possible-bug-in-the-remove-function Feel free to add yourself as a watcher there, and we'll report back here when it's been resolved. This is part of an evaluation of Tender that we're conducting, to better support issue resolution. The mailing list is still the place to start conversations, which we'll bump up to help.neo4j.org as appropriate, and attach issue tracking with Lighthouse (also being evaluated) as needed. Of course, we're still working out the mechanics that will make this a good experience. Please let us know what you think. Cheers, Andreas On Jun 13, 2011, at 2:55 PM, Peter Neubauer wrote: Rick, good finds! Mattias is back from vacation next week, meanwhile, I will keep this open in order to ask him and get back here. Chris, do you have any info on this? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Mon, Jun 13, 2011 at 10:19 PM, Rick Bullotta rick.bullo...@thingworx.com wrote: Hi, all. A couple index framework questions: 1) There does not seem to be a built-in API call to determine whether or not a node exists in a specific IndexNode. After looking into the index structures with Luke (the must-have tool for anyone working with Lucene), it appears that each node has a virtual field named _id_ that could be used in with index.get(_id_,nodeID) to do this check. Is this a reasonable approach? Can we count on the _id_ field name being consistent in future versions of Neo4J 2) The remove(node) method is supposed to remove the node from the index. In the case where the node wasn't in the index, what it appears to do is actually *add* the node to the index as an empty document. If it was in the index, it deletes all of the field values except the _id_ field but leaves the node (document) in the index. This seems like it will lead to ever decreasing performance as these ghost nodes or ghost documents accumulate. We are testing w/1.4M04, if that matter. Thanks, Rick ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] No index provider 'lucene' found
I've came back to look at it, and I had deliberatley taken the jar files out of the war folder for a smaller transfer between us. I had also specified that in the email that contained the original archive. I will get back to you and send the project some time tommorrow. Cheers, Ian. Emailed from my iPhone On 2011-06-15, at 04:06, Michael Hunger michael.hun...@neotechnology.com wrote: Ian, I used your fragments to rebuild your setup, please send a minimal functional application (it missed the entry-points and also the startup-html files) next time, that will help a lot in actually looking for the problem instead taking a lot of time setting it up. It turned out that your war/WEB-INF/lib didn't contain the libraries but the gwt (jetty) server requires them there for the service-location to work, other IDE's automatically setup the WEB-INF/lib directory with the libraries used. I don't know why eclipse and/or the gwt plugin don't do it. The simple fix was to provide all the libraries used in the war/WEB-INF/lib directory. So that issue had nothing to do with Neo4j's service lookup mechanism but just with the webapp library configuration of your project. Cheers Michael Am 12.06.2011 um 22:02 schrieb Ian Bussières: I was wondering if anybody had a workaround or fix to my issue. Otherwise, is there a new or other way to accomplish indexing, as that portion of work would take me very little time to recode. Thanks ! 2011/6/10 Ian Bussières ian.bussie...@gmail.com: Here you go, took out src, war, lib, test, test-classes and also cleaned some of the stuff in the war folder. Oh and took out all the .jar files there too, otherwise this was just way to huge to share. Hope this will shine some light on the problem :S On Fri, Jun 10, 2011 at 07:18, Michael Hunger michael.hun...@neotechnology.com wrote: Eclipse .project and .classpath files. In general if you copy the project and remove all *.java files, that would be sufficient. Thanks so much Michael Am 10.06.2011 um 13:11 schrieb Ian Bussières: GWT 2.3 and Eclipse Helios. I can't share too much information from the project itself, I suppose I could share the config stuff. What config are you referring to? -- Ian Bussières Étudiant en Génie des Technologies de l'Information École de technologie supérieure [514] 814-0136 On Fri, Jun 10, 2011 at 01:31, Michael Hunger michael.hun...@neotechnology.com wrote: Ian, can you share your project somehow at least config and the startup class? What are eclipse and gwt plugin versions? M Sent from my iBrick4 Am 10.06.2011 um 03:33 schrieb Ian Bussières ian.bussie...@gmail.com: The web server is slightly ambiguous, because it's a GWT eclipse plugin that is used to run. I do believe it to be a Tomcat server. I'm trying to find a source that confirms that. Nothing about OSGi, no... The messages.log looks ok, apart from a few non clean shutdowns but successful recoveries. Ian Bussières Étudiant en Génie des Technologies de l'Information École de technologie supérieure [514] 814-0136 On Thu, Jun 9, 2011 at 21:03, Michael Hunger michael.hun...@neotechnology.com wrote: Ian, is that web-server a stock tomcat or jetty or something running an OSGi environment? So do you see something about OSGi containers being started in your console output (felix, equinox)? Could you have a look at the messages.log file in your graphdb directory if it shows anything unusual? Thanks Michael Am 10.06.2011 um 02:49 schrieb Ian Bussières: New buildpath, same problem. C:\Users\Ian\Dev\PFE\pfe\src;C:\Users\Ian\Dev\PFE\pfe\lib;C:\Users\Ian\Dev\PFE\pfe\test-classes;C:\Users\Ian\Dev\PFE\pfe\war\WEB-INF\classes;C:\Program Files (x86)\Eclipse\plugins\com.google.gwt.eclipse.sdkbundle_2.3.0.r36v201104261928\gwt-2.3.0\gwt-user.jar;C:\Program Files (x86)\Eclipse\plugins\com.google.gwt.eclipse.sdkbundle_2.3.0.r36v201104261928\gwt-2.3.0\gwt-dev.jar;C:\Program Files (x86)\Eclipse\plugins\com.google.gwt.eclipse.sdkbundle_2.3.0.r36v201104261928\gwt-2.3.0\validation-api-1.0.0.GA-sources.jar;C:\Program Files (x86)\Eclipse\plugins\com.google.gwt.eclipse.sdkbundle_2.3.0.r36v201104261928\gwt-2.3.0\validation-api-1.0.0.GA.jar;C:\Users\Ian\Dev\PFE\pfe\lib\geronimo-jta_1.1_spec-1.1.1.jar;C:\Users\Ian\Dev\PFE\pfe\lib\neo4j-community-1.4.M03.jar;C:\Users\Ian\Dev\PFE\pfe\lib\neo4j-graph-algo-1.4.M03.jar;C:\Users\Ian\Dev\PFE\pfe\lib\neo4j-jmx-1.4.M03.jar;C:\Users\Ian\Dev\PFE\pfe\lib\neo4j-kernel-1.4.M03.jar;C:\Users\Ian\Dev\PFE\pfe\lib\neo4j-lucene-index-1.4.M03.jar;C:\Users\Ian\Dev\PFE\pfe\lib\lucene-core-3.1.0.jar;C:\Users\Ian\Dev\PFE\pfe\lib\org.apache.servicemix.bundles.jline-0.9.94_1.jar;C:\Users\Ian\Dev\PFE\pfe\lib\neo4j-shell-1.4.M03.jar;C:\Users\Ian\Dev\PFE\pfe\lib\neo4j-udc-1.4.M03.jar;C:\Users\Ian\Dev\PFE\pfe\lib\server-api-1.4.M03.jar On Thu, Jun 9, 2011 at 20:32, Michael Hunger
Re: [Neo4j] User Digest, Vol 51, Issue 96
Dear all / Im' sorry if I cant use the user@lists properly, I am indeed lost :-( Neo4J would be so much better as a forum or a stackOverNeo4J :-)***/ Allow me to say, that the 50K magic number is not very useful for real practical modern Social Network apps. What if there's simply a couple of million Person nodes that may LIKE the Movie nodes? And what if I have a few million of Movies and many million of Persons ? Its a typical case a movie having a few 100K rating/votes. And imagine if I have Song, Book Product nodes! I think this issue is *MAJOR* and it needs to be promoted to a high priority to the neo4j team. The proxy solution sounds wonderful, but it can be quite a hassle if its not rightly encapsulated transparent. I think all Traversals will become quite hacked I can't even think what will happen to Object mapping etc. I imagine it COULD be part of an upcoming version of the new amazing Spring Data Graph framework (check it out!), where a simple Annotation such as @NodeWithProxy along with information for what *RelationshipTypes / Directions *should go to the real or the proxy Node, could do all of the proxy magic!!! But, the *RelationshipType/Direction indexing *I proposed, I dare say, could be a more generic and cleaner idea, and also a quicker hack! All we need is a method TraversalDescription.*index(myIndex);* where we can declare which index should be used for looking up the (few) RelationshipTypes/Directions among the millions on the Node. The best thing is that we have already declared those on TraversalDescription.*relationships(*MyRelationshipType.hasPart, Direction.OUTGOING). The *Traversal *would then follow (only) those found on the index! Bingo We could also have a *.followIndexedOnly(false) *and even *recreateFollowedIndexes(true) *to save us next time! In any case, something must be implemented! Without being an expert on neo4j, I think there is a lot of Indexing optimization needed yet! Michael what do you think ? Could you please see this being promoted to the team while sharing their views? Agelos Date: Wed, 15 Jun 2011 17:57:55 +0200 From: Balazs E. Pataki pat...@dsd.sztaki.hu Subject: Re: [Neo4j] Slow Traversals on Nodes with too many Relationships To: Neo4j user discussions user@lists.neo4j.org Message-ID: 4df8d683.8010...@dsd.sztaki.hu Content-Type: text/plain; charset=ISO-8859-1; format=flowed Hi, when we started to evaluate neo4j we made some measurements and for us it seemed that 50.000 is a magical number: this many relationships and properties on one node seemed to be a limit, which once reached makes things slow. But we didn't actually need that much relationship/property in our case, so we could live with it, or could make workarounds (eg. storing things in properties and doing indexed lookups instead of using relationships) An automatic indexed lookup on relationship types and directions would be awsome, definitely. Regards, --- balazs Date: Wed, 15 Jun 2011 23:19:32 +0800 From: Craig Taverner cr...@amanzi.com Subject: Re: [Neo4j] Slow Traversals on Nodes with too many Relationships To: Neo4j user discussions user@lists.neo4j.org Message-ID: banlktine_mk5+9damh07tsrq6nnxifo...@mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 Could this also be related to the possibility that in order to determine relationship type and direction, the relationships need to be loaded from disk? If so, then having a large number of relationships on the same node would decrease performance, if the number was large enough to affect the disk io caching. If this is the case, perhaps adding a proxy node for the incoming relationships would work-around the problem? Of course then you have doubled the number of part nodes (two for each part, one part and one containers proxy). Date: Wed, 15 Jun 2011 18:40:05 +0300 From: Agelos Pikoulas agelos.pikou...@gmail.com Subject: Re: [Neo4j] Slow Traversals on Nodes with too many Relationships To: user@lists.neo4j.org Message-ID: banlktinsmadbatf1rglaj4wqngkfekj...@mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 Re: [Neo4j] Slow Traversals on Nodes with too many Relationships I have to respectfully agree with Rick Bullotta. I was suspecting the big-O is not linear for this case. To verify I added x4 Container nodes (400.000) and their appropriate Relationships, and it is now *unbelievably* slow : It does not take x4 more, but it takes more than 30-40 seconds for each next() Remind you 100K nodes = ~2secs for each next() !!! And only to make matters worse, the subsequent runs weren't fast either - they actually took more time than the first (1st TotalTraversalTime= 389936ms, 2nd TotalTraversalTime= 443948ms) The whole setup is running on Eclipse 3.6, with -Xmx512m on JavaVM, Windows2003 VMWare machine with 4GB, running on a fast 2nd gen SSD (OCZ Vertex 2). The neo4J data resides on this SSD. The 100.000 nodes data files were ~250MB, the 400.000 one is ~1GB.
[Neo4j] gremlin rest plugin and map
Hello neo4j,
I'm trying to run via the gremlin server plugin a query like:
m=[:];n.outE.inV.inE.outV.groupCount(m);m.sort{a,b - b.value =
a.value}; m;
This doesn't output anything. I tried m.keySet() and that doesn't return
anything either.
Any ideas how I could make this work?
-m.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] gremlin rest plugin and map
Hi,
The trick to Gremlin is to know that an expression is an Iterator and thus,
must be iterated.
n.outE.inV.inE.outV.groupCount(m);m.sort{a,b - b.value = a.value}
Should be:
n.outE.inV.inE.outV.groupCount(m);m.sort{a,b - b.value = a.value}
-1
-1 is equivalent to while(itty.hasNext()) { itty.next() }
When you do sutff in the Gremlin terminal, the terminal will say: oh, you
returned an iterator, let me iterate through the results for you.
See the first point in:
https://github.com/tinkerpop/gremlin/wiki/Troubleshooting
Question: where are you setting the variable 'n' ? Is that Gremlin plugin thing?
Hope that helps,
Marko.
http://markorodriguez.com
On Jun 15, 2011, at 1:08 PM, Marcelo Barbudas wrote:
Hello neo4j,
I'm trying to run via the gremlin server plugin a query like:
m=[:];n.outE.inV.inE.outV.groupCount(m);m.sort{a,b - b.value =
a.value}; m;
This doesn't output anything. I tried m.keySet() and that doesn't return
anything either.
Any ideas how I could make this work?
-m.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] gremlin rest plugin and map
Hey Marko, Wow, that was an *unbelievably* fast response. Amazing. Question: where are you setting the variable 'n' ? Is that Gremlin plugin thing? It's unrelated to the neo4j plugin. Just an easy way for me to set up a starting node. -m. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] gremlin rest plugin and map
Oops.
n.outE.inV.inE.outV.groupCount(m) -1;m.sort{a,b - b.value = a.value}
NOT
n.outE.inV.inE.outV.groupCount(m);m.sort{a,b - b.value = a.value} -1
I didn't see the semicolon.
Marko.
On Jun 15, 2011, at 1:12 PM, Marko Rodriguez wrote:
Hi,
The trick to Gremlin is to know that an expression is an Iterator and thus,
must be iterated.
n.outE.inV.inE.outV.groupCount(m);m.sort{a,b - b.value = a.value}
Should be:
n.outE.inV.inE.outV.groupCount(m);m.sort{a,b - b.value = a.value}
-1
-1 is equivalent to while(itty.hasNext()) { itty.next() }
When you do sutff in the Gremlin terminal, the terminal will say: oh, you
returned an iterator, let me iterate through the results for you.
See the first point in:
https://github.com/tinkerpop/gremlin/wiki/Troubleshooting
Question: where are you setting the variable 'n' ? Is that Gremlin plugin
thing?
Hope that helps,
Marko.
http://markorodriguez.com
On Jun 15, 2011, at 1:08 PM, Marcelo Barbudas wrote:
Hello neo4j,
I'm trying to run via the gremlin server plugin a query like:
m=[:];n.outE.inV.inE.outV.groupCount(m);m.sort{a,b - b.value =
a.value}; m;
This doesn't output anything. I tried m.keySet() and that doesn't return
anything either.
Any ideas how I could make this work?
-m.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] gremlin rest plugin and map
n.outE.inV.inE.outV.groupCount(m) -1;m.sort{a,b - b.value = a.value}
Yep, this versions runs and returns.
However the returned values from the web interface are different from
the ones via the gremlin plugin.
-m.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] gremlin rest plugin and map
Marcelo,
the Gremlin plugin is transforming the results from a query into Neo4j
REST conform structures, while the console is a pure command line
interpreter, outputting strings that are not consumable by clients.
Does that make sense?
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Jun 15, 2011 at 9:28 PM, Marcelo Barbudas nos...@gmail.com wrote:
n.outE.inV.inE.outV.groupCount(m) -1;m.sort{a,b - b.value = a.value}
Yep, this versions runs and returns.
However the returned values from the web interface are different from
the ones via the gremlin plugin.
-m.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cannot execute Cypher
On Tue, Jun 14, 2011 at 9:11 AM, Andres Taylor andres.tay...@neotechnology.com wrote: Hi Pierre, On Tue, Jun 14, 2011 at 1:48 PM, Pierre De Wilde pierredewi...@gmail.com wrote: start n=(1) return n Nothing is returned... Could you look in the logs for any clues? They are in neo4j install dir/data/log This will most likely reveal the source of the problem. You could also install the firebug plugin for firefox, and use it to see what the raw response from the server is. Andrés ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] gremlin rest plugin and map
Hey Peter,
I think I understand.
So that means I can get a sort listed of nodes (the contents of m keys),
but not the actual number on which the sorting was based (m values)?
-m.
On 06/15/2011 10:46 PM, Peter Neubauer wrote:
Marcelo,
the Gremlin plugin is transforming the results from a query into Neo4j
REST conform structures, while the console is a pure command line
interpreter, outputting strings that are not consumable by clients.
Does that make sense?
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/- Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Jun 15, 2011 at 9:28 PM, Marcelo Barbudas nos...@gmail.com wrote:
n.outE.inV.inE.outV.groupCount(m) -1;m.sort{a,b - b.value = a.value}
Yep, this versions runs and returns.
However the returned values from the web interface are different from
the ones via the gremlin plugin.
-m.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] gremlin rest plugin and map
Well,
you get as the result from your expression whatever the last statement
is, since from a script, there is only one return construct. You could
try putting things into a map or so ( both the count and the list of
nodes) and return that?
Alternatively, we could send back all bound variables in the return,
but that is very cryptic in its side effects and IMHO not a clean way
to do this. WDYT?
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Jun 15, 2011 at 9:51 PM, Marcelo Barbudas nos...@gmail.com wrote:
Hey Peter,
I think I understand.
So that means I can get a sort listed of nodes (the contents of m keys),
but not the actual number on which the sorting was based (m values)?
-m.
On 06/15/2011 10:46 PM, Peter Neubauer wrote:
Marcelo,
the Gremlin plugin is transforming the results from a query into Neo4j
REST conform structures, while the console is a pure command line
interpreter, outputting strings that are not consumable by clients.
Does that make sense?
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Jun 15, 2011 at 9:28 PM, Marcelo Barbudas nos...@gmail.com wrote:
n.outE.inV.inE.outV.groupCount(m) -1;m.sort{a,b - b.value = a.value}
Yep, this versions runs and returns.
However the returned values from the web interface are different from
the ones via the gremlin plugin.
-m.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] gremlin rest plugin and map
I was doing something wrong on my end. I could see the resulting map but
with different values in the web console from the rest plugin. It turns
out there was a lingering variable in the web console that I forgot to
reset and now I am getting the same values.
Thanks Peter.
-m.
On 06/15/2011 11:03 PM, Peter Neubauer wrote:
Well,
you get as the result from your expression whatever the last statement
is, since from a script, there is only one return construct. You could
try putting things into a map or so ( both the count and the list of
nodes) and return that?
Alternatively, we could send back all bound variables in the return,
but that is very cryptic in its side effects and IMHO not a clean way
to do this. WDYT?
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/- Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Jun 15, 2011 at 9:51 PM, Marcelo Barbudas nos...@gmail.com wrote:
Hey Peter,
I think I understand.
So that means I can get a sort listed of nodes (the contents of m keys),
but not the actual number on which the sorting was based (m values)?
-m.
On 06/15/2011 10:46 PM, Peter Neubauer wrote:
Marcelo,
the Gremlin plugin is transforming the results from a query into Neo4j
REST conform structures, while the console is a pure command line
interpreter, outputting strings that are not consumable by clients.
Does that make sense?
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/- Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Jun 15, 2011 at 9:28 PM, Marcelo Barbudas nos...@gmail.com wrote:
n.outE.inV.inE.outV.groupCount(m) -1;m.sort{a,b - b.value = a.value}
Yep, this versions runs and returns.
However the returned values from the web interface are different from
the ones via the gremlin plugin.
-m.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] gremlin rest plugin and map
Great!
I plan to upgrade to Gremlin 1.1 tomorrow if the source is with me,
and add support even for the table() return construct, so you can do
the full shebang there :)
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Jun 15, 2011 at 10:10 PM, Marcelo Barbudas nos...@gmail.com wrote:
I was doing something wrong on my end. I could see the resulting map but
with different values in the web console from the rest plugin. It turns
out there was a lingering variable in the web console that I forgot to
reset and now I am getting the same values.
Thanks Peter.
-m.
On 06/15/2011 11:03 PM, Peter Neubauer wrote:
Well,
you get as the result from your expression whatever the last statement
is, since from a script, there is only one return construct. You could
try putting things into a map or so ( both the count and the list of
nodes) and return that?
Alternatively, we could send back all bound variables in the return,
but that is very cryptic in its side effects and IMHO not a clean way
to do this. WDYT?
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Jun 15, 2011 at 9:51 PM, Marcelo Barbudas nos...@gmail.com wrote:
Hey Peter,
I think I understand.
So that means I can get a sort listed of nodes (the contents of m keys),
but not the actual number on which the sorting was based (m values)?
-m.
On 06/15/2011 10:46 PM, Peter Neubauer wrote:
Marcelo,
the Gremlin plugin is transforming the results from a query into Neo4j
REST conform structures, while the console is a pure command line
interpreter, outputting strings that are not consumable by clients.
Does that make sense?
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Jun 15, 2011 at 9:28 PM, Marcelo Barbudas nos...@gmail.com wrote:
n.outE.inV.inE.outV.groupCount(m) -1;m.sort{a,b - b.value =
a.value}
Yep, this versions runs and returns.
However the returned values from the web interface are different from
the ones via the gremlin plugin.
-m.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] User Digest, Vol 51, Issue 96
Agelos, we are just testing help.neo4j.org, you might try to start a discussion there if that is more convenient? It's web based and should read more like a forum if you like that style. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Wed, Jun 15, 2011 at 8:53 PM, Agelos Pikoulas agelos.pikou...@gmail.com wrote: Dear all / Im' sorry if I cant use the user@lists properly, I am indeed lost :-( Neo4J would be so much better as a forum or a stackOverNeo4J :-)***/ Allow me to say, that the 50K magic number is not very useful for real practical modern Social Network apps. What if there's simply a couple of million Person nodes that may LIKE the Movie nodes? And what if I have a few million of Movies and many million of Persons ? Its a typical case a movie having a few 100K rating/votes. And imagine if I have Song, Book Product nodes! I think this issue is *MAJOR* and it needs to be promoted to a high priority to the neo4j team. The proxy solution sounds wonderful, but it can be quite a hassle if its not rightly encapsulated transparent. I think all Traversals will become quite hacked I can't even think what will happen to Object mapping etc. I imagine it COULD be part of an upcoming version of the new amazing Spring Data Graph framework (check it out!), where a simple Annotation such as @NodeWithProxy along with information for what *RelationshipTypes / Directions *should go to the real or the proxy Node, could do all of the proxy magic!!! But, the *RelationshipType/Direction indexing *I proposed, I dare say, could be a more generic and cleaner idea, and also a quicker hack! All we need is a method TraversalDescription.*index(myIndex);* where we can declare which index should be used for looking up the (few) RelationshipTypes/Directions among the millions on the Node. The best thing is that we have already declared those on TraversalDescription.*relationships(*MyRelationshipType.hasPart, Direction.OUTGOING). The *Traversal *would then follow (only) those found on the index! Bingo We could also have a *.followIndexedOnly(false) *and even *recreateFollowedIndexes(true) *to save us next time! In any case, something must be implemented! Without being an expert on neo4j, I think there is a lot of Indexing optimization needed yet! Michael what do you think ? Could you please see this being promoted to the team while sharing their views? Agelos Date: Wed, 15 Jun 2011 17:57:55 +0200 From: Balazs E. Pataki pat...@dsd.sztaki.hu Subject: Re: [Neo4j] Slow Traversals on Nodes with too many Relationships To: Neo4j user discussions user@lists.neo4j.org Message-ID: 4df8d683.8010...@dsd.sztaki.hu Content-Type: text/plain; charset=ISO-8859-1; format=flowed Hi, when we started to evaluate neo4j we made some measurements and for us it seemed that 50.000 is a magical number: this many relationships and properties on one node seemed to be a limit, which once reached makes things slow. But we didn't actually need that much relationship/property in our case, so we could live with it, or could make workarounds (eg. storing things in properties and doing indexed lookups instead of using relationships) An automatic indexed lookup on relationship types and directions would be awsome, definitely. Regards, --- balazs Date: Wed, 15 Jun 2011 23:19:32 +0800 From: Craig Taverner cr...@amanzi.com Subject: Re: [Neo4j] Slow Traversals on Nodes with too many Relationships To: Neo4j user discussions user@lists.neo4j.org Message-ID: banlktine_mk5+9damh07tsrq6nnxifo...@mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 Could this also be related to the possibility that in order to determine relationship type and direction, the relationships need to be loaded from disk? If so, then having a large number of relationships on the same node would decrease performance, if the number was large enough to affect the disk io caching. If this is the case, perhaps adding a proxy node for the incoming relationships would work-around the problem? Of course then you have doubled the number of part nodes (two for each part, one part and one containers proxy). Date: Wed, 15 Jun 2011 18:40:05 +0300 From: Agelos Pikoulas agelos.pikou...@gmail.com Subject: Re: [Neo4j] Slow Traversals on Nodes with too many Relationships To: user@lists.neo4j.org Message-ID: banlktinsmadbatf1rglaj4wqngkfekj...@mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 Re: [Neo4j] Slow Traversals on Nodes with too many Relationships I have to respectfully agree
Re: [Neo4j] gremlin rest plugin and map
Tables look great. Just right now googled them.
Will they return something JSON encoded like [ row, row ],
where row is [ hash, hash ]?
-m.
On 06/15/2011 11:13 PM, Peter Neubauer wrote:
Great!
I plan to upgrade to Gremlin 1.1 tomorrow if the source is with me,
and add support even for the table() return construct, so you can do
the full shebang there :)
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/- Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Jun 15, 2011 at 10:10 PM, Marcelo Barbudas nos...@gmail.com wrote:
I was doing something wrong on my end. I could see the resulting map but
with different values in the web console from the rest plugin. It turns
out there was a lingering variable in the web console that I forgot to
reset and now I am getting the same values.
Thanks Peter.
-m.
On 06/15/2011 11:03 PM, Peter Neubauer wrote:
Well,
you get as the result from your expression whatever the last statement
is, since from a script, there is only one return construct. You could
try putting things into a map or so ( both the count and the list of
nodes) and return that?
Alternatively, we could send back all bound variables in the return,
but that is very cryptic in its side effects and IMHO not a clean way
to do this. WDYT?
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/- Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Jun 15, 2011 at 9:51 PM, Marcelo Barbudas nos...@gmail.com wrote:
Hey Peter,
I think I understand.
So that means I can get a sort listed of nodes (the contents of m keys),
but not the actual number on which the sorting was based (m values)?
-m.
On 06/15/2011 10:46 PM, Peter Neubauer wrote:
Marcelo,
the Gremlin plugin is transforming the results from a query into Neo4j
REST conform structures, while the console is a pure command line
interpreter, outputting strings that are not consumable by clients.
Does that make sense?
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/- Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Jun 15, 2011 at 9:28 PM, Marcelo Barbudas nos...@gmail.com
wrote:
n.outE.inV.inE.outV.groupCount(m) -1;m.sort{a,b - b.value =
a.value}
Yep, this versions runs and returns.
However the returned values from the web interface are different from
the ones via the gremlin plugin.
-m.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] gremlin rest plugin and map
Yes,
I am thinking along the lines of the Cypher return tables, see
http://docs.neo4j.org/chunked/snapshot/cypher-plugin.html in which
there is a column with Node representations, and a column with just a
String, etc.
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Jun 15, 2011 at 10:20 PM, Marcelo Barbudas nos...@gmail.com wrote:
Tables look great. Just right now googled them.
Will they return something JSON encoded like [ row, row ],
where row is [ hash, hash ]?
-m.
On 06/15/2011 11:13 PM, Peter Neubauer wrote:
Great!
I plan to upgrade to Gremlin 1.1 tomorrow if the source is with me,
and add support even for the table() return construct, so you can do
the full shebang there :)
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Jun 15, 2011 at 10:10 PM, Marcelo Barbudas nos...@gmail.com wrote:
I was doing something wrong on my end. I could see the resulting map but
with different values in the web console from the rest plugin. It turns
out there was a lingering variable in the web console that I forgot to
reset and now I am getting the same values.
Thanks Peter.
-m.
On 06/15/2011 11:03 PM, Peter Neubauer wrote:
Well,
you get as the result from your expression whatever the last statement
is, since from a script, there is only one return construct. You could
try putting things into a map or so ( both the count and the list of
nodes) and return that?
Alternatively, we could send back all bound variables in the return,
but that is very cryptic in its side effects and IMHO not a clean way
to do this. WDYT?
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Jun 15, 2011 at 9:51 PM, Marcelo Barbudas nos...@gmail.com wrote:
Hey Peter,
I think I understand.
So that means I can get a sort listed of nodes (the contents of m keys),
but not the actual number on which the sorting was based (m values)?
-m.
On 06/15/2011 10:46 PM, Peter Neubauer wrote:
Marcelo,
the Gremlin plugin is transforming the results from a query into Neo4j
REST conform structures, while the console is a pure command line
interpreter, outputting strings that are not consumable by clients.
Does that make sense?
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph
database.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Wed, Jun 15, 2011 at 9:28 PM, Marcelo Barbudas nos...@gmail.com
wrote:
n.outE.inV.inE.outV.groupCount(m) -1;m.sort{a,b - b.value =
a.value}
Yep, this versions runs and returns.
However the returned values from the web interface are different from
the ones via the gremlin plugin.
-m.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
Re: [Neo4j] Most Efficient way to query in my use cases
Hi Manav, I think there's a relationship missing here. Pen--SOLD_BY--shop That way it's easy to find all the pens that a shop sold, and who them sold them to. In general modelling your domain expressively does not come at an increase cost with Neo4j (caveat: you can still create write hotspots). Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Cannot find a path
Hi Josh, It's a safe assumption that JSON keys will be underscore delimited. If you find something that isn't consider it a bug and let us know. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Most Efficient way to query in my use cases
Another common thing to do in this case is create a node for the purchase action. This node would be related to the purchaser (user), item (pen) and shop, and would contain data appropriate to the purchase (date/time, price, etc). Then traverse from the shop or the pen to all purchase actions that reference the other one (shop or pen). On Thu, Jun 16, 2011 at 4:48 AM, Jim Webber j...@neotechnology.com wrote: Hi Manav, I think there's a relationship missing here. Pen--SOLD_BY--shop That way it's easy to find all the pens that a shop sold, and who them sold them to. In general modelling your domain expressively does not come at an increase cost with Neo4j (caveat: you can still create write hotspots). Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Most Efficient way to query in my use cases
Absolutely! Funny how entities start falling about the woodwork when you think graphy. There's a purchase entity that links the specific pen, shopper, and shop too. You might even boil it down to line items if that makes sense in this domain. And don't get me going on where the pen was (ethically) sourced - so many interesting things to explore. Jim On 16 Jun 2011, at 00:34, Craig Taverner wrote: Another common thing to do in this case is create a node for the purchase action. This node would be related to the purchaser (user), item (pen) and shop, and would contain data appropriate to the purchase (date/time, price, etc). Then traverse from the shop or the pen to all purchase actions that reference the other one (shop or pen). On Thu, Jun 16, 2011 at 4:48 AM, Jim Webber j...@neotechnology.com wrote: Hi Manav, I think there's a relationship missing here. Pen--SOLD_BY--shop That way it's easy to find all the pens that a shop sold, and who them sold them to. In general modelling your domain expressively does not come at an increase cost with Neo4j (caveat: you can still create write hotspots). Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Slow Traversals on Nodes with too many Relationships
I understood that on windows the memory mapped sizes needed to be included in the heap, since they are not allocated outside the heap as they are on linux/mac. So in this case he needs a larger heap (and make sure the memory mapped files are much smaller than the heap). The relevant part of the configuration settings doc says: When running Neo4j on Windows the size of the memory-mapped nioneo configurations need to be added to the heap size parameter. On Linux and Unix-systems memory mapped IO is not included in the heap size. I still think that the solution to this case is to group the different relationship types into separate sub-graphs, so that the performance of traversing HAS_ONE is not affected by the number of relationships of CONTAINS. Of course traversing the CONTAINS will still be slow without increasing the cache, as you suggest. On Thu, Jun 16, 2011 at 12:07 AM, Michael Hunger michael.hun...@neotechnology.com wrote: Agelos, sorry, didn't want to sound that way. 512M ram is not very much for larger graphs. Neo4j has to cache nodes, relationships in the heap as well as you own datastructures. The memory mapped files for the datastores are kept outside the heap. Normally with your 4G I'd suggest using about 1.5G for heap and 1.5G for the memory mapped files. http://wiki.neo4j.org/content/Configuration_Settings Do you have a small test-case available that creates your graph and runs your traversal? Then I could have a look at that and also do some profiling to determine the issues for this slowdown. The indexing doesn't help as it also has to hit caches or disk. The graph traversal is normally a very efficient operation that shouldn't experience this bad performance. Cheers Michael P.S. I just use my mail client for handling the mailing list and it works fine for me. Imho Gmail groups threads automatically. Am 15.06.2011 um 17:40 schrieb Agelos Pikoulas: Re: [Neo4j] Slow Traversals on Nodes with too many Relationships I have to respectfully agree with Rick Bullotta. I was suspecting the big-O is not linear for this case. To verify I added x4 Container nodes (400.000) and their appropriate Relationships, and it is now *unbelievably* slow : It does not take x4 more, but it takes more than 30-40 seconds for each next() Remind you 100K nodes = ~2secs for each next() !!! And only to make matters worse, the subsequent runs weren't fast either - they actually took more time than the first (1st TotalTraversalTime= 389936ms, 2nd TotalTraversalTime= 443948ms) The whole setup is running on Eclipse 3.6, with -Xmx512m on JavaVM, Windows2003 VMWare machine with 4GB, running on a fast 2nd gen SSD (OCZ Vertex 2). The neo4J data resides on this SSD. The 100.000 nodes data files were ~250MB, the 400.000 one is ~1GB. I wonder what would happen if the Container nodes were a few million (which will be my case) - it will run forever. Could you please looking into my suggestion - i.e Using a 'smart' behind the scenes Indexing on both *RelationshipType* and *Direction* that Traversals actually use to boost things up ? To another topic, how does one use this mailing list - I use it through gmail and I am utterly lost - is there a better client/UI to actually post/reply into threads ? ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Slow Traversals on Nodes with too many Relationships
Angelos, I spoke with Johan and he said that traversal iterations than scale not linearly with the numbers of rels indicate a configuration issue or are a bug. Craig reminded me that on Windows the memory mapped buffers are in the heap. Could you please increase your heap to 2 or 3 GB an rerun your test. And I want to urge you to provide you test code so that we can reproduce the issue. Thanks for your detailed suggestions. As outlined above this should not happen at all. So I don't think these optimizations are necessary. Cheers Michael Sent from my iBrick4 Am 15.06.2011 um 17:40 schrieb Agelos Pikoulas agelos.pikou...@gmail.com: Re: [Neo4j] Slow Traversals on Nodes with too many Relationships I have to respectfully agree with Rick Bullotta. I was suspecting the big-O is not linear for this case. To verify I added x4 Container nodes (400.000) and their appropriate Relationships, and it is now *unbelievably* slow : It does not take x4 more, but it takes more than 30-40 seconds for each next() Remind you 100K nodes = ~2secs for each next() !!! And only to make matters worse, the subsequent runs weren't fast either - they actually took more time than the first (1st TotalTraversalTime= 389936ms, 2nd TotalTraversalTime= 443948ms) The whole setup is running on Eclipse 3.6, with -Xmx512m on JavaVM, Windows2003 VMWare machine with 4GB, running on a fast 2nd gen SSD (OCZ Vertex 2). The neo4J data resides on this SSD. The 100.000 nodes data files were ~250MB, the 400.000 one is ~1GB. I wonder what would happen if the Container nodes were a few million (which will be my case) - it will run forever. Could you please looking into my suggestion - i.e Using a 'smart' behind the scenes Indexing on both *RelationshipType* and *Direction* that Traversals actually use to boost things up ? To another topic, how does one use this mailing list - I use it through gmail and I am utterly lost - is there a better client/UI to actually post/reply into threads ? ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
