Re: [Neo4j] Facetted search in neo4j
Hi, Nils. We've implemented something like this here at ThingWorx, but we had to do so at a layer above Neo4J since there, of course, were no built-in faceting capabilities. One of the major challenges we faced was the opaque connection from the Neo4J API to the underlying Lucene engine. There are quite a few things we would/could have done differently if we could access the underlying Lucene IndexReaders directly, but that isn't possible. Nor would it probably be wise, since Neo4J's indexing framework does a lot of transaction isolation magic for Lucene that you'd run the risk of interfering with. Also, you'll probably find that you need to implement faceting at a higher level anyway, since in Neo4J, the concept of a document in an index is merely its node or relationship ID - not the set of properties associated with it, so you'll need to do most of that work of reconstituting a document-oriented resultset elsewhere. A few other things to consider: - Cypher and/or Gremlin might provide ways to calculate individual facets via their aggregate functions. The only problem with this approach is that you'd probably need to make multiple traversals to calculate multiple unrelated facets. In code we can calculate those facets in a single pass - We use HashSets quite pervasively to keep lists of unique results that were the intersection of multiple searches and to manage collections of facets. The LinkedHashSet has been very helpful since it preserves the order with which items were added - Neo4J's index framework does not make it easy to (efficiently) get an exhaustive list of facet values/index values, if you're more interested in the key/value pairs and their frequency rather than the documents themselves I think it would be a nice feature to the Neo4J Lucene index implementation if we could have access to the IndexReader/IndexWriter with some specific caveats on what is safe to do. We contemplated hacking our own mods to the index framework to get access to this, but since we'd be missing the second part (the caveats), we felt like we'd just get into trouble. Hope this helps. Rick -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Nils Kaiser Sent: Thursday, December 08, 2011 6:59 AM To: user@lists.neo4j.org Subject: [Neo4j] Facetted search in neo4j Hi, I'm new to neo4j and graph databases and currently experimenting with a video sharing application. One thing I don't see mention on the neo4j list a lot is facetted search. Is it because it is trivial to implement using graphs? I've used solr before, where I can return both search results and facet info (number of objects that would match that facet given the current search terms / facet choices) on a single call, so quite a luxury and really easy to implement. What would be the best approach with neo4j? I probably need to write a traversal that changes its stop conditions according to already entered search terms or facets, right? Anybody some examples or things to consider? Thanks, Nils -- Nils Kaiser MSc in Information Systems ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ NOTICE: THIS MAILING LIST IS BEING SWITCHED TO GOOGLE GROUPS, please register and consider posting at https://groups.google.com/forum/#!forum/neo4j Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j for a CMS backend
FWIW, we use Neo4J also as our dependency graph in ThingWorx. We can determine deep, two-way dependencies quite easily, so that when any object in ThingWorx is modified, we can intelligently do hot updates to affected entities without having to stop and restart the entire server. It would have been all but impossible to map in a traditional data store. From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Peter Neubauer [peter.neuba...@neotechnology.com] Sent: Wednesday, December 07, 2011 4:15 AM To: ne...@googlegroups.com Cc: user@lists.neo4j.org Subject: Re: [Neo4j] Neo4j for a CMS backend Haha, sorry, my bad. I meant CMDB actually :) Thanks for pointing that out Axel! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer brew install neo4j neo4j start heroku addons:add neo4j On Wed, Dec 7, 2011 at 9:30 AM, Axel Morgner a...@morgner.de wrote: Hi Peter, seems like you confused CMDB (Configuration Management Database) with CMS (Content Management System)? While Neo4j is perfect for building both, the use cases don't have very much in common ... Greetings Axel Am 07.12.2011 09:26, schrieb Peter Neubauer: Hi folks, thought this might be interesting for some of you ... http://www.dzone.com/links/r/why_im_pretty_excited_about_using_neo4j_for_a_cmd.html Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer brew install neo4j neo4j start heroku addons:add neo4j ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Standalone server and transactions
...though there are perfectly good reasons to include business logic on the server as well (e.g. stored procedures, which Neo4J can support in its own way via server-side extensions). -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Jim Webber Sent: Friday, December 02, 2011 6:36 AM To: Neo4j user discussions Subject: Re: [Neo4j] Standalone server and transactions It is a long topic on itself: Where the business logic belongs to - the server or the client. But the point is that far the most common use-case is to write the business logic on the client, not on the server. The business logic on the server has already failed multiple times in the history (think of stored procedures on the RDBMS). Server side logic works well when the DB is part of the app (e.g. embedded) and server/client code is often indistinguishable. I don't believe that we are talking about business logic on the server - we're talking about data access logic (queries). Business logic binds to that data over the network - that the data is sourced through a plugin is an implementation detail. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Standalone server and transactions
...or you can re-think the definition of client. As you know, it is very common in an SOA model to have actors functioning as both clients and servers. For example, to the client making a reservation, the server might be the airline's reservation site. That site might have server-side logic to manage the complexity of the reservations process, which is interacting with external billing sites as well as one or more external data stores or other services (in this case, perhaps a Neo4J database via the REST API). In that scenario, the reservations server is the client to the database server. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Dmytrii Nagirniak Sent: Friday, December 02, 2011 7:04 AM To: Neo4j user discussions Subject: Re: [Neo4j] Standalone server and transactions On 02/12/2011, at 10:36 PM, Jim Webber wrote: I don't believe that we are talking about business logic on the server - we're talking about data access logic (queries). Business logic binds to that data over the network - that the data is sourced through a plugin is an implementation detail. Well, this scenario (Reserving a seat) includes a lot of business logic, for example. User enters the details of the reservation and clicks Place reservation. Then the system has to: Find available seat. Validate user details agains that seat (it may include a lot of logic in itself). Place reservation and mark the seat as unavailable. No double reservations allowed. Now, how can you do it with REST API in a transaction if you keep the logic on the client? The only way to do this, is to move the whole logic over to the server. And this leads to the points from my previous post. We just have to accept that some scenarios can't be handled over REST with *reasonable* effort. In theory, you can write the whole app on the server (it would be embedded). But this defeats the purpose. This is about trade-off: need transactions - use embedded DB, need multiple clients for the DB - use REST. If you need both, reevaluate. If still you do. Then either pay for Enterprise or implement your own locking system. Cheers. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Standalone server and transactions
FWIW, at present Neo4J has among the most comprehensive ACID support of the major NoSQL databases. The fact that you might have to write a bit of code or script to run on the server shouldn't necessarily be viewed as a negative - the fact that you *can* do this at all is a big plus! Most NoSQL DB's take a very atomic view of transactions - usually single object additions, deletions, or modifications. Neo4J is one of the few that uses real JTA transactions across multiple database modifications. I also agree with you that it would be nice to have transaction support over a remote API, but that's not a trivial thing to accomplish in a web scenario when there is not any thread affinity or even necessarily server affinity between requests that occur within a transaction. Knowing when to abandon a pending transaction in a scenario where the client(s) do not maintain a persistent connection is also quite challenging. One possibility might be to leverage another RPC mechanism or perhaps even something like Web Sockets in the not too distant future. But I think the goal of a language neutral binding also needs to be preserved if possible. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of zolv Sent: Friday, December 02, 2011 8:33 AM To: user@lists.neo4j.org Subject: Re: [Neo4j] Standalone server and transactions dnagir wrote On 02/12/2011, at 11:28 PM, Rick Bullotta wrote: Doesn't matter how you play the words, you need transactions somewhere. And with REST it is only available as a database plugin. For most people it is just not worth the effort (especially who's not coming from Java background) and they will turn around. Exactly. Unfortunately I am such person who must find some other NoSQL DB :/ because of these 2 mentioned cases. But I promise to come back later and check if standalone + transactions are ready :) Cheers. -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Standalone-server-and-transactions-tp3553058p3554646.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Heroku, Neo4j and Google Spreadsheet in 10min. Flat.
Nice! Show me how to access those services and I'll build you a killer UI for it using ThingWorx - in 10 minutes. ;-) -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Peter Neubauer Sent: Friday, December 02, 2011 11:51 AM To: Neo4j user discussions; neo4jrb Subject: [Neo4j] Heroku, Neo4j and Google Spreadsheet in 10min. Flat. Hi all, todays lab project: http://vimeo.com/33032604 Thought you might enjoy it. Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer brew install neo4j neo4j start heroku addons:add neo4j ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Will there ever be a native SDK for Node.js?
Why not take a look at RingoJS? Would be uber easy to integrate native Neo4J APIs. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of yobi Sent: Wednesday, November 30, 2011 10:59 PM To: user@lists.neo4j.org Subject: [Neo4j] Will there ever be a native SDK for Node.js? Have been reading through the Java documentation. Having a native SDK seems to be a big win when interacting with Neo4j. I wonder if you have any plans on making a native SDK for Node.js since apps on Heroku could run on Node.js as well. Thanks Johnny -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Will-there-ever-be-a-native-SDK-for-Node-js-tp3550518p3550518.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] collation and wild card queries
We don't use Solr, but we use some of Solr's analyzers and filters. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Niels Hoogeveen Sent: Monday, November 28, 2011 2:14 PM To: user@lists.neo4j.org Subject: Re: [Neo4j] collation and wild card queries Hi Rick, Thanks for the response. I will look into this. It seems there are quite a number of features to Solr that could be of interest. Are you using Neo4j in conjunction with Solr, and if so, how have you been able to integrate the two? Niels From: rick.bullo...@thingworx.com To: user@lists.neo4j.org Date: Mon, 28 Nov 2011 09:14:20 -0700 Subject: Re: [Neo4j] collation and wild card queries Hi, Niels. Does this help? http://lucene.472066.n3.nabble.com/trouble-with-CollationKeyFilter-td3532170.html Seems there may be a patch out there to provide some additional wildcard/range capabilities for these Analyzers. Note also that in general, Solr analyzers/filters can be used with Lucene by themselves. Rick -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Niels Hoogeveen Sent: Monday, November 28, 2011 10:26 AM To: user@lists.neo4j.org Subject: [Neo4j] collation and wild card queries In order to have proper sort order for Strings with diacritical characters, I started using Lucene's ICUCollationKeyAnalyzer. This indeed gives the proper sort order for queries, but for some reason wild card queries no longer seem to work. This applies for both the normal CollationKeyAnalyzer and for the ICU variant. Exact queries work, but as soon as a wild card is added the query no longer returns any results. Does anyone have an idea how to solve this? I'd like to be able to have an index that allows both diacritics-aware sort order and support for wild cards. Niels ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j child threads segfaulting?
Could really only be a VM bug I'd think!? No native code in Neo4J that I'm aware of. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Taylor Phillips Sent: Monday, November 28, 2011 6:12 PM To: user@lists.neo4j.org Subject: [Neo4j] Neo4j child threads segfaulting? Hi all, Our Neo4j server has been up and running for several months with over a million nodes and properties. Today, we noticed a significant response latency and a run of strace on the neo4j server yielded several segfaulting child threads. We don't have more details at the moment, but were curious if anyone else had seen this behavior. We are running Neo4j 1.5 Community server on RedHat 4.1.2-50. Cheers, Taylor ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] How to boost performance?
The companion piece of this test is how fast you can get data *into* your
database. Have you run a comparison of those two scenarios with MySQL vs
Neo4J? We found Neo4J to be substantially faster with inserting/updating data
as long as we executed our transactions in blocks of 50-1000.
-Original Message-
From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On
Behalf Of Vinicius Carvalho
Sent: Wednesday, November 23, 2011 6:55 AM
To: Neo4j user discussions
Subject: Re: [Neo4j] How to boost performance?
Hi Peter, thanks, that indeed boosts to lower 10 ms, but we really need to
access the nodes to perform the operations.
I know that to be fair on a test, we should be running an embedded version
of neo4j inside the appserver, after all the cache resides in the same JVM.
But I'm not worried with performance against cache, just would like to have
a faster access versus the RDBMS.
I'm working on a version using an embedded read only db pointing to the
server data files, hope it boosts performance a lot :). I'll be fare and
give the same amount of memory as I would give to the cache, so I can
benefit from Object caching on neo as well.
Regards
On Wed, Nov 23, 2011 at 11:44 AM, Peter Neubauer
peter.neuba...@neotechnology.com wrote:
Vinicius,
in order to cut down on the REST JSON overhead (which you don't have
in the RDBMS case), maybe you could look at just counting the results,
something like
START n = node(3)
MATCH n--()--(x)
return count(x)
And see what happens?
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - NOSQL for the Enterprise.
http://startupbootcamp.org/- Öresund - Innovation happens HERE.
On Wed, Nov 23, 2011 at 11:54 AM, Vinicius Carvalho
java.vinic...@gmail.com wrote:
Hi there, I've posted a few days ago about the POC I'm doing here at my
company. I have some initial numbers and I'd like to ask for some help
here
in order to promote neo4j here in LMI Ericsson.
I've loaded a mySQL db with a really simple entity, that pretty much only
represents a node and relations (only properties it has is an UID and x/y
space coordinate for each node)
The DB contains 250.000 cells and 19. relations stored in a myISAM table,
indexed only by it's primary key. Please find the DDL for the two tables.
CREATE TABLE `pci`.`cells` (
`id` varchar(32) collate utf8_bin NOT NULL,
`x_pos` double default NULL,
`y_pos` double default NULL,
`pci` smallint(6) default '0',
PRIMARY KEY (`id`)
)
CREATE TABLE `pci`.`relations` (
`id` int(11) NOT NULL auto_increment,
`source` varchar(32) collate utf8_bin default NULL,
`target` varchar(32) collate utf8_bin default NULL,
PRIMARY KEY (`id`),
KEY `src_idx` (`source`),
KEY `src_target` (`target`)
)
So as you can see, a simple secondary table contains the relationship
with
source and targets pointing to the cells table.
I've loaded this exact same DB into a neoserver running on the same
machine: A Blade with 26 cpus (6 cores each) and 16gb RAM.
One of the requirements we have is to find all associations of my
associations. Something that in neo I did like this:
START n = node(3)
MATCH n--()--(x)
return x
For this specific node it returns 6475 nodes.
I have tested this before using Hibernate in two modes: without a L2
cache,
and with an L2 Cache (Ehcache standalone no replication).
Here's a snippet of the code that loads it, so you can understand what's
going under the hood:
@Override
public ListCell loadCellWithRealtions(String... ids) {
Session session = (Session) em.getDelegate();
Criteria c = session.createCriteria(Cell.class)
.setFetchMode(incomingRelations, FetchMode.SELECT)
.setFetchMode(outgoingRelations, FetchMode.SELECT)
.add(Restrictions.in(id, Arrays.asList(ids)));
ListCell results = c.list();
for(Cell cell : results){
Hibernate.initialize(cell.getIncomingRelations());
Hibernate.initialize(cell.getOutgoingRelations());
}
return results;
}
@Override
public ListCell loadCellWithNeighbourRelations(String... ids) {
ListCell cells = loadCellWithRealtions(ids);
for(Cell c : cells){
for(Relation r : c.getIncomingRelations()){
Hibernate.initialize(r.getSource().getIncomingRelations());
Hibernate.initialize(r.getSource().getOutgoingRelations());
}
for(Relation r : c.getOutgoingRelations()){
Hibernate.initialize(r.getTarget().getIncomingRelations());
Hibernate.initialize(r.getTarget().getOutgoingRelations());
}
}
return cells;
}
So the first method executes one query and 2 subselects to find a cell
and
all relations, the second method, iterate over each relation and do the
same. So I pretty much will have something like 3+r*3 selects on db,
where
Re: [Neo4j] Is it possible to have relationship attributes?
Absolutely. And fairly unique to Neo4J. Plus you can even index and traverse relationships on those attributes (properties)! -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Stefan Arentz Sent: Wednesday, November 23, 2011 12:34 PM To: user@lists.neo4j.org Subject: [Neo4j] Is it possible to have relationship attributes? Is it possible to add attributes to relationships? I would like to for example have a relationship like Person VISITED Building but then add a timestamp or additional attributes to the VISITED relationship between those two nodes. If this is not supported, what is a good alternative in the world of graphs? S. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Activity Streams and Twitter Sample App
You might also want to take a look at the videos from the Twitter presentations from QCon London. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Aseem Kishore Sent: Monday, November 21, 2011 4:43 PM To: Neo4j user discussions Subject: Re: [Neo4j] Activity Streams and Twitter Sample App This is very interesting -- thanks Peter for the link, and thanks maxdemarzi for starting this conversation. In our social network -- which has extremely little load, it's just in beta -- we currently use basically (a), and it works just fine. We use Cypher to do the sorting/trimming on the server, e.g. roughly this query: START user=(123) MATCH (user) -[:FOLLOWS]- (friend) -[:EVENT]- (event) RETURN event ORDER BY event.timestamp DESC LIMIT 20 This obviously will not scale, since Neo4j has to read every single potential event from disk and process it in memory. We too have been thinking about this in the back of our minds, and the obvious solution in our minds is indeed to use a linked list timeline for events, and have a custom traverser that merge sorts in realtime. Since this'd be a linked list where you want the newest events at the head of the list, there will indeed be a small write-time cost of having to update two relationships whenever a new event is created (i.e. inserting it at the head of the list). Rene mentions this approach as the first (baseline) approach he tried. It's at 3:08 in the video in that post. He decides that approach isn't good enough, however, because merge sorting depends on the number of friends you have (# users you follow), which he wants to avoid at read-time. IMHO however, that's not worth optimizing for. It's rare that a user follows a very high number of other people. E.g. it's usually around let's say 100-1000 people. The flip side -- the # of *followers* -- can be very high. And interestingly, Rene's solution suffers from *that* at write-time. E.g. a new event has to update every *followers'* Graphity index. I'll provide Rene this feedback, but FWIW, I have the feeling that a simple timeline will easily be good enough, in fact optimal, for a social network on Neo4j: - O(1) writes - O(d) reads in space performance - O(d log d) reads in time performance Where d is the out-degree -- # of users you follow -- which is generally small. Cheers, Aseem On Mon, Nov 21, 2011 at 6:07 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: You might even be interested in Rene Pickards work on a full solution (albeit with some write-time overhead), see http://www.rene-pickhardt.de/graphity-an-efficient-graph-model-for-retrieving-the-top-k-news-feeds-for-users-in-social-networks/ Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - NOSQL for the Enterprise. http://startupbootcamp.org/- Öresund - Innovation happens HERE. On Thu, Nov 3, 2011 at 10:36 PM, maxdemarzi maxdema...@gmail.com wrote: I had not considered imperfect solutions, and in some activity stream scenarios a sampling of the last few messages could work. The sample would have to be taken from all Person nodes because if we sample from the Tweets in general and we encounter a chatty person node early on, it would take up all the sample space. Person1 - Person 2 - 10 tweets Person1 - Person3 - 1000 tweets = Sample Size reached, traversal stops Person1 - Person4 - 10 tweets Person1 - PersonX - 10 tweets Person3 would prevent Person4 to PersonX's tweets from ever making it to the sample. Some applications in domains like Financials or Network Monitoring may require the last known status and sampling might not be acceptable. -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Activity-Streams-and-Twitter-Sample-App-tp3477669p3478477.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Batch Insert : poooor performance
That seems about normal. The good news is that it is much faster (usually) than an RDBMS on the same hardware. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Krzysztof Raczynski Sent: Friday, November 18, 2011 6:47 AM To: Neo4j user discussions Subject: Re: [Neo4j] Batch Insert : pr performance Btw, inserting 600k nodes over REST with about 8 properties in batches of 100 takes 20-30minutes for me. It's not awesomely fast, but it's not slow either. What settings are affecting insertion speeds, Peter? ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Scalability Roadmap
...but I'm sure the community will come up with a wide range of sharding patterns, code, and best practices! On Nov 18, 2011, at 5:46 PM, Jim Webber j...@neotechnology.com wrote: Hey Matt, Not to nitpick, but that's for an ideal graph partitioning, not graph sharding overall, right? Eg the problem is solvable in many specific domains? You're right - it's the general case. I was just making the point that sharding isn't something that's an afternoon's hacking to complete. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Gremlin plugin and script engine question
It iterates.
Sorry, couldn't resist. ;-)
-Original Message-
From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On
Behalf Of Alfredas Chmieliauskas
Sent: Thursday, November 17, 2011 12:31 PM
To: Neo4j user discussions
Subject: Re: [Neo4j] Gremlin plugin and script engine question
Thanks!
What exactly does iterate() do?
A
On Thu, Nov 17, 2011 at 4:07 PM, Marko Rodriguez okramma...@gmail.com wrote:
Hey Alfredas,
Be sure to iterate your pipeline
x = []; g.v(1).out(from).out(to).aggregate(x).loop(3){it.loops
4}.iterate(); x
* NOTE: You can also do:
g.v(1).out(from).out(to).aggregate(x).loop(3){it.loops 4} -1
but the convention is no longer with us in Gremlin 1.4-SNAPSHOT.
HTH,
Marko.
http://markorodriguez.com
On Nov 17, 2011, at 7:21 AM, Alfredas Chmieliauskas wrote:
Dear all,
This concerns gremlin plugin and the script engine.
Maybe there's an explanation for this behavior:
1) gremlin x = [];
g.v(1).out(from).out(to).aggregate(x).loop(3){it.loops 4};
gremlin x;
== v[7]
== v[3]
== v[5]
2) gremlin x = [];
g.v(1).out(from).out(to).aggregate(x).loop(3){it.loops 4}; x;
returns nothing...
Thanks,
Alfredas
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Lucene sort with diacritic characters
You probably need to create a custom analyzer using one of Lucene's collation filters (which you will provide as a parameter to the Neo4J index creation method). Unfortunately, you can't apply a new analyzer after the fact. I think you'll need to delete and regenerate the index. Lucene has some built-in language specific collation filters, but there is also a contributed package, ICUCollationKeyFilter, which may have some advantages in terms of performance. Unfortunately, I do not direct experience in using either, but hopefully this will help get you pointed in the right direction. Rick From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Niels Hoogeveen [pd_aficion...@hotmail.com] Sent: Friday, November 11, 2011 9:27 AM To: user@lists.neo4j.org Subject: Re: [Neo4j] Lucene sort with diacritic characters anyone? From: pd_aficion...@hotmail.com To: user@lists.neo4j.org Date: Thu, 10 Nov 2011 20:20:46 +0100 Subject: [Neo4j] Lucene sort with diacritic characters When retrieving items from a Lucene index, using the sort method, it seems the order doesn't abide proper rules for sorting diacritic characters. For example, Århus comes later in the list than Zürich and Ḩalab comes later than Žužemberk. Can someone help me solve this? Niels ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Lucene sort with diacritic characters
Excellent! A suggestion for the Neo4J team: enhance the Index framework to allow the Map of property values that is passed to the index creation method to be propagated/passed to the constructor for custom analyzers. Niels, there's a trick I use as a workaround, which is to set the parameters in a static class member in our wrapper prior to creating the Neo4J indices, then the no-arg constructor can use those parameters as needed. Hacky but functional. From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Niels Hoogeveen [pd_aficion...@hotmail.com] Sent: Friday, November 11, 2011 11:33 AM To: user@lists.neo4j.org Subject: Re: [Neo4j] Lucene sort with diacritic characters It works like a dream. One note for others needing this functionality. The ICUCollationKeyAnalyzer has a constructor which takes a Collator (from icu4j) as argument. Neo4j's index requires a constructor without arguments, so it's necessary to wrap the ICUCollationKeyAnalyzer and provide it the appropriate Collator in the constructor. For me Collator.SECONDARY was the best choice. Niels From: rick.bullo...@thingworx.com To: user@lists.neo4j.org Date: Fri, 11 Nov 2011 07:33:44 -0700 Subject: Re: [Neo4j] Lucene sort with diacritic characters You probably need to create a custom analyzer using one of Lucene's collation filters (which you will provide as a parameter to the Neo4J index creation method). Unfortunately, you can't apply a new analyzer after the fact. I think you'll need to delete and regenerate the index. Lucene has some built-in language specific collation filters, but there is also a contributed package, ICUCollationKeyFilter, which may have some advantages in terms of performance. Unfortunately, I do not direct experience in using either, but hopefully this will help get you pointed in the right direction. Rick From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Niels Hoogeveen [pd_aficion...@hotmail.com] Sent: Friday, November 11, 2011 9:27 AM To: user@lists.neo4j.org Subject: Re: [Neo4j] Lucene sort with diacritic characters anyone? From: pd_aficion...@hotmail.com To: user@lists.neo4j.org Date: Thu, 10 Nov 2011 20:20:46 +0100 Subject: [Neo4j] Lucene sort with diacritic characters When retrieving items from a Lucene index, using the sort method, it seems the order doesn't abide proper rules for sorting diacritic characters. For example, Århus comes later in the list than Zürich and Ḩalab comes later than Žužemberk. Can someone help me solve this? Niels ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Numeric index update
The call to remove probably removes the node from the index, nit the key from
the node. You need to remove then completely reminded the node, including the
other two keys.
From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of
ptashcka [ptash...@gmail.com]
Sent: Tuesday, November 08, 2011 7:31 AM
To: user@lists.neo4j.org
Subject: [Neo4j] Numeric index update
Hi.
I have a problem with following test (perform it on an empty database).
First we add same node to the index with different keys and values. One of
them is numeric. After that - check that node may be found using these keys
and values. Then - update value for one of the keys and try to get node from
indexes. For text index node is not null, but for numeric index node is not
found, though the query is the same.
public class DBTest {
public static final String TEXT = text;
public static final String NUMERIC = numeric;
public static final String TEXT_1 = text_1;
@Autowired GraphDatabaseService db;
@Test
public void updateIndex() throws Exception {
Transaction tx = db.beginTx();
try {
Node n = db.createNode();
getIndex().add(n, NUMERIC, new ValueContext(5).indexNumeric());
getIndex().add(n, TEXT, text);
getIndex().add(n, TEXT_1, text);
tx.success();
} catch (Exception e) {
tx.failure();
} finally {
tx.finish();
}
Node n = getIndex().query(QueryContext.numericRange(NUMERIC, 5, 5,
true, true)).getSingle();
assertTrue(n != null);
n = getIndex().get(TEXT_1, text).getSingle();
assertNotNull(n);
tx = db.beginTx();
try {
// Following line may be commented, it's addition of node that
causes the problem
getIndex().remove(n, TEXT, text);
getIndex().add(n, TEXT, text 1);
tx.success();
} catch (Exception e) {
tx.failure();
} finally {
tx.finish();
}
n = getIndex().get(TEXT_1, text).getSingle();
assertNotNull(n);
n = getIndex().query(QueryContext.numericRange(NUMERIC, 5, 5, true,
true)).getSingle();
// Test fails here
assertNotNull(n);
}
private IndexNode getIndex() {
return db.index().forNodes(index);
}
}
--
View this message in context:
http://neo4j-community-discussions.438527.n3.nabble.com/Numeric-index-update-tp3490124p3490124.html
Sent from the Neo4j Community Discussions mailing list archive at Nabble.com.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Property loading in Neo4J 1.4
If a node is accessed, are *all* of its properties loaded into memory? Thanks, Rick ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Node Id generation deadlock
FWIW, you might be better off pipelining these writes through a single worker thread/queue. It helps with a few performance issues: 1) you can avoid synchronization concerns 2) you can manage block writes (e.g. a number of writes in a single transaction) more easily and 3) you can (if you choose) implement throttling/queueing to support burst mode scenarios where the # requests to write data exceed your ability to process them (and to provide responsiveness for queries). This is what we do with our activity stream engine in ThingWorx. Note that the same applies for deleting entries - we try to do bulk deletes as well. From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Mattias Persson [matt...@neotechnology.com] Sent: Saturday, November 05, 2011 9:14 AM To: Neo4j user discussions Subject: Re: [Neo4j] Node Id generation deadlock You'd have to go with another solution then. Is your application this critical to write throughput or are you just thinking ahead and making sure that it some day might need to support an amount of write throughput? 2011/11/3 Yaniv Ben Yosef yani...@gmail.com Hi, I've also been wondering about this subject. According to the Neo4J design guide ( http://wiki.neo4j.org/content/Design_Guide) the factory node and id generator patterns are the way to implement sequential ID generation. However, according to this thread, it sounds like in a multi-threaded environment I have one of two choices: 1. lock the factory node. However this will force transaction serialization. That's not practical in my app. 2. reducing the granularity of the transactions - in the sense that each transaction should only contain one node creation (of the same type, i.e. uses the same factory node). That's not practical for me either, because in several cases I would like to create more than one node in the same transaction. Since the design guide recommends the factory node pattern, I'm wondering if there's anything I'm missing here, or that I should just avoid this pattern and use some other ID generation mechanism. Thanks, Yaniv On Thu, Nov 3, 2011 at 11:13 AM, Mattias Persson matt...@neotechnology.comwrote: 2011/11/3 Cres cre...@gmail.com This solution would have been ok if I had only one node created from that factory in each transaction. It doesn't matter... after factory.setProperty is run that transaction has got a write lock on that factory node which is held until the transaction committs. The benefit you get from my proposal would be that you make sure you read the correct value. however, as shown in the sample code I posted in the original message, I have multiple nodes created in one transaction, and multiple such transactions in multiple threads. So while the creation of the actual nodes will of course be serialized, one thread's transaction will have to wait for the other thread's transaction to finish completely, and so if the first thread has some processing to do between the creation of the first and second nodes, the other third won't be able to create its two nodes in the meanwhile, because the first thread will have the lock on the factory node until the entire transaction completes. I'm looking for a way to do this having the nodes creation serialized but without having the entire transactions serialized, possibly by somehow releasing the lock on the factory node in mid-transaction, or by any other method. Thanks again, Ran. -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Node-Id-generation-deadlock-tp3473118p3476498.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Node Id generation deadlock
Oh, and one more benefit - if you want to implement a distributed/sharded storage engine, this abstraction through a write queue makes it much, much simpler and more transparent to the application. From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Rick Bullotta [rick.bullo...@thingworx.com] Sent: Saturday, November 05, 2011 11:24 AM To: Neo4j user discussions Subject: Re: [Neo4j] Node Id generation deadlock FWIW, you might be better off pipelining these writes through a single worker thread/queue. It helps with a few performance issues: 1) you can avoid synchronization concerns 2) you can manage block writes (e.g. a number of writes in a single transaction) more easily and 3) you can (if you choose) implement throttling/queueing to support burst mode scenarios where the # requests to write data exceed your ability to process them (and to provide responsiveness for queries). This is what we do with our activity stream engine in ThingWorx. Note that the same applies for deleting entries - we try to do bulk deletes as well. From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Mattias Persson [matt...@neotechnology.com] Sent: Saturday, November 05, 2011 9:14 AM To: Neo4j user discussions Subject: Re: [Neo4j] Node Id generation deadlock You'd have to go with another solution then. Is your application this critical to write throughput or are you just thinking ahead and making sure that it some day might need to support an amount of write throughput? 2011/11/3 Yaniv Ben Yosef yani...@gmail.com Hi, I've also been wondering about this subject. According to the Neo4J design guide ( http://wiki.neo4j.org/content/Design_Guide) the factory node and id generator patterns are the way to implement sequential ID generation. However, according to this thread, it sounds like in a multi-threaded environment I have one of two choices: 1. lock the factory node. However this will force transaction serialization. That's not practical in my app. 2. reducing the granularity of the transactions - in the sense that each transaction should only contain one node creation (of the same type, i.e. uses the same factory node). That's not practical for me either, because in several cases I would like to create more than one node in the same transaction. Since the design guide recommends the factory node pattern, I'm wondering if there's anything I'm missing here, or that I should just avoid this pattern and use some other ID generation mechanism. Thanks, Yaniv On Thu, Nov 3, 2011 at 11:13 AM, Mattias Persson matt...@neotechnology.comwrote: 2011/11/3 Cres cre...@gmail.com This solution would have been ok if I had only one node created from that factory in each transaction. It doesn't matter... after factory.setProperty is run that transaction has got a write lock on that factory node which is held until the transaction committs. The benefit you get from my proposal would be that you make sure you read the correct value. however, as shown in the sample code I posted in the original message, I have multiple nodes created in one transaction, and multiple such transactions in multiple threads. So while the creation of the actual nodes will of course be serialized, one thread's transaction will have to wait for the other thread's transaction to finish completely, and so if the first thread has some processing to do between the creation of the first and second nodes, the other third won't be able to create its two nodes in the meanwhile, because the first thread will have the lock on the factory node until the entire transaction completes. I'm looking for a way to do this having the nodes creation serialized but without having the entire transactions serialized, possibly by somehow releasing the lock on the factory node in mid-transaction, or by any other method. Thanks again, Ran. -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Node-Id-generation-deadlock-tp3473118p3476498.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com
Re: [Neo4j] Newbie issues with indexing
...because you have a space character in the name. From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Bill Baker [bill...@billbak.com] Sent: Wednesday, November 02, 2011 11:33 AM To: Neo4j user discussions Subject: Re: [Neo4j] Newbie issues with indexing Thanks Mattias. One node is a Foo node and one is a Bar node. Maybe I am being too fancy; I could just use 'NAME' for any type of node. I'm a little confused about the query string below. Should it perhaps be NodeType:Bar AND Name:Bar 1 ? I'm not getting why the property name is inside the quotes for one predicate and not the other (below.) Thanks! -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Mattias Persson Sent: Wednesday, November 02, 2011 3:56 AM To: Neo4j user discussions Subject: Re: [Neo4j] Newbie issues with indexing Hi Bill, Why BarName in one and FooName in the other? I'm assuming you have NodeType/Name for both ok? So... a query like this would look like: nodeIndex.query( NodeType:Bar AND Name:\Bar 1\ ); 2011/11/1 Bill Baker bill...@billbak.com Hello, I have an index and add my nodes to it as I create them. I index two properties, nodeType and nodeName. Later I want to see if a node already exists of a given type and a given name. If it does, I'll use it; otherwise I'll create it (and index it.) (All this is in the Java API.) IndexManager indexMgr = graphDB.index(); IndexNode nodeIndex = indexMgr.forNodes(NodeType); Later on I create a node (newNode) with properties NodeType = Bar and BarName = Bar 1 and another with NodeType = Foo and FooName = Foo 1. nodeIndex.add (newNode, NodeType, Bar); nodeIndex.add (newNode, BarName, Bar 1); nodeIndex.add (newNode2, NodeType, Foo); nodeIndex.add)newNode2, FooName, Foo 1); etc. Still later, I want to see if I already have a node of type Bar with name Bar 1. I read up on the query syntax in 7.7 ( http://docs.neo4j.org/chunked/stable/indexing-search.html) and noticed that the syntax listed there does not support any white space in the query string. So I am looking into how to use Lucene query objects to form my query. But now I'm stumped. I want to get a list (IndexHits is fine) of any nodes in the system that have these two properties (NodeType and BarType) set to specific values. I think there might be more to Lucene Query objects than just TermQuery. Is that in the Neo4j docs, the Lucene docs? Any examples handy? Thanks in advance! Bill -- Bill Baker, Investor, Advisor, Board Member 206-619-0928 My other house is a data warehouse ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Tree structure
I've used d3. You do need a bit of JQuery/Javascript skills to munge the data into a form the d3 libraries expect it, but the results are impressive if you do. From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Andreas Kollegger [andreas.kolleg...@neotechnology.com] Sent: Wednesday, November 02, 2011 1:52 PM To: Neo4j user discussions Subject: Re: [Neo4j] Tree structure Hi Emil, Are you interested in displaying the tree structure in a web interface? You could either take a widget-approach using something like jstree (http://www.jstree.com/) or a more model visualization using D3 (http://mbostock.github.com/d3/ex/). Cheers, Andreas On Nov 2, 2011, at 9:36 AM, Emil Dombagolla wrote: Hi all, i need to display a tree structure , based on the nodes i retrieved from the database through traverse. Nodes are gathered from different levels of the graph. some of the nodes are having parent / child relation in the graph. so if my result containing such a relation i want to display it in a tree structure. Please help me on this and please give me your ideas how to solve this. Thanks a lot. Emil Dombagolla, ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] SDN w/ write and readonly Graph Database Service?
Probably the opposite (if you could even do it). You'd lose the LRU caching across the boundary. Is the data being written the same as the data being read, or is there a natural segmentation? If so you could implement a crude form of sharding/partioning to avoid hot spots (concurrency related) during these periods. As always, SSD + plenty of RAM. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Tero Paananen Sent: Tuesday, November 01, 2011 12:16 PM To: user@lists.neo4j.org Subject: [Neo4j] SDN w/ write and readonly Graph Database Service? I'm using SDN to build a graph db I'm expecting to have quite a heavy volume of write and read activity at peak times. We're not ready to start using HA at this point. I was wondering, if using Neo4J using two instances of the Graph Database Service, one EmbeddedGraphDatabase (writes), and one or more EmbeddedReadOnlyGraphDatabase (reads) would make any difference? Are there any benefits for splitting the db access that way? Or would accessing the db through the writable instance for all operations be pretty much the same? -TPP ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] SDN w/ write and readonly Graph Database Service?
Cool. If you can control access to the data (e.g. no access until the entire subgraph/dataset has been imported) you can probably avoid a lot of the concurrency issues. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Tero Paananen Sent: Tuesday, November 01, 2011 12:49 PM To: Neo4j user discussions Subject: Re: [Neo4j] SDN w/ write and readonly Graph Database Service? On Tue, Nov 1, 2011 at 12:26 PM, Rick Bullotta rick.bullo...@thingworx.com wrote: Probably the opposite (if you could even do it). You'd lose the LRU caching across the boundary. Is the data being written the same as the data being read, or is there a natural segmentation? If so you could implement a crude form of sharding/partioning to avoid hot spots (concurrency related) during these periods. It's largely going to be the same data. It's definitely the same type of data. Basically there will be discreet sets of same type of data (kinda like sub-graphs, but they can be connected) inserted in a batch-like manner. Those discreet sets of data will then immediately be consumed by processes that use it to calculate additional data and store the results to various data stores, incl. the same Neo4J db. After the initial consumption, the data sets will be consumed on-demand, but I'm not sure about how frequently at this point. It'll likely be something like once or twice a day or even less frequently, but I'll have to see how the usage patterns emerge after the solution goes live to be sure. Thanks for the answer Rick. I'll do some quick-and-dirty testing later this week to guide the decision making on this. I'll see if I can post what I find on this thread afterwards. -TPP ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Lucene uberfast indexing
Hi, Neo team. Have you looked into the work Mike McCandless was doing with DocumentsWriterPerThread? It seems at first glance to fit in nicely with the Neo transaction isolation model and could have a significantly positive effect on (already fast) performance. http://www.searchworkings.org/blog/-/blogs/gimme-all-resources-you-have-i-can-use-them!/ Thoughts? Rick ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] REST traverse deprecation
...but maybe offer different install bundles. The other danger of forcing them into the server distro is that it increases the likelihood of dependency conflicts. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Rick Bullotta Sent: Friday, October 28, 2011 7:16 AM To: Neo4j user discussions Subject: Re: [Neo4j] REST traverse deprecation Don't Bloat. This also allows the plug-ins and language bindings to evolve at a difference pace/cycle than the server. I'd say keep them separate. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Jim Webber Sent: Friday, October 28, 2011 7:11 AM To: Neo4j user discussions Subject: Re: [Neo4j] REST traverse deprecation If we do that, I propose that we move the Cypher and Gremlin plugins into server. WDYT? I'm ambivalent about that. What end-user benefits are there versus shipping them as plugins? Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] neo4j-advanced-1.5.M02 can`t run in Windows 7 64 bit
The error messages indicates that the database is already open. Is Neoclipse open or another instance of the server running? Or some code that is opening the database in embedded mode? -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of rbuckland Sent: Thursday, October 27, 2011 7:55 AM To: user@lists.neo4j.org Subject: Re: [Neo4j] neo4j-advanced-1.5.M02 can`t run in Windows 7 64 bit What JDK are you using ? Where is the path E:\document\neo4j\Neo4j\neo4j-advanced-1.5.M02\data\graph.db\neostore ? Is that a network path as the error message looks a bit suspicious. I do have Windows 7 64bit so can try it later. -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/neo4j-advanced-1-5-M02-can-t-run-in-Windows-7-64-bit-tp3440480p3457394.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Why doInternalRecovery is necessary?
That's a great question, Balazs. I never understood why it replayed all of the logical logs rather than just the most recent. Aren't all but the active log already committed? -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Balazs E. Pataki Sent: Wednesday, October 26, 2011 9:41 AM To: Neo4j user discussions Subject: [Neo4j] Why doInternalRecovery is necessary? Hi, after crashing my application the next time I start it up I get the usual Oct 26, 2011 3:28:42 PM org.neo4j.kernel.impl.transaction.xaframework.XaLogicalLog doInternalRecovery INFO: Non clean shutdown detected on log [/db/nioneo_logical.log.1]. Recovery started ... INFO: Non clean shutdown detected on log [/db/index/lucene.log.1]. Recovery started ... kind of messages. I have a 8GB database and now this recovery has been running for quite some time, and I started to thinking why is this recovery is necessary at all? AS far as I'm aware the transactions are ACID, so they either commit, or rollback, in both case the state of the database must be OK. In this case I could see no reason why to do any recovery (from transactional point of view). If the DB crash happens while transactions are open, then there coudl be some problems. But in my case I could live with loosing such transactions and just not have recovery at all. To sum up my question: is this doInternalRecovery process essential? And if not, is there a way to avoid it at startup? Thanks, --- balazs ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Default Analyzer in Index Framework
Anyone able to provide some insights on this? Thanks. From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Rick Bullotta [rick.bullo...@thingworx.com] Sent: Monday, October 24, 2011 6:16 PM To: Neo4j user discussions Subject: [Neo4j] Default Analyzer in Index Framework When not using fulltext indexing, what Lucene Analyzer class does Neo4J use? It seems that non-fulltext index searches are case sensitive - we'd like to change that behavior. Thanks for any help/guidance/examples! Rick ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Urgent: 1.4.2 Github tag does not seem to match the 1.4.2 distro JAR
When attempting to debug an issue with the index framework, the debugger is clearly on the wrong source lines, so I suspect there's some type of mismatch. Thoughts? ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Urgent: 1.4.2 Github tag does not seem to match the 1.4.2 distro JAR
Sorry for not reporting back. It was an eclipse issue. All good now! On Oct 25, 2011, at 7:21 PM, Mattias Persson matt...@neotechnology.com wrote: Odd, could you just give a sample of a line that is wrong? just to help me get started looking at this 2011/10/25 Rick Bullotta rick.bullo...@thingworx.com When attempting to debug an issue with the index framework, the debugger is clearly on the wrong source lines, so I suspect there's some type of mismatch. Thoughts? ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Default Analyzer in Index Framework
Hi, Mattias.
That's exactly what we did. One interesting note: the query and get methods
seemed to work without lower casing the search term (maybe the analyzer is used
to parse the query?), but for native lucene queries we needed to lowercase
them. All good now! Thanks for the tips.
Rick
On Oct 25, 2011, at 7:08 PM, Mattias Persson matt...@neotechnology.com
wrote:
Hi Rick,
yes you can do that, but not in a super easy way. What you'd have to do
right now to get it working is to make sure you create an index with a
special analyzer which converts everything to lower case (both additions and
queries), effectively making it case insensitive. So create a class like
this:
public class LowerCaseAnalyzer extends Analyzer
{
@Override
public TokenStream tokenStream( String fieldName, Reader reader )
{
return new LowerCaseFilter( Version.LUCENE_31, new
KeywordTokenizer( reader ) );
}
}
and make sure you create your index with a configuration map like:
IndexNode index = graphDb.index().forNodes(
myCaseInsensitiveIndex, MapUtil.stringMap( analyzer,
LowerCaseAnalyzer.class.getName() ) );
then this will work:
index.add( node, name, Rick Bullotta );
index.query( name:\rick bullotta\ ); // == returns that node.
2011/10/25 Rick Bullotta rick.bullo...@thingworx.com
Anyone able to provide some insights on this?
Thanks.
From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On
Behalf Of Rick Bullotta [rick.bullo...@thingworx.com]
Sent: Monday, October 24, 2011 6:16 PM
To: Neo4j user discussions
Subject: [Neo4j] Default Analyzer in Index Framework
When not using fulltext indexing, what Lucene Analyzer class does Neo4J
use? It seems that non-fulltext index searches are case sensitive - we'd
like to change that behavior.
Thanks for any help/guidance/examples!
Rick
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Odd Result Comparing Nodes...
I guess I never tried/noticed this before, but if two Node objects refer to the
same node (getId() == the same), shouldn't the following evaluate as true?
Node node1;
Node node2;
somehow they get set...
If(node1 == node2) {
}
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Default Analyzer in Index Framework
When not using fulltext indexing, what Lucene Analyzer class does Neo4J use? It seems that non-fulltext index searches are case sensitive - we'd like to change that behavior. Thanks for any help/guidance/examples! Rick ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Odd Result Comparing Nodes...
Thanks! I had forgotten that.
On Oct 24, 2011, at 7:39 PM, Michael Hunger
michael.hun...@neotechnology.com wrote:
No because they might be two different objects (node-proxies pointing to
node-manager) in memory.
But node1.equals(node2) evaluates to true.
Michael
Am 24.10.2011 um 15:22 schrieb Rick Bullotta:
I guess I never tried/noticed this before, but if two Node objects refer to
the same node (getId() == the same), shouldn't the following evaluate as
true?
Node node1;
Node node2;
somehow they get set...
If(node1 == node2) {
}
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] [bug?] Unicode node property not returned correctly from bulk REST index search
I doubt it, since a GET works fine. It's probably an encoding issue somewhere
in the batch processing pipeline.
-Original Message-
From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On
Behalf Of Daniel Fitzpatrick
Sent: Thursday, October 20, 2011 10:37 PM
To: Neo4j user discussions
Subject: Re: [Neo4j] [bug?] Unicode node property not returned correctly from
bulk REST index search
Possibly an issue with the client code not understanding unicode. Is there
something you could use as a baseline to rule the database out eg maybe the
web admin?
On Oct 20, 2011 7:48 PM, Nuo Yan yan@gmail.com wrote:
I have nodes with data properties with unicode (Chinese/Japanese) characters
such as:
{uid = 12345, name = 例子}
I index such nodes with their id, so that by doing this (where users_index
is the index, uid is the key, 12345 is the value):
GET to /index/node/users_index/uid/12345
I can get back the right result:
{indexed=
http://localhost:7474/db/data/index/node/users_node/uid/12345/6638;,
outgoing_relationships=
http://localhost:7474/db/data/node/6638/relationships/out;,
* data={uid=12345, name=例子}, *
traverse=http://localhost:7474/db/data/node/6638/traverse/{returnType}
,
all_typed_relationships=
http://localhost:7474/db/data/node/6638/relationships/all/{-list||types},
property=http://localhost:7474/db/data/node/6638/properties/{key};,
self=http://localhost:7474/db/data/node/6638;, properties=
http://localhost:7474/db/data/node/6638/properties;,
outgoing_typed_relationships=
http://localhost:7474/db/data/node/6638/relationships/out/{-list||types},
incoming_relationships=
http://localhost:7474/db/data/node/6638/relationships/in;, extensions={},
create_relationship=
http://localhost:7474/db/data/node/6638/relationships;, paged_traverse=
http://localhost:7474/db/data/node/6638/paged/traverse/{returnType}{?pageSize,leaseTime}
,
all_relationships=
http://localhost:7474/db/data/node/6638/relationships/all;,
incoming_typed_relationships=
http://localhost:7474/db/data/node/6638/relationships/in/{-list||types}}
However, if I do the same search query as a part of a bulk REST request:
POST to /batch:
[{method = GET,
to = /index/node/users_index/uid/12345,
body = {},
id = 0}]
Returns the node in the body, however, with bad characters in the data
field:
[{id=0, body=[{indexed=
http://localhost:7474/db/data/index/node/users_node/uid/12345/6638;,
outgoing_relationships=
http://localhost:7474/db/data/node/6638/relationships/out;,
*data={uid=12345, name=￥ﾤﾧ￥ﾂﾻ\uFFE7ﾓﾜ}, *
traverse=http://localhost:7474/db/data/node/6638/traverse/{returnType};,
all_typed_relationships=
http://localhost:7474/db/data/node/6638/relationships/all/{-list||types},
property=http://localhost:7474/db/data/node/6638/properties/{key};,
self=http://localhost:7474/db/data/node/6638;, properties=
http://localhost:7474/db/data/node/6638/properties;,
outgoing_typed_relationships=
http://localhost:7474/db/data/node/6638/relationships/out/{-list||types},
incoming_relationships=
http://localhost:7474/db/data/node/6638/relationships/in;, extensions={},
create_relationship=
http://localhost:7474/db/data/node/6638/relationships;, paged_traverse=
http://localhost:7474/db/data/node/6638/paged/traverse/{returnType}{?pageSize,leaseTime}
,
all_relationships=
http://localhost:7474/db/data/node/6638/relationships/all;,
incoming_typed_relationships=
http://localhost:7474/db/data/node/6638/relationships/in/{-list||types}
}],
from=/index/node/users_node/uid/12345}]
Do you think if this is a bug or is there anything I can change to make the
bulk request return the correct Chinese/Japanese characters? I can reproduce
this all the time.
Thanks,
Nuo
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] WebAdmin visualization tool for large DB ?
We're doing something similar as part of a visual search engine for ThingWorx. Our UX designers might be able to provide some recommendations/input on Neo4J DB visualization also. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Jacob Hansson Sent: Monday, October 17, 2011 8:30 AM To: Neo4j user discussions Subject: Re: [Neo4j] WebAdmin visualization tool for large DB ? On Mon, Oct 17, 2011 at 12:49 PM, Rick Bullotta rick.bullo...@thingworx.com wrote: I think that both Neo4J and webadmin should have the option of specifying first N relationships as a limiter. This limit should be on a per-relationship basis (e.g. the first N of each type, rather than simply the first N relationships). This way, you would be able to visualize all of the relationship types that go in/out of the node, but not be overwhelmed with one specific relationship type. Also, it would be ideal if some visual cue that there were more relationships of a given type could be provided (e.g. a relationship line to a box labeled ...and 10,679 more From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Pablo Pareja [ppar...@era7.com] Sent: Monday, October 17, 2011 6:44 AM To: Neo4j user discussions Subject: Re: [Neo4j] WebAdmin visualization tool for large DB ? Hi Mattias, what about the hypothetical scenario where there are ~ 1.000.000 relationships going out from a node, would the visualization tool try to plot all theses relationships? If not, would it first fetch them all and then plot a 1.000.000 group node? Or would it rather plot something like* more than 'x' relationships coming out from here*? Currently, it would pull down all relationships, and render a group. Since we've added support for paging traversals since that code was written, we could do something like what Rick is suggesting. It would be fun to sketch a bit on how a UI like that could work.. There is quite a bit of stuff in the webadmin backlog, so this won't get done right now, but it will eventually bubble to the top of the list :) Cheers, Pablo On Mon, Oct 17, 2011 at 12:38 PM, Mattias Persson matt...@neotechnology.com wrote: Hi Pablo, I don't think if there are any hard limits in the webadmin visualizer, It mostly depends on your computer and browser (what with the implementation of web workers and all) I'd say. 2011/10/13 Pablo Pareja ppar...@era7.com Hi, I was looking for a way to visualize and interact with small subsets of large Neo4j databases, *(somehow* *navigating through the data in order to have a general idea of how things are organized at specific parts* *of my DB)*. So before starting to create my own solution to that, I'd like to know which are the limits/constraints of the new WebAdmin tool ?* (in terms of number of rels/nodes in the neighborhood/ total DB)* Thanks, Pablo -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] WebAdmin visualization tool for large DB ?
Of course! -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Jacob Hansson Sent: Monday, October 17, 2011 8:53 AM To: Neo4j user discussions Subject: Re: [Neo4j] WebAdmin visualization tool for large DB ? That would be really interesting! I'll make a note of it and ping you when this gets up to the top of the backlog, if that's ok with you? /Jake On Mon, Oct 17, 2011 at 2:50 PM, Rick Bullotta rick.bullo...@thingworx.comwrote: We're doing something similar as part of a visual search engine for ThingWorx. Our UX designers might be able to provide some recommendations/input on Neo4J DB visualization also. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Jacob Hansson Sent: Monday, October 17, 2011 8:30 AM To: Neo4j user discussions Subject: Re: [Neo4j] WebAdmin visualization tool for large DB ? On Mon, Oct 17, 2011 at 12:49 PM, Rick Bullotta rick.bullo...@thingworx.com wrote: I think that both Neo4J and webadmin should have the option of specifying first N relationships as a limiter. This limit should be on a per-relationship basis (e.g. the first N of each type, rather than simply the first N relationships). This way, you would be able to visualize all of the relationship types that go in/out of the node, but not be overwhelmed with one specific relationship type. Also, it would be ideal if some visual cue that there were more relationships of a given type could be provided (e.g. a relationship line to a box labeled ...and 10,679 more From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Pablo Pareja [ppar...@era7.com] Sent: Monday, October 17, 2011 6:44 AM To: Neo4j user discussions Subject: Re: [Neo4j] WebAdmin visualization tool for large DB ? Hi Mattias, what about the hypothetical scenario where there are ~ 1.000.000 relationships going out from a node, would the visualization tool try to plot all theses relationships? If not, would it first fetch them all and then plot a 1.000.000 group node? Or would it rather plot something like* more than 'x' relationships coming out from here*? Currently, it would pull down all relationships, and render a group. Since we've added support for paging traversals since that code was written, we could do something like what Rick is suggesting. It would be fun to sketch a bit on how a UI like that could work.. There is quite a bit of stuff in the webadmin backlog, so this won't get done right now, but it will eventually bubble to the top of the list :) Cheers, Pablo On Mon, Oct 17, 2011 at 12:38 PM, Mattias Persson matt...@neotechnology.com wrote: Hi Pablo, I don't think if there are any hard limits in the webadmin visualizer, It mostly depends on your computer and browser (what with the implementation of web workers and all) I'd say. 2011/10/13 Pablo Pareja ppar...@era7.com Hi, I was looking for a way to visualize and interact with small subsets of large Neo4j databases, *(somehow* *navigating through the data in order to have a general idea of how things are organized at specific parts* *of my DB)*. So before starting to create my own solution to that, I'd like to know which are the limits/constraints of the new WebAdmin tool ?* (in terms of number of rels/nodes in the neighborhood/ total DB)* Thanks, Pablo -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pablo Pareja Tobes My site http://about.me/pablopareja LinkedInhttp://www.linkedin.com/in/pabloparejatobes Twitter http://www.twitter.com/pablopareja Creator of Bio4j -- http://www.bio4j.com http://www.ohnosequences.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User
Re: [Neo4j] Trying to use Neo4J with Atomikos transaction manager, issues with Lucene index
If I remember correctly, neo has an implicit field on each index, and the name is either id or _id. On Oct 8, 2011, at 1:58 PM, Chris Gioran chris.gio...@neotechnology.com wrote: Hi Thibaut, I noticed the following snippet in your test case code: long id = node.getId(); node.setProperty(testProp, test); index.add(node, testProp, test); Assert.assertEquals(node.getProperty(testProp), test); // Lookup By id Assert.assertNotNull(graphDb.getNodeById(id)); // Now via Index Node found = index.get(id, id).getSingle(); Assert.assertNotNull(found); Assert.assertEquals(found.getId(), id); which leads to the first test failure with a null result returned. Here you are indexing a node with key testProp and value test but you go on to ask it from the index with key id and value its id. Since you haven't added that key/value pair in the index, rightfully you don't get any results - the test fails in my setup even with the native Neo4j transaction manager. I changed that to ask the node from the index with the proper key/value pair and the test passes - the same thing happens a bit further down where you ask the index for a node with a key/value of testProp/prop (instead of testProp/test). Moreover, it passes with both the native Transaction Manager and the Atomikos implementation. Also, you are not deleting the db for the test - this means that since you add to the index nodes with the same key/value the test will pass only on the first run as getSingle() on the index hits will find the previously added nodes and it will fail. Could you try that out and see if it solves your issues? I am glad that this functionality is being used in production - please provide any feedback on any problems and what you think of this feature. hope that helps, CG On Thu, Oct 6, 2011 at 7:50 PM, tcolar tco...@colar.net wrote: I've been trying to make this work for a few days but while it mostly works, the Lucene index just won't. We have a fairly complex setup including neo4j, mysql, mongo and JMS ad trying to do transactions accross all that with Atomikos. Since that is quite complex, I've made a smaller unit test just using Atomikos and Neo4j to demonstrate the issue: https://bitbucket.org/tcolar/stuff/src/ddd17191e9a4/AtomikosNeo4j The main test is here: https://bitbucket.org/tcolar/stuff/src/ddd17191e9a4/AtomikosNeo4j/src/test/java/net/colar/atomikosNeo4j/AtomikosNeo4jTest.java and results: https://bitbucket.org/tcolar/stuff/src/ddd17191e9a4/AtomikosNeo4j/test.log I used http://digitalstain.blogspot.com/2010/11/using-jotm-as-transactionmanager-in.html to get me going. And my code for that part is here: https://bitbucket.org/tcolar/stuff/src/ddd17191e9a4/AtomikosNeo4j/src/main/java/net/colar/atomikosNeo4j I'm thinking that's probably where I'm doing something wrong maybe ? I'm probably not doing something right but i can't seem to find what's going on with Lucene. The test works if not using Atomikos but just plain Neo4j Transactions. Anybody as an idea what is wrong? Thanks. -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Trying-to-use-Neo4J-with-Atomikos-transaction-manager-issues-with-Lucene-index-tp3400319p3400319.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Best practise for multi language properties
Property names can have special characters in them, so we use a pattern like: Description Description.EN Description.DE This way we can keep any number of localiz(able) expressions on a node. Based on the user's language, we first try to get the value in the local language, then the default. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Michael Hunger Sent: Friday, October 07, 2011 8:07 AM To: Neo4j user discussions Subject: Re: [Neo4j] Best practise for multi language properties You mean something like internationalization? I don't know if that itself is a responsibility of the datamode/graph. You could probably use string arrays to store those. Or do you mean multi-language property names? Cheers Michael, Dresden,Germany Am 07.10.2011 um 09:24 schrieb Moootron: Best Practice for multilanguage properties Just a few month I am working with neo4j. We are modeling graphs for new Applications combining ontologies and data for production data. The most important thing actually is to have multi language properties for nodes and relations with the following most important use cases: - language amount of each node could differ, this means that node 123 could have EN and DE and node 234 EN and SP. - it should be possible to have a simple update process, like language packages, that we can fire to the graph Are there any best practices, base concepts or contacts to solve this? ... Thanks and regards from Regensburg-Germany, Ottmar ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Problem with lucene indexing
Classpath? -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of andrew ton Sent: Saturday, October 01, 2011 5:27 AM To: Neo4j_user Subject: [Neo4j] Problem with lucene indexing Hello, I have a problem with indexing when using the Neo4J embedded database. I'm using neo4j-1.5M01, neo4j-lucene-index-1.5M01. I have tried both lucene-core-3.1.0 and lucene-core-3.4.0. Can somebody show me what is wrong here? I create an instance of EmbeddedGraphDatabase as graphDb = new EmbeddedGraphDatabase(mystore); and create index: nodeIndex = graphDb.index().forNodes(nodes, MapUtil.stringMap(provider, lucene, type, fulltext)); When I run unit test I have a problem: com.ericsson.research.semantic.PersistentEmbeddedOntology -Exception caugth while instantiating. No index provider 'lucene' found java.lang.IllegalArgumentException: No index provider 'lucene' found and this is the messages.log in my store Sat Oct 01 01:41:20 PDT 2011: Failed to load index provider lucene LUCENE_31 java.lang.NoSuchFieldError: LUCENE_31 at org.neo4j.index.impl.lucene.LuceneDataSource.clinit(LuceneDataSource.java:88) at org.neo4j.index.impl.lucene.LuceneIndexImplementation.init(LuceneIndexImplementation.java:72) at org.neo4j.index.lucene.LuceneIndexProvider.load(LuceneIndexProvider.java:37) at org.neo4j.kernel.KernelData.loadIndexImplementations(KernelData.java:146) at org.neo4j.kernel.EmbeddedGraphDbImpl$2.initializeIndexProviders(EmbeddedGraphDbImpl.java:180) at org.neo4j.kernel.GraphDbInstance.start(GraphDbInstance.java:159) at org.neo4j.kernel.EmbeddedGraphDbImpl.init(EmbeddedGraphDbImpl.java:190) at org.neo4j.kernel.EmbeddedGraphDatabase.init(EmbeddedGraphDatabase.java:80) at org.neo4j.kernel.EmbeddedGraphDatabase.init(EmbeddedGraphDatabase.java:64) Thank you, ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] InvalidRecordException: Record[1983624] not in useRecord[1983624] not in use
Why are you running a beta version?
On Sep 28, 2011, at 9:03 AM, René Pickhardt r.pickha...@googlemail.com
wrote:
Neo1.4.M04 community edition no self build.
2011/9/28 Mattias Persson matt...@neotechnology.com
Which version of neo4j did you use to import your data, was it a
downloadable package, from maven or building yourself from a branch?
2011/9/28 René Pickhardt r.pickha...@googlemail.com
hey everyone
I got a strange error message that a record is not in use if I call
hasRelationship() function even though I checked against null before! My
real purpose was to start the traverser but it exited with the same error
message. That is why I put the n.hasRelationship which also would not
work.
I imported a graph with 50 mio. nodes and about the same size of
relationships. (though I am about to import more relationships later on)
I
put 2 mio. nodes in a lucence index called article.
the counting variable t has the value 2385 (so the first 2385 calls work
fine) if I skip this node the same problem accoures again at t = 3239.
maybe it is usefull to say that I inserted the graph using the batch
inserter.
IndexHitsNode res = article.get(key, values[0]);
if (res!=null){
Node n = res.getSingle();
if (n!=null){
if (n.hasRelationship(DynamicRelationshipType.withName( UPDATE ),
Direction.BOTH)){
//n.traverse(Order.BREADTH_FIRST,StopEvaluator.DEPTH_ONE,
ReturnableEvaluator.ALL_BUT_START_NODE,DynamicRelationshipType.withName(
UPDATE ) );
t++;
}
}
callstack:
org.neo4j.kernel.impl.nioneo.store.InvalidRecordException:
Record[1983624]
not in useRecord[1983624] not in use
at
org.neo4j.kernel.impl.nioneo.store.RelationshipStore.getRecord(RelationshipStore.java:230)
at
org.neo4j.kernel.impl.nioneo.store.RelationshipStore.getChainRecord(RelationshipStore.java:337)
at
org.neo4j.kernel.impl.nioneo.xa.ReadTransaction.getMoreRelationships(ReadTransaction.java:114)
at
org.neo4j.kernel.impl.nioneo.xa.ReadTransaction.getMoreRelationships(ReadTransaction.java:97)
at
org.neo4j.kernel.impl.persistence.PersistenceManager.getMoreRelationships(PersistenceManager.java:108)
at
org.neo4j.kernel.impl.core.NodeManager.getMoreRelationships(NodeManager.java:603)
at
org.neo4j.kernel.impl.core.NodeImpl.getMoreRelationships(NodeImpl.java:351)
at
org.neo4j.kernel.impl.core.NodeImpl.loadInitialRelationships(NodeImpl.java:318)
at
org.neo4j.kernel.impl.core.NodeImpl.ensureRelationshipMapNotNull(NodeImpl.java:305)
at
org.neo4j.kernel.impl.core.NodeImpl.getAllRelationshipsOfType(NodeImpl.java:156)
at
org.neo4j.kernel.impl.core.NodeImpl.getRelationships(NodeImpl.java:236)
at
org.neo4j.kernel.impl.core.NodeImpl.hasRelationship(NodeImpl.java:576)
at
org.neo4j.kernel.impl.core.NodeProxy.hasRelationship(NodeProxy.java:109)
at IndexEvaluation.GenerateStreams(IndexEvaluation.java:124)
at IndexEvaluation.init(IndexEvaluation.java:98)
at EntryPoint.main(EntryPoint.java:20)
--
--
mobile: +49 (0)176 6433 2481
Skype: +49 (0)6131 / 4958926
Skype: rene.pickhardt
www.rene-pickhardt.de
http://www.beijing-china-blog.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
--
mobile: +49 (0)176 6433 2481
Skype: +49 (0)6131 / 4958926
Skype: rene.pickhardt
www.rene-pickhardt.de
http://www.beijing-china-blog.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] InvalidRecordException: Record[1983624] not in useRecord[1983624] not in use
Hi, René.
I recognized your error almost immediately, since we encountered it in an early
1.4 beta. ;-)
I think you'll have a good result if you can switch to the 1.4.1 release!
Best,
Rick
-Original Message-
From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On
Behalf Of René Pickhardt
Sent: Wednesday, September 28, 2011 10:29 AM
To: Neo4j user discussions
Subject: Re: [Neo4j] InvalidRecordException: Record[1983624] not in
useRecord[1983624] not in use
Hey guys,
I don't know how I found the old milestone. I was downloading it once I
started using neo4j.
I will just migrate to the current stable version and tell you afterwards if
I still have problems. by your comments I guess the problem will be solved.
Thanks so far!
best regards René
2011/9/28 Mattias Persson matt...@neotechnology.com
I'm curious to know how you found that old milestone also :) you picked the
one out of extremely few having some kind of data problems.
Den onsdagen den 28:e september 2011 skrev Chris Gioran
chris.gio...@neotechnology.com:
Hi Rene,
According to git log, 1,4.M04 was tagged 10/06/2011, while the fix for
a bug in the BatchInserterImpl that caused corrupted stores which gave
the message you are seeing is
https://github.com/neo4j/community/commit/71fbe4ac1174fad16a7bb154c9f78d3f1b377248
which has a date of 04/07/2011.
This makes 1.4.M06 the earliest release to use that does not have this
issue or, even better, go for 1.4.1 which is the latest stable
release. Since you are comfortable using milestones, 1.5.M01 might be
of interest to you.
http://neo4j.org/download has the links that you need.
hope that helped,
CG
2011/9/28 René Pickhardt r.pickha...@googlemail.com:
Neo1.4.M04 community edition no self build.
2011/9/28 Mattias Persson matt...@neotechnology.com
Which version of neo4j did you use to import your data, was it a
downloadable package, from maven or building yourself from a branch?
2011/9/28 René Pickhardt r.pickha...@googlemail.com
hey everyone
I got a strange error message that a record is not in use if I call
hasRelationship() function even though I checked against null before!
My
real purpose was to start the traverser but it exited with the same
error
message. That is why I put the n.hasRelationship which also would not
work.
I imported a graph with 50 mio. nodes and about the same size of
relationships. (though I am about to import more relationships later
on)
I
put 2 mio. nodes in a lucence index called article.
the counting variable t has the value 2385 (so the first 2385 calls
work
fine) if I skip this node the same problem accoures again at t =
3239.
maybe it is usefull to say that I inserted the graph using the batch
inserter.
IndexHitsNode res = article.get(key, values[0]);
if (res!=null){
Node n = res.getSingle();
if (n!=null){
if (n.hasRelationship(DynamicRelationshipType.withName( UPDATE ),
Direction.BOTH)){
//n.traverse(Order.BREADTH_FIRST,StopEvaluator.DEPTH_ONE,
ReturnableEvaluator.ALL_BUT_START_NODE,DynamicRelationshipType.withName(
UPDATE ) );
t++;
}
}
callstack:
org.neo4j.kernel.impl.nioneo.store.InvalidRecordException:
Record[1983624]
not in useRecord[1983624] not in use
at
org.neo4j.kernel.impl.nioneo.store.RelationshipStore.getRecord(RelationshipStore.java:230)
at
org.neo4j.kernel.impl.nioneo.store.RelationshipStore.getChainRecord(RelationshipStore.java:337)
at
org.neo4j.kernel.impl.nioneo.xa.ReadTransaction.getMoreRelationships(ReadTransaction.java:114)
at
org.neo4j.kernel.impl.nioneo.xa.ReadTransaction.getMoreRelationships(ReadTransaction.java:97)
at
org.neo4j.kernel.impl.persistence.PersistenceManager.getMoreRelationships(PersistenceManager.java:108)
at
org.neo4j.kernel.impl.core.NodeManager.getMoreRelationships(NodeManager.java:603)
at
org.neo4j.kernel.impl.core.NodeImpl.getMoreRelationships(NodeImpl.java:351)
at
org.neo4j.kernel.impl.core.NodeImpl.loadInitialRelationships(NodeImpl.java:318)
at
org.neo4j.kernel.impl.core.NodeImpl.ensureRelationshipMapNotNull(NodeImpl.java:305)
at
org.neo4j.kernel.impl.core.NodeImpl.getAllRelationshipsOfType(NodeImpl.java:156)
at
org.neo4j.kernel.impl.core.NodeImpl.getRelationships(NodeImpl.java:236)
at
org.neo4j.kernel.impl.core.NodeImpl.hasRelationship(NodeImpl.java:576)
at
org.neo4j.kernel.impl.core.NodeProxy.hasRelationship(NodeProxy.java:109)
at IndexEvaluation.GenerateStreams(IndexEvaluation.java:124)
at IndexEvaluation.init(IndexEvaluation.java:98)
at EntryPoint.main(EntryPoint.
--
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
___
Neo4j mailing list
User@lists.neo4j.org
Re: [Neo4j] add property to all relationships
Or you could simply call the alternate version of getProperty with a default value of zero... On Sep 26, 2011, at 5:44 AM, sometime dons...@gmail.com wrote: Hi! I have 1.2M nodes and 5M relationships. Around 20 000 relationships are two properties: prop1 prop2, the remaining relationships have only one property: prop1. I've got all the relationships that have only one property - add prop2 with value 0. It should run on all relationships and add this property? If yes - then I have to run the robot and it will run at least 3 days. Can be somehow easier to solve this problem so as not to spend too much time? -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/add-property-to-all-relationships-tp3368779p3368779.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Modelling with neo4j
That's a great summary, Niels. Very similar to how we've applied Neo4J here at ThingWorx, though we've done most of the type system work (nodes and relationships are all typed/subtyped) in our application domain layer. A few other items that we leveraged in our implementation that you may wish to consider: - A common pattern we encountered was a collection of typed entities (e.g. a typed collection), and we implemented a specific model using supernodes for this. This also allowed us to rapidly and easily iterate/search collections and also to organize nodes in a human comprehensible way that can be readily viewed with something like Neoclipse for troubleshooting purposes. Also, if the type was truck, we stamped the node with the type truck as a property (using enumerations with a custom int member) and used that same enum as the relationship type between the node and the collection node. In our model, an entity has a single type, but we implemented the concept of supertyping/subtyping in our domain model - We found quite a few examples where a one-way relationship was more than adequate and, instead of incurring the overhead of a relationship (particularly when millions of these relationships were attached to a single supernode), we used a *property* on a node containing the node id of the node it references. Sounds like a hack, but it actually has substantial performance advantages, particularly if you are frequently adding/removing relationships to/from the supernode - We overlaid our own REST API on our domain model, and wanted to come up with a simple way to resolve the URI for any given node/entity. For that, we used a pattern for which each node can have an optional parent node type. Example: a blog comment is always attached to a blog entry or other blog comment. A blog entry is always attached to a blog. A blog is always attached to the blogs collection, and so on. Each node has a name and/or an ID. Because those relationship patterns are well known, it is a trival matter to create the URI to any entity given only its node, e.g.: /Blogs/MyBlog/Entries/103/Comments/204 Of course, it works the other way as well - easy to parse and traverse. - We often found that there were data structures in our application domain for which it was OK to be opaque - e.g. although the structures were deep and complex, they did not require searchability or traversability (e.g. they were kind like object blobs), so in our metamodel, they are not stored as nodes, relationships, and properties, but rather, as a JSON blob, serialized as a string to a node property. That has worked out really well. When we do need to filter/manipulate those, we do them at the domain level Just wanted to share some more examples. Rick From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Niels Hoogeveen [pd_aficion...@hotmail.com] Sent: Saturday, September 24, 2011 9:14 AM To: user@lists.neo4j.org Subject: Re: [Neo4j] Modelling with neo4j You raise interesting questions, most of them very much related to the work I did on Enhanced API. Let me start with the distinction between Node and Relationship, which in my opinion too is a bit artificial. I understand when creating a graph database, it is helpful to have something like vertices and edges, but indeed see those more as modalities of the elements of the graph than as clearly separated types. This was one of the reasons to unify all elements of the graph with one underlying type. At the time, I saw two option: a) make the graph bipartite, so that all relationships and properties become nodes and use relationships only as a hidden linking feature b) create shadow nodes for relationships and properties when needed and let the API handle that transparently I chose for option b for performance reasons. There are likely many applications where most of the relationships are simple, ie. link two nodes while possibly having some properties. Using a bipartite layout for such relationships adds nothing, but it takes twice as many links to traverse. The shadow node solution only treats relationships and properties as special (having relationships to them) when that is needed. Now to the typing issues. Neo4j has chosen not to add typing features to the database and I actually like that. It allows for optional type systems that can be used but are not enforced to be used. Type systems are nice beasts, especially when dealing with large and complex applications, but they impose a development overhead, mostly felt in small quick and dirty applications. This is true for programming languages, where many people prefer to use an untyped language such as Javascript, Python, Ruby and PHP over a typed language such as Java, Scala, C# or Haskell and I think it is also true for databases. I think one of the reasons NOSQL became so popular is because the type system
Re: [Neo4j] Modelling with neo4j
We're using Neo4J to model the real world with things here at ThingWorx as well. See my responses to Niels for some specifics. From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of loldrup [lold...@gmail.com] Sent: Saturday, September 24, 2011 1:52 AM To: user@lists.neo4j.org Subject: [Neo4j] Modelling with neo4j I'm trying to figure out how to model the world most flexibly (okay, so I'm sticking to modelling organisations for now, but still). My main problem seems to occur when I want to allow the model to naturally expand in complexity. Say we have the following relationship: Joe is a janitor at the school. This can easily be modelled with two entities and a relationship. Now say I have some common properties for janitors. I would have to make a link from the janitor-relation to some node denoting the type 'janitor' which could then hold information on these common things. Unfortunately, relationships doesn't support that. Long story short: the problem is that sometimes I want my things to act as things, sometimes as types, sometimes as interfaces, and I cannot know in advance which of these modalities I'm going to need. Therefore, I'm considering going with this model: Imagine a graph in three layers. The lower layer represents things, the middle layer represents types and the upper layer represents interfaces. Initially i populate only the lowest layer, but as need arise I go back and promote various things to also be types or interfaces. These then crop up in the second and third layer of the graph, respectively. When this happens, a vertical relationship is added between the element in the lower layer and its new type/interface in three higher layers. Now the question is: how to model this scheme in neo4j? A number of challenges pops up: * Neo4j relationships cannot be n-ary, so every relationship must be modelled with a hyperrelationship, thus allowing future relations to the second and third layers. * In a modalities-are-a-changing-paradigm it doesn't really make sense to distinguish between relations and entities; at different points in time, one element may have to act in the roles of both. Neo4j however makes a fundamental destinction between the two things. I could choose too model all relationships as nodes, but will that not make graph traversals messy? * Neo4j doesn't come with a type strong destinction between such three layers of modalityy -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Modelling-with-neo4j-tp3363823p3363823.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] List all indexed key/values for an entity
It would be nice to have this exposed in the embedded index framework as well. On Sep 21, 2011, at 7:05 PM, Tim McNamara paperl...@timmcnamara.co.nz wrote: +1 But the plural of index is indices. On 22/09/2011 1:56 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Josh, very reasonable request. Could you please put that into an issue? I suspect this is in line with better auto indexing support, I am not sure how much of it there already is, so let's keep track of it in the issue. Thanks! /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Mon, Sep 19, 2011 at 4:15 PM, jadell josh.ad...@gmail.com wrote: Is there any way via the REST API to list all the indexes and keys/values that a given entity is indexed on? Something like: GET http://0.0.0.0:7474/db/data/node/123/indexes returns: [ http://0.0.0.0:7474/db/data/index/node/favorites/color/blue;, http://0.0.0.0:7474/db/data/index/node/favorites/food/pizza;, http://0.0.0.0:7474/db/data/index/node/users/name/josh;, http://0.0.0.0:7474/db/data/index/node/authors/post/456;, ] Does anything exist that gives this functionality or similar? -- Josh Adell -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/List-all-indexed-key-values-for-an-entity-tp3348997p3348997.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Design help for G+ like app
If you really think your application will grow large, you might want to design your own sharding scheme across multiple servers for the posts, which will represent your largest # of nodes and properties. You can look at some of the QCon presentations from the Twitter team as to how they've attacked the problem. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Antoine Pouch Sent: Wednesday, September 21, 2011 10:21 AM To: Neo4j user discussions Subject: Re: [Neo4j] Design help for G+ like app Thanks for the reply Peter. So no direct user-to-user relationship, and each user has a group/circle called AllMyFriends. Makes sense. Regarding the timeline index(es), should I make a huge one in Lucene with all of users posts and properties associating them to users and circles or is it better to make a lot of small indexes inside Neo4j? Antoine Pouch Architecte chef programmeur Woozworld Inc. Tél : 514-281-8999 #212 Cell : 514-463-1588 On Wed, Sep 21, 2011 at 9:38 AM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Antoine, the design sounds good from the initial in-the-head look. The only thing I see is the modeling of groups. I would imagine something like http://yuml.me/diagram/scruffy/class/[user1]-circle%3E[friendsU1],[user2]-partOf%3E[friendsU1],[user1]-circle%3E[jobU1],[user2]-partOf%3E[jobU1] where User2 is part of two of User1 circles. I agree that, if all relationships are through circles, you probably don't need users-user relationships at all. Does that make sense? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Tue, Sep 20, 2011 at 7:29 PM, ant-1 apo...@woozworld.com wrote: Hi, I'm our company software architect, and I'm new to GraphDBs. But as we're building a Google+-like, we realized the need for something like Neo4j. And as this community seems the best, we settle for you guys :) Anyway. Onto the design. Call us fools, but we're trying to redo Google+ (except for kids). I need help with the design, for starters. Here's the Domain: - Users - Users have friends - Users can place friends in one or more group (circle for G+), groups being only visible to the user creating them. - Users can create posts, which are visible either by all his friends or only one or more groups. I realize the hardest part is to retrieve feeds. For example, I want the posts feed for user X for his group G. Here's what I envision: - User are nodes - Users have FRIEND_WITH relationships (direction being the initial requester to the other) - Groups are nodes. - Group has a CREATED_BY relationship to user - Group has BELONGS_TO relationships to multiple users - Post are nodes - Post has CREATED_BY relationship to the user - Post has VISIBLE_TO relationship to one or more groups - PostingEvent is a node with a timestamp property - PostingEvent has a RELATED_TO relationship to the user and the post And we would have a timeline index (Lucene or B-tree, I have no idea) for feeds retrieval. 1. Do you see issues with my design? 2. What to do with postings to All my friends, do I create a All friends group? In that case do I still need the user-to-user relationships? 3. I never worked with timeline indexes and such, so I could use some readings on the subject, even theorical ones, even dead-tree books. Please don't hesitate to make recommendations. Thanks ! Antoine -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Design-help-for-G-like-app-tp3353185p3353185.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] How to backup with neo4j community?
If on Linux, you could use DRBD, I suppose. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of McKinley Sent: Wednesday, September 21, 2011 11:21 AM To: Neo4j user discussions Subject: Re: [Neo4j] How to backup with neo4j community? This came up recently on the list in August http://neo4j.org/forums/#nabble-td3274919. The last message in the thread has contact info and sounds like there is room for hearing your specific requirements. Cheers, McKinley On Wed, Sep 21, 2011 at 8:03 AM, Rugal ru...@email.it wrote: Hi, I'm starting a little side project in a startup which will use neo4j. How can I set the server to backup the data without stopping the neo4j server and without going with the enterprise version? as mush as I'd like to, but really, 2000 usd/month are too much for a bootstrapping startup (I can't neither release the source code) Thank you ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Recommended way to index and lookup paired properties
I think this falls into the category of best to try it. I would simulate a couple million items and see what kind of performance you get in both scenarios. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Aseem Kishore Sent: Wednesday, September 14, 2011 3:41 PM To: Neo4j user discussions Subject: Re: [Neo4j] Recommended way to index and lookup paired properties Can anyone help w/ this question? =) Thanks! Aseem On Thu, Sep 8, 2011 at 9:58 AM, Aseem Kishore aseem.kish...@gmail.comwrote: Hey guys, quick question on indexing. We track Amazon products in our db, and the way Amazon identifies its products is with an Amazon Standard Identification Number (ASIN). The only catch is that ASINs are only unique within a particular locale, and Amazon has a few different locales (e.g. us, uk, de, jp, etc.). Because of this, when we index and lookup Amazon products, we need to index and lookup with *both* properties together. E.g. we really want to index the pair. As such -- and because we use Neo4j's auto-indexing -- we created a shadow locale+asin property that sits alongside the regular locale and asin properties, and it's a concatenation of the two values, e.g. us+A123456789. We thus index and perform lookups via this shadow property. Recently, though, I saw that you can query even exact indexes with complex Lucene syntax, so in theory, we could ditch the shadow property and query something like (locale:us AND asin:A123456789). Will this be slower though, if there many items w/ the same locale? Generalizing, then, which of these two routes is the recommended way of indexing paired properties? (Or is there a third way?) Thanks! Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Querying a full text index
While I don't know that it will change anything, any reason that you're using
M06 and not 1.4.1? There have been quite a few important fixes. Also, the
analyzer that is used to tokenize both the indexed content and the query have
an effect on the query processing. In any case, I would update to 1.4.1 so
that diagnosing the issues would be significantly easier.
From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of
Yaniv Ben Yosef [yani...@gmail.com]
Sent: Wednesday, September 07, 2011 6:16 AM
To: Neo4j user discussions
Subject: [Neo4j] Querying a full text index
Hi,
This question may be Lucene related, but since I'm using it via Neo4J I'm
asking here first. I'm using Neo4J 1.4 M06.
I have a graph representing people, with a few properties about each person
(e.g., their name and job title).
Now I'd like to create a search form that will allow the user to enter
either the person's first name, last name, title, or any combination. For
example, the query [john director] should result with all the people whose
name or title contain both john and director.
To play with that, I created this little psvm:
public class FullTextIndexTest
{
public static void main(String[] args)
{
GraphDatabaseService graphDb =
GraphDatabaseServiceFactory.createGraphDatabase(target/var/db);
Transaction t = graphDb.beginTx();
Node n1 = graphDb.createNode();
n1.setProperty(name, John Smith);
n1.setProperty(title, Directory Manager);
Node n2 = graphDb.createNode();
n2.setProperty(name, Johnny Malkovich);
n2.setProperty(title, Director of RD);
Node n3 = graphDb.createNode();
n3.setProperty(name, John Horovich);
n3.setProperty(title, Sr. Director);
IndexManager index = graphDb.index();
IndexNode fulltextPerson = index.forNodes(person-fulltext,
MapUtil.stringMap(IndexManager.PROVIDER, lucene, type,
fulltext));
fulltextPerson.add(n1, combined, n1.getProperty(name) + +
n1.getProperty(title));
fulltextPerson.add(n2, combined, n2.getProperty(name) + +
n2.getProperty(title));
fulltextPerson.add(n3, combined, n3.getProperty(name) + +
n3.getProperty(title));
t.success();
t.finish();
// search in the fulltext index
IndexHitsNode hits = fulltextPerson.query(combined, director
john);
System.out.printf(Found %d results:\n, hits.size());
for (Node node : hits)
{
System.out.println(node.getProperty(name) + , +
node.getProperty(title));
}
}
}
I expected this program to return 1 result: John Horovich, Sr. Director
Instead, I'm getting 3:
John Horovich, Sr. Director
John Smith, Directory Manager
Johnny Malkovich, Director of RD
It seems that Lucene will accept terms that contain a query term (e.g,
Directory and Johnny) even if I'm not using any wildcards in my query. How
do I turn this behavior off? I'd like the results to contain only people
whose name or title *contain* the word john, but not johnny.
Thanks!
--- Yaniv
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Community - count me in!
Congratulations, Peter! There couldn't be a better champion for Neo4J and the community. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Peter Neubauer Sent: Tuesday, September 06, 2011 10:33 AM To: Neo4j user discussions; gremlin-users; neo4jrb Subject: [Neo4j] Community - count me in! Hi all, Over the last year, the graph community has seen a fantastic surge of interest and activity. We feel that we can and must support all of you more seriously, so I am stepping up as the first of the Neo Technology community team, dedicating more of my time to explicit community support. http://blog.neo4j.org/2011/09/neo4j-community-team-announcement.html Good days to come everyone, it's an honor to work with you! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Websocket Server instead of REST for Neo4J for access a DB
That sounds like a good approach. Let's get the streaming protocol working, and then support a variety of transports. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Thomas Fritz Sent: Monday, August 29, 2011 11:02 AM To: Neo4j user discussions Subject: Re: [Neo4j] Websocket Server instead of REST for Neo4J for access a DB Of course. I am completly with you. Just wanted to say that websocket is not only usable from browsers. But i also think that first there has to be a binary protocol and then build a websocket protocol on top of it. --- Thomas FRITZ web http://fritzthomas.com twitter http://twitter.com/thomasf 2011/8/29 Jacob Hansson jacob.hans...@neotechnology.com: On Mon, Aug 29, 2011 at 4:35 PM, Thomas Fritz fritz...@gmail.com wrote: Hi! Keep in mind. Websocket is not only for the browser! Java: http://code.google.com/p/weberknecht/ Python: http://pypi.python.org/pypi/websocket-client/0.4 .net: https://github.com/sta/websocket-sharp and JavaScript / Node: socket.io for instance Absolutely, my point is that there are many other higher level transport protocols that would be a better fit for out-of-browser environments. The only benefit, that I can see, to using websockets is easy use from the browser (I may be wrong though, I'm not super-familiar with websocket details). Since the vast majority of use case environments are outside of the browser, it feels important to choose a transport protocol that considers those as first class citizens. Don't get me wrong, I'm a huge proponent for top-notch web support as well. That's what I meant by saying we might be able to design a binary protocol that is flexible enough to fit through many different transport protocols. /Jake --- Thomas FRITZ web http://fritzthomas.com twitter http://twitter.com/thomasf 2011/8/29 Jacob Hansson jacob.hans...@neotechnology.com: In my mind websockets is not necessarily something we want to focus on initially for a binary protocol. It would only benefit in-browser javascript access to the server, while setting back every other programming language, forcing them to add extra layers there to conform to the websocket protocol. It would be interesting to take this into consideration when designing the binary protocol, however. Perhaps we can put something together that would be able to use websockets as a transport protocol at a later point in time. It would be cool to, once a certain level of authorization/authentication logic has been implemented, build high performance in-browser clients that talk directly to the server.. /Jake On Mon, Aug 29, 2011 at 12:06 PM, Peter Neubauer peter.neuba...@neotechnology.com wrote: Thomas, a websocket interface would be a great addition! We haven't come around to do a proper binary protocol to Neo4j yet (pushed to Neo4j 1.6), but as Rick mentions, a websocket interface would also mean to have streaming semantics on the client. If you wanna prototype one with some trivial serialization that can be changed later, that would rock! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/ - Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Mon, Aug 29, 2011 at 12:01 PM, Thomas Fritz fritz...@gmail.com wrote: Hi! I hope you had some good bear friday! ;) Do you think a (web)socket interface to neo4j could be done and integrated into the core? I do not like the idea of using a high performance graph database with fast indices (like neo4j is) and than the only interface to it is a HTTP RESTful API to it. Or are my assumptions and fears wrong? kind regards --- Thomas FRITZ web http://fritzthomas.com twitter http://twitter.com/thomasf 2011/8/26 Rick Bullotta rick.bullo...@thingworx.com: We're doing some similar things using XMPP pub-sub and BOSH... -Original Message- From: user-boun...@lists.neo4j.org [mailto: user-boun...@lists.neo4j.org] On Behalf Of Thomas Fritz Sent: Friday, August 26, 2011 10:49 AM To: Neo4j user discussions Subject: Re: [Neo4j] Websocket Server instead of REST for Neo4J for access a DB If you like partial results take a look at this paper: http://research.microsoft.com/apps/pubs/default.aspx?id=131524 and this cast: http://channel9.msdn.com/Events/MIX/MIX11/RES04 It uses rabin fingerprinting to detect changed chunks and only transfers these changed chunks (like rsync does). In this case the connecting side - the client - would also need
Re: [Neo4j] More batch vs. single operation benchmarks
Wow. That's surprising data. With Neo4J embedded, we can usually get about 100X of that performance (including Lucene indexing of the nodes), so there clearly seem to be some big impacts from using the REST interface versus Neo4J embedded. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of jadell Sent: Sunday, August 28, 2011 3:55 PM To: user@lists.neo4j.org Subject: [Neo4j] More batch vs. single operation benchmarks Here are the latest benchmarks of batch vs. individual entity creation using the Neo4jPHP library. Most of the processing time is spent on the server, so I believe that these numbers are probably not specific to Neo4jPHP. I'm not implying that there is anything wrong or to be fixed; I just thought the results might be of interest to others. I'd love to see results from others using a REST client in any language. 3 scenarios were run 10 times each for different batch sizes. The scenarios and averages of the 10 runs of each batch size are below. First column is the batch size, second column is the average time in seconds to create that many entities in a batch, third column is the average time in seconds to create that many entities with individual calls. Benchmark script can be found here: http://gist.github.com/1177100 Results: Scenario 1: Create nodes sizebatchsingle 10 00 100 00.8 250 0.1 1.5 500 0.9 1.7 10001.5 3.8 25006.7 9.6 500023.5 13.2 Scenario 2: Create relationships sizebatchsingle 10 00 100 0.2 0.2 250 0.3 0.5 500 0.6 1.1 10001.3 3 25007.1 9.7 500025.3 22.2 Scenario 3: Create 2 nodes and a relationship between them sizebatchsingle 10 00.1 100 0.4 1.3 250 0.8 3 500 2.9 5.2 10009.2 11.9 250054.3 29.9 5000710.359.6 Note: in the 5000 run of the last scenario, 15000 operations are sent in a single HTTP request. 10 runs of the 5000 batch size (15 operations total operations in 10 batches) took almost 2 hours to complete. -- Josh Adell -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/More-batch-vs-single-operation-benchmarks-tp3291092p3291092.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Set reference node through REST API
No, it isn't needed. It is very helpful as an attachment point for other nodes and point of entry for traversals, but it isn't required. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Javier de la Rosa Sent: Saturday, August 27, 2011 12:14 PM To: Neo user discussions Subject: [Neo4j] Set reference node through REST API Hi all, As far I know, the REST API always returns a reference node. But in case of I delete it, there's no option to set it again through REST API. I read in this list that you could do this in Java like next: graph.delete(); newRoot = neo4j.createNode(); neo4j.setReferenceNode(newRoot); But I don't know the way to do this using the REST API. Even if I don't delete it anything, I would have to be able to set the reference node to a node of my preference. I know that was discussed previously, but is the reference node really needed? Regards. -- Javier de la Rosa http://versae.es ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Websocket Server instead of REST for Neo4J for access a DB
Let's hope that one day soon all of these issues will be non-issues! Having cool technology always coming soon reminds me of this sign: http://www.rareirishstuff.com/media/13/a20792b12af7736b49978d_m.JPG -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Thomas Fritz Sent: Friday, August 26, 2011 10:17 AM To: Neo4j user discussions Subject: Re: [Neo4j] Websocket Server instead of REST for Neo4J for access a DB Hi. Thanks for your fast reply. 2011/8/26 Rick Bullotta rick.bullo...@thingworx.com: A few potential challenges and reasons to wait on this: - the Websocket API is still in a state of change thats true. But i think, since the latest security issues are solved now with the latest protocol version, hopefully there are no breaking changes anymore - the existing REST API is synchronous request/response, so there's not a lot of benefit to switching to websockets without a lot of rework I think thats a real challenge. Since locking and concurrent writes come into play - Except you would allow only one websocket connection - The real benefit will be when you Javascript and other web clients can use a pure binary protocol (not possible today) I pretty sure it is in the latest protocol definition. But i think no Browser at this time of writing has implemented the binary parts. regards Tom Rick -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Thomas Fritz Sent: Friday, August 26, 2011 9:52 AM To: Neo4j user discussions Subject: [Neo4j] Websocket Server instead of REST for Neo4J for access a DB Hi all! Have anyone thought, or is it possible to implement a Websocket Endpoint in Neo4J Server so one can use this instead of the RESTful API. Which would be more performant and scalable than the RESTful HTTP API. It is possible to use such a Websocket through Java, Node and any other Server Side Language and even JavaScript on the client side. What do you think? Kind regards --- Thomas FRITZ web http://fritzthomas.com twitter http://twitter.com/thomasf ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Change multiple relations in one operation
A couple questions: - What are you changing? The properties of the relationship(s) or the relationships themselves? - Are you relationships many-to-many, one-to-many, or one-to-one? -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of cnordenfelt Sent: Thursday, August 11, 2011 1:55 PM To: user@lists.neo4j.org Subject: Re: [Neo4j] Change multiple relations in one operation Thanks For now I'm just prototyping and the case might be that there are several millions of relationshids that has to be changed in one transaction. However, it is a very uncommon operation and it might also be that the graph design should be changed to limit the number of relationships? //Carl On Thu, Aug 11, 2011 at 6:04 PM, Peter Neubauer [via Neo4j Community Discussions] ml-node+3246384-1725270690-421...@n3.nabble.com wrote: Carl, this is what you can do. Relationships are immutable, that is, once created you have to delete them and create new ones in order to change endpoints. Not much choice there. What is the size of your reassignments and the number of relationships in one transaction? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer a href=http://www.neo4j.org http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, Aug 11, 2011 at 4:47 PM, cnordenfelt [hidden email]http://user/SendEmail.jtp?type=nodenode=3246384i=0 wrote: Hi I have started to digg into neo4j and have not found a nice way to change many relationships from one node to another. For example, I have n1 with a relationship to m1, m2 and m3. Now I would like to change the relationship to point from n2 to m1, m2 and m3 instead. I could of course fetch all relationships from n1 and change each relationship individually to n2 but this will make me iterate over all relationships, which could be quite many. Is there any way to replace a node with another or to reassign multiple relationships with one operation? I'm currently using the Java API. Thanks Carl -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Change-multiple-relations-in-one-operation-tp3246141p3246141.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list [hidden email] http://user/SendEmail.jtp?type=nodenode=3246384i=1 https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list [hidden email] http://user/SendEmail.jtp?type=nodenode=3246384i=2 https://lists.neo4j.org/mailman/listinfo/user -- If you reply to this email, your message will be added to the discussion below: http://neo4j-community-discussions.438527.n3.nabble.com/Change-multiple-relations-in-one-operation-tp3246141p3246384.html To unsubscribe from Change multiple relations in one operation, click herehttp://neo4j-community-discussions.438527.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_codenode=3246141code=Y2FybC5ub3JkZW5mZWx0QGdtYWlsLmNvbXwzMjQ2MTQxfDQwNDcxMjI=. -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Change-multiple-relations-in-one-operation-tp3246141p3246722.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Graph database index | query across indices
or of course you could just put it all in one index.
-Original Message-
From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On
Behalf Of Mattias Persson
Sent: Wednesday, August 10, 2011 3:31 PM
To: Neo4j user discussions
Subject: Re: [Neo4j] Graph database index | query across indices
Compound query between two indices isn't possible, but you could just query
both and intersect the node set with the end or start node set from the
relationships.
2011/7/27 sambodhi sagi sambodhi.s...@gmail.com
Hi !
I am facing some problem with indexing. My graph looks like this:
Say, I want to query products for a particular category (say 'Phone,Mobiles
and telecoms') AND having text 'attractive' in 'details' field. I created
two indices: one for Relationship 'RelProductIndex' and one for product
'ProductIndex'. Algo for what I exactly want to achieve :
typeHits = relProdctIndex.query( type:Product, phoneAndTelecomCatNode,
null );
for each typeHits {
productsInCatList.add(typeHits.getEndNode());
}
productsTextSearchHits = productIndex.query(details, attractive);
resultantProducts = common set between productsInCatList and
productsTextSearchHits
My query is something like this
QUERY-1 AND QUERY-2
QUERY-1 : EndNodes (say end node is a product) of a relationship of type
'Product' (this query is for RelProductIndex)
QUERY-2 : products which has text 'mobile' (this query is for
ProductsIndex)
So basically what it requires is *compound query across two indices*.
Normally we donot do compound queries across indices but am not sure how to
handle this.* One solution* I can think of is to add 'details' property to
the relationship Product also. In this case I can use only relationship
index:
typeHits = relProdctIndex.query( type:Product AND details:attractive,
phoneAndTelecomCatNode, null );
Problem I see with this approach is, I would end up indexing same field(or
key) in two different indices (Product node and Product relationship). This
might hit the performance.* Can there be a better approach to handle this?*
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Mattias Persson, [matt...@neotechnology.com]
Hacker, Neo Technology
www.neotechnology.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] cant use index after commit
Which version? 1.4 or 1.4.1? From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of ahmed.elsharkasy [ahmed.elshark...@gmail.com] Sent: Sunday, August 07, 2011 8:42 AM To: user@lists.neo4j.org Subject: [Neo4j] cant use index after commit i deleted some nodes and committed and was successfuly deleted but in the same run i tried to start my batch index again and i faced this exception Exception in thread main java.lang.RuntimeException: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: nativefsl...@write.lock at org.neo4j.index.impl.lucene.LuceneBatchInserterIndex.instantiateWriter(LuceneBatchInserterIndex.java:240) at org.neo4j.index.impl.lucene.LuceneBatchInserterIndex.init(LuceneBatchInserterIndex.java:72) at org.neo4j.index.impl.lucene.LuceneBatchInserterIndexProvider.index(LuceneBatchInserterIndexProvider.java:135) at org.neo4j.index.impl.lucene.LuceneBatchInserterIndexProvider.nodeIndex(LuceneBatchInserterIndexProvider.java:102) at databaseManagers.Neo4jDatabaseManager.startIndex(Neo4jDatabaseManager.java:266) at databaseManagers.Neo4jDatabaseManager.deleteDocument(Neo4jDatabaseManager.java:155) at databaseManagers.Neo4jDatabaseManager.main(Neo4jDatabaseManager.java:450) Caused by: org.apache.lucene.store.LockObtainFailedException: Lock obtain timed out: nativefsl...@...write.lock at org.apache.lucene.store.Lock.obtain(Lock.java:84) at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1115) at org.neo4j.index.impl.lucene.LuceneBatchInserterIndex.instantiateWriter(LuceneBatchInserterIndex.java:235) ... 6 more what is the problem? -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/cant-use-index-after-commit-tp3233038p3233038.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] cant use index after commit
Give 1.4.1 a try. On Aug 7, 2011, at 10:36 AM, ahmed.elsharkasy ahmed.elshark...@gmail.com wrote: 1.4 -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/cant-use-index-after-commit-tp3233038p3233172.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Batch find
You'll probably want to use an Index for this. Either a Lucene index or in-graph index. I would recommend a Lucene index, since you can also leverage Lucene (and even Solr's) analyzers and parsers to process your document. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of ahmed.elsharkasy Sent: Wednesday, August 03, 2011 7:15 AM To: user@lists.neo4j.org Subject: Re: [Neo4j] Batch find I am trying to insert a document containing list of words , and i wont to check whether some of this words are already in my graph and in this case i will update their properties otherwise i will create new nodes with the new words -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Batch-find-tp3221634p3221964.html Sent from the Neo4j Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Synchronization of EmbeddedReadOnlyGraphDatabase - Bug?
FWIW, I really like the idea of a remote API. Not only for tooling, but as an alternative to bulkier and more abstracted REST APIs. One way to think of it could be a high performance, binary-formatted REST API - if the overhead of HTTP isn't too much (and I seriously doubt that it would be - I implemented a solution at my previous company, Lighthammer, using HTTP + a binary stream protocol and it was considerably faster than native binary protocols for databases and other similar scenarios in most if not all use cases). This API would allow you to view a running instance from the P.O.V. as if you were another thread accessing the same embedded instance. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Tobias Ivarsson Sent: Monday, August 01, 2011 2:53 PM To: Neo4j user discussions Subject: Re: [Neo4j] Synchronization of EmbeddedReadOnlyGraphDatabase - Bug? I think a bit of elaboration might be in order. EmbeddedReadOnlyGraphDatabase was created for one specific purpose: Being able to interactively introspect a graph without having to shut down the application that uses it. Specifically the tools that we wanted to support with this were the Neo4j shell and Neoclipse. EmbeddedReadOnlyGraphDatabase (EROGD) has two major issues with way caching is done internally in Neo4j (one issue with each cache): - When the EROGD reads data from the file system it will, like a normal EGD, cache the node and relationship objects. If a normal EGD modifies the graph under the feet of the EROGD, there is no way for the EROGD to know that the data in cache is now stale, which will lead to an inconsistent view of the graph. If for example the EROGD has cached Node[15] with the information that it is connected to some other node through Relationship[344], and Relationship[344] is deleted you will get InvalidRecordException (as you described). And of course if relationships are added to Node[15] these will not be seen at all by the EROGD (until Node[15] is evicted from the cache due to not being used for a while). - Neo4j also caches data on the filesystem level by memory mapping (mmap) hot regions of the store files. Writes to these regions will not be flushed to the actual file until the mmapped window is evicted due to being less hot than other windows, or when the transaction log for Neo4j is rotated. This means that from the p.o.v. of the EROGD the actual data written to disk will look inconsistent. Which would also lead to InvalidRecordExcaption. This situation is actually made even more complicated by the fact that unix operating systems will attempt to share memory mapped data from the same file between multiple processes, but the normal EGD and the EROGD will not make the same decisions on which regions to mmap, they might not even decide on the same size for mmap windows. We haven't tested how well different operating systems deal with reading data that was written to an mmap region through non-mmap syscalls from a different process, most likely this varies from OS to OS. The second of these problems is of course the worst, since it cannot be worked around. The first one can be mitigated by configuring Neo4j to not use the object cache, by passing the cache_type=none parameter to the constructor of the EROGD. This should really be made default for EROGD, unless we decide to completely remove EROGD. I hope that sheds some light on the reasons why you experience these problems with EmbeddedReadOnlyGraphDatabase, and what the intention of creating it was. As a side note I can mention that I had a different idea for how to solve the introspection-of-live-graph problem at the time EmbeddedReadOnlyGraphDatabase was created: Create network based implementation of the GraphDatabaseService API and connect directly to the running instance. This would completely avoid the cache staleness problem, but at the cost of network overhead for each graph operation, which is probably fine for tooling purposes. With the JVM agent attach protocol it would be possible to inject such a server into a running graph database that wasn't originally configured for it. I in fact implemented this as the RemoteGraphDatabase subproject. Since my colleagues did not share my vision about that idea, this project didn't receive much attention after its initial inception. It was also never really used for these purposes, but rather misused for building applications, leading us to deprecate the project. When we then later discovered a severe bug in the implementation of the remote transaction handling logic, we completely removed the project. I still believe this to be a superior model for tools, but would build it differently if I were to build it today. -tobias On Mon, Aug 1, 2011 at 4:48 PM, Jim Webber j...@neotechnology.com wrote: Hi Mathias, EmbeddedReadOnlyGraphDatabase is not quite what it
Re: [Neo4j] Paged traversal REST API suggestion for improvement
If you don't keep state paging will not work properly if the data changes often. What may have been record #21 when you are viewing the first page of 20 result might not be record #21 when you go to fetch the 2nd page. If you're only concerned with pages of 20 or so, you should absolutely, positively do the paging on the browser. Anything up to a couple thousand rows can *easily* be handled by any modern browser. Keep the parsed JSON data on the client side and page it there. This approach will be very performant on both the client and server, and won't require any state. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Aseem Kishore Sent: Thursday, July 28, 2011 3:01 PM To: Neo4j user discussions Subject: Re: [Neo4j] Paged traversal REST API suggestion for improvement Jim, thanks for the explanation. I understand your constraints, but thinking about it more, I'm actually even more baffled -- how can we actually make use of this paged traversal API in a web scenario? [neo4j db] [web server] [web client, e.g. browser] If I want to show the results of a search, let's say, in a way that supports paging, it just seems like I can't take advantage of the Neo4j API at all here. If the user wants to be able to refresh the page, or click Prev, the browser isn't keeping state across page changes, so it has to rely on the web server to give it the right page. If the server now has to keep state of all results, it defeats the purpose of using the Neo4j paging API altogether: the server necessarily has to fetch everything and handle paging itself. Am I missing something? Or is this (typical in our case) scenario not something you guys are designing for? (And if not, what *is* the use case you guys envisioned for this?) It seems to me that the root cause of this dilemma is the IterableT constraint you mentioned -- it only goes forward. Perhaps a better API/model then would be to use an offset parameter. This is different than page because it doesn't require the server to keep state; it just tells the server, return me the results starting *after* this result (which would generally be the last result I saw on my previous page of results). E.g. the following requests: - Get me the results of this traverse would return everything - Get me the results of this traverse, paged with size 20 would return 20 results - Get me the results of this traverse, paged with size 20, starting with the node/rel after this one [given by an ID] would return the next 20 results - etc. Aseem On Thu, Jul 28, 2011 at 5:09 AM, Jim Webber j...@neotechnology.com wrote: Hi Aseem, When you GET a resource, you're asking for its state at a particular instant in time. GET's don't always return the same content, what's important about them is that clients can't be held responsible for any (unintended) side-effects of a GET operation. For example, if you GET the BBC new page now, and then GET it again in 20 minutes, the bytes you receive will almost certainly be different. I think what's perhaps a bit odd in the paged traverser case is that clients do in fact trigger a state transition on GET which materially changes the response. Adding /next to the paged traverser URI does not change this, it just lengthens the URI. Under the covers, the REST API is just using the traversal API and so gets back an IterableT which is forward-only. We can't (or rather won't) do things like ?page=2 (or similar) because that would involve holding all the traversal state on the server which is one of the things paged traversers were meant to avoid (being kind to both the server and clients). If you want to remember all traversal state on the client, that's OK. You can page through your own local data structures. In fact given the number of clients is likely to be larger than the number of servers, this makes sense since the memory burden is shared amongst the many clients, rather than being placed on the single server. So, I'll buy that we've conflated a little too much into a single GET, but it's motivated by a sincere trade off to not overburden the server. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Paged traversal REST API suggestion for improvement
BTW, paging is a relic of the dial-up modem days, IMNSHO. If a machine is the client of the REST API call, it should get all the data in a single, atomic call. If it is a browser or mobile app, there's a natural limit to the # of items that a human will bother paging through, which is fairly small (20 - 500 typically), so again, it is painless to get all the data in a single call. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Rick Bullotta Sent: Thursday, July 28, 2011 6:11 PM To: Neo4j user discussions Subject: Re: [Neo4j] Paged traversal REST API suggestion for improvement If you don't keep state paging will not work properly if the data changes often. What may have been record #21 when you are viewing the first page of 20 result might not be record #21 when you go to fetch the 2nd page. If you're only concerned with pages of 20 or so, you should absolutely, positively do the paging on the browser. Anything up to a couple thousand rows can *easily* be handled by any modern browser. Keep the parsed JSON data on the client side and page it there. This approach will be very performant on both the client and server, and won't require any state. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Aseem Kishore Sent: Thursday, July 28, 2011 3:01 PM To: Neo4j user discussions Subject: Re: [Neo4j] Paged traversal REST API suggestion for improvement Jim, thanks for the explanation. I understand your constraints, but thinking about it more, I'm actually even more baffled -- how can we actually make use of this paged traversal API in a web scenario? [neo4j db] [web server] [web client, e.g. browser] If I want to show the results of a search, let's say, in a way that supports paging, it just seems like I can't take advantage of the Neo4j API at all here. If the user wants to be able to refresh the page, or click Prev, the browser isn't keeping state across page changes, so it has to rely on the web server to give it the right page. If the server now has to keep state of all results, it defeats the purpose of using the Neo4j paging API altogether: the server necessarily has to fetch everything and handle paging itself. Am I missing something? Or is this (typical in our case) scenario not something you guys are designing for? (And if not, what *is* the use case you guys envisioned for this?) It seems to me that the root cause of this dilemma is the IterableT constraint you mentioned -- it only goes forward. Perhaps a better API/model then would be to use an offset parameter. This is different than page because it doesn't require the server to keep state; it just tells the server, return me the results starting *after* this result (which would generally be the last result I saw on my previous page of results). E.g. the following requests: - Get me the results of this traverse would return everything - Get me the results of this traverse, paged with size 20 would return 20 results - Get me the results of this traverse, paged with size 20, starting with the node/rel after this one [given by an ID] would return the next 20 results - etc. Aseem On Thu, Jul 28, 2011 at 5:09 AM, Jim Webber j...@neotechnology.com wrote: Hi Aseem, When you GET a resource, you're asking for its state at a particular instant in time. GET's don't always return the same content, what's important about them is that clients can't be held responsible for any (unintended) side-effects of a GET operation. For example, if you GET the BBC new page now, and then GET it again in 20 minutes, the bytes you receive will almost certainly be different. I think what's perhaps a bit odd in the paged traverser case is that clients do in fact trigger a state transition on GET which materially changes the response. Adding /next to the paged traverser URI does not change this, it just lengthens the URI. Under the covers, the REST API is just using the traversal API and so gets back an IterableT which is forward-only. We can't (or rather won't) do things like ?page=2 (or similar) because that would involve holding all the traversal state on the server which is one of the things paged traversers were meant to avoid (being kind to both the server and clients). If you want to remember all traversal state on the client, that's OK. You can page through your own local data structures. In fact given the number of clients is likely to be larger than the number of servers, this makes sense since the memory burden is shared amongst the many clients, rather than being placed on the single server. So, I'll buy that we've conflated a little too much into a single GET, but it's motivated by a sincere trade off to not overburden the server. Jim ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo
Re: [Neo4j] Paged traversal REST API suggestion for improvement
Not really, unless the JSON content is HUGE - which is rare. The small price in bandwidth versus a responsive UI and the need for not managing state is a good tradeoff. The only exception might be mobile apps, but in that case, I'd just fetch a smaller block (maybe 4 or 5 pages worth). Very few users have the patience to page through more than a few pages worth of information. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Daniel Gasienica Sent: Thursday, July 28, 2011 6:20 PM To: Neo4j user discussions Subject: Re: [Neo4j] Paged traversal REST API suggestion for improvement Sure but isn’t it a huge waste of bandwidth if you load hundreds of results and the user only looks at the first dozen? On Thu, Jul 28, 2011 at 15:16, Rick Bullotta rick.bullo...@thingworx.comwrote: BTW, paging is a relic of the dial-up modem days, IMNSHO. If a machine is the client of the REST API call, it should get all the data in a single, atomic call. If it is a browser or mobile app, there's a natural limit to the # of items that a human will bother paging through, which is fairly small (20 - 500 typically), so again, it is painless to get all the data in a single call. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Rick Bullotta Sent: Thursday, July 28, 2011 6:11 PM To: Neo4j user discussions Subject: Re: [Neo4j] Paged traversal REST API suggestion for improvement If you don't keep state paging will not work properly if the data changes often. What may have been record #21 when you are viewing the first page of 20 result might not be record #21 when you go to fetch the 2nd page. If you're only concerned with pages of 20 or so, you should absolutely, positively do the paging on the browser. Anything up to a couple thousand rows can *easily* be handled by any modern browser. Keep the parsed JSON data on the client side and page it there. This approach will be very performant on both the client and server, and won't require any state. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Aseem Kishore Sent: Thursday, July 28, 2011 3:01 PM To: Neo4j user discussions Subject: Re: [Neo4j] Paged traversal REST API suggestion for improvement Jim, thanks for the explanation. I understand your constraints, but thinking about it more, I'm actually even more baffled -- how can we actually make use of this paged traversal API in a web scenario? [neo4j db] [web server] [web client, e.g. browser] If I want to show the results of a search, let's say, in a way that supports paging, it just seems like I can't take advantage of the Neo4j API at all here. If the user wants to be able to refresh the page, or click Prev, the browser isn't keeping state across page changes, so it has to rely on the web server to give it the right page. If the server now has to keep state of all results, it defeats the purpose of using the Neo4j paging API altogether: the server necessarily has to fetch everything and handle paging itself. Am I missing something? Or is this (typical in our case) scenario not something you guys are designing for? (And if not, what *is* the use case you guys envisioned for this?) It seems to me that the root cause of this dilemma is the IterableT constraint you mentioned -- it only goes forward. Perhaps a better API/model then would be to use an offset parameter. This is different than page because it doesn't require the server to keep state; it just tells the server, return me the results starting *after* this result (which would generally be the last result I saw on my previous page of results). E.g. the following requests: - Get me the results of this traverse would return everything - Get me the results of this traverse, paged with size 20 would return 20 results - Get me the results of this traverse, paged with size 20, starting with the node/rel after this one [given by an ID] would return the next 20 results - etc. Aseem On Thu, Jul 28, 2011 at 5:09 AM, Jim Webber j...@neotechnology.com wrote: Hi Aseem, When you GET a resource, you're asking for its state at a particular instant in time. GET's don't always return the same content, what's important about them is that clients can't be held responsible for any (unintended) side-effects of a GET operation. For example, if you GET the BBC new page now, and then GET it again in 20 minutes, the bytes you receive will almost certainly be different. I think what's perhaps a bit odd in the paged traverser case is that clients do in fact trigger a state transition on GET which materially changes the response. Adding /next to the paged traverser URI does not change this, it just lengthens the URI. Under the covers, the REST API is just using the traversal API
Re: [Neo4j] Paged traversal REST API suggestion for improvement
Um, I'm guessing that you aren't aware of how Google's UI/API works... Open up Firebug, Chrome Tools, or Fiddler, and you'll see that the biggest chunk of traffic when you switch from one page of Google search results to another page (via the main Google search page) is a JSON packet...which uses their search API...which returns JSON. Do they page? Yes. Can you call the API and get hundreds of results at once? Yes... -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Aseem Kishore Sent: Thursday, July 28, 2011 6:42 PM To: Neo4j user discussions Subject: Re: [Neo4j] Paged traversal REST API suggestion for improvement Sorry, I don't understand. Are you going to tell Google and Bing to send JSON for their search results also, and page on the client side? There is a very valid use case for not wanting to require JavaScript to page. Aseem On Thu, Jul 28, 2011 at 3:25 PM, Rick Bullotta rick.bullo...@thingworx.comwrote: Not really, unless the JSON content is HUGE - which is rare. The small price in bandwidth versus a responsive UI and the need for not managing state is a good tradeoff. The only exception might be mobile apps, but in that case, I'd just fetch a smaller block (maybe 4 or 5 pages worth). Very few users have the patience to page through more than a few pages worth of information. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Daniel Gasienica Sent: Thursday, July 28, 2011 6:20 PM To: Neo4j user discussions Subject: Re: [Neo4j] Paged traversal REST API suggestion for improvement Sure but isn't it a huge waste of bandwidth if you load hundreds of results and the user only looks at the first dozen? On Thu, Jul 28, 2011 at 15:16, Rick Bullotta rick.bullo...@thingworx.com wrote: BTW, paging is a relic of the dial-up modem days, IMNSHO. If a machine is the client of the REST API call, it should get all the data in a single, atomic call. If it is a browser or mobile app, there's a natural limit to the # of items that a human will bother paging through, which is fairly small (20 - 500 typically), so again, it is painless to get all the data in a single call. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Rick Bullotta Sent: Thursday, July 28, 2011 6:11 PM To: Neo4j user discussions Subject: Re: [Neo4j] Paged traversal REST API suggestion for improvement If you don't keep state paging will not work properly if the data changes often. What may have been record #21 when you are viewing the first page of 20 result might not be record #21 when you go to fetch the 2nd page. If you're only concerned with pages of 20 or so, you should absolutely, positively do the paging on the browser. Anything up to a couple thousand rows can *easily* be handled by any modern browser. Keep the parsed JSON data on the client side and page it there. This approach will be very performant on both the client and server, and won't require any state. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Aseem Kishore Sent: Thursday, July 28, 2011 3:01 PM To: Neo4j user discussions Subject: Re: [Neo4j] Paged traversal REST API suggestion for improvement Jim, thanks for the explanation. I understand your constraints, but thinking about it more, I'm actually even more baffled -- how can we actually make use of this paged traversal API in a web scenario? [neo4j db] [web server] [web client, e.g. browser] If I want to show the results of a search, let's say, in a way that supports paging, it just seems like I can't take advantage of the Neo4j API at all here. If the user wants to be able to refresh the page, or click Prev, the browser isn't keeping state across page changes, so it has to rely on the web server to give it the right page. If the server now has to keep state of all results, it defeats the purpose of using the Neo4j paging API altogether: the server necessarily has to fetch everything and handle paging itself. Am I missing something? Or is this (typical in our case) scenario not something you guys are designing for? (And if not, what *is* the use case you guys envisioned for this?) It seems to me that the root cause of this dilemma is the IterableT constraint you mentioned -- it only goes forward. Perhaps a better API/model then would be to use an offset parameter. This is different than page because it doesn't require the server to keep state; it just tells the server, return me the results starting *after* this result (which would generally be the last result I saw on my previous page of results). E.g. the following requests: - Get me the results of this traverse would return
Re: [Neo4j] Concurrent access to Index
You don't need to open multiple instances. Simply share the instance across threads - it is thread safe. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of paolo.forte Sent: Wednesday, July 27, 2011 9:14 AM To: user@lists.neo4j.org Subject: [Neo4j] Concurrent access to Index Hello, I'm testing neo4j (1.4) with a massive and continuous process: I have an instance of EmbeddedGraphDatabase where each node must be unique (a node identify a userid). My current problem is that when I try to open another instance of graphdb with EmbeddedReadOnlyGraphDatabase in order to query the graph (meanwhile the first instance is still inserting new data) I experienced the following error: Exception in thread main java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)Caused by: java.lang.RuntimeException: java.io.FileNotFoundException: /data/neo4j/metadata.db/index/lucene/node/users_id/_22j1m.cfs (No such file or directory) at org.neo4j.index.impl.lucene.LuceneDataSource.getIndexWriter(LuceneDataSource.java:601) at org.neo4j.index.impl.lucene.LuceneDataSource.getIndexSearcher(LuceneDataSource.java:484) at org.neo4j.index.impl.lucene.LuceneIndex.query(LuceneIndex.java:248) at org.neo4j.index.impl.lucene.LuceneIndex.get(LuceneIndex.java:187) at nosql.metadata.test.Neo4JQueryTest.main(Neo4JQueryTest.java:117) ... 5 moreCaused by: java.io.FileNotFoundException: /data/neo4j/metadata.db/index/lucene/node/users_id/_22j1m.cfs (No such file or directory) at java.io.RandomAccessFile.open(Native Method) at java.io.RandomAccessFile.init(RandomAccessFile.java:233) at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput$Descriptor.init(SimpleFSDirectory.java:69) at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.init(SimpleFSDirectory.java:90) at org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.init(NIOFSDirectory.java:91) at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:78) at org.apache.lucene.index.CompoundFileReader.init(CompoundFileReader.java:66) at org.apache.lucene.index.CompoundFileReader.init(CompoundFileReader.java:55) at org.apache.lucene.index.IndexWriter.getFieldInfos(IndexWriter.java:1193) at org.apache.lucene.index.IndexWriter.getCurrentFieldInfos(IndexWriter.java:1213) at org.apache.lucene.index.IndexWriter.init(IndexWriter.java:1149) at org.neo4j.index.impl.lucene.LuceneDataSource.getIndexWriter(LuceneDataSource.java:588) ... 9 more I can recover the db index renaming the index segment (probably with loss of data) and try to recover it with CheckIndex Probably is my lack on knolodge on lucene indexing, but how can I (MULTIPLE) query in safe way the db without corrupting the lucene index segments? I also noted that if I start the server in read only mode (in order to connect to an EmbeddedGraphDatabase that continuously ingest new data) the webadmin interface doesn't update the number of nodes, edges and relationships are the two problems correlated? Thanks in advance for any suggestions and help. Paolo Forte As a matter of fact I would like to understand how it is possible to write and read (query in read only mode) the db. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Number of properties on a PropertyContainer
That's a perfect use case for a super trivial Lucene index, no? Would seem to be a much easier solution with much faster lookup...and built into Neo4J. From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Niels Hoogeveen [pd_aficion...@hotmail.com] Sent: Friday, July 22, 2011 8:21 AM To: user@lists.neo4j.org Subject: [Neo4j] Number of properties on a PropertyContainer Last night while working on some enhancements on the Neo4J API, I set up a method to lookup the name of a relationship, to facilitate the creation of associated nodes. The method I use, is to create a node (which can be found through the reference node) and on this node create a property for each relationship name. The value of the property is the node id of the associated node. My question is if this scales properly. It is unlikely any Neo4J store will contain millions of relationship types, but it is conceivable that there may be over ten thousand relationship types in a store. Will this solution still work under such conditions, or is it better to create a relationship for each relationship type and store the node id of the associated node in the relation. Niels ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Evaluation questions : Neo4j features
Hi, Stefan. FWIW, we use Neo4J for time series data, at reasonably high volumes of near-real-time storage. What storage rates are you looking for/volume of data in your application? Rick From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of acmeguy [ste...@gmail.com] Sent: Thursday, July 21, 2011 5:27 AM To: user@lists.neo4j.org Subject: [Neo4j] Evaluation questions : Neo4j features Hi, I'm relatively new to graphs and Neo4j but I have gathered a list of questions I hoping this community can answer or point me onward towards them. 1. Typed edges - What is available in Neo4j to tackle semantics (as in hierarchical structure of edge types) - What are the best practices when dealing with semantics? 2. Versioning - What are the best practices when dealing with data versioning? 3. Hypergraphs - What is the practical advantage of supporting multi target edges other than to reduce the edge count in a graph? (I ask only because I'm trying to better understand the architectural difference between Neo4j and HypergraphDB) 4. Property count and property data - Is there a practical limit to the number of properties per node? - Would anyone store binary content in them (are they all loaded into memory as a part of the graph)? 5. MQL I have used MQL (the Freebase.com graph query language) and I have, in my limited use of it, found it quite intuitive. I have tried Cypher and SPARQL as well. - What is the state of single industry wide We don't have to deal with data in the I-have-to-have-automatic-sharding range so I leave performance and scalability issues out for now. We are planing to use Neo4j for a sub-CRMish structure and HBase for event logs (time series etc.) and realtime analytics/statistics. Would anyone claim that a craph database is suitable for that? Best regards and thank's for the great job, -Stefan -- View this message in context: http://neo4j-community-discussions.438527.n3.nabble.com/Evaluation-questions-Neo4j-features-tp3188069p3188069.html Sent from the Neo4J Community Discussions mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Auto-index fulltext?
Chris, I think that auto indexing is another great reason for a formal concept of node type. It could provide an unambiguous link between a node and its indexing strategy(ies). - Reply message - From: Chris Gioran chris.gio...@neotechnology.com Date: Mon, Jul 18, 2011 6:16 am Subject: [Neo4j] Auto-index fulltext? To: Neo4j user discussions user@lists.neo4j.org No, that is not what i meant. The main idea is to provide the means to configure at least some aspects of the auto index instead of relying on the default settings only. There will still be one auto index for each primitive category. However, one feature under consideration is to actually allow an arbitrary number of auto indexes, each of which will allow for individual configuration. So, when that comes along you will have what you described (and more, actually). thanks, CG On Mon, Jul 18, 2011 at 12:29 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Awesome to hear, Chris, thanks. Just to clarify/confirm then: in the future, we will be able to have *both* an exact auto-index and a fulltext auto-index side-by-side? Cheers, Aseem On Mon, Jul 18, 2011 at 3:18 AM, Chris Gioran chris.gio...@neotechnology.com wrote: Hi Aseem, On Sat, Jul 16, 2011 at 9:46 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Is the 1.4 auto-index only exact? Or can it be configured to be a fulltext index? Yes, currently the auto-indexes are only exact, there is no straightforward way to configure them explicitly. This is a known shortcoming and will be remedied pretty soon. Another addition that is coming, by the way, is the visibility of changes of the auto index within the transaction, instead of waiting for commit time as is now. (Btw, it would be awesome if we could have two auto-indexes: one exact, one full-text. It would be great in general if all indexing could be auto. Not sure when you would ever want/need manual indexing.) Well, a lot of use cases call for manual indexing, when for example it is conditional or the use of multiple indexes is required. If, however, auto indexing covers all your needs then by all means, do just that. cheers, CG Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Auto-index fulltext?
I was merely suggesting incremental steps towards nirvana. ;-) - Reply message - From: Peter Neubauer peter.neuba...@neotechnology.com Date: Mon, Jul 18, 2011 8:19 am Subject: [Neo4j] Auto-index fulltext? To: Neo4j user discussions user@lists.neo4j.org But then, how about conditionals and more complex stuff like relationships connected to nodes, inferred type because of in-graph reasoning etc? Node Type as a simple String is not adding much over other recognizable patterns, and much less usable other than crude scenarios. If you want that, why not set a TYPE property and then (in future versions) link conditional auto-indexing to that? Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Mon, Jul 18, 2011 at 2:15 PM, Rick Bullotta rick.bullo...@thingworx.com wrote: Chris, I think that auto indexing is another great reason for a formal concept of node type. It could provide an unambiguous link between a node and its indexing strategy(ies). - Reply message - From: Chris Gioran chris.gio...@neotechnology.com Date: Mon, Jul 18, 2011 6:16 am Subject: [Neo4j] Auto-index fulltext? To: Neo4j user discussions user@lists.neo4j.org No, that is not what i meant. The main idea is to provide the means to configure at least some aspects of the auto index instead of relying on the default settings only. There will still be one auto index for each primitive category. However, one feature under consideration is to actually allow an arbitrary number of auto indexes, each of which will allow for individual configuration. So, when that comes along you will have what you described (and more, actually). thanks, CG On Mon, Jul 18, 2011 at 12:29 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Awesome to hear, Chris, thanks. Just to clarify/confirm then: in the future, we will be able to have *both* an exact auto-index and a fulltext auto-index side-by-side? Cheers, Aseem On Mon, Jul 18, 2011 at 3:18 AM, Chris Gioran chris.gio...@neotechnology.com wrote: Hi Aseem, On Sat, Jul 16, 2011 at 9:46 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Is the 1.4 auto-index only exact? Or can it be configured to be a fulltext index? Yes, currently the auto-indexes are only exact, there is no straightforward way to configure them explicitly. This is a known shortcoming and will be remedied pretty soon. Another addition that is coming, by the way, is the visibility of changes of the auto index within the transaction, instead of waiting for commit time as is now. (Btw, it would be awesome if we could have two auto-indexes: one exact, one full-text. It would be great in general if all indexing could be auto. Not sure when you would ever want/need manual indexing.) Well, a lot of use cases call for manual indexing, when for example it is conditional or the use of multiple indexes is required. If, however, auto indexing covers all your needs then by all means, do just that. cheers, CG Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Auto-index fulltext?
We manage that today using Neo. Node types are represented by an array property on the node - therefore a node can be of many types. We then utilize relationships to provide an implements relationship that enables arbitrarily deep and complex inheritance/implementation scenarios. My core point was that something better than primitive type is clearly needed, and a node type is a step in the right direction. As Peter points out, it's one. That said, if you make it overly complex to support every possible scenario, then you've pretty much defeated the benefit of auto indexing in the first place. The logic to decide which index to use (and the associated code behind) could easily be as complex or more complex to maintain than application level code. Callbacks and such are just another form of code, so it really doesn't seem to me to be the auto indexing scenario. Auto indexing should be declarative, shouldn't it? I prefer to make the easy stuff easy and the hard stuff possible - and that implies using simple, easy-to-implement and easy-to-maintain patterns for the stuff that 95% of applications need, and API-based approaches for the other 5%. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Niels Hoogeveen Sent: Monday, July 18, 2011 8:50 AM To: user@lists.neo4j.org Subject: Re: [Neo4j] Auto-index fulltext? Rick, I think adding a node type to neo4j is not a good idea. Different applications have different typing needs. My own application for example, supports multiple node types per node, while node types can be subtyped as well. This creates a forest of types for each node, that needs to be traversed each time the typing information of a node is requested. Morever the INSTANCE_OF relationships are stored as and Indexed Relationship (https://github.com/peterneubauer/graph-collections/tree/master/src/main/java/org/neo4j/collections/indexedrelationship), because certain types can have millions of instances. I am certain that such a typing construct is not everybody's cup of tea and I wouldn't want to see it formally supported in core. Neither would I want to see support for typing constructs other than the one I use. Being a schemaless database is one of the strengths of Neo4J, and typing shouldn't become part of the core distribution.That said, there are good reasons to create different typing libraries as a layer on top of Neo4J, the meta model component, though in need of some attention, is one such layer. For auto-indexing purposes, I would much rather see the addition of an installable call back function that takes:the node, the key, the value and the index as input and returns the value to be indexed or null if no indexing should take place. Niels From: rick.bullo...@thingworx.com To: chris.gio...@neotechnology.com; user@lists.neo4j.org Date: Mon, 18 Jul 2011 05:15:08 -0700 Subject: Re: [Neo4j] Auto-index fulltext? Chris, I think that auto indexing is another great reason for a formal concept of node type. It could provide an unambiguous link between a node and its indexing strategy(ies). - Reply message - From: Chris Gioran chris.gio...@neotechnology.com Date: Mon, Jul 18, 2011 6:16 am Subject: [Neo4j] Auto-index fulltext? To: Neo4j user discussions user@lists.neo4j.org No, that is not what i meant. The main idea is to provide the means to configure at least some aspects of the auto index instead of relying on the default settings only. There will still be one auto index for each primitive category. However, one feature under consideration is to actually allow an arbitrary number of auto indexes, each of which will allow for individual configuration. So, when that comes along you will have what you described (and more, actually). thanks, CG On Mon, Jul 18, 2011 at 12:29 PM, Aseem Kishore aseem.kish...@gmail.com wrote: Awesome to hear, Chris, thanks. Just to clarify/confirm then: in the future, we will be able to have *both* an exact auto-index and a fulltext auto-index side-by-side? Cheers, Aseem On Mon, Jul 18, 2011 at 3:18 AM, Chris Gioran chris.gio...@neotechnology.com wrote: Hi Aseem, On Sat, Jul 16, 2011 at 9:46 AM, Aseem Kishore aseem.kish...@gmail.com wrote: Is the 1.4 auto-index only exact? Or can it be configured to be a fulltext index? Yes, currently the auto-indexes are only exact, there is no straightforward way to configure them explicitly. This is a known shortcoming and will be remedied pretty soon. Another addition that is coming, by the way, is the visibility of changes of the auto index within the transaction, instead of waiting for commit time as is now. (Btw, it would be awesome if we could have two auto-indexes: one exact, one full-text. It would be great in general if all indexing could be auto. Not sure when you would ever want/need manual indexing.)
Re: [Neo4j] [Blog Post] Graphs, Brains, and Gremlin
Cool post, Marko! Added a comment and a book recommendation to your blog. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Marko Rodriguez Sent: Thursday, July 14, 2011 4:32 PM To: gremlin-us...@googlegroups.com Cc: Neo4j user discussions; orient-database Subject: [Neo4j] [Blog Post] Graphs, Brains, and Gremlin Howdy ho, If a graph database was simply another way to evaluate the same style of queries/computations as relational/document/etc. database, then my mind would find solace elsewhere. Fortunately, graph databases offer a unique vantage point on data. Data can be slithered through like water rushing through streams and as such, has various correlations to natural information processing systems such as the brain. To articulate this perspective, I dropped a blog post. http://markorodriguez.com/2011/07/14/graphs-brains-and-gremlin/ Boo yea, Marko. http://markorodriguez.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Getting sorted results from a traversal
The DB would do it in memory too, wouldn't it? In the case of a complex
traversal, indexes don't really apply, since the ordering and the traversal
order are unrelated, so you'd generally need to sort in memory anyway. Whether
you do it as you add elements to the traversed list of stuff or do it after
the fact is another discussion, but I think in either case, it needs to be done
after the fact.
-Original Message-
From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On
Behalf Of Pere Urbon Bayes
Sent: Friday, July 15, 2011 11:05 AM
To: Neo4j user discussions
Subject: Re: [Neo4j] Getting sorted results from a traversal
Well, this is great if I want to do all the math in memory, but I expect to
do the computation by the db.
/ purbon
On 15 July 2011 16:10, Marko Rodriguez okramma...@gmail.com wrote:
Hi Pere,
To sort you need to have all your results.
Thus, in Gremlin (and hopefully you can do the mapping to the core Neo4j
traverser framework),
results = []
g.v(1).out('friend').out('likes') results // what my friends like
results.sort{a,b - a.name = b.name} // sort resultant vertices by name
In short, once you have the result of your traversal, you can then apply a
comparator to the Collection to sort it as you please --- its just Java
comparators.
See ya,
Marko.
http://markorodriguez.com
On Jul 15, 2011, at 8:06 AM, Pere Urbon Bayes wrote:
HI!
I am on the situation of having to traverse neo4j, and then expect the
resultset returned to be ordered in a certain order. I've been
researching a
bit over the traversal API, but I did not find anything related to that.
I
really will appreciate any tip on that!!
BTW I expect to be possible right?, as we have in relational the
ordering,
or on redis, etc...
/purbon
--
Pere Urbon-Bayes
moviepilot GmbH | Mehringdamm 33 | 10961 Berlin | Germany
Telefon +49 30 616 512 -110 | Fax +49 30 616 512 -133
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Pere Urbon-Bayes
moviepilot GmbH | Mehringdamm 33 | 10961 Berlin | Germany
Telefon +49 30 616 512 -110 | Fax +49 30 616 512 -133
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Getting sorted results from a traversal
I would think that the graph structure definitely matters, in that there may be
optimizations that can be achieved via indexing/querying vs traversal and
sorting (or a hybrid of the two) depending on the specifics.
-Original Message-
From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On
Behalf Of Pere Urbon Bayes
Sent: Friday, July 15, 2011 11:14 AM
To: Neo4j user discussions
Subject: Re: [Neo4j] Getting sorted results from a traversal
Well the graph structure is not relevant here, I think. The property I
expect to be sorting is on the destination node, so I can do the traversal
and then expect to run the sorting.
Please, tell me how the data structure can help me to deal with that order,
please?
/ purbon
On 15 July 2011 17:10, David Montag david.mon...@neotechnology.com wrote:
Hi Pere,
Can you elaborate on your graph structure?
Thanks,
David
On Fri, Jul 15, 2011 at 8:04 AM, Pere Urbon Bayes p...@moviepilot.com
wrote:
Well, this is great if I want to do all the math in memory, but I expect
to
do the computation by the db.
/ purbon
On 15 July 2011 16:10, Marko Rodriguez okramma...@gmail.com wrote:
Hi Pere,
To sort you need to have all your results.
Thus, in Gremlin (and hopefully you can do the mapping to the core
Neo4j
traverser framework),
results = []
g.v(1).out('friend').out('likes') results // what my friends like
results.sort{a,b - a.name = b.name} // sort resultant vertices by
name
In short, once you have the result of your traversal, you can then
apply
a
comparator to the Collection to sort it as you please --- its just Java
comparators.
See ya,
Marko.
http://markorodriguez.com
On Jul 15, 2011, at 8:06 AM, Pere Urbon Bayes wrote:
HI!
I am on the situation of having to traverse neo4j, and then expect
the
resultset returned to be ordered in a certain order. I've been
researching a
bit over the traversal API, but I did not find anything related to
that.
I
really will appreciate any tip on that!!
BTW I expect to be possible right?, as we have in relational the
ordering,
or on redis, etc...
/purbon
--
Pere Urbon-Bayes
moviepilot GmbH | Mehringdamm 33 | 10961 Berlin | Germany
Telefon +49 30 616 512 -110 | Fax +49 30 616 512 -133
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Pere Urbon-Bayes
moviepilot GmbH | Mehringdamm 33 | 10961 Berlin | Germany
Telefon +49 30 616 512 -110 | Fax +49 30 616 512 -133
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
David Montag david.mon...@neotechnology.com
Neo Technology, www.neotechnology.com
Cell: 650.556.4411
Skype: ddmontag
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
--
Pere Urbon-Bayes
moviepilot GmbH | Mehringdamm 33 | 10961 Berlin | Germany
Telefon +49 30 616 512 -110 | Fax +49 30 616 512 -133
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Getting sorted results from a traversal
A few questions/thoughts: - Do you have any expectation/prediction of the number of nodes satisfying the query? - Do you care about *all* results or only the top n (I ask this because in most cases, a dataset of a zillion ordered records is of little value - often only the edges have meaningful content) If the answer to the first question is a couple million or so, I would probably sort in memory. If the answer to the second question is yes, then the problem because very easy, in that you can maintain a sorted list of the top n during the traversal process fairly easily. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Pere Urbon Bayes Sent: Friday, July 15, 2011 11:17 AM To: Neo4j user discussions Subject: Re: [Neo4j] Getting sorted results from a traversal Well in fact, I expect to order the resulting set of nodes by a property on that end nodes. But This class cold also help on some use cases I think. / purbon On 15 July 2011 16:43, Niels Hoogeveen pd_aficion...@hotmail.com wrote: Hi,You basically have two options. If your result set is not too big, you can sort it in memory, though that approach may consume too much memory if the result set is very large. In that case you can use SortedTree (see: https://github.com/peterneubauer/graph-collections/tree/master/src/main/java/org/neo4j/collections/sortedtree). This allows you to layout the relationships you want to traverse in sorted order. Niels Date: Fri, 15 Jul 2011 16:06:44 +0200 From: p...@moviepilot.com To: user@lists.neo4j.org Subject: [Neo4j] Getting sorted results from a traversal HI! I am on the situation of having to traverse neo4j, and then expect the resultset returned to be ordered in a certain order. I've been researching a bit over the traversal API, but I did not find anything related to that. I really will appreciate any tip on that!! BTW I expect to be possible right?, as we have in relational the ordering, or on redis, etc... /purbon -- Pere Urbon-Bayes moviepilot GmbH | Mehringdamm 33 | 10961 Berlin | Germany Telefon +49 30 616 512 -110 | Fax +49 30 616 512 -133 ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Pere Urbon-Bayes moviepilot GmbH | Mehringdamm 33 | 10961 Berlin | Germany Telefon +49 30 616 512 -110 | Fax +49 30 616 512 -133 ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] [Blog Post] Graphs, Brains, and Gremlin
Nice. It's a very though provoking (pun clearly intended) blog post. I need to read it a couple times to fully grok it. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Marko Rodriguez Sent: Friday, July 15, 2011 11:11 AM To: Neo4j user discussions Subject: Re: [Neo4j] [Blog Post] Graphs, Brains, and Gremlin Hi Rick, Added a comment and a book recommendation to your blog. Thanks. On Intelligence is the first reference in the references section of the post --- along with some other goodies. Thanks for reading and commenting, Marko. http://markorodriguez.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Best practice to avoid Unable to lock store error?
Why would you do that? Keep the database open as a singleton instead. - Reply message - From: noppanit noppani...@gmail.com Date: Fri, Jul 15, 2011 7:00 pm Subject: [Neo4j] Best practice to avoid Unable to lock store error? To: user@lists.neo4j.org user@lists.neo4j.org Such as. Unable to lock store [/Users/xxx/db/neostore], this is usually caused by another Neo4j kernel already running in this JVM for this particular store I'm getting this error when the the user tries to insert into the database really fast (e.g, multiple clicks). The design that I use is to open and shutdown after each commit. So, sometime may be the database is not finish and the user clicks again. I think one way would be to disable the button after one click, but is there a better way to deal with that? -- View this message in context: http://neo4j-user-list.438527.n3.nabble.com/Best-practice-to-avoid-Unable-to-lock-store-error-tp3173592p3173592.html Sent from the Neo4J User List mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neo4j with Tomcat...need a transaction manager
It's built in for neo4j. Full jta support! - Reply message - From: etc3 e...@nextideapartners.com Date: Fri, Jul 15, 2011 7:38 pm Subject: [Neo4j] Neo4j with Tomcat...need a transaction manager To: user@lists.neo4j.org user@lists.neo4j.org We plan on using neo4j as the database for our website, which will run on Tomcat. Also, we'll be using ActiveMQ for sending messages within neo4j transactions, so that leaves the question of what to use for the transaction manager. Atomikos, JBossTM comes to mind, are we missing anything obvious? Does neo4j provide anything out-of-box for transactions management? Thanks Jaf ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] how setup lucene fulltext search filter
Couldn't you simply use QueryFilter/FilteredQuery and pass it to the query method as a native query object? -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Mattias Persson Sent: Wednesday, July 13, 2011 9:05 AM To: Neo4j user discussions Cc: h...@neo4j.org Subject: Re: [Neo4j] how setup lucene fulltext search filter 2011/7/13 Balazs E. Pataki pat...@dsd.sztaki.hu Passing a Filter would be great, because in many cases I can decide based on Lucene Documents (indexed fields), which objects I need as Nodes (and selecting/filtering/sorting Documents is much faster than first turning each Document to a Node and then sorting/filtering the nodes) But as Edgar pointed out, it isn't seemed to be supported. It should be easy to add: it could be a simple configuration option like the analyzer. Are there plans to add something like this? (Asking the neotechnology wizzards. :-) ) That could very well be added, there just hasn't been a call for it yet... until now. I'm redirecting this thread into our support discussion forum. --- balazs On 7/12/11 3:42 PM, 刘运杰 wrote: I like to filter fulltext search result,but don't kown where to set lucene search filter. I notice file LuceneIndex.java , line 355, Hits hits = new Hits( searcher, query, null, sorting, forceScore ); line 370, Hits hits = new Hits( additionsSearcher, query, null ); all pass null for filter. So Neo4j disable lucene search filter? how setup lucene fulltext search filter? Thanks in advance. Edgar, ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Constraints on index key names?
Yes, there is. Just escape it with the \ character. See: http://ifdefined.com/btnet/lucene_syntax.html#+ -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Aseem Kishore Sent: Wednesday, July 13, 2011 5:02 PM To: Neo4j user discussions Subject: Re: [Neo4j] Constraints on index key names? Thanks all! Re: lucene syntax - is there no way to escape the plus in queries? Aseem On Wed, Jul 13, 2011 at 4:56 PM, Mattias Persson matt...@neotechnology.comwrote: Jacob, you're talking about index names... I think the initial question was regarding keys in key/value pairs, for which neo4j adds no restrictions. 2011/7/13 Jacob Hansson ja...@voltvoodoo.com I'm not sure if there is anything keeping you from using them, but the names 'node_auto_index' and 'relationship_auto_index' are used by the auto indexing subsystem. On Wed, Jul 13, 2011 at 8:35 AM, Mattias Persson matt...@neotechnology.comwrote: Yes, the neo4j integration doesn't impose any additional restrictions other than those (if any) of Lucene. 2011/7/12 Rick Bullotta rick.bullo...@thingworx.com That would conflict with lucene's query syntax, I'd think. From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Aseem Kishore [aseem.kish...@gmail.com] Sent: Monday, July 11, 2011 7:41 PM To: Neo4j user discussions Subject: [Neo4j] Constraints on index key names? Is there any constraint on what an index key is allowed to be named? E.g. can it have a '+' in its name? Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Jacob Hansson Phone: +46 (0) 763503395 Twitter: @jakewins ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] What do people do for complex properties?
We do both. ;-)
We use qualified names, just as you do, when the complex data is a single
row. We use JSON when it is tabular data or a deeper data structure.
-Original Message-
From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On
Behalf Of Aseem Kishore
Sent: Wednesday, July 13, 2011 5:51 PM
To: Neo4j user discussions
Subject: [Neo4j] What do people do for complex properties?
Node/relationship properties are simple key/value pairs, where the values
are simple primitive values (exception: array of a single type of primitive
value), so I was wondering what people do to store more complex properties.
One solution is of course to store a JSON string. We'd like to avoid that
monolithic route though as it's a hack that'll make maintenance difficult.
E.g. we can't just go into the web admin and tweak a property as needed.
We currently have a property schema that looks something like this, for
one of our nodes:
title:
description:
image:
url:
width: #
height: #
We've mapped that to this schema in Neo4j:
title:
description:
image.url:
image.width: #
image.height: #
You can see we've flattened the object to derive the flat list of keys and
values. This works okay, but it makes discovery of image properties
difficult. E.g. with a regular JS object, you could just do this to see if
this node had an image:
if (node.image) {
...
}
But the equivalent in Neo4j won't work (pseudocode):
if node has property image
...
So you have to check for specific subproperties. Not a huge annoyance for
simple structures, but for deep structures that could get pretty silly, so
you would then have to iterate over all properties and see if any begin with
image. for example.
Anyway, just wanted to see if any of you guys have come up with any better /
more creative solutions for this problem. =D Would love to hear if so!
Thanks.
Cheers,
Aseem
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Constraints on index key names?
That would conflict with lucene's query syntax, I'd think. From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of Aseem Kishore [aseem.kish...@gmail.com] Sent: Monday, July 11, 2011 7:41 PM To: Neo4j user discussions Subject: [Neo4j] Constraints on index key names? Is there any constraint on what an index key is allowed to be named? E.g. can it have a '+' in its name? Aseem ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Index got deleted when deploy to production
Luke versions are somewhat specific to lucene versions. Which version of Luke and which version of neo are you using??? - Reply message - From: noppanit noppani...@gmail.com Date: Sun, Jul 10, 2011 6:40 pm Subject: [Neo4j] Index got deleted when deploy to production To: user@lists.neo4j.org user@lists.neo4j.org Hi! sorry if I keep bothering you. I point to db/index/lucene/node/myindex now. I got this error. Unknown format version: -11 Thanks :) -- View this message in context: http://neo4j-user-list.438527.n3.nabble.com/Index-got-deleted-when-deploy-to-production-tp3157195p3157622.html Sent from the Neo4J User List mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] web application and neo4j hosting
...if you want durable storage, via EBS, yes. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of noppanit Sent: Saturday, July 09, 2011 10:37 AM To: user@lists.neo4j.org Subject: [Neo4j] web application and neo4j hosting Hi, I'm writing an web application with neo4j, but I'm not sure if any hosting support read/write local files? Does amazon ec2 support that? Thanks. -- View this message in context: http://neo4j-user-list.438527.n3.nabble.com/web-application-and-neo4j-hosting-tp3154806p3154806.html Sent from the Neo4J User List mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] web application and neo4j hosting
Simplest difference is that EBS volumes are like storage devices that can be mounted to a running instance and provide file system functionality. S3 is more a general purpose durable storage engine, but doesn't directly allow mounting it is a file system. There are some hacks/3rd party ways to do it, but I'd avoid them for Neo4J usage. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of noppanit Sent: Saturday, July 09, 2011 10:51 AM To: user@lists.neo4j.org Subject: Re: [Neo4j] web application and neo4j hosting Would you mind if I ask this question, what is the difference between S3 and EBS? Thank you. -- View this message in context: http://neo4j-user-list.438527.n3.nabble.com/web-application-and-neo4j-hosting-tp3154806p3154841.html Sent from the Neo4J User List mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] web application and neo4j hosting
Oops - meant to say S3 doesn't directly allow mounting it *as* a file system. Also, S3 can be used as a backup/snapshot mechanism for EBS volumes. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Rick Bullotta Sent: Saturday, July 09, 2011 10:59 AM To: Neo4j user discussions Subject: Re: [Neo4j] web application and neo4j hosting Simplest difference is that EBS volumes are like storage devices that can be mounted to a running instance and provide file system functionality. S3 is more a general purpose durable storage engine, but doesn't directly allow mounting it is a file system. There are some hacks/3rd party ways to do it, but I'd avoid them for Neo4J usage. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of noppanit Sent: Saturday, July 09, 2011 10:51 AM To: user@lists.neo4j.org Subject: Re: [Neo4j] web application and neo4j hosting Would you mind if I ask this question, what is the difference between S3 and EBS? Thank you. -- View this message in context: http://neo4j-user-list.438527.n3.nabble.com/web-application-and-neo4j-hosting-tp3154806p3154841.html Sent from the Neo4J User List mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] web application and neo4j hosting
Can't use S3. Can use EBS. And no, Neo4J doesn't require a shared disk, and in fact, would probably gack if you used one. Each HA instance would have its own synchronized/eventually consistent storage, which could be backed by EBS if needed. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of etc3 Sent: Saturday, July 09, 2011 11:22 AM To: 'Neo4j user discussions' Subject: Re: [Neo4j] web application and neo4j hosting Does neo4j require a shared-disk across all nodes in a cluster? Is that the problem with S3? -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Rick Bullotta Sent: Saturday, July 09, 2011 11:03 AM To: Neo4j user discussions Subject: Re: [Neo4j] web application and neo4j hosting Oops - meant to say S3 doesn't directly allow mounting it *as* a file system. Also, S3 can be used as a backup/snapshot mechanism for EBS volumes. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Rick Bullotta Sent: Saturday, July 09, 2011 10:59 AM To: Neo4j user discussions Subject: Re: [Neo4j] web application and neo4j hosting Simplest difference is that EBS volumes are like storage devices that can be mounted to a running instance and provide file system functionality. S3 is more a general purpose durable storage engine, but doesn't directly allow mounting it is a file system. There are some hacks/3rd party ways to do it, but I'd avoid them for Neo4J usage. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of noppanit Sent: Saturday, July 09, 2011 10:51 AM To: user@lists.neo4j.org Subject: Re: [Neo4j] web application and neo4j hosting Would you mind if I ask this question, what is the difference between S3 and EBS? Thank you. -- View this message in context: http://neo4j-user-list.438527.n3.nabble.com/web-application-and-neo4j-hostin g-tp3154806p3154841.html Sent from the Neo4J User List mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] web application and neo4j hosting
We do that today. It works fine. Here's a simple version of what we do: - We created an AMI that contains our app server + app (which embeds neo) - For each instance that we provision on EC2 using that AMI, we create an EBS volume, and mount it to that instance - We use a consistent mounting/naming scheme so that we can point the neo database to use that path Rick -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of noppanit Sent: Saturday, July 09, 2011 11:16 AM To: user@lists.neo4j.org Subject: Re: [Neo4j] web application and neo4j hosting Thank you very much, however, anyone has experience deploying neo4j app to EC2 and EBS? May be someone could share some thoughts. -- View this message in context: http://neo4j-user-list.438527.n3.nabble.com/web-application-and-neo4j-hosting-tp3154806p3154887.html Sent from the Neo4J User List mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Add relationships dynamically
Take a look at the RelationshipType interface. If you implement that (which is really simple - just a name() property), you can have your own class that can have relationships with any names you want. They do need to be unique, however. From: user-boun...@lists.neo4j.org [user-boun...@lists.neo4j.org] On Behalf Of noppanit [noppani...@gmail.com] Sent: Thursday, July 07, 2011 4:01 PM To: user@lists.neo4j.org Subject: [Neo4j] Add relationships dynamically Is there anyway I can add relationships on-the-fly or programmatically? Because sometime I might not know the relationships and I want to add that to the database. Cheers, -- View this message in context: http://neo4j-user-list.438527.n3.nabble.com/Add-relationships-dynamically-tp3149437p3149437.html Sent from the Neo4J User List mailing list archive at Nabble.com. ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Performance issue on nodes with lots of relationships
Hi, Michael. Are you thinking maybe of lazily loading relationships in 1.5? That might be a huge boost. Rick -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Michael Hunger Sent: Wednesday, July 06, 2011 10:32 AM To: Neo4j user discussions Subject: Re: [Neo4j] Performance issue on nodes with lots of relationships Andrew, could you please also try to access the graph via the latest Milestone 1.4.M06 to see if things have improved. Does this behaviour only effect the supernodes or every node in your graph (e.g. when you access, cd, ls a person-node?) We've been discussing some changes to the initial loading/caching that might improve performance on heavily connected (super-)nodes. If our changes and tests are successful these change will be integrated in early 1.5. Milestones. Cheers Michael Am 06.07.2011 um 16:15 schrieb Andrew White: I have a graph with roughly 10M nodes. Some of these nodes are highly connected to other nodes. For example I may have a single node with 1M+ relationships. A good analogy is a population that has a lives-in relationship to a state. Now the problem... Both neoclipse or neo4j-shell are terribly slow when working with these nodes. In the shell I would expect a `cd node-id` to be very fast, much like selecting via a rowid in a standard DB. Instead, I usually see several seconds delay. Doing a `ls` takes so long that I usually have to just kill the process. In fact `ls` never outputs anything which is odd since I would expect it to stream the output as it found it. I have very similar performance issues with neoclipse. I am using Neo4j 1.3 embedded on Ubuntu 10.04 with 4GB of RAM. Disclaimer, I am new to Neo4j. Thanks, Andrew ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] reify links with other neo4j databases located on different distributed servers
Actually the opposite. We use nodes in both databases, of course, but we use node references via properties in the *data* graph to point back to entities in the *model* graph. - Reply message - From: Aliabbas Petiwala aliabba...@gmail.com Date: Tue, Jul 5, 2011 1:10 am Subject: [Neo4j] reify links with other neo4j databases located on different distributed servers To: Neo4j user discussions user@lists.neo4j.org Thanks for sharing with us the details, Your solutions sounds innovative and interesting but You have stored the relationships only in the model graph. does it mean you have a fixed schema and how to store new relationships between different nodes. Its really difficult to understand how can you store relationships only once in a model graph and only nodes in the data graph? Regards, Ali On 7/3/11, Rick Bullotta rick.bullo...@thingworx.com wrote: Our approach is very application-specific, but it can be summarized by: - We keep our model database on one server and our run time data (somewhat like activity streams) on another server - A long value (node id) source property on data nodes that identify a model node in the other graph - Long value (node id) server property on data nodes that identify a node in the same graph, which contains logical server information stored as properties (logical name + domain name/IP address + port + protocol) - Lucene indices on the data nodes that index the data by tag(s), source, and time - Relationships in the model graph that describe inter-entity model relationships (inheritance, reference), dependencies and usage references, etc. - Lucene indices on the model nodes that index the model entities by type, tag(s) - Lucene indices on the tagging vocabularies on both the model and data graph(s) We avoided using relationships in the data graph due to the fact that we are constantly adding and deleting potentially thousands of items per second, and this could create concurrency and performance issues when there are potentially millions of relationships on a node We didn't originally design it this way. The original approach was a single (embedded) database, using relationships for all node-node connections. We're in the process of moving to our new design in phases, the first of which was a logical separation of model + data, though in the same graph, and switching from relationships to the node id property approach for some specific scenarios. I have to think there are substantial performance implications *if* you are trying to do complex cross-shard or cross-graph traversals, which we generally do not need to do. Rather, we can deal with this at the application layer. -Original Message- From: Aliabbas Petiwala [mailto:aliabba...@gmail.com] Sent: Sunday, July 03, 2011 2:54 AM To: Neo4j user discussions Subject: Re: [Neo4j] reify links with other neo4j databases located on different distributed servers Thanks a lot Rick can you please provide more details on issues which you faced while using this approach and share some code with us . Had you decided about this at design time itself and designed your graph db schema accordingly? Is there much perceived performance penalties if there are a large number of such references spanning physical boundaries? On 7/2/11, Rick Bullotta rick.bullo...@thingworx.com wrote: We are using node-id property references (the node id as a property), qualified with a logical server reference, to provide this type of binding across graphs. If you combine these with an index, you can actually get a lot of the functionality of relationships cross graph, spanning physical boundaries. Of course, as Craig points out, this all has to be done at the application level, including dealing with cascading deletes when a node is removed from one graph, ensuring that references to it in another graph are removed/redirected. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Craig Taverner Sent: Saturday, July 02, 2011 6:03 AM To: Neo4j user discussions Subject: Re: [Neo4j] reify links with other neo4j databases located on different distributed servers As far as I know there is no internal support for transparent traversals across shards. Generally people are doing that in the application layer. However, I think there might be a middle ground of sorts. I we modify the relationship expander, I could imagine that relationships that are between shards could be modified to return node on the other shard. This would make the traversal return nodes across shards, but since I've not tried this myself, I am uncertain if there are other consequences. On Sat, Jul 2, 2011 at 4:03 AM, Aliabbas Petiwala aliabba...@gmail.comwrote: Hi, I cannot figure out how my application logic can reify links with other neo4j databases located on different distributed servers? hence , how can i make
Re: [Neo4j] polyglot persistence for neo4j
Another blog entry on the topic: http://blog.stavi.sh/polyglot-persistence-integrating-low-latency -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Marko Rodriguez Sent: Tuesday, July 05, 2011 10:07 AM To: Neo4j user discussions Subject: Re: [Neo4j] polyglot persistence for neo4j Hi, I have a few projects that mix and match Neo4j + (Oracle/MySQL). Basically, its these two architectures (not necessarily for recommendation in all projects): http://markorodriguez.com/services/development/recommendation-system/ (see the last two diagrams). Good luck, Marko. http://markorodriguez.com On Jul 5, 2011, at 3:35 AM, Aliabbas Petiwala wrote: Thanks for the info, It would help us a lot in using neo4j for our social network website if we can talk with the researchers involved in testing neo4j with cassandra as backend. Hence please let us know of them. Greetings for neo4j team, aliabbas On 7/5/11, Peter Neubauer neubauer.pe...@gmail.com wrote: Hi there, There have been experiments with Cassandra as a backend, but I don't think they are running in production. What are you thinking of? /peter Sent from my phone. On Jul 5, 2011 7:02 AM, Aliabbas Petiwala aliabba...@gmail.com wrote: i guess no body has used polyglot persistence with neo4j in production environment? On 7/1/11, Aliabbas Petiwala aliabba...@gmail.com wrote: i would be interested in knowing about anyone who has combined neo4j graph db with other dbs like Cassandra ,orient db what is an optimal configuration to do so? any other configurations for polyglot persistence to achieve high scalability and massive parallelism? -- Aliabbas Petiwala M.Tech CSE -- Aliabbas Petiwala M.Tech CSE ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Aliabbas Petiwala M.Tech CSE ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Neoclipse broken?
Seems like all versions of Neoclipse now display *all* properties on nodes, regardless of what you configure for property names in the graph decorations configuration dialog. This pretty much renders Neoclipse unusable. :( ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Neoclipse broken?
Ah, I found a new configuration option hidden in a secret tiny View Menu called expanded node mode, which is on by default...turned that off and voila, back to the wondrous purity of the old Neoclipse. ;-) -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Rick Bullotta Sent: Tuesday, July 05, 2011 8:54 PM To: Neo4j user discussions Subject: [Neo4j] Neoclipse broken? Importance: High Seems like all versions of Neoclipse now display *all* properties on nodes, regardless of what you configure for property names in the graph decorations configuration dialog. This pretty much renders Neoclipse unusable. :( ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] reify links with other neo4j databases located on different distributed servers
Our approach is very application-specific, but it can be summarized by: - We keep our model database on one server and our run time data (somewhat like activity streams) on another server - A long value (node id) source property on data nodes that identify a model node in the other graph - Long value (node id) server property on data nodes that identify a node in the same graph, which contains logical server information stored as properties (logical name + domain name/IP address + port + protocol) - Lucene indices on the data nodes that index the data by tag(s), source, and time - Relationships in the model graph that describe inter-entity model relationships (inheritance, reference), dependencies and usage references, etc. - Lucene indices on the model nodes that index the model entities by type, tag(s) - Lucene indices on the tagging vocabularies on both the model and data graph(s) We avoided using relationships in the data graph due to the fact that we are constantly adding and deleting potentially thousands of items per second, and this could create concurrency and performance issues when there are potentially millions of relationships on a node We didn't originally design it this way. The original approach was a single (embedded) database, using relationships for all node-node connections. We're in the process of moving to our new design in phases, the first of which was a logical separation of model + data, though in the same graph, and switching from relationships to the node id property approach for some specific scenarios. I have to think there are substantial performance implications *if* you are trying to do complex cross-shard or cross-graph traversals, which we generally do not need to do. Rather, we can deal with this at the application layer. -Original Message- From: Aliabbas Petiwala [mailto:aliabba...@gmail.com] Sent: Sunday, July 03, 2011 2:54 AM To: Neo4j user discussions Subject: Re: [Neo4j] reify links with other neo4j databases located on different distributed servers Thanks a lot Rick can you please provide more details on issues which you faced while using this approach and share some code with us . Had you decided about this at design time itself and designed your graph db schema accordingly? Is there much perceived performance penalties if there are a large number of such references spanning physical boundaries? On 7/2/11, Rick Bullotta rick.bullo...@thingworx.com wrote: We are using node-id property references (the node id as a property), qualified with a logical server reference, to provide this type of binding across graphs. If you combine these with an index, you can actually get a lot of the functionality of relationships cross graph, spanning physical boundaries. Of course, as Craig points out, this all has to be done at the application level, including dealing with cascading deletes when a node is removed from one graph, ensuring that references to it in another graph are removed/redirected. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Craig Taverner Sent: Saturday, July 02, 2011 6:03 AM To: Neo4j user discussions Subject: Re: [Neo4j] reify links with other neo4j databases located on different distributed servers As far as I know there is no internal support for transparent traversals across shards. Generally people are doing that in the application layer. However, I think there might be a middle ground of sorts. I we modify the relationship expander, I could imagine that relationships that are between shards could be modified to return node on the other shard. This would make the traversal return nodes across shards, but since I've not tried this myself, I am uncertain if there are other consequences. On Sat, Jul 2, 2011 at 4:03 AM, Aliabbas Petiwala aliabba...@gmail.comwrote: Hi, I cannot figure out how my application logic can reify links with other neo4j databases located on different distributed servers? hence , how can i make the traversals and graph algorithms transparent to the location of the different databases ? -- Aliabbas Petiwala M.Tech CSE ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user -- Aliabbas Petiwala M.Tech CSE ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] reify links with other neo4j databases located on different distributed servers
We are using node-id property references (the node id as a property), qualified with a logical server reference, to provide this type of binding across graphs. If you combine these with an index, you can actually get a lot of the functionality of relationships cross graph, spanning physical boundaries. Of course, as Craig points out, this all has to be done at the application level, including dealing with cascading deletes when a node is removed from one graph, ensuring that references to it in another graph are removed/redirected. -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Craig Taverner Sent: Saturday, July 02, 2011 6:03 AM To: Neo4j user discussions Subject: Re: [Neo4j] reify links with other neo4j databases located on different distributed servers As far as I know there is no internal support for transparent traversals across shards. Generally people are doing that in the application layer. However, I think there might be a middle ground of sorts. I we modify the relationship expander, I could imagine that relationships that are between shards could be modified to return node on the other shard. This would make the traversal return nodes across shards, but since I've not tried this myself, I am uncertain if there are other consequences. On Sat, Jul 2, 2011 at 4:03 AM, Aliabbas Petiwala aliabba...@gmail.comwrote: Hi, I cannot figure out how my application logic can reify links with other neo4j databases located on different distributed servers? hence , how can i make the traversals and graph algorithms transparent to the location of the different databases ? -- Aliabbas Petiwala M.Tech CSE ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] neo4j-graph-collections
That's basically what we do - wrappers around our Location, InfoTable (2D data table), DateTime (JODA), JSON, and XML entities to perform those types of comparator actions for filtering and sorting. We also expose methods in the wrappers (which also include Double, Boolean, String, etc.) to deal with persistence/de-persistence in a variety of formats, including Neo, JSON, XML, and simple toString()/fromString(). -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Niels Hoogeveen Sent: Wednesday, June 29, 2011 8:25 AM To: user@lists.neo4j.org Subject: Re: [Neo4j] neo4j-graph-collections At this moment Btree only supports the primitive datatype long, while Rtree only supports the datatype double. For Btree it makes sense to at least support strings, floats, doubles and ints too. Use cases for these data types are pretty obvious and are Btree backed in (almost) every RDBMS product around.I think the best solution would be to create Comparator objects wrapping these primitive data types and store the class name of the comparator in root of the index tree. This allows users to create their own comparators for datatypes not covered yet. It would make sense people would want to store BigInt and BigDecimal objects in a Btree too, others may want to store dates (instead of datetime), fractions, complex numbers or even more exotic data types. Niels From: sxk1...@hotmail.com To: user@lists.neo4j.org Date: Tue, 28 Jun 2011 22:43:24 -0700 Subject: Re: [Neo4j] neo4j-graph-collections I've read through this thread in more detail and have a few thoughts, when you talk about type I am assuming that you are referring to an interface that both (Btree,Rtree) can implement, for the data types I'd like to understand the use cases first before implementing the different data types, maybe we could store types of Object instead of Long or Double and implement comparators in a more meaningful fashion. Also I was wondering if unit tests would need to be extracted out of the spatial component and embedded inside the graph-collections component as well or whether we'd potentially need to write brand new unit tests as well. Craig as I mentioned I'd love to help, let me know if it would be possible to fork a repo or to talk in more detail this week. Regards From: pd_aficion...@hotmail.com To: user@lists.neo4j.org Date: Wed, 29 Jun 2011 01:35:43 +0200 Subject: Re: [Neo4j] neo4j-graph-collections As to the issue of n-dim doubles, it would be interesting to consider creating a set of classes of type Orderable (supporting , =, , = operations), this we can use in both Rtree and Btree. Right now Btree only supports datatype Long. This should also become more generic. A first step we can take is at least wrap the common datatypes in Orderable classes. Niels Date: Wed, 29 Jun 2011 00:32:15 +0200 From: cr...@amanzi.com To: user@lists.neo4j.org Subject: Re: [Neo4j] neo4j-graph-collections The RTree in principle should be generalizable, but the current implementation in neo4j-spatial does make a few assumptions specific to spatial data, and makes use of spatial envelopes for the tree node bounding boxes. It is also specific to 2D. We could make a few improvements first, like generalizing to n-dimensions, replacing the recursive search with a traverser and generalizing the bounding boxes to be simple double-arrays. Then the only thing left would be to decide if it is ok for it to be based on n-dim doubles or should be generalized to more types. On Tue, Jun 28, 2011 at 11:14 PM, Saikat Kanjilal sxk1...@hotmail.comwrote: I would be interested in helping out with this, let me know next steps. Sent from my iPhone On Jun 28, 2011, at 8:49 AM, Niels Hoogeveen pd_aficion...@hotmail.com wrote: A couple of weeks ago Peter Neubauer set up a repository for in-graph datastructures: https://github.com/peterneubauer/graph-collections. At this time of writing only the Btree/Timeline index is part of this component. In my opinion it would be interesting to move the Rtree parts of neo-spatial to neo4j-graph-collections too. I looked at the code but don't feel competent to seperate out those classes that support generic Rtrees from those classes that are clearly spatial related. Is there any enthusiasm for such a project and if so, who is willing and able to do this? Niels ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list
Re: [Neo4j] Unexpected error
Wow. Neo4J is optimized for keys to find certain nodes or relationships which
normally are not more than a dozen. That's quite a surprise to me, and I hope
it not to be the case!
-Original Message-
From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On
Behalf Of Michael Hunger
Sent: Tuesday, June 28, 2011 7:58 AM
To: Neo4j user discussions
Subject: Re: [Neo4j] Unexpected error
Hi Massimo,
still the lucene document holds an ArrayList with all fields which grows
immensely as you add millions of fields.
package org.apache.lucene.document;
public final class Document implements java.io.Serializable {
ListFieldable fields = new ArrayListFieldable();
}
So perhaps the difference is that in Neo4j we have one Document per
node/relationship, where all keys + values are added.
Which builds up those large documents.
It is unusual for Neo4j to have a single document that contains that many
entries as you normally just index the keys to find certain nodes or
relationships
which normally are not more than a dozen. And that's why the neo4j lucene index
implementation is and will be optimized for exactly this case.
So I'd suggest as Mattias said you create your own datasource and handle the
adding there (obviously just adding node-ids + hashes to an arbitrary lucene
index.
(with one doc per node-id (or hash).
Cheers
Michael
Am 28.06.2011 um 10:21 schrieb Massimo Lusetti:
On Mon, Jun 27, 2011 at 11:56 PM, Michael Hunger
michael.hun...@neotechnology.com wrote:
Massimo,
could you please look into the Lucene Document instance that you add all the
fields to?
You're right... I add only the NodeId and my own hash... Which fields
do you add?
If it also contains this ultralarge ArrayList with all the Fields ?
As I said It only contains the Fields relative to the NodeId and the Hash
And which version of lucene did you use for your standalone testing?
I use 3.1.0.
BTW I'm testing resin and even couchdb for the index part but I don't
like it that much cause that index phase will be out of the node4j
Transaction and I want to isolate all that operations in a Tranaction.
Have you hint for me on how to extend the node4j Transaction... is
that extensible?
Thanks
--
Massimo
http://meridio.blogspot.com
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
