Re: [Neo4j] Index Framework - Query Question
Rick, hacking in Sweden on indexing would be a very cool option. Need to coordinate that but let me know how we can do it best! Meanwhile, we are deciding on an issue tracking system better than Trac. Until then, I will add this to our Google Spreadsheet so we don't loose your suggestions here! Cheers, /peter neubauer GTalk: neubauer.peter Skype peter.neubauer Phone +46 704 106975 LinkedIn http://www.linkedin.com/in/neubauer Twitter http://twitter.com/peterneubauer http://www.neo4j.org - Your high performance graph database. http://startupbootcamp.org/- Öresund - Innovation happens HERE. http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party. On Thu, May 12, 2011 at 5:20 PM, Rick Bullotta wrote: > Hi, Mattias. > > I'll definitely give it a try. > > Ultimately, I'd like to achieve the following scenario (and maybe you can > give us some guidance on how to implement it): > > As a use case: we want to index the node associated with a "collaboration > entry" (that has a timestamp, tag(s), location, textual content, and > potentially some numeric values as properties) on a combination of what is > currently many different types of indexes in Neo - fulltext, timeline, > spatial, and "plain" (key/value field(s)).We'd like to search/query on > any combination of those elements. > > We've been considering a few options: > > - use relationships (that wouldn't meet our performance/concurrency > requirements) > - use the current index framework with separate indexes, query multiple > times, and do the set comparison/intersection in our code > - bypass Neo's indexing model altogether and go directly to Lucene, though > we'd lose the transaction capabilities > - hack the Neo code for the various Lucene index implementations and create > our own "composite index" index type that supports fulltext fields, > geospatial fields, simple data types (string, number, boolean), timeline, > and multi-valued fields > > I like option #4, but we'd definitely need some help and input to build it > in a way that wasn't too fragile from version to version of Neo. I think it > is a capability that would be broadly useful for a lot of Neo users, though, > and would fit nicely with some of the work Anders is doing in a Neo query > language/syntax. > > I do think the ability to have more control at the "field" level could lead > to some cool capabilities and significant optimizations both in storage and > retrieval. I also think you've managed to work around the transactioning > issues, so it would be foolish to re-invent the wheel there. The basic idea > would be a composite index that perhaps had a new "add" method which > accepted a Field definition instead of just a simple string field name. The > Lucene index implementation could leverage this to exploit a lot of the > power of Lucene to meet the overall goal of a very flexible indexing/query > subsystem for Neo4J. Additionally, the Field definition for full-text > fields could have an analyzer associated with it, rather than an Analyzer at > the index level only. > > If you would be up for it, maybe we come over to Sweden for a couple days > and hack #4 together, or we could do it "virtually"? > > Thoughts? > > Rick > > -Original Message- > From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] > On Behalf Of Mattias Persson > Sent: Thursday, May 12, 2011 3:55 AM > To: Neo4j user discussions > Subject: Re: [Neo4j] Index Framework - Query Question > > Could you perhaps create an index with a custom Analyzer which treats > titles > as fulltext and others as non-analyzed? > > 2011/5/11 Rick Bullotta > > > Is there currently any way to have a composite index consisting of > fulltext > > and non-analyzed (simple fields), and to query them in a single query > > statement? e.g. : title:Reloaded AND year:1999 > > > > In this case, I'd be using a Lucene Analyzer on the title, but indexing > the > > primitive types without using the analyzer/norms/etc...I know how to do > it > > with Lucene directly, but not clear if it can be done w/Neo4J. > > > > Thanks! > > > > Rick > > > > ___ > > Neo4j mailing list > > User@lists.neo4j.org > > https://lists.neo4j.org/mailman/listinfo/user > > > > > > -- > Mattias Persson, [matt...@neotechnology.com] > Hacker, Neo Technology > www.neotechnology.com > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Index Framework - Query Question
Hi, Mattias. I'll definitely give it a try. Ultimately, I'd like to achieve the following scenario (and maybe you can give us some guidance on how to implement it): As a use case: we want to index the node associated with a "collaboration entry" (that has a timestamp, tag(s), location, textual content, and potentially some numeric values as properties) on a combination of what is currently many different types of indexes in Neo - fulltext, timeline, spatial, and "plain" (key/value field(s)).We'd like to search/query on any combination of those elements. We've been considering a few options: - use relationships (that wouldn't meet our performance/concurrency requirements) - use the current index framework with separate indexes, query multiple times, and do the set comparison/intersection in our code - bypass Neo's indexing model altogether and go directly to Lucene, though we'd lose the transaction capabilities - hack the Neo code for the various Lucene index implementations and create our own "composite index" index type that supports fulltext fields, geospatial fields, simple data types (string, number, boolean), timeline, and multi-valued fields I like option #4, but we'd definitely need some help and input to build it in a way that wasn't too fragile from version to version of Neo. I think it is a capability that would be broadly useful for a lot of Neo users, though, and would fit nicely with some of the work Anders is doing in a Neo query language/syntax. I do think the ability to have more control at the "field" level could lead to some cool capabilities and significant optimizations both in storage and retrieval. I also think you've managed to work around the transactioning issues, so it would be foolish to re-invent the wheel there. The basic idea would be a composite index that perhaps had a new "add" method which accepted a Field definition instead of just a simple string field name. The Lucene index implementation could leverage this to exploit a lot of the power of Lucene to meet the overall goal of a very flexible indexing/query subsystem for Neo4J. Additionally, the Field definition for full-text fields could have an analyzer associated with it, rather than an Analyzer at the index level only. If you would be up for it, maybe we come over to Sweden for a couple days and hack #4 together, or we could do it "virtually"? Thoughts? Rick -Original Message- From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On Behalf Of Mattias Persson Sent: Thursday, May 12, 2011 3:55 AM To: Neo4j user discussions Subject: Re: [Neo4j] Index Framework - Query Question Could you perhaps create an index with a custom Analyzer which treats titles as fulltext and others as non-analyzed? 2011/5/11 Rick Bullotta > Is there currently any way to have a composite index consisting of fulltext > and non-analyzed (simple fields), and to query them in a single query > statement? e.g. : title:Reloaded AND year:1999 > > In this case, I'd be using a Lucene Analyzer on the title, but indexing the > primitive types without using the analyzer/norms/etc...I know how to do it > with Lucene directly, but not clear if it can be done w/Neo4J. > > Thanks! > > Rick > > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] Index Framework - Query Question
Could you perhaps create an index with a custom Analyzer which treats titles as fulltext and others as non-analyzed? 2011/5/11 Rick Bullotta > Is there currently any way to have a composite index consisting of fulltext > and non-analyzed (simple fields), and to query them in a single query > statement? e.g. : title:Reloaded AND year:1999 > > In this case, I'd be using a Lucene Analyzer on the title, but indexing the > primitive types without using the analyzer/norms/etc...I know how to do it > with Lucene directly, but not clear if it can be done w/Neo4J. > > Thanks! > > Rick > > ___ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
[Neo4j] Index Framework - Query Question
Is there currently any way to have a composite index consisting of fulltext and non-analyzed (simple fields), and to query them in a single query statement? e.g. : title:Reloaded AND year:1999 In this case, I'd be using a Lucene Analyzer on the title, but indexing the primitive types without using the analyzer/norms/etc...I know how to do it with Lucene directly, but not clear if it can be done w/Neo4J. Thanks! Rick ___ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
