> I've worked a lot recently on the Hibernate Lucene integration. Here > are the concepts, the new features and the todo list. > Please comment and give feedbacks. > > My work is commited in branches/Lucene_Integration because we'll > probably need to be based on Hibernate 3.3 > > *Concepts* > Each time you change an object state, the lucene index is updated and > kept in sync. This is done through the Hibernate event system. > Whether an entity is indexed or not and whether a property is indexed > or not is defined through annotations. > You can also search through your domain model using Lucene and > retrieve managed objects. The whole idea here is to do a nice > integration between the search engine and the ORM without loosing the > search engine power, hence most of the API remains. To sum up, query > Lucene, get managed object back. > > *Mapping* > A given entity is mapped to an index. A lucene index is stored in a > Directory, a Directory is a Lucnee abstract concept for index storage > system. It can be a memory directory (RAMDirectory), a file system > directory (FSDirectory) or any other kind of backend. Hibernate Lucene > introduce the notion of DirectoryProvider that you can configure and > define on a per entity basis (and wich is defaulted defaulted). The > concept is very similar to ConnectionProvider. > > Lucene only works with Strings, so you can define a @FieldBridge which > transform a java property into a Lucene Field (and potentially > vice-versa). A more simple (useful?) version handle the transformation > of a java property into a String. > Some built-in FieldBrigde exists. @FieldBridge is very much like an > Hibernate Type. Esp I introduced the notion of precision in dates > (year, month, .. second, millisecond). This FieldBridge and > StringBridge gives a lot of flexibility in the way to design the > property indexing. > > > *Querying* > I've introduced the notion of LuceneSession which implements Session > and actually delegates to a regular Hibernate Session. This lucene > session has a /createLuceneQuery()/ method and a /index()/ method. > > /session.createLuceneQuery(lucene.Query, Class[])/ takes a Lucene > query as a parameter and the list of targeted entities. Using a Lucene > query as a parameter gives the full Lucene flexibility (no abstraction > on top of it). An /org.hibernate.Query/ object is returned. > You can (must) use pagination. A Lucene query also return the number > of matching results (regardless of the pagination): query.resultSize() > sort of count(*). > /list()/ returns the list of matching objects. It heavily depends > on batch-size to be efficient (ie the proxy are created for all the > results and then we initialize them. > There might be alternative strategies here (select ... where id in ( , > , , ) ), but the real benefit would come if combined with the dynamic > fetching profile we talked about a while ago. > /iterate()/ has the same semantic as the regular method in > hibernate, meaning initialize the objects one by one. > /scroll()/ allows an efficient navigation into the resultset, > (objects are loaded one by one though). > Having the dynamic fetch profile would definitely be a killer pair > (searching the lucene index, and fetching the appropriate object graph) > > /session.//index(Object)/ is currently not implemented it requires > some modifications of SessionImpl or of LuceneSession. This feature is > useful to initialize / refresh the index in a batch way (ie loading > the data and applying the indexing process on this set of data). > Basically the object is added to the index queue. At flush() time, the > queue is processed. > > design considerations: > The delegation vs subclassing strategy for LuceneSession (ie > LuceneSession delegating to a regular Session allowing simple wrapping > or the LuceneSessionImpl being a subclass of SessionImpl is an ongoing > discussion. > Using a subclassing model would allow the LuceneSession to keep > operation queues (for batch indexing either through object changes or > through session.index() ), but it does not allow a potential Hibernate > - XXX integration on the same subclassing model. Batching is essential > in Lucene for performance reasons. > Using the delegation model requires some SessionImpl modifications to > be able to keep track of a generic context. This context will keep the > operation queues. > > > *ToDo* > Argue on the LuceneSession design are pick up one (Steve/Emmanuel/Feel > free to join the danse) > > Find a way to keep the DocumentBuilder (sort of EntityPersister) at > the SessionFactory level rather than the EventListener level > (Steve/Emmanuel) > > Implement the use of FieldBridge for all properties. It is currently > used for the id property only (trivial). > > Batch changes: to do that I need to be able to keep a session related > queue of all insert/update changes. I can't in the current design > because SessionImpl does not have such concept and because the > LuceneSession is build on the delegation model. We need to discuss the > strategy here (delegation vs subclassing) > > Massive batch changes: in some system, we don't really bother with > "real time" index synchronization, for those a common batch changes > queue (ie across several sessions) would make sense with a queue > flushing happening every hour for example. > > Clustered Directory: think about that. A JDBC Directory might not be > the perfect solution actually. > > fetch profile > > Align the field indexing annotations to Lucene 2.0 > > Think aboud Analyser to give the same flexibility @Boost provide > > Make Lucene query parameterizable query.setParameter(); > > implements additional strategies to load object on query.list() >
------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ hibernate-devel mailing list hibernate-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/hibernate-devel
