Oops, forgot to paste the link from which the quote was taken: http://engineering.twitter.com/2010/10/twitters-new-search-architecture.html
Shai On Thu, Oct 7, 2010 at 8:46 AM, Shai Erera <[email protected]> wrote: > I came across this post today: > http://techcrunch.com/2010/10/06/new-twitter-search > > And continued reading these: > http://techcrunch.com/2010/10/06/twitter-search-lives/ > > From the latter: > > Modified Lucene > > Lucene is great, but in its current form it has several shortcomings for > real-time search. That’s why we rewrote big parts of the > core in-memory data structures, especially the posting lists, while still > supporting Lucene’s standard APIs. This allows us to > use Lucene’s search layer almost unmodified. Some of the highlights of our > changes include: > > - significantly improved garbage collection performance > - lock-free data structures and algorithms > - posting lists, that are traversable in reverse order > - efficient early query termination > > We believe that the architecture behind these changes involves several > interesting topics that pertain to software engineering in > general (not only search). We hope to continue to share more on these > improvements. > > And, before you ask, we’re planning on contributing all these changes back > to Lucene; some of which have already made it into > Lucene’s trunk and its new realtime branch. > > > And as you can read at the bottom of the last post, Michael B. is behind > all this :-). > > FYI > Shai >
