Well ... I could argue that it's you who miss the point :). I completely don't buy the "all the new features" comment --> how many new features are in a major release which force you to consider reindexing? Yet there are many of them that change the API. How will I know whether a release supports my index or not? Why do I need to work hard to back-port all the new developed issues onto a branch I use? How many of those branches will exist? Will they all run nightly unit tests? Can I cut a release of such branch myself? Or will I need the PMC or a VOTE? This will get complicated pretty fast ...
Lucene is not a "do it yourself" kit - we try so hard to have the best defaults, best out of the box experience ... best everything for our users. Even w/ Analyzers we try so damn hard. While we could have simply componentize everything and tell the users "you can use those filters, tokenizers, segment mergers, policies etc. to make up your indexing application" ... And I don't think there are features out there that exist and are not contributed because people are afraid of the index format changes ... obviously if they have done it, they're passed the fear of handling index format ... I'd like to hear of one such feature. I'd bet there are such out there that are not contributed for IP, Business and Laziness reasons. Shai On Thu, Apr 15, 2010 at 3:56 PM, Robert Muir <rcm...@gmail.com> wrote: > I think you guys miss the entire point. > > The idea that you can keep getting "all the new features" without > reindexing is merely an illusion > > Instead, features simply aren't being added at all, because the policy > makes it too cumbersome. > > Why is it problematic to have a different SVN branch/release, with lots of > new features, but requires you to reindex and change your app? > > If its too difficult to reindex, it doesnt break your app that features > exist elsewhere that you cannot access. > Its the same as it is today, there are features you cannot access, except > they do not even exist in apache SVN at all, even trunk, because of these > problems. > > On Thu, Apr 15, 2010 at 8:42 AM, Earwin Burrfoot <ear...@gmail.com> wrote: > >> I like the idea of index conversion tool over silent online upgrade >> because it is >> 1. controllable - with online upgrade you never know for sure when >> your index is completely upgraded, even optimize() won't help here, as >> it is a noop for already-optimized indexes >> 2. way easier to write - as flex shows, index format changes are >> accompanied by API changes. Here you don't have to emulate new APIs >> over old structures (can be impossible for some cases?), you only have >> to, well, convert. >> >> On Thu, Apr 15, 2010 at 16:32, Danil ŢORIN <torin...@gmail.com> wrote: >> > All I ask is a way to migrate existing indexes to newer format. >> > >> > >> > On Thu, Apr 15, 2010 at 15:21, Robert Muir <rcm...@gmail.com> wrote: >> >> >> >> its open source, if you feel this way, you can put the work to add >> >> features to some version branch from trunk in a backwards compatible >> way. >> >> Then this branch can have a backwards-compatible minor release with new >> >> features, but nothing ground-breaking. >> >> but this kinda stuff shouldnt hinder development on trunk. >> >> >> >> On Thu, Apr 15, 2010 at 8:17 AM, Danil ŢORIN <torin...@gmail.com> >> wrote: >> >>> >> >>> Sometimes it's REALLY impossible to reindex, or has absolutely >> >>> prohibitive cost to do in a running production system (i can't shut it >> down >> >>> for maintainance, so i need a lot of hardware to reindex ~5 billion >> >>> documents, i have no idea what are the costs to retrieve that data all >> over >> >>> again, but i estimate it to be quite a lot) >> >>> And providing a way to migrate existing indexes to new lucene is >> crucial >> >>> from my point of view. >> >>> I don't care what this way is: calling optimize() with newer lucene or >> >>> running some tool that takes 5 days, it's ok with me. >> >>> Just don't put me through full reindexing as I really don't have all >> that >> >>> data anymore. >> >>> It's not my data, i just receive it from clients, and provide a search >> >>> interface. >> >>> It took years to build those indexes, rebuilding is not an option, and >> >>> staying with old lucene forever just sucks. >> >>> >> >>> Danil. >> >>> On Thu, Apr 15, 2010 at 14:57, Robert Muir <rcm...@gmail.com> wrote: >> >>>> >> >>>> >> >>>> On Thu, Apr 15, 2010 at 7:52 AM, Shai Erera <ser...@gmail.com> >> wrote: >> >>>>> >> >>>>> Well ... I must say that I completely disagree w/ dropping index >> >>>>> structure back-support. Our customers will simply not hear of >> reindexing 10s >> >>>>> of TBs of content because of version upgrades. Such a decision is >> key to >> >>>>> Lucene adoption in large-scale projects. It's entirely not about >> whether >> >>>>> Lucene is a content store or not - content is stored on other >> systems, I >> >>>>> agree. But that doesn't mean reindexing it is tolerable. >> >>>>> >> >>>> >> >>>> I don't understand how its helpful to do a MAJOR version upgrade >> without >> >>>> reindexing... what in the world do you stand to gain from that? >> >>>> The idea here, is that development can be free of such hassles. >> >>>> Development should be this way. >> >>>> If you, Shai, need some feature X.Y.Z from Version 4 and don't want >> to >> >>>> reindex, and are willing to do the work to port it back to Version 3 >> in a >> >>>> completely backwards compatible way, then under this new scheme it >> can >> >>>> happen. >> >>>> >> >>>> -- >> >>>> Robert Muir >> >>>> rcm...@gmail.com >> >>> >> >> >> >> >> >> >> >> -- >> >> Robert Muir >> >> rcm...@gmail.com >> > >> > >> >> >> >> -- >> Kirill Zakharenko/Кирилл Захаренко (ear...@gmail.com) >> Home / Mobile: +7 (495) 683-567-4 / +7 (903) 5-888-423 >> ICQ: 104465785 >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-dev-h...@lucene.apache.org >> >> > > > -- > Robert Muir > rcm...@gmail.com >
