I'm realize that just transforming old index won't give me anything new. The applications usually evolve.
Let's take as example 2.9 (relatively few changes in index structure, but Trie was a nice addition, per segment search and reload was a bless): - There are 4 billion documents which don't have numeric ranges (but those still got faster reopen) - But for next 1 billion documents in another index i do have numeric ranges. The whole application works in ONE environment from same codebase. Splitting it into several environments based on whatever version of lucene happend to be current at index creation date, and maintaining branches of code would be quite a PITA for a developer (and very error prone) So yeah, I won't get new features for old indexes if i transform them to new format, but new indexes will be able to use them. And my application as a whole will be much cleaner and easier to maintain (I'm a lazy developer that thinks that he is already overworked) I just want my system as a whole to evolve together with lucene without dropping the indexes I already have and keeping tens of branches of code and remembering how things worked back in 2005 to slightly modify the analyzer because data in 2010 changed a bit. Danil. On Thu, Apr 15, 2010 at 15:56, Robert Muir <rcm...@gmail.com> wrote: > I think you guys miss the entire point. > > The idea that you can keep getting "all the new features" without > reindexing is merely an illusion > > Instead, features simply aren't being added at all, because the policy > makes it too cumbersome. > > Why is it problematic to have a different SVN branch/release, with lots of > new features, but requires you to reindex and change your app? > > If its too difficult to reindex, it doesnt break your app that features > exist elsewhere that you cannot access. > Its the same as it is today, there are features you cannot access, except > they do not even exist in apache SVN at all, even trunk, because of these > problems. > > On Thu, Apr 15, 2010 at 8:42 AM, Earwin Burrfoot <ear...@gmail.com> wrote: > >> I like the idea of index conversion tool over silent online upgrade >> because it is >> 1. controllable - with online upgrade you never know for sure when >> your index is completely upgraded, even optimize() won't help here, as >> it is a noop for already-optimized indexes >> 2. way easier to write - as flex shows, index format changes are >> accompanied by API changes. Here you don't have to emulate new APIs >> over old structures (can be impossible for some cases?), you only have >> to, well, convert. >> >> On Thu, Apr 15, 2010 at 16:32, Danil ŢORIN <torin...@gmail.com> wrote: >> > All I ask is a way to migrate existing indexes to newer format. >> > >> > >> > On Thu, Apr 15, 2010 at 15:21, Robert Muir <rcm...@gmail.com> wrote: >> >> >> >> its open source, if you feel this way, you can put the work to add >> >> features to some version branch from trunk in a backwards compatible >> way. >> >> Then this branch can have a backwards-compatible minor release with new >> >> features, but nothing ground-breaking. >> >> but this kinda stuff shouldnt hinder development on trunk. >> >> >> >> On Thu, Apr 15, 2010 at 8:17 AM, Danil ŢORIN <torin...@gmail.com> >> wrote: >> >>> >> >>> Sometimes it's REALLY impossible to reindex, or has absolutely >> >>> prohibitive cost to do in a running production system (i can't shut it >> down >> >>> for maintainance, so i need a lot of hardware to reindex ~5 billion >> >>> documents, i have no idea what are the costs to retrieve that data all >> over >> >>> again, but i estimate it to be quite a lot) >> >>> And providing a way to migrate existing indexes to new lucene is >> crucial >> >>> from my point of view. >> >>> I don't care what this way is: calling optimize() with newer lucene or >> >>> running some tool that takes 5 days, it's ok with me. >> >>> Just don't put me through full reindexing as I really don't have all >> that >> >>> data anymore. >> >>> It's not my data, i just receive it from clients, and provide a search >> >>> interface. >> >>> It took years to build those indexes, rebuilding is not an option, and >> >>> staying with old lucene forever just sucks. >> >>> >> >>> Danil. >> >>> On Thu, Apr 15, 2010 at 14:57, Robert Muir <rcm...@gmail.com> wrote: >> >>>> >> >>>> >> >>>> On Thu, Apr 15, 2010 at 7:52 AM, Shai Erera <ser...@gmail.com> >> wrote: >> >>>>> >> >>>>> Well ... I must say that I completely disagree w/ dropping index >> >>>>> structure back-support. Our customers will simply not hear of >> reindexing 10s >> >>>>> of TBs of content because of version upgrades. Such a decision is >> key to >> >>>>> Lucene adoption in large-scale projects. It's entirely not about >> whether >> >>>>> Lucene is a content store or not - content is stored on other >> systems, I >> >>>>> agree. But that doesn't mean reindexing it is tolerable. >> >>>>> >> >>>> >> >>>> I don't understand how its helpful to do a MAJOR version upgrade >> without >> >>>> reindexing... what in the world do you stand to gain from that? >> >>>> The idea here, is that development can be free of such hassles. >> >>>> Development should be this way. >> >>>> If you, Shai, need some feature X.Y.Z from Version 4 and don't want >> to >> >>>> reindex, and are willing to do the work to port it back to Version 3 >> in a >> >>>> completely backwards compatible way, then under this new scheme it >> can >> >>>> happen. >> >>>> >> >>>> -- >> >>>> Robert Muir >> >>>> rcm...@gmail.com >> >>> >> >> >> >> >> >> >> >> -- >> >> Robert Muir >> >> rcm...@gmail.com >> > >> > >> >> >> >> -- >> Kirill Zakharenko/Кирилл Захаренко (ear...@gmail.com) >> Home / Mobile: +7 (495) 683-567-4 / +7 (903) 5-888-423 >> ICQ: 104465785 >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-dev-h...@lucene.apache.org >> >> > > > -- > Robert Muir > rcm...@gmail.com >
