Re: Proposal about Version API relaxation

2010-04-16 Thread Michael McCandless
Getting back to the stable/experimental branches... I think, with separate stable experimental branches, development would/should be active on both branches. It'd depend on the feature... Eg today we'd have 3.x stable branch and the experimental branch (= trunk). Small features, bug fixes,

Re: Proposal about Version API relaxation

2010-04-16 Thread Mark Miller
I'd be for this plan if I really thought the stable branch would get similar attention to the experimental branch - but I have some doubts about that. Its a fairly small dev community in comparison to other projects that do this ... Dev on the experimental latest greatest fun branch, or the

Re: Proposal about Version API relaxation

2010-04-16 Thread Robert Muir
On Fri, Apr 16, 2010 at 12:12 PM, Mark Miller markrmil...@gmail.com wrote: I'd be for this plan if I really thought the stable branch would get similar attention to the experimental branch - but I have some doubts about that. Its a fairly small dev community in comparison to other projects

Re: Proposal about Version API relaxation

2010-04-16 Thread Michael McCandless
I agree that's a risk... but this being open source, I think it'd balance out? So eg right now I'm looking @ speeding up PhraseQuery (thanks to Robert's prodding ;) ). These changes are all under the hood, so, I would do this on the stable branch. There's no reason not to. There are also good

Re: Proposal about Version API relaxation

2010-04-16 Thread Mark Miller
On 04/16/2010 12:16 PM, Robert Muir wrote: On Fri, Apr 16, 2010 at 12:12 PM, Mark Miller markrmil...@gmail.com mailto:markrmil...@gmail.com wrote: I'd be for this plan if I really thought the stable branch would get similar attention to the experimental branch - but I have some

Re: Proposal about Version API relaxation

2010-04-16 Thread Robert Muir
On Fri, Apr 16, 2010 at 12:45 PM, Mark Miller markrmil...@gmail.com wrote: It's not a sign that users don't care about it. Lately I think you have taken the stance, users be damned, Lucene dev should just be geared towards devs. I'm not a fan of that kind of attitude when it comes to Lucene

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
We can remove Version, because all incompatible changes go straight to a new major release, which we release more often, yes. 3.x is going to be released after 4.0 if bugs are found and fixed, or if people ask to backport some (minor?) features, and some dev has time for this. The question of

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
Well ... I think that version numbers mean more than we'd like them to mean, as people perceive them. Let's discuss the format X.Y.Z: When X is changed, it should mean something 'big' happened - index structure has changed (e.g. the flexible scoring work), new Java version supported (Java 1.6)

Re: Proposal about Version API relaxation

2010-04-15 Thread Michael McCandless
2010/4/15 Shai Erera ser...@gmail.com: One way is to define 'major' as X and minor X.Y, and another is to define major as 'X.Y' and minor as 'X.Y.Z'. I prefer the latter but don't have any strong feelings against the former. I prefer X.Y, ie, changes to Y only is a minor release (mostly bug

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
Well ... I must say that I completely disagree w/ dropping index structure back-support. Our customers will simply not hear of reindexing 10s of TBs of content because of version upgrades. Such a decision is key to Lucene adoption in large-scale projects. It's entirely not about whether Lucene is

Re: Proposal about Version API relaxation

2010-04-15 Thread Robert Muir
On Thu, Apr 15, 2010 at 7:52 AM, Shai Erera ser...@gmail.com wrote: Well ... I must say that I completely disagree w/ dropping index structure back-support. Our customers will simply not hear of reindexing 10s of TBs of content because of version upgrades. Such a decision is key to Lucene

Re: Proposal about Version API relaxation

2010-04-15 Thread Danil ŢORIN
Sometimes it's REALLY impossible to reindex, or has absolutely prohibitive cost to do in a running production system (i can't shut it down for maintainance, so i need a lot of hardware to reindex ~5 billion documents, i have no idea what are the costs to retrieve that data all over again, but i

Re: Proposal about Version API relaxation

2010-04-15 Thread Robert Muir
its open source, if you feel this way, you can put the work to add features to some version branch from trunk in a backwards compatible way. Then this branch can have a backwards-compatible minor release with new features, but nothing ground-breaking. but this kinda stuff shouldnt hinder

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
I think an index upgrade tool is okay? While you still definetly have to code it, things like if idxVer==m doOneStuff elseif idxVer==n doOtherStuff else blowUp are kept away from lucene innards and we all profit? On Thu, Apr 15, 2010 at 16:21, Robert Muir rcm...@gmail.com wrote: its open source,

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
Thanks Danil - you reminded me of another reason why reindexing is impossible - fetching the data, even if it's available is too damn costly. Robert, I think you're driven by Analyzers changes ... been too much around them I'm afraid :). A major version upgrade is a move to Java 1.5 for example.

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
I can live w/ that Earwin ... I prefer the ongoing upgrades still, but I won't hold off the back-compat policy change vote because of that. Shai On Thu, Apr 15, 2010 at 3:30 PM, Earwin Burrfoot ear...@gmail.com wrote: I think an index upgrade tool is okay? While you still definetly have to

Re: Proposal about Version API relaxation

2010-04-15 Thread Danil ŢORIN
All I ask is a way to migrate existing indexes to newer format. On Thu, Apr 15, 2010 at 15:21, Robert Muir rcm...@gmail.com wrote: its open source, if you feel this way, you can put the work to add features to some version branch from trunk in a backwards compatible way. Then this branch

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
I like the idea of index conversion tool over silent online upgrade because it is 1. controllable - with online upgrade you never know for sure when your index is completely upgraded, even optimize() won't help here, as it is a noop for already-optimized indexes 2. way easier to write - as flex

Re: Proposal about Version API relaxation

2010-04-15 Thread Robert Muir
I think you guys miss the entire point. The idea that you can keep getting all the new features without reindexing is merely an illusion Instead, features simply aren't being added at all, because the policy makes it too cumbersome. Why is it problematic to have a different SVN branch/release,

Re: Proposal about Version API relaxation

2010-04-15 Thread Yonik Seeley
Seamless online upgrades have their place too... say you are upgrading one server at a time in a cluster. -Yonik Apache Lucene Eurocon 2010 18-21 May 2010 | Prague On Thu, Apr 15, 2010 at 8:42 AM, Earwin Burrfoot ear...@gmail.com wrote: I like the idea of index conversion tool over silent

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
Well ... I could argue that it's you who miss the point :). I completely don't buy the all the new features comment -- how many new features are in a major release which force you to consider reindexing? Yet there are many of them that change the API. How will I know whether a release supports my

Re: Proposal about Version API relaxation

2010-04-15 Thread Danil ŢORIN
I'm realize that just transforming old index won't give me anything new. The applications usually evolve. Let's take as example 2.9 (relatively few changes in index structure, but Trie was a nice addition, per segment search and reload was a bless): - There are 4 billion documents which don't

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
On Thu, Apr 15, 2010 at 17:17, Yonik Seeley yo...@lucidimagination.com wrote: Seamless online upgrades have their place too... say you are upgrading one server at a time in a cluster. Nothing here that can't be solved with an upgrade tool. Down one server, upgrade index, upgrade sofware, up.

Re: Proposal about Version API relaxation

2010-04-15 Thread Danil ŢORIN
True. Just need the tool. On Thu, Apr 15, 2010 at 16:39, Earwin Burrfoot ear...@gmail.com wrote: On Thu, Apr 15, 2010 at 17:17, Yonik Seeley yo...@lucidimagination.com wrote: Seamless online upgrades have their place too... say you are upgrading one server at a time in a cluster.

Re: Proposal about Version API relaxation

2010-04-15 Thread Yonik Seeley
On Thu, Apr 15, 2010 at 9:39 AM, Earwin Burrfoot ear...@gmail.com wrote: On Thu, Apr 15, 2010 at 17:17, Yonik Seeley yo...@lucidimagination.com wrote: Seamless online upgrades have their place too... say you are upgrading one server at a time in a cluster. Nothing here that can't be solved

Re: Proposal about Version API relaxation

2010-04-15 Thread Robert Muir
wrong, it doesnt fix the analyzers problem. you need to reindex. On Thu, Apr 15, 2010 at 9:39 AM, Earwin Burrfoot ear...@gmail.com wrote: On Thu, Apr 15, 2010 at 17:17, Yonik Seeley yo...@lucidimagination.com wrote: Seamless online upgrades have their place too... say you are upgrading

Re: Proposal about Version API relaxation

2010-04-15 Thread Grant Ingersoll
+1 On Apr 14, 2010, at 5:22 PM, Michael McCandless wrote: On Wed, Apr 14, 2010 at 12:06 AM, Marvin Humphrey mar...@rectangular.com wrote: Essentially, we're free to break back compat within Lucy at any time, but we're not able to break back compat within a stable fork like Lucy1, Lucy2,

Re: Proposal about Version API relaxation

2010-04-15 Thread Grant Ingersoll
I do think major versions should be able to read the previous version index. Still, even being able to do that is no guarantee that it will produce correct results. Likewise, even having an upgrade tool is no guarantee that correct results will be produced. So, my take is that we strive for

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
On Thu, Apr 15, 2010 at 17:49, Robert Muir rcm...@gmail.com wrote: wrong, it doesnt fix the analyzers problem. you need to reindex. On Thu, Apr 15, 2010 at 9:39 AM, Earwin Burrfoot ear...@gmail.com wrote: On Thu, Apr 15, 2010 at 17:17, Yonik Seeley yo...@lucidimagination.com wrote:

Re: Proposal about Version API relaxation

2010-04-15 Thread Danil ŢORIN
Agree. However I don't see how lucene could suddenly change that even a conversion tool is impossible to create. After all it's all about terms, positions and frequencies. Yeah..some additions as payloads may appear, disappear, or evolve into something new, but those are on user's side anyway.

Re: Proposal about Version API relaxation

2010-04-15 Thread Yonik Seeley
On Wed, Apr 14, 2010 at 5:22 PM, Michael McCandless luc...@mikemccandless.com wrote:  * There is no back compat across major releases (index nor APIs),    but full back compat within branches. This would match how many other projects work (KS/Lucy, as Marvin describes above; Apache Tomcat;

Re: Proposal about Version API relaxation

2010-04-15 Thread Erick Erickson
Coming in late to the discussion, and without really understanding the underlying Lucene issues, but... The size of the problem of reindexing is under-appreciated I think. Somewhere in my company is the original data I indexed. But the effort it would take to resurrect it is O(unknown). An

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
reasonable, but changing APIs around when there's not a good reason behind it (other than someone liked the name a little better) should still be approached with caution. Changing names is a good enough reason :) They make a darn difference between having to read a book to be able to use some

Re: Proposal about Version API relaxation

2010-04-15 Thread Mark Miller
If you absolutely cannot re-index, and you have *no* access to the data again - you are one ballsy mofo to upgrade to a new version of Lucene for features. It means you likely BASE jump in your free time? On 04/15/2010 10:14 AM, Erick Erickson wrote: Coming in late to the discussion, and

Re: Proposal about Version API relaxation

2010-04-15 Thread Erick Erickson
'Cause some exec finally noticed the product was losing market share. Or got a wild hair strategically placed. My point is only that we should be clear that some number of Lucene users *will* be in such a position. I'm actually fine with a decision that we're not going to support such a scenario,

Re: Proposal about Version API relaxation

2010-04-15 Thread Danil ŢORIN
The app is not rotted, it's alive and kicking, and gets a lot of TLC. There are some older indexes that use some features and there are newer indexes that will benefit greatly from newer features. All running in one freaking big distributed application. Leveraging lucene version by updating to

Re: Proposal about Version API relaxation

2010-04-15 Thread DM Smith
On 04/15/2010 09:49 AM, Robert Muir wrote: wrong, it doesnt fix the analyzers problem. you need to reindex. On Thu, Apr 15, 2010 at 9:39 AM, Earwin Burrfoot ear...@gmail.com mailto:ear...@gmail.com wrote: On Thu, Apr 15, 2010 at 17:17, Yonik Seeley yo...@lucidimagination.com

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
First, the index format. IMHO, it is a good thing for a major release to be able to read the prior major release's index. And the ability to convert it to the current format via optimize is also good. Whatever is decided on this thread should take this seriously. Optimize is a bad way to

Re: Proposal about Version API relaxation

2010-04-15 Thread Robert Muir
On Thu, Apr 15, 2010 at 1:30 PM, DM Smith dmsmith...@gmail.com wrote: Another behavior change is an upgrade in Java version. By forcing users to go to Java 5 with Lucene 3, the version of Unicode changed. This in itself causes a change in some token streams. ... It is my observation,

Re: Proposal about Version API relaxation

2010-04-15 Thread DM Smith
On 04/15/2010 01:50 PM, Earwin Burrfoot wrote: First, the index format. IMHO, it is a good thing for a major release to be able to read the prior major release's index. And the ability to convert it to the current format via optimize is also good. Whatever is decided on this thread should take

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
I'd like to remind that Mike's proposal has stable branches. We can branch off preflex trunk right now and wrap it up as 3.1. Current trunk is declared as future 4.0 and all backcompat cruft is removed from it. If some new features/bugfixes appear in trunk, and they don't break stuff - we

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
I seriously don't understand the fuss around index format back compat. How many times is this changed such that it is too much to ask to keep X support X-1? I prefer to have ongoing segment merging but can live w/ a manual converter tool. Thing is - I'll probably not be able to develop one myself

Re: Proposal about Version API relaxation

2010-04-15 Thread Sanne Grinovero
Hello, I think some compatibility breaks should really be accepted, otherwise these requirements are going to kill the technological advancement: the effort in backwards compatibility will grow and be more timeconsuming and harder every day. A mayor release won't happen every day, likely not even

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
BTW Earwin, we can come up w/ a migrate() method on IW to accomplish manual migration on the segments that are still on old versions. That's not the point about whether optimize() is good or not. It is the difference between telling the customer to run a 5-day migration process, or a couple

Re: Proposal about Version API relaxation

2010-04-15 Thread DM Smith
On 04/15/2010 03:04 PM, Earwin Burrfoot wrote: BTW Earwin, we can come up w/ a migrate() method on IW to accomplish manual migration on the segments that are still on old versions. That's not the point about whether optimize() is good or not. It is the difference between telling the customer to

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
On Thu, Apr 15, 2010 at 23:07, DM Smith dmsmith...@gmail.com wrote: On 04/15/2010 03:04 PM, Earwin Burrfoot wrote: BTW Earwin, we can come up w/ a migrate() method on IW to accomplish manual migration on the segments that are still on old versions. That's not the point about whether

Re: Proposal about Version API relaxation

2010-04-15 Thread jm
Not sure if plain users are allowed/encouraged to post in this list, but wanted to mention (just an opinion from a happy user), as other users have, that not all of us can reindex just like that. It would not be 10 min for one of our installations for sure... First, i would need to implement some

Re: Proposal about Version API relaxation

2010-04-15 Thread Robert Muir
but seriously... are you moving across major lucene releases every single day? if you are using 3.x, how does it hurt you if there is a version 4.x that you can't use without re-indexing? why wouldn't you just stay on your stable branch (say 3.x)? 2010/4/15 jm jmugur...@gmail.com Not sure if

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
Not sure if plain users are allowed/encouraged to post in this list, but wanted to mention (just an opinion from a happy user), as other users have, that not all of us can reindex just like that. It would not be 10 min for one of our installations for sure... First, i would need to implement

Re: Proposal about Version API relaxation

2010-04-15 Thread DM Smith
On 04/15/2010 03:12 PM, Earwin Burrfoot wrote: On Thu, Apr 15, 2010 at 23:07, DM Smithdmsmith...@gmail.com wrote: On 04/15/2010 03:04 PM, Earwin Burrfoot wrote: BTW Earwin, we can come up w/ a migrate() method on IW to accomplish manual migration on the segments that are still on

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
The reason Earwin why online migration is faster is because when u finally need to *fully* migrate your index, most chances are that most of the segments are already on the newer format. Offline migration will just keep the application idle for some amount of time until ALL segments are migrated.

Re: Proposal about Version API relaxation

2010-04-15 Thread DM Smith
On 04/15/2010 03:25 PM, Shai Erera wrote: We should create a migrate() API on IW which will touch just those segments and not incur a full optimize. That API can also be used for an offline migration tool, if we decide that's what we want. What about an index that has already called

RE: Proposal about Version API relaxation

2010-04-15 Thread Uwe Schindler
.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Earwin Burrfoot [mailto:ear...@gmail.com] Sent: Thursday, April 15, 2010 8:15 PM To: java-dev@lucene.apache.org Subject: Re: Proposal about Version API relaxation I'd like

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
2010/4/15 Shai Erera ser...@gmail.com: The reason Earwin why online migration is faster is because when u finally need to *fully* migrate your index, most chances are that most of the segments are already on the newer format. Offline migration will just keep the application idle for some

Re: Proposal about Version API relaxation

2010-04-15 Thread Michael McCandless
Unfortunately, live searching against an old index can get very hairy. EG look at what I had to do for the flex API on pre-flex index flex emulation layer. It's also not great because it gives the illusion that all is good, yet, you've taken a silent hit (up to ~10% or so) in your search perf.

Re: Proposal about Version API relaxation

2010-04-15 Thread Robert Muir
2010/4/15 Michael McCandless luc...@mikemccandless.com I realize the migration tool has issues -- it fixes the hard changes but silently allows the soft changes to break (ie, your analyzers my not produce the same tokens, until we move all core analyzers outside of core, so they are

Re: Proposal about Version API relaxation

2010-04-15 Thread Grant Ingersoll
From IRC: why do I get the feeling that everyone is in heated agreement on the Version thread? there are some cases that mean people will have to reindex in those cases, we should tell people they will have to reindex then they can decide to upgrade or not all other cases, just do the sensible

Re: Proposal about Version API relaxation

2010-04-15 Thread Earwin Burrfoot
I think this should split off the mega-thread :) On Thu, Apr 15, 2010 at 23:28, Uwe Schindler u...@thetaphi.de wrote: Hi Earwin, I am strongly +1 on this. I would also make the Release Manager for 3.1, if nobody else wants to do this. I would like to take the preflex tag or some revisions

Re: Proposal about Version API relaxation

2010-04-15 Thread Andi Vajda
On Thu, 15 Apr 2010, Robert Muir wrote: 2010/4/15 Michael McCandless luc...@mikemccandless.com I realize the migration tool has issues -- it fixes the hard changes but silently allows the soft changes to break (ie, your analyzers my not produce the same tokens,

RE: Proposal about Version API relaxation

2010-04-15 Thread Uwe Schindler
Sent: Thursday, April 15, 2010 9:46 PM To: java-dev@lucene.apache.org Subject: Re: Proposal about Version API relaxation From IRC: why do I get the feeling that everyone is in heated agreement on the Version thread? there are some cases that mean people will have to reindex in those cases

Re: Proposal about Version API relaxation

2010-04-15 Thread Robert Muir
3. put analyzers in their own versioned jar files. Yes, every analyzer needs to have its own version and thus, jar file. Putting all analyzers into one versioned jar file joins them at the hip and suffers from the same versioning and compat problems we're currently facing in core. Andi..

Re: Proposal about Version API relaxation

2010-04-15 Thread Michael McCandless
On Thu, Apr 15, 2010 at 3:50 PM, Robert Muir rcm...@gmail.com wrote: for now simply moving analyzers to its own jar filE would be a great step! +1 -- why not consolidate all analyzers now? (And fix indexer to require a minimal API = TokenStream minus reset close). Mike

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
+1 on the Analyzers as well. Earwin, I think I don't mind if we introduce migrate() elsewhere rather than on IW. What I meant to say is that if we stick w/ index format back-compat and ongoing migration, then such a method would be useful on IW for customers to call to ensure they're on the

Re: Proposal about Version API relaxation

2010-04-15 Thread Grant Ingersoll
On Apr 15, 2010, at 4:21 PM, Shai Erera wrote: +1 on the Analyzers as well. Earwin, I think I don't mind if we introduce migrate() elsewhere rather than on IW. What I meant to say is that if we stick w/ index format back-compat and ongoing migration, then such a method would be useful on

Re: Proposal about Version API relaxation

2010-04-15 Thread Robert Muir
On Thu, Apr 15, 2010 at 4:21 PM, Shai Erera ser...@gmail.com wrote: Actually, I'd like to know if people like Robert (basically those who have no problem to reindex and don't understand the fuss around it) will want to change the index format - can I count on them to be asked to provide such

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
Grant ... you've made it - the 100th response to that thread. Do we keep records somewhere? :) Ok I'm simply proposing to define 'index back-compat' as index format back-compat. With that, we don't 'wait' for something to happen, we just say up front that if that changes, we provide a migration

Re: Proposal about Version API relaxation

2010-04-15 Thread Sanne Grinovero
+1 on the Analyzers split, But would like to point out that it's not very different than having a non final static version field. Just a much better solution as you keep your code manageable. 2010/4/15 Grant Ingersoll gsing...@apache.org: On Apr 15, 2010, at 4:21 PM, Shai Erera wrote: +1 on

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
Robert ... I'm sorry but changes to Analyzers don't *force* people to reindex. They can simply choose not to use the latest version. They can choose not to upgrade a Unicode version. They can copy the entire Analyzer code to match their needs. Index format changes is what I'm worried about because

Re: Proposal about Version API relaxation

2010-04-15 Thread Robert Muir
If you really believe this. then you have no problem if i remove all Version from all core and contrib analyzers right now. On Thu, Apr 15, 2010 at 4:50 PM, Shai Erera ser...@gmail.com wrote: Robert ... I'm sorry but changes to Analyzers don't *force* people to reindex. They can simply choose

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
By all means Robert ... by all means :). Remember who started that thread, and for what reason :D. Shai On Fri, Apr 16, 2010 at 12:01 AM, Robert Muir rcm...@gmail.com wrote: If you really believe this. then you have no problem if i remove all Version from all core and contrib analyzers right

Re: Proposal about Version API relaxation

2010-04-15 Thread Robert Muir
As soon as I have removed version, then we can fix StandardTokenizer too! On Thu, Apr 15, 2010 at 5:13 PM, Shai Erera ser...@gmail.com wrote: By all means Robert ... by all means :). Remember who started that thread, and for what reason :D. Shai On Fri, Apr 16, 2010 at 12:01 AM, Robert

Re: Proposal about Version API relaxation

2010-04-15 Thread DM Smith
On Apr 15, 2010, at 4:50 PM, Shai Erera wrote: Robert ... I'm sorry but changes to Analyzers don't *force* people to reindex. They can simply choose not to use the latest version. They can choose not to upgrade a Unicode version. They can copy the entire Analyzer code to match their

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
DM I think ICU is great. But currently we use JFlex and you can run Java 10 if you want, but as long as JFlex is compiled w/ Java 1.4, that's what you'll get. Luckily Uwe and Robert recently bumped it up to Java 1.5. Such a change should be clearly documented in CHANGES so people are aware of

Re: Proposal about Version API relaxation

2010-04-15 Thread DM Smith
On Apr 15, 2010, at 5:28 PM, Shai Erera wrote: DM I think ICU is great. But currently we use JFlex and you can run Java 10 if you want, but as long as JFlex is compiled w/ Java 1.4, that's what you'll get. Luckily Uwe and Robert recently bumped it up to Java 1.5. Such a change should be

Re: Proposal about Version API relaxation

2010-04-15 Thread Robert Muir
On Thu, Apr 15, 2010 at 6:50 PM, DM Smith dmsmith...@gmail.com wrote: Robert has already started one. (1488 I think). and it could work with this new scheme... because then you could use an older icu jar file with an older lucene-analyzer-icu.jar or whatever and you have it more under

Re: Proposal about Version API relaxation

2010-04-14 Thread Grant Ingersoll
On Apr 14, 2010, at 12:49 AM, Robert Muir wrote: On Wed, Apr 14, 2010 at 12:06 AM, Marvin Humphrey mar...@rectangular.com wrote: New class names would work, too. I only mention that for the sake of completeness, though -- it's not a suggestion. Right, to me this is just as bad. In

Re: Proposal about Version API relaxation

2010-04-14 Thread Earwin Burrfoot
The thread somehow got sidetracked. So, let's get this carriage back on its rails? Let me remind - we have an API on hands that is mandatory and tends to be cumbersome. Proposed solution does indeed have ultrascary word static in it. But if you brace yourself and look closer - the use of said

Re: Proposal about Version API relaxation

2010-04-14 Thread Robert Muir
Its not sidetracked at all. there seem to be more compelling alternatives to achieve the same thing, so we should consider alternative solutions, too. On Wed, Apr 14, 2010 at 8:54 AM, Earwin Burrfoot ear...@gmail.com wrote: The thread somehow got sidetracked. So, let's get this carriage back

Re: Proposal about Version API relaxation

2010-04-14 Thread DM Smith
On 04/14/2010 09:13 AM, Robert Muir wrote: Its not sidetracked at all. there seem to be more compelling alternatives to achieve the same thing, so we should consider alternative solutions, too. Maybe have the index store the version(s) and use that when constructing a reader or writer? Given

Re: Proposal about Version API relaxation

2010-04-14 Thread Yonik Seeley
On Wed, Apr 14, 2010 at 10:39 AM, DM Smith dmsmith...@gmail.com wrote: Maybe have the index store the version(s) and use that when constructing a reader or writer? That would cause a reindex to change behavior (among other problems). -Yonik Apache Lucene Eurocon 2010 18-21 May 2010 | Prague

Re: Proposal about Version API relaxation

2010-04-14 Thread Marvin Humphrey
On Wed, Apr 14, 2010 at 08:30:14AM -0400, Grant Ingersoll wrote: The thing I keep going back to is that somehow Lucene has managed for years (and I mean lots of years) w/o stuff like Version and all this massive back compatibility checking. Non-constant global variables are an anti-pattern.

Re: Proposal about Version API relaxation

2010-04-14 Thread Robert Muir
On Wed, Apr 14, 2010 at 12:29 PM, Marvin Humphrey mar...@rectangular.comwrote: I also am not sure whether it in the past we just missed/ignored more back compatibility issues or whether now we are creating more back compat. issues due to more rapid change. It would be hard to search

Re: Proposal about Version API relaxation

2010-04-14 Thread Mark Miller
On 04/14/2010 12:29 PM, Marvin Humphrey wrote: On Wed, Apr 14, 2010 at 08:30:14AM -0400, Grant Ingersoll wrote: The thing I keep going back to is that somehow Lucene has managed for years (and I mean lots of years) w/o stuff like Version and all this massive back compatibility checking.

Re: Proposal about Version API relaxation

2010-04-14 Thread Andi Vajda
On Apr 14, 2010, at 7:45, Yonik Seeley yo...@lucidimagination.com wrote: On Wed, Apr 14, 2010 at 10:39 AM, DM Smith dmsmith...@gmail.com wrote: Maybe have the index store the version(s) and use that when constructing a reader or writer? That would cause a reindex to change behavior

Re: Proposal about Version API relaxation

2010-04-14 Thread Marvin Humphrey
On Wed, Apr 14, 2010 at 12:49:52AM -0400, Robert Muir wrote: its very unnatural for release 3.0 to be almost a no-op and for release 3.1 to provide a new default index format and support for customizing how the index is stored. And now we are looking at providing flexibility in scoring that

RE: Proposal about Version API relaxation

2010-04-14 Thread Uwe Schindler
-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Mark Miller [mailto:markrmil...@gmail.com] Sent: Wednesday, April 14, 2010 6:58 PM To: java-dev@lucene.apache.org Subject: Re: Proposal about Version API relaxation On 04/14/2010 12:29 PM

RE: Proposal about Version API relaxation

2010-04-14 Thread Uwe Schindler
And 2.9's backwards compatibility layer in TokenStream was significantly slower. I protest! No, it was not slower, only at the beginning because of missing reflection caching! But this also affected the *new* API. With 2.9.x and old TokenStreams there is no speed difference, really. Uwe

Re: Proposal about Version API relaxation

2010-04-14 Thread Robert Muir
On Wed, Apr 14, 2010 at 2:49 PM, Uwe Schindler u...@thetaphi.de wrote: And 2.9's backwards compatibility layer in TokenStream was significantly slower. I protest! No, it was not slower, only at the beginning because of missing reflection caching! But this also affected the *new* API.

Re: Proposal about Version API relaxation

2010-04-14 Thread Michael McCandless
On Wed, Apr 14, 2010 at 12:06 AM, Marvin Humphrey mar...@rectangular.com wrote: Essentially, we're free to break back compat within Lucy at any time, but we're not able to break back compat within a stable fork like Lucy1, Lucy2, etc. So what we'll probably do during normal development with

Re: Proposal about Version API relaxation

2010-04-14 Thread Chris Male
On Wed, Apr 14, 2010 at 11:22 PM, Michael McCandless luc...@mikemccandless.com wrote: On Wed, Apr 14, 2010 at 12:06 AM, Marvin Humphrey mar...@rectangular.com wrote: Essentially, we're free to break back compat within Lucy at any time, but we're not able to break back compat within a

Re: Proposal about Version API relaxation

2010-04-14 Thread Robert Muir
+1 On Wed, Apr 14, 2010 at 5:22 PM, Michael McCandless luc...@mikemccandless.com wrote: On Wed, Apr 14, 2010 at 12:06 AM, Marvin Humphrey mar...@rectangular.com wrote: Essentially, we're free to break back compat within Lucy at any time, but we're not able to break back compat within a

Re: Proposal about Version API relaxation

2010-04-14 Thread Earwin Burrfoot
Can't believe my eyes. +1 On Thu, Apr 15, 2010 at 01:22, Michael McCandless luc...@mikemccandless.com wrote: On Wed, Apr 14, 2010 at 12:06 AM, Marvin Humphrey mar...@rectangular.com wrote: Essentially, we're free to break back compat within Lucy at any time, but we're not able to break back

Re: Proposal about Version API relaxation

2010-04-14 Thread Andi Vajda
On Thu, 15 Apr 2010, Earwin Burrfoot wrote: Can't believe my eyes. +1 Likewise. +1 ! Andi.. On Thu, Apr 15, 2010 at 01:22, Michael McCandless luc...@mikemccandless.com wrote: On Wed, Apr 14, 2010 at 12:06 AM, Marvin Humphrey mar...@rectangular.com wrote: Essentially, we're free to

Re: Proposal about Version API relaxation

2010-04-14 Thread Shai Erera
Ahh ... a dream finally comes true ... what a great way to start a day :). +1 !!! I have some questions/comments though: * Index back compat should be maintained between major releases, like it is today, STRUCTURE-wise. So apps get a chance to incrementally upgrade their segments when they move

Re: Proposal about Version API relaxation

2010-04-14 Thread Shai Erera
Also, we will still need to maintain the Backwards section in CHANGES (or move it to API Changes), to help people upgrade from release to release. Just pointing that out as well. Shai On Thu, Apr 15, 2010 at 7:05 AM, Shai Erera ser...@gmail.com wrote: Ahh ... a dream finally comes true ...

Re: Proposal about Version API relaxation

2010-04-14 Thread Mark Miller
I don't read what you wrote and what Mike wrote as even close to the same. - Mark http://www.lucidimagination.com (mobile) On Apr 15, 2010, at 12:05 AM, Shai Erera ser...@gmail.com wrote: Ahh ... a dream finally comes true ... what a great way to start a day :). +1 !!! I have some

Re: Proposal about Version API relaxation

2010-04-14 Thread Shai Erera
So then I don't understand this: {quote} * A major release always bumps the major release number (2.x - 3.0), and, starts a new branch for all minor (3.1, 3.2, 3.3) releases along that branch * There is no back compat across major releases (index nor APIs), but full back compat within

Proposal about Version API relaxation

2010-04-13 Thread Shai Erera
Hi I'd like to propose a relaxation on the Version API. Uwe, please read the entire email before you reply :). I was thinking, following a question on the user list, that the Version-based API may not be very intuitive to users, especially those who don't care about versioning, as well as very

Re: Proposal about Version API relaxation

2010-04-13 Thread Earwin Burrfoot
I wholeheartedly support this anti-version riot :) On Tue, Apr 13, 2010 at 19:27, Shai Erera ser...@gmail.com wrote: Hi I'd like to propose a relaxation on the Version API. Uwe, please read the entire email before you reply :). I was thinking, following a question on the user list, that the

Re: Proposal about Version API relaxation

2010-04-13 Thread Tim Williams
On Tue, Apr 13, 2010 at 11:27 AM, Shai Erera ser...@gmail.com wrote: Hi I'd like to propose a relaxation on the Version API. Uwe, please read the entire email before you reply :). I was thinking, following a question on the user list, that the Version-based API may not be very intuitive to

  1   2   >