doap.rdf not updated for 6.0 release

[Anshum Gupta]

Hi,

While working on the 5.5.1 release, I realized that doap.rdf was not
updated for both Lucene and Solr for 6.0 release. I was inclined to update
it but wanted to be sure that it wasn't deliberate.

Mike, can you please confirm ?

-- 
Anshum Gupta

Re: 6.0 Release

[Shawn Heisey]

On 3/31/2016 10:02 AM, Ishan Chattopadhyaya wrote:
> AFAICT, I think we can continue to have Trie* fields well into 7.x (by
> pulling the Legacy numerics into Solr at 7.0 release), so as not to
> force users to change schema or reindex if they don't want to.
> Additional Points* fields would be there (at some 6.x onwards) for
> whoever wants performance benefits, and we can make that as the
> default in a 7.0 schema.

I've been poking around in master trying to figure out how to do this
successfully.

There are some very intricate interdependencies that I am not sure how
to resolve.  In particular, there are a bunch of deprecated enum values
in UninvertingReader that will disappear in 7.0... but UninvertingReader
itself is not deprecated.  UninvertingReader is used quite a bit in Solr.

I did see that in Lucene, UninvertingReader is only used in *tests*, so
one possible solution would be to move UninvertingReader to Solr and
nuke the Lucene test code that uses it.  That solution ignores all
third-party software, so it's *probably* not viable.

I'm completely clueless about internals at this level of detail, and I
suspect that the overall deprecation picture is going to make it
extremely difficult to pull the legacy numeric classes into Solr.  Any
thoughts?

Thanks,
Shawn


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: 6.0 Release

[Jack Krupansky]

I was looking at Jira, where
https://issues.apache.org/jira/browse/LUCENE-7075 is still open even now
and the most recent comment says "still left" and lists four areas,
including Join. Silly me for thinking that Jira was accurate. Digging
deeper, I can see that the four linked issues are each marked fixed, so
maybe it was just an oversight not to mark the umbrella issue - the only
one mentioned here - as fixed as well. Sloppy Jira handling sure doesn't
help people understand what's going on.

But I'll stand by my observation that points is still a new and
experimental feature and that deprecating scalar numerics was premature.

And I haven't see any evidence to suggest that physics is to blame for the
failure to articulate a semi-sane migration strategy for existing indexes.
Is it really true that there is not sufficient information in the current
(5.x) numeric indexes to extract the numeric data and generate the points
index?

I will note that no senior committers have supported any of my complaints
in this matter, so that alone suggests that you guys would be on solid
ground to simply ignore at least most if not all of my complaints.

-- Jack Krupansky

On Fri, Apr 1, 2016 at 6:30 AM, Michael McCandless <
luc...@mikemccandless.com> wrote:

> Jack I think you misunderstood Robert's summary: Lucene and its
> modules has already switched from legacy numerics to points, thanks to
> the hard work of multiple devs (thank you all!). This was very
> healthy: it uncovered exciting bugs.  What Rob is describing is how we
> now go about iteratively removing legacy numerics in master, but that
> doesn't block the 6.0 release.
>
> Most recent points-related work (geo improvements, low level BKD
> optimizations) have been going to 6.1 not 6.0.  The 6.0 changes really
> have generally tapered off, and so it makes sense to release now (it
> made sense even a week ago).  The timing here shouldn't be surprising:
> it was nearly 7 weeks ago that I suggested we get the ball rolling for
> 6.0.0.
>
> For Solr I think there are multiple options, already enumerated in
> this thread.  As Adrien pointed out, since legacy numerics builds on
> postings, Solr can easily keep them "alive" even after Lucene has
> removed them in master, if that's really necessary.
>
> At the end of the day, whether you use Elasticsearch, Solr, or Lucene
> directly, you'll need to both update your code and reindex previous
> documents, to take advantage of points.  There is no magic bullet ...
> it comes down to physics really ;)
>
> That said, a magical migration tool would be neat; maybe you can work
> up a patch?  I haven't heard anyone else suggest this idea ...
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Thu, Mar 31, 2016 at 11:39 AM, Jack Krupansky
> <jack.krupan...@gmail.com> wrote:
> > Robert's detailing of the remaining work to get the rest of Lucene off of
> > current (current release, soon to be legacy) numerics is enlightening.
> >
> > Personally, I had thought that it was Solr that was holding up an
> imminent
> > Lucene/Solr 6.0 release, but now I'm thinking:
> >
> > 1. The new "point stuff" (did I mention that I didn't like or approve of
> the
> > current name?) seems more like a work in progress...
> > 2. I'd label the "point stuff" as experimental for 6.0.
> > 3. I wouldn't hold up 6.0 for any further baking of the "point stuff" or
> > migration of other Lucene features off current numeric types.
> > 4. The rest of Lucene can be weaned off the current numerics at a more
> > leaisurely pace, like for 6.1 or 6.2.
> > 5. Once the new "point stuff" is finally full baked, and the rest of
> Lucene
> > is migrated off current numerics, and... Solr has made "point stuff" its
> > default numeric type (6.1 or 6.2?), AND Lucene or Solr comes up with a
> sound
> > migration plan and/or index migration tool for current numerics, only
> THEN
> > should the current numerics become deprecated.
> > 6. I'm not absolutely certain, but I think the 6.0 changes to Solr to use
> > the Lucene LegacyXXX numeric field types should be fine for an initial
> 6.0
> > release, meaning backcompat is assured for 6.x.
> > 7. I'm imagining that with a manually-invoked index upgrade tool a
> current
> > (5.x) numeric field can be migrated to a "point stuff" field type. A
> Lucene
> > heavy will have to confirm that feasibility.
> > 8. I'm imagining that a typical Solr site would be okay with the
> requirement
> > that they have to explicitly, manually run such an index upgrade tool to
> > migrate from current (5.x) numerics to "point stuff". And that they could
>

Re: 6.0 Release

[Michael McCandless]

Jack I think you misunderstood Robert's summary: Lucene and its
modules has already switched from legacy numerics to points, thanks to
the hard work of multiple devs (thank you all!). This was very
healthy: it uncovered exciting bugs.  What Rob is describing is how we
now go about iteratively removing legacy numerics in master, but that
doesn't block the 6.0 release.

Most recent points-related work (geo improvements, low level BKD
optimizations) have been going to 6.1 not 6.0.  The 6.0 changes really
have generally tapered off, and so it makes sense to release now (it
made sense even a week ago).  The timing here shouldn't be surprising:
it was nearly 7 weeks ago that I suggested we get the ball rolling for
6.0.0.

For Solr I think there are multiple options, already enumerated in
this thread.  As Adrien pointed out, since legacy numerics builds on
postings, Solr can easily keep them "alive" even after Lucene has
removed them in master, if that's really necessary.

At the end of the day, whether you use Elasticsearch, Solr, or Lucene
directly, you'll need to both update your code and reindex previous
documents, to take advantage of points.  There is no magic bullet ...
it comes down to physics really ;)

That said, a magical migration tool would be neat; maybe you can work
up a patch?  I haven't heard anyone else suggest this idea ...

Mike McCandless

http://blog.mikemccandless.com

On Thu, Mar 31, 2016 at 11:39 AM, Jack Krupansky
<jack.krupan...@gmail.com> wrote:
> Robert's detailing of the remaining work to get the rest of Lucene off of
> current (current release, soon to be legacy) numerics is enlightening.
>
> Personally, I had thought that it was Solr that was holding up an imminent
> Lucene/Solr 6.0 release, but now I'm thinking:
>
> 1. The new "point stuff" (did I mention that I didn't like or approve of the
> current name?) seems more like a work in progress...
> 2. I'd label the "point stuff" as experimental for 6.0.
> 3. I wouldn't hold up 6.0 for any further baking of the "point stuff" or
> migration of other Lucene features off current numeric types.
> 4. The rest of Lucene can be weaned off the current numerics at a more
> leaisurely pace, like for 6.1 or 6.2.
> 5. Once the new "point stuff" is finally full baked, and the rest of Lucene
> is migrated off current numerics, and... Solr has made "point stuff" its
> default numeric type (6.1 or 6.2?), AND Lucene or Solr comes up with a sound
> migration plan and/or index migration tool for current numerics, only THEN
> should the current numerics become deprecated.
> 6. I'm not absolutely certain, but I think the 6.0 changes to Solr to use
> the Lucene LegacyXXX numeric field types should be fine for an initial 6.0
> release, meaning backcompat is assured for 6.x.
> 7. I'm imagining that with a manually-invoked index upgrade tool a current
> (5.x) numeric field can be migrated to a "point stuff" field type. A Lucene
> heavy will have to confirm that feasibility.
> 8. I'm imagining that a typical Solr site would be okay with the requirement
> that they have to explicitly, manually run such an index upgrade tool to
> migrate from current (5.x) numerics to "point stuff". And that they could
> either do that once Solr adds support for "point stuff" fields or when they
> migrate from 6.x to 7.x. Bonus points if Solr can have a variation of the
> index upgrade tool that discovers and upgrades all current numeric fields.
>
> What else? (I'll ask some questions about Elasticsearch plans in a separate
> message.)
>
>
>
>
> -- Jack Krupansky
>
> On Thu, Mar 31, 2016 at 12:31 AM, David Smiley <david.w.smi...@gmail.com>
> wrote:
>>
>> That was an excellent summary Rob; thanks.
>> Minor nit: BBoxSpatialStrategy isn't/wasn't deprecated.  It was enhanced
>> to use PointValues.
>>
>> I too would like to see the legacy numerics stay in "backwards-codecs" as
>> you describe with precisionStep specified on the Analyzer.
>>
>> I disagree with Shawn about #5, that a user with a Solr 6.0 index must be
>> able to upgrade straight to 7.0.  Perhaps this has been the case for every
>> major release in the past, and it would be nice if it continues if for no
>> other reason than consistency.  But, IMO, that's kind of cosmetic -- it
>> isn't important.  What matters is that an eventual 6.x release occurs that
>> allows someone to upgrade to 7.0 -- that there's a path forward.  And that
>> one can always upgrade from one 6.x release to any greater 6.x release.
>>
>> Quoting Adrien:
>> bq. Detour: In the future I wonder that we should consider having separate
>> release cycles again. In addition to giving Solr more time t

Re: 6.0 Release

[Shawn Heisey]

On 3/31/2016 10:02 AM, Ishan Chattopadhyaya wrote:
> > What bothers me is that the 6.0 user will have to change their
> > schema and completely reindex, even if they upgrade to a newer 6.x
> > version before going to 7.0.
>
> AFAICT, I think we can continue to have Trie* fields well into 7.x (by
> pulling the Legacy numerics into Solr at 7.0 release), so as not to
> force users to change schema or reindex if they don't want to.
> Additional Points* fields would be there (at some 6.x onwards) for
> whoever wants performance benefits, and we can make that as the
> default in a 7.0 schema.

Moving the legacy numerics into Solr is not my ideal solution to the
6.0->7.0 compatibility issue, but there isn't much of a downside, so I'm
willing to work with it.  I also think it's the solution least likely to
encounter opposition.  Should I start a VOTE thread so Solr can make a
formal decision?

If moving classes is the plan, I believe we should do this in master at
the same time we implement points-based field classes in 6.x.  They
should remain deprecated, to be completely removed in 8.0.

IMHO, switching to new points-based field classes in example schemas
should happen in a 6.x release, but it should not happen in the same
release that introduces the new classes.  Waiting at least two minor
releases seems prudent.  We need time to battle-test them before we make
them default.  I do think that they should be tested further before 7.0,
which making them default in a later 6.x would accomplish.

On the subject of a VOTE thread, I think there are at least two separate
things to discuss and decide:

* Separating Lucene/Solr releases.
I've thought of a couple of approaches for this, things that need
discussing.

* Solr: deciding how to solve 6.0->7.0 compatibility.

Thanks,
Shawn


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: lucene-solr 6.0 Release Notes

[Cassandra Targett]

I don't have time this moment to edit the Solr release notes, but just
wanted to make a note that the Solr release notes only copy the upgrade
notes from CHANGES.txt and don't mention any new features, so we probably
want to make sure those are updated. It also mentions that the Solr Ref
Guide is available, which it may not be (it's not ready today!).

I'll try to make some edits when I can tomorrow, just wanted to note it
here in case anyone else has time to take a look before I can.

On Thu, Mar 31, 2016 at 2:35 PM, Nicholas Knize  wrote:

> Version 1 of the release notes for lucene and solr are now available:
>
> https://wiki.apache.org/lucene-java/ReleaseNote60
> https://wiki.apache.org/solr/ReleaseNote60
>
> Please review and make edits as necessary.
>
> - Nick Knize
>

lucene-solr 6.0 Release Notes

[Nicholas Knize]

Version 1 of the release notes for lucene and solr are now available:

https://wiki.apache.org/lucene-java/ReleaseNote60
https://wiki.apache.org/solr/ReleaseNote60

Please review and make edits as necessary.

- Nick Knize

Re: 6.0 Release

[Joel Bernstein]

So what's the latest plan on the release?

I just back ported SOLR- to branch_6x. If the release is imminent I'll
hold off on releasing until 6.1. But if this release continues to be
delayed I'd like to backport to branch_6_0.

Joel Bernstein
http://joelsolr.blogspot.com/

On Thu, Mar 31, 2016 at 12:16 PM, Jack Krupansky <jack.krupan...@gmail.com>
wrote:

> Reindexing for the proposed changes in numeric fields...
>
> We in Solr land have this split personality about reindexing - sometimes
> blithely telling users "oh, if you want to make that schema change, then
> you will have to reindex (all of your data)" and then insisting on index
> compatibility and that installing a new release will "not require
> reindexing." When I was at Lucid with their packaged version of Solr,
> automatically reindexing was a single-click operation, so it required nary
> a second thought. But in raw Solr land reindexing is a cause of great
> concern, anxiety, pain, and in many cases outright impossibility. The later
> typically because using store="TRUE" for all of your fields is considered a
> very bad thing.
>
> But what's the story in Elasticsearch land? First, they have this concept
> of a "_source" field which can keep a fully copy of the entire original
> input document, so that any document can always be fully updated and...
> reindexed. They also have a scrolling feature to make it easy to bulk copy
> from one index into another. Again, making it easy to reindex or migrate
> from an old index to a fresh new one.
>
> And now, we read in a recent blog post that "Reindex is coming!", making
> reindexing even easier than ever in ES:
> https://www.elastic.co/blog/reindex-is-coming
>
> In short, reindexing is much less of a huge deal in ES land than it is in
> Solr. IOW, telling users "you must reindex" is not the end of the world for
> ES users.
>
> So, the point of all of this is to ask a question of the Elasticsearch
> guys (also known as Lucene guys): How does Elasticsearch plan on dealing
> with this transition from current numerics to dimensional points? The
> recent blog post is here:
> https://www.elastic.co/blog/lucene-points-6.0
>
> It merely says "As of this writing, Elasticsearch has not yet exposed
> points, but I expect that will change soon." The question I have is will ES
> simply tell users "you must reindex" to use a future release of ES that is
> based on Lucene 6.0, or... will ES offer some index migration tool, or...
> will ES automatically and transparently upgrade existing ES numeric fields
> to dimensional points, or... will ES support both numeric formats, or...
> will ES have some JSON syntax for selecting between the two numeric formats?
>
> Not that I actually expect ES to fully disclose future product plans here,
> but... do they actually have some kind of secret plan to make users fully
> happy with this transition of numeric formats, or do they simply plan to
> say "you must (manually) reindex", or... have they in fact not yet thought
> through these migration issues?
>
> The real point here is that it will be senseless for the Solr guys to work
> through and propose a sensible migration plan (including the decision as to
> when and how the current numerics will be deprecated) if ES staff have some
> hidden plan in play. If it really is simply a matter that ES hasn't thought
> through the migration process or that easy reindexing is the ES answer,
> then fine, but it would be helpful to state that explicitly. I'm not making
> any presumption about which of these scenarios is the truth, just trying to
> figure out what's really going on, and whether easy reindexing in ES is at
> the root of this mad push to deprecate current numerics before dimensional
> points is fully baked.
>
>
> -- Jack Krupansky
>
> On Thu, Mar 31, 2016 at 11:39 AM, Jack Krupansky <jack.krupan...@gmail.com
> > wrote:
>
>> Robert's detailing of the remaining work to get the rest of Lucene off of
>> current (current release, soon to be legacy) numerics is enlightening.
>>
>> Personally, I had thought that it was Solr that was holding up an
>> imminent Lucene/Solr 6.0 release, but now I'm thinking:
>>
>> 1. The new "point stuff" (did I mention that I didn't like or approve of
>> the current name?) seems more like a work in progress...
>> 2. I'd label the "point stuff" as experimental for 6.0.
>> 3. I wouldn't hold up 6.0 for any further baking of the "point stuff" or
>> migration of other Lucene features off current numeric types.
>> 4. The rest of Lucene can be weaned off the current numerics at a more
>> leais

Re: 6.0 Release

[Jack Krupansky]

Reindexing for the proposed changes in numeric fields...

We in Solr land have this split personality about reindexing - sometimes
blithely telling users "oh, if you want to make that schema change, then
you will have to reindex (all of your data)" and then insisting on index
compatibility and that installing a new release will "not require
reindexing." When I was at Lucid with their packaged version of Solr,
automatically reindexing was a single-click operation, so it required nary
a second thought. But in raw Solr land reindexing is a cause of great
concern, anxiety, pain, and in many cases outright impossibility. The later
typically because using store="TRUE" for all of your fields is considered a
very bad thing.

But what's the story in Elasticsearch land? First, they have this concept
of a "_source" field which can keep a fully copy of the entire original
input document, so that any document can always be fully updated and...
reindexed. They also have a scrolling feature to make it easy to bulk copy
from one index into another. Again, making it easy to reindex or migrate
from an old index to a fresh new one.

And now, we read in a recent blog post that "Reindex is coming!", making
reindexing even easier than ever in ES:
https://www.elastic.co/blog/reindex-is-coming

In short, reindexing is much less of a huge deal in ES land than it is in
Solr. IOW, telling users "you must reindex" is not the end of the world for
ES users.

So, the point of all of this is to ask a question of the Elasticsearch guys
(also known as Lucene guys): How does Elasticsearch plan on dealing with
this transition from current numerics to dimensional points? The recent
blog post is here:
https://www.elastic.co/blog/lucene-points-6.0

It merely says "As of this writing, Elasticsearch has not yet exposed
points, but I expect that will change soon." The question I have is will ES
simply tell users "you must reindex" to use a future release of ES that is
based on Lucene 6.0, or... will ES offer some index migration tool, or...
will ES automatically and transparently upgrade existing ES numeric fields
to dimensional points, or... will ES support both numeric formats, or...
will ES have some JSON syntax for selecting between the two numeric formats?

Not that I actually expect ES to fully disclose future product plans here,
but... do they actually have some kind of secret plan to make users fully
happy with this transition of numeric formats, or do they simply plan to
say "you must (manually) reindex", or... have they in fact not yet thought
through these migration issues?

The real point here is that it will be senseless for the Solr guys to work
through and propose a sensible migration plan (including the decision as to
when and how the current numerics will be deprecated) if ES staff have some
hidden plan in play. If it really is simply a matter that ES hasn't thought
through the migration process or that easy reindexing is the ES answer,
then fine, but it would be helpful to state that explicitly. I'm not making
any presumption about which of these scenarios is the truth, just trying to
figure out what's really going on, and whether easy reindexing in ES is at
the root of this mad push to deprecate current numerics before dimensional
points is fully baked.


-- Jack Krupansky

On Thu, Mar 31, 2016 at 11:39 AM, Jack Krupansky <jack.krupan...@gmail.com>
wrote:

> Robert's detailing of the remaining work to get the rest of Lucene off of
> current (current release, soon to be legacy) numerics is enlightening.
>
> Personally, I had thought that it was Solr that was holding up an imminent
> Lucene/Solr 6.0 release, but now I'm thinking:
>
> 1. The new "point stuff" (did I mention that I didn't like or approve of
> the current name?) seems more like a work in progress...
> 2. I'd label the "point stuff" as experimental for 6.0.
> 3. I wouldn't hold up 6.0 for any further baking of the "point stuff" or
> migration of other Lucene features off current numeric types.
> 4. The rest of Lucene can be weaned off the current numerics at a more
> leaisurely pace, like for 6.1 or 6.2.
> 5. Once the new "point stuff" is finally full baked, and the rest of
> Lucene is migrated off current numerics, and... Solr has made "point stuff"
> its default numeric type (6.1 or 6.2?), AND Lucene or Solr comes up with a
> sound migration plan and/or index migration tool for current numerics, only
> THEN should the current numerics become deprecated.
> 6. I'm not absolutely certain, but I think the 6.0 changes to Solr to use
> the Lucene LegacyXXX numeric field types should be fine for an initial 6.0
> release, meaning backcompat is assured for 6.x.
> 7. I'm imagining that with a manually-invoked index upgrade tool a current
> (5.x) numeric fie

Re: 6.0 Release

[Ishan Chattopadhyaya]

> What bothers me is that the 6.0 user will have to change their
> schema and completely reindex, even if they upgrade to a newer 6.x
> version before going to 7.0.

AFAICT, I think we can continue to have Trie* fields well into 7.x (by
pulling the Legacy numerics into Solr at 7.0 release), so as not to force
users to change schema or reindex if they don't want to. Additional Points*
fields would be there (at some 6.x onwards) for whoever wants performance
benefits, and we can make that as the default in a 7.0 schema.


On Thu, Mar 31, 2016 at 9:24 PM, Shawn Heisey  wrote:

> On 3/30/2016 10:31 PM, David Smiley wrote:
> > I disagree with Shawn about #5, that a user with a Solr 6.0 index must
> > be able to upgrade straight to 7.0.  Perhaps this has been the case
> > for every major release in the past, and it would be nice if it
> > continues if for no other reason than consistency.  But, IMO, that's
> > kind of cosmetic -- it isn't important.  What matters is that an
> > eventual 6.x release occurs that allows someone to upgrade to 7.0 --
> > that there's a path forward.  And that one can always upgrade from one
> > 6.x release to any greater 6.x release.
>
> I'm not bothered by the idea that somebody might need to do a two-stage
> upgrade.  What bothers me is that the 6.0 user will have to change their
> schema and completely reindex, even if they upgrade to a newer 6.x
> version before going to 7.0.
>
> For me, this isn't really a worry, because I always reindex when
> upgrading.  We've got at least one user with a 10TB SolrCloud index
> containing billions of documents.  Reindexing on that scale might take a
> few months.
>
> > Quoting Adrien:
> > bq. Detour: In the future I wonder that we should consider having
> > separate release cycles again. In addition to giving Solr more time to
> > use new Lucene features like here, it would also remove the issue that
> > we had when releasing 5.3.2 after 5.4.0, which makes perfectly sense
> > from a Solr perspective but not from Lucene since it introduces blind
> > spots in the testing of index backward compatibility.
> >
> > +1 to that!  I've had that thought.  It would be awesome for Solr to
> > release when it feels it's right, independently of Lucene.  If that's
> > too difficult/problematic then perhaps keep synchronizing releases but
> > allow Lucene & Solr's release version to vary.Then we'd be having
> > a Solr 5.6 release here.
>
> I'm ambivalent about a separate release cycle.  I like the synchronized
> releases, but I can also see the advantages to separating them.  It
> would help with LUCENE-5589 and friends.
>
> Thanks,
> Shawn
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: 6.0 Release

[Shawn Heisey]

On 3/30/2016 10:31 PM, David Smiley wrote:
> I disagree with Shawn about #5, that a user with a Solr 6.0 index must
> be able to upgrade straight to 7.0.  Perhaps this has been the case
> for every major release in the past, and it would be nice if it
> continues if for no other reason than consistency.  But, IMO, that's
> kind of cosmetic -- it isn't important.  What matters is that an
> eventual 6.x release occurs that allows someone to upgrade to 7.0 --
> that there's a path forward.  And that one can always upgrade from one
> 6.x release to any greater 6.x release.

I'm not bothered by the idea that somebody might need to do a two-stage
upgrade.  What bothers me is that the 6.0 user will have to change their
schema and completely reindex, even if they upgrade to a newer 6.x
version before going to 7.0.

For me, this isn't really a worry, because I always reindex when
upgrading.  We've got at least one user with a 10TB SolrCloud index
containing billions of documents.  Reindexing on that scale might take a
few months.

> Quoting Adrien:
> bq. Detour: In the future I wonder that we should consider having
> separate release cycles again. In addition to giving Solr more time to
> use new Lucene features like here, it would also remove the issue that
> we had when releasing 5.3.2 after 5.4.0, which makes perfectly sense
> from a Solr perspective but not from Lucene since it introduces blind
> spots in the testing of index backward compatibility.
>
> +1 to that!  I've had that thought.  It would be awesome for Solr to
> release when it feels it's right, independently of Lucene.  If that's
> too difficult/problematic then perhaps keep synchronizing releases but
> allow Lucene & Solr's release version to vary.Then we'd be having
> a Solr 5.6 release here.

I'm ambivalent about a separate release cycle.  I like the synchronized
releases, but I can also see the advantages to separating them.  It
would help with LUCENE-5589 and friends.

Thanks,
Shawn


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: 6.0 Release

[Jack Krupansky]

Robert's detailing of the remaining work to get the rest of Lucene off of
current (current release, soon to be legacy) numerics is enlightening.

Personally, I had thought that it was Solr that was holding up an imminent
Lucene/Solr 6.0 release, but now I'm thinking:

1. The new "point stuff" (did I mention that I didn't like or approve of
the current name?) seems more like a work in progress...
2. I'd label the "point stuff" as experimental for 6.0.
3. I wouldn't hold up 6.0 for any further baking of the "point stuff" or
migration of other Lucene features off current numeric types.
4. The rest of Lucene can be weaned off the current numerics at a more
leaisurely pace, like for 6.1 or 6.2.
5. Once the new "point stuff" is finally full baked, and the rest of Lucene
is migrated off current numerics, and... Solr has made "point stuff" its
default numeric type (6.1 or 6.2?), AND Lucene or Solr comes up with a
sound migration plan and/or index migration tool for current numerics, only
THEN should the current numerics become deprecated.
6. I'm not absolutely certain, but I think the 6.0 changes to Solr to use
the Lucene LegacyXXX numeric field types should be fine for an initial 6.0
release, meaning backcompat is assured for 6.x.
7. I'm imagining that with a manually-invoked index upgrade tool a current
(5.x) numeric field can be migrated to a "point stuff" field type. A Lucene
heavy will have to confirm that feasibility.
8. I'm imagining that a typical Solr site would be okay with the
requirement that they have to explicitly, manually run such an index
upgrade tool to migrate from current (5.x) numerics to "point stuff". And
that they could either do that once Solr adds support for "point stuff"
fields or when they migrate from 6.x to 7.x. Bonus points if Solr can have
a variation of the index upgrade tool that discovers and upgrades all
current numeric fields.

What else? (I'll ask some questions about Elasticsearch plans in a separate
message.)




-- Jack Krupansky

On Thu, Mar 31, 2016 at 12:31 AM, David Smiley <david.w.smi...@gmail.com>
wrote:

> That was an excellent summary Rob; thanks.
> Minor nit: BBoxSpatialStrategy isn't/wasn't deprecated.  It was enhanced
> to use PointValues.
>
> I too would like to see the legacy numerics stay in "backwards-codecs" as
> you describe with precisionStep specified on the Analyzer.
>
> I disagree with Shawn about #5, that a user with a Solr 6.0 index must be
> able to upgrade straight to 7.0.  Perhaps this has been the case for every
> major release in the past, and it would be nice if it continues if for no
> other reason than consistency.  But, IMO, that's kind of cosmetic -- it
> isn't important.  What matters is that an eventual 6.x release occurs that
> allows someone to upgrade to 7.0 -- that there's a path forward.  And that
> one can always upgrade from one 6.x release to any greater 6.x release.
>
> Quoting Adrien:
> bq. Detour: In the future I wonder that we should consider having separate
> release cycles again. In addition to giving Solr more time to use new
> Lucene features like here, it would also remove the issue that we had when
> releasing 5.3.2 after 5.4.0, which makes perfectly sense from a Solr
> perspective but not from Lucene since it introduces blind spots in the
> testing of index backward compatibility.
>
> +1 to that!  I've had that thought.  It would be awesome for Solr to
> release when it feels it's right, independently of Lucene.  If that's too
> difficult/problematic then perhaps keep synchronizing releases but allow
> Lucene & Solr's release version to vary.Then we'd be having a Solr 5.6
> release here.
>
> ~ David
>
> On Wed, Mar 30, 2016 at 9:39 PM Robert Muir <rcm...@gmail.com> wrote:
>
>> On Wed, Mar 30, 2016 at 12:43 PM, Adrien Grand <jpou...@gmail.com> wrote:
>> > Hi Shawn,
>> >
>> > I think marking the legacy fields/queries as deprecated in Lucene in
>> 6.0 is
>> > the right thing to do in order to encourage users to migrate to the new
>> > points API. If Solr needs to keep them around for 7.x, it would be fine
>> to
>> > move them to solr/ instead of lucene/ instead of a hard removal. Given
>> that
>> > it works on top of the postings API, it would not break.
>>
>> Also see my issue (https://issues.apache.org/jira/browse/LUCENE-7075)
>> where I proposed to at least get things headed to the backwards/ jar.
>> And the uninverting issue is still being discussed. If you look at
>> linked issues you will see the deprecated encoding is involved with
>> the following modules:
>>
>> * core (not just field/query/utils classes, but stuff like
>> precisionStep in the .document api!)

Re: 6.0 Release

[David Smiley]

That was an excellent summary Rob; thanks.
Minor nit: BBoxSpatialStrategy isn't/wasn't deprecated.  It was enhanced to
use PointValues.

I too would like to see the legacy numerics stay in "backwards-codecs" as
you describe with precisionStep specified on the Analyzer.

I disagree with Shawn about #5, that a user with a Solr 6.0 index must be
able to upgrade straight to 7.0.  Perhaps this has been the case for every
major release in the past, and it would be nice if it continues if for no
other reason than consistency.  But, IMO, that's kind of cosmetic -- it
isn't important.  What matters is that an eventual 6.x release occurs that
allows someone to upgrade to 7.0 -- that there's a path forward.  And that
one can always upgrade from one 6.x release to any greater 6.x release.

Quoting Adrien:
bq. Detour: In the future I wonder that we should consider having separate
release cycles again. In addition to giving Solr more time to use new
Lucene features like here, it would also remove the issue that we had when
releasing 5.3.2 after 5.4.0, which makes perfectly sense from a Solr
perspective but not from Lucene since it introduces blind spots in the
testing of index backward compatibility.

+1 to that!  I've had that thought.  It would be awesome for Solr to
release when it feels it's right, independently of Lucene.  If that's too
difficult/problematic then perhaps keep synchronizing releases but allow
Lucene & Solr's release version to vary.Then we'd be having a Solr 5.6
release here.

~ David

On Wed, Mar 30, 2016 at 9:39 PM Robert Muir  wrote:

> On Wed, Mar 30, 2016 at 12:43 PM, Adrien Grand  wrote:
> > Hi Shawn,
> >
> > I think marking the legacy fields/queries as deprecated in Lucene in 6.0
> is
> > the right thing to do in order to encourage users to migrate to the new
> > points API. If Solr needs to keep them around for 7.x, it would be fine
> to
> > move them to solr/ instead of lucene/ instead of a hard removal. Given
> that
> > it works on top of the postings API, it would not break.
>
> Also see my issue (https://issues.apache.org/jira/browse/LUCENE-7075)
> where I proposed to at least get things headed to the backwards/ jar.
> And the uninverting issue is still being discussed. If you look at
> linked issues you will see the deprecated encoding is involved with
> the following modules:
>
> * core (not just field/query/utils classes, but stuff like
> precisionStep in the .document api!)
> * spatial (Deprecated GeoPoint encoding etc)
> * spatial-extras (Deprecated Bbox encoding etc)
> * misc (UninvertingReader)
> * queryparser (flexible and xml)
> * join
>
> The purpose of that issue is to make sure people have the stuff they
> need to move their code of the old encoding. I personally thought this
> would make the transition easier, and it was finding bugs/problems in
> points and improving the apis. I imagined it would just be me, but i
> created a ton of linked issues all up front just in case. I did not
> think anyone else would really be excited to work on these, because
> its not particularly exciting stuff, but thanks Nick, David, Martijn,
> etc who did. I didn't try to plan any grandiose schemes of *actually
> pulling the old encoding out* because this was plenty on its own. I
> tried to work on the fieldcache only because I was talking to Tomas
> and he mentioned it as a difficulty in cutting over solr. But I bailed
> after encountering complexity, and don't think it is the way to go,
> read the issue for my explanation.
>
> To me, this is why we have a backwards compatibility policy for N-1,
> it has to be a volunteer thing for some of this stuff: can't all be on
> Mike.
>
> I do personally think it is enough to release, "removing" or "moving"
> deprecations is something to worry about for master branch.
>
> I did mention in the issue an idea for a first step would be to get
> the core/ stuff pulled out somewhere better.  Maybe the core/ stuff
> should go to the backwards-codec jar if we can detangle the
> deprecations from the .document api (e.g. maybe precisionStep can be a
> parameter on a tokenizer or analyzer or something, so its a little bit
> harder to use, but still works and not holding back core/'s .document
> api). But what to do about the other stuff?
>
> If i wanted to start removing deprecations now, I would be trying to
> just factor out the core/ NumericRangeQuery/NumericField stuff out to
> the backwards-codec jar. I hate modules depending on other ones, I
> really do, but just to iterate, I'd temporarily make all those other
> modules depend on backwards-codec/ jar and then remove deprecations
> from each one-by-one. Its too much to do all at once. I think we can
> do it this way iteratively without breaking solr.
>
> If solr wants to hang on to e.g. some spatial field with old numerics
> for an additional time (since it was still using it for 6.0), then the
> deprecated spatial field can be moved to solr. If not, lets nuke it.
>

Re: 6.0 Release

[Robert Muir]

On Wed, Mar 30, 2016 at 12:43 PM, Adrien Grand  wrote:
> Hi Shawn,
>
> I think marking the legacy fields/queries as deprecated in Lucene in 6.0 is
> the right thing to do in order to encourage users to migrate to the new
> points API. If Solr needs to keep them around for 7.x, it would be fine to
> move them to solr/ instead of lucene/ instead of a hard removal. Given that
> it works on top of the postings API, it would not break.

Also see my issue (https://issues.apache.org/jira/browse/LUCENE-7075)
where I proposed to at least get things headed to the backwards/ jar.
And the uninverting issue is still being discussed. If you look at
linked issues you will see the deprecated encoding is involved with
the following modules:

* core (not just field/query/utils classes, but stuff like
precisionStep in the .document api!)
* spatial (Deprecated GeoPoint encoding etc)
* spatial-extras (Deprecated Bbox encoding etc)
* misc (UninvertingReader)
* queryparser (flexible and xml)
* join

The purpose of that issue is to make sure people have the stuff they
need to move their code of the old encoding. I personally thought this
would make the transition easier, and it was finding bugs/problems in
points and improving the apis. I imagined it would just be me, but i
created a ton of linked issues all up front just in case. I did not
think anyone else would really be excited to work on these, because
its not particularly exciting stuff, but thanks Nick, David, Martijn,
etc who did. I didn't try to plan any grandiose schemes of *actually
pulling the old encoding out* because this was plenty on its own. I
tried to work on the fieldcache only because I was talking to Tomas
and he mentioned it as a difficulty in cutting over solr. But I bailed
after encountering complexity, and don't think it is the way to go,
read the issue for my explanation.

To me, this is why we have a backwards compatibility policy for N-1,
it has to be a volunteer thing for some of this stuff: can't all be on
Mike.

I do personally think it is enough to release, "removing" or "moving"
deprecations is something to worry about for master branch.

I did mention in the issue an idea for a first step would be to get
the core/ stuff pulled out somewhere better.  Maybe the core/ stuff
should go to the backwards-codec jar if we can detangle the
deprecations from the .document api (e.g. maybe precisionStep can be a
parameter on a tokenizer or analyzer or something, so its a little bit
harder to use, but still works and not holding back core/'s .document
api). But what to do about the other stuff?

If i wanted to start removing deprecations now, I would be trying to
just factor out the core/ NumericRangeQuery/NumericField stuff out to
the backwards-codec jar. I hate modules depending on other ones, I
really do, but just to iterate, I'd temporarily make all those other
modules depend on backwards-codec/ jar and then remove deprecations
from each one-by-one. Its too much to do all at once. I think we can
do it this way iteratively without breaking solr.

If solr wants to hang on to e.g. some spatial field with old numerics
for an additional time (since it was still using it for 6.0), then the
deprecated spatial field can be moved to solr. If not, lets nuke it.

To me this seems the least controversial path, and its something that
can be done iteratively. It has the downside of keeping "core"
deprecated legacy numerics around for an extra major release in the
backwards-codec jar. I think this "extra" back compat is ok in this
case. Uwe made clean code :)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: 6.0 Release

[Jack Krupansky]

Adrien, sure, Solr could take ownership of the so-called legacy (trie)
numerics, but then what would Elasticsearch do for its users when ES
upgrades to Lucene 7.0 which would then no longer have the code to handle
existing Trie-based indexes? Would ES then start depending on Solr?! Or is
ES planning on somehow automatically migrating ES 2.x indexes from Trie
numeric fields to PointValues?

Is ES going to do its own Trie to PV converter?

Will Solr also need to do its own Trie to PV converter for 7.0?

It does seem very odd to me that the Lucene guys have made this switch to
PV without any migration tool for existing Trie indexes.

-- Jack Krupansky

On Wed, Mar 30, 2016 at 6:56 PM, Adrien Grand  wrote:

> Le mer. 30 mars 2016 à 22:56, Shawn Heisey  a écrit :
>
>> These are the choices I see to address the problem, in decreasing order
>> of personal preference:
>>
>> 1) Revert LUCENE-6917 in the 6.x versions, move the deprecation to master.
>> 2) Delay the Lucene/Solr 6.0.0 release so Solr has new field classes and
>> updated examples.
>> 3) Keep to the schedule for the Lucene 6.0.0 release, but do NOT release
>> Solr 6.0.0.  Do a synchronized Lucene/Solr release of 6.0.1 or 6.1.0
>> with new Solr classes and examples.
>> 4) Move the deprecated Lucene classes to the Solr 7.0 package space
>> (still deprecated) as suggested by Adrien.  Fully remove them in 8.0.
>> 5) Compromise Solr's historical guarantees of major version backward
>> compatibility.
>>
>
> I am confused why you put 1 before 4: to me they are the same from a Solr
> perspective, and 4 is better than 1 from a Lucene perspective since it
> makes the path forward clearer?
>
> I think the only reasonable alternative to 4 is 2, which like you said
> would be disappointing. I don't think anybody wants 5, and 3 feels awkward
> to me. Detour: In the future I wonder that we should consider having
> separate release cycles again. In addition to giving Solr more time to use
> new Lucene features like here, it would also remove the issue that we had
> when releasing 5.3.2 after 5.4.0, which makes perfectly sense from a Solr
> perspective but not from Lucene since it introduces blind spots in the
> testing of index backward compatibility.
>
> Back to the current issue, my preference would go for 4. I could be wrong
> but I think it is also consistent with the fact that Solr historically kept
> compatibility for a longer time than Lucene (eg. by still supporting
> IntField or allowing uninverting out of the box).
>

Re: 6.0 Release

[Adrien Grand]

Le mer. 30 mars 2016 à 22:56, Shawn Heisey  a écrit :

> These are the choices I see to address the problem, in decreasing order
> of personal preference:
>
> 1) Revert LUCENE-6917 in the 6.x versions, move the deprecation to master.
> 2) Delay the Lucene/Solr 6.0.0 release so Solr has new field classes and
> updated examples.
> 3) Keep to the schedule for the Lucene 6.0.0 release, but do NOT release
> Solr 6.0.0.  Do a synchronized Lucene/Solr release of 6.0.1 or 6.1.0
> with new Solr classes and examples.
> 4) Move the deprecated Lucene classes to the Solr 7.0 package space
> (still deprecated) as suggested by Adrien.  Fully remove them in 8.0.
> 5) Compromise Solr's historical guarantees of major version backward
> compatibility.
>

I am confused why you put 1 before 4: to me they are the same from a Solr
perspective, and 4 is better than 1 from a Lucene perspective since it
makes the path forward clearer?

I think the only reasonable alternative to 4 is 2, which like you said
would be disappointing. I don't think anybody wants 5, and 3 feels awkward
to me. Detour: In the future I wonder that we should consider having
separate release cycles again. In addition to giving Solr more time to use
new Lucene features like here, it would also remove the issue that we had
when releasing 5.3.2 after 5.4.0, which makes perfectly sense from a Solr
perspective but not from Lucene since it introduces blind spots in the
testing of index backward compatibility.

Back to the current issue, my preference would go for 4. I could be wrong
but I think it is also consistent with the fact that Solr historically kept
compatibility for a longer time than Lucene (eg. by still supporting
IntField or allowing uninverting out of the box).

Re: 6.0 Release

[Jack Krupansky]

Does Solr need its own MIGRATE.txt file? Maybe not for 6.0, but probably
for 6.x when any Solr field type changes occur.

Right now, MIGRATE.txt belongs to the lucene package:
https://github.com/apache/lucene-solr/blob/branch_6_0/lucene/MIGRATE.txt

The migration comments related to numeric fields don't really say anything
related to Solr users directly:

"# PointValues replaces NumericField (LUCENE-6917)

PointValues provides faster indexing and searching, a smaller
index size, and less heap used at search time. See
org.apache.lucene.index.PointValues
for an introduction.

Legacy numeric encodings from previous versions of Lucene are
deprecated as LegacyIntField, LegacyFloatField, LegacyLongField, and
LegacyDoubleField,
and can be searched with LegacyNumericRangeQuery."

I mean, Solr users deal with solr.TrieIntField, et al, which are Solr
classes that internally map to Lucene field classes.

Since the 6.0 Solr code has already been "migrated" to use the
LegacyTrieXxxField types, which should be fully compatible with 5.x, Solr
should initially be fine with 6.0.

See the createField method of Solr's TrieField:
https://github.com/apache/lucene-solr/blob/branch_6_0/solr/core/src/java/org/apache/solr/schema/TrieField.java


-- Jack Krupansky

On Wed, Mar 30, 2016 at 4:56 PM, Shawn Heisey  wrote:

> On 3/30/2016 10:43 AM, Adrien Grand wrote:
> > I think marking the legacy fields/queries as deprecated in Lucene in
> > 6.0 is the right thing to do in order to encourage users to migrate to
> > the new points API. If Solr needs to keep them around for 7.x, it
> > would be fine to move them to solr/ instead of lucene/ instead of a
> > hard removal. Given that it works on top of the postings API, it would
> > not break.
>
> Encouraging users to migrate to better APIs is a great idea.
>
> I'm sure we'll update Solr's examples to use new classes that leverage
> the points API, but at the moment, we do not have time before the 6.0.0
> release to do this.  This puts Solr in an awkward position.
>
> These are the choices I see to address the problem, in decreasing order
> of personal preference:
>
> 1) Revert LUCENE-6917 in the 6.x versions, move the deprecation to master.
> 2) Delay the Lucene/Solr 6.0.0 release so Solr has new field classes and
> updated examples.
> 3) Keep to the schedule for the Lucene 6.0.0 release, but do NOT release
> Solr 6.0.0.  Do a synchronized Lucene/Solr release of 6.0.1 or 6.1.0
> with new Solr classes and examples.
> 4) Move the deprecated Lucene classes to the Solr 7.0 package space
> (still deprecated) as suggested by Adrien.  Fully remove them in 8.0.
> 5) Compromise Solr's historical guarantees of major version backward
> compatibility.
>
> Option 1 should not be a major hardship for Lucene.  Users can still use
> the new API, and such use can be encouraged by examples, documentation,
> and community activity.  Version 7.0 would provide further
> encouragement, and I don't see any reason we can't work towards a quick
> 7.0 release and a relatively short lifetime for 6.x.  5.x had a much
> shorter lifetime than 4.x did.
>
> Option 2 would be disappointing for everyone, and even though 6.0 would
> probably STILL be the fastest major release in recent project history
> even with this delay, I am expecting significant pressure against it.
>
> Option 3 would be a major disappointment for Solr, but I think it would
> be better for the integrity of the project than our current trajectory.
>
> Option 4 is more than a little ugly, which is why it's in 4th place.
> Assuming the postings API would work correctly after the package move,
> it would fix the problem.
>
> I do not like option 5 at all.
>
> This community works by consensus, so my personal preference on the
> options above is not worth very much.
>
> Thanks,
> Shawn
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: 6.0 Release

[Shawn Heisey]

On 3/30/2016 10:43 AM, Adrien Grand wrote:
> I think marking the legacy fields/queries as deprecated in Lucene in
> 6.0 is the right thing to do in order to encourage users to migrate to
> the new points API. If Solr needs to keep them around for 7.x, it
> would be fine to move them to solr/ instead of lucene/ instead of a
> hard removal. Given that it works on top of the postings API, it would
> not break.

Encouraging users to migrate to better APIs is a great idea.

I'm sure we'll update Solr's examples to use new classes that leverage
the points API, but at the moment, we do not have time before the 6.0.0
release to do this.  This puts Solr in an awkward position.

These are the choices I see to address the problem, in decreasing order
of personal preference:

1) Revert LUCENE-6917 in the 6.x versions, move the deprecation to master.
2) Delay the Lucene/Solr 6.0.0 release so Solr has new field classes and
updated examples.
3) Keep to the schedule for the Lucene 6.0.0 release, but do NOT release
Solr 6.0.0.  Do a synchronized Lucene/Solr release of 6.0.1 or 6.1.0
with new Solr classes and examples.
4) Move the deprecated Lucene classes to the Solr 7.0 package space
(still deprecated) as suggested by Adrien.  Fully remove them in 8.0.
5) Compromise Solr's historical guarantees of major version backward
compatibility.

Option 1 should not be a major hardship for Lucene.  Users can still use
the new API, and such use can be encouraged by examples, documentation,
and community activity.  Version 7.0 would provide further
encouragement, and I don't see any reason we can't work towards a quick
7.0 release and a relatively short lifetime for 6.x.  5.x had a much
shorter lifetime than 4.x did.

Option 2 would be disappointing for everyone, and even though 6.0 would
probably STILL be the fastest major release in recent project history
even with this delay, I am expecting significant pressure against it.

Option 3 would be a major disappointment for Solr, but I think it would
be better for the integrity of the project than our current trajectory.

Option 4 is more than a little ugly, which is why it's in 4th place. 
Assuming the postings API would work correctly after the package move,
it would fix the problem.

I do not like option 5 at all.

This community works by consensus, so my personal preference on the
options above is not worth very much.

Thanks,
Shawn


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: 6.0 Release

[Adrien Grand]

Hi Shawn,

I think marking the legacy fields/queries as deprecated in Lucene in 6.0 is
the right thing to do in order to encourage users to migrate to the new
points API. If Solr needs to keep them around for 7.x, it would be fine to
move them to solr/ instead of lucene/ instead of a hard removal. Given that
it works on top of the postings API, it would not break.

Le mer. 30 mars 2016 à 17:25, Jack Krupansky  a
écrit :

> (And of course we're going to pretend that the
> PointValues/DimensionalPoints/DimensionalValues/Numeric/PointField
> backcompat issue doesn't exist -
> https://issues.apache.org/jira/browse/SOLR-8396. Sigh. But... nothing
> that holds up 6.0 per se. It just would have been better to get that tangle
> straightened out before a release that deprecates the existing numeric
> field types that Solr currently has no replacement for, let alone a
> backcompat story for new Solr indexes that will be be created with 6.0 but
> with a type that won't exist in 7.0, at least with the current plan.)
>
> -- Jack Krupansky
>
> On Wed, Mar 30, 2016 at 11:12 AM, Erick Erickson 
> wrote:
>
>> Also, SOLR-8812 needs to be resolved and SOLR-8725 I think. For those
>> two I want to be sure we make a conscious decision to change or not
>> for 6.0, so consider the "blocker" status a bit tentative as far as
>> I'm concerned.
>>
>> Erick
>>
>> On Wed, Mar 30, 2016 at 8:06 AM, David Smiley 
>> wrote:
>> > FYI Steve Rowe is in-progress reviewing my patches to SOLR-8903 &
>> SOLR-8904
>> > (related to date-time parsing/formatting) that really ought to be done
>> in a
>> > major release.
>> >
>> > And there's LUCENE-7094 that you of course know about... I'm going to
>> send
>> > an updated patch to consider Rob's input now.
>> >
>> > ~ David
>> >
>> > On Wed, Mar 30, 2016 at 10:49 AM Nicholas Knize 
>> wrote:
>> >>
>> >> It appears all 6.0 blocker issues are coming to a close. If there are
>> no
>> >> objections I will begin the RC process today.
>> >
>> > --
>> > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
>> > LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
>> > http://www.solrenterprisesearchserver.com
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>
>

Re: 6.0 Release

[Shawn Heisey]

On 3/30/2016 8:49 AM, Nicholas Knize wrote:
> It appears all 6.0 blocker issues are coming to a close. If there are
> no objections I will begin the RC process today.

I debated with myself about whether or not to even send this message. 
Jack Krupansky mentioned it, so I'm going ahead.

I think that LUCENE-6917 should be reverted in branch_6x and branch_6_0,
but left in place in master.  Alternately, we could delay the removal of
the legacy types until 8.0.

I'm not trying to suggest that the dimensional types aren't good, or
that the older types shouldn't be removed eventually.  I'm just saying
that we need to push it back one major release -- allow both sets of
classes to exist as peers in 6.x, with the older classes deprecated in
7.0and removed in 8.0.

Side issue: If we do revert in 6x, the class rename has potential to
cause pain for backports from master to stable.  I'm not sure a rename
is necessary -- isn't deprecation enough?

This is of particular importance for Solr.  As things stand right now,
Solr 6.0 will ship with all its numeric types using deprecated Lucene
code.  This means that a new user who starts with an example config from
Solr 6.0 will not be able to upgrade to Solr 7.0 without big config
changes and a complete reindex, because the code required to read their
index data will be gone.

Thanks,
Shawn


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: 6.0 Release

[Jack Krupansky]

(And of course we're going to pretend that the
PointValues/DimensionalPoints/DimensionalValues/Numeric/PointField
backcompat issue doesn't exist -
https://issues.apache.org/jira/browse/SOLR-8396. Sigh. But... nothing that
holds up 6.0 per se. It just would have been better to get that tangle
straightened out before a release that deprecates the existing numeric
field types that Solr currently has no replacement for, let alone a
backcompat story for new Solr indexes that will be be created with 6.0 but
with a type that won't exist in 7.0, at least with the current plan.)

-- Jack Krupansky

On Wed, Mar 30, 2016 at 11:12 AM, Erick Erickson 
wrote:

> Also, SOLR-8812 needs to be resolved and SOLR-8725 I think. For those
> two I want to be sure we make a conscious decision to change or not
> for 6.0, so consider the "blocker" status a bit tentative as far as
> I'm concerned.
>
> Erick
>
> On Wed, Mar 30, 2016 at 8:06 AM, David Smiley 
> wrote:
> > FYI Steve Rowe is in-progress reviewing my patches to SOLR-8903 &
> SOLR-8904
> > (related to date-time parsing/formatting) that really ought to be done
> in a
> > major release.
> >
> > And there's LUCENE-7094 that you of course know about... I'm going to
> send
> > an updated patch to consider Rob's input now.
> >
> > ~ David
> >
> > On Wed, Mar 30, 2016 at 10:49 AM Nicholas Knize 
> wrote:
> >>
> >> It appears all 6.0 blocker issues are coming to a close. If there are no
> >> objections I will begin the RC process today.
> >
> > --
> > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> > LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
> > http://www.solrenterprisesearchserver.com
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: 6.0 Release

[Erick Erickson]

Also, SOLR-8812 needs to be resolved and SOLR-8725 I think. For those
two I want to be sure we make a conscious decision to change or not
for 6.0, so consider the "blocker" status a bit tentative as far as
I'm concerned.

Erick

On Wed, Mar 30, 2016 at 8:06 AM, David Smiley  wrote:
> FYI Steve Rowe is in-progress reviewing my patches to SOLR-8903 & SOLR-8904
> (related to date-time parsing/formatting) that really ought to be done in a
> major release.
>
> And there's LUCENE-7094 that you of course know about... I'm going to send
> an updated patch to consider Rob's input now.
>
> ~ David
>
> On Wed, Mar 30, 2016 at 10:49 AM Nicholas Knize  wrote:
>>
>> It appears all 6.0 blocker issues are coming to a close. If there are no
>> objections I will begin the RC process today.
>
> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
> http://www.solrenterprisesearchserver.com

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: 6.0 Release

[David Smiley]

FYI Steve Rowe is in-progress reviewing my patches to SOLR-8903 & SOLR-8904
(related to date-time parsing/formatting) that really ought to be done in a
major release.

And there's LUCENE-7094 that you of course know about... I'm going to send
an updated patch to consider Rob's input now.

~ David

On Wed, Mar 30, 2016 at 10:49 AM Nicholas Knize  wrote:

> It appears all 6.0 blocker issues are coming to a close. If there are no
> objections I will begin the RC process today.
>
-- 
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com

6.0 Release

[Nicholas Knize]

It appears all 6.0 blocker issues are coming to a close. If there are no
objections I will begin the RC process today.

[jira] [Resolved] (SOLR-8764) Remove all deprecated methods and classes from master prior to the 6.0 release

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe resolved SOLR-8764.
--
Resolution: Fixed

> Remove all deprecated methods and classes from master prior to the 6.0 release
> --
>
> Key: SOLR-8764
> URL: https://issues.apache.org/jira/browse/SOLR-8764
> Project: Solr
>  Issue Type: Task
>Reporter: Steve Rowe
>Assignee: Steve Rowe
> Fix For: 6.0
>
> Attachments: SOLR-8764.patch, SOLR-8764.patch
>
>
> Code marked as deprecated with {{@Deprecated}} and/or {{@deprecated}} should 
> be removed from master, unless it's being used internally, or the annotations 
> are there as markers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8764) Remove all deprecated methods and classes from master prior to the 6.0 release

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177973#comment-15177973
 ] 

Steve Rowe commented on SOLR-8764:
--

These changes exposed a bug in Solr's CSV handling code - see SOLR-8778.

> Remove all deprecated methods and classes from master prior to the 6.0 release
> --
>
> Key: SOLR-8764
> URL: https://issues.apache.org/jira/browse/SOLR-8764
> Project: Solr
>  Issue Type: Task
>Reporter: Steve Rowe
>Assignee: Steve Rowe
> Fix For: 6.0
>
> Attachments: SOLR-8764.patch, SOLR-8764.patch
>
>
> Code marked as deprecated with {{@Deprecated}} and/or {{@deprecated}} should 
> be removed from master, unless it's being used internally, or the annotations 
> are there as markers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8764) Remove all deprecated methods and classes from master prior to the 6.0 release

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177946#comment-15177946
 ] 

ASF subversion and git services commented on SOLR-8764:
---

Commit 84d2bb604207e6f07914fd37da5337b867c0cee9 in lucene-solr's branch 
refs/heads/branch_6_0 from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=84d2bb6 ]

SOLR-8764: test schema-latest.xml spatial dist units should be kilometers
(no test uses yet?)
(cherry picked from commit deb6a49)
(cherry picked from commit 6dcb01c)


> Remove all deprecated methods and classes from master prior to the 6.0 release
> --
>
> Key: SOLR-8764
> URL: https://issues.apache.org/jira/browse/SOLR-8764
> Project: Solr
>  Issue Type: Task
>Reporter: Steve Rowe
>Assignee: Steve Rowe
> Fix For: 6.0
>
> Attachments: SOLR-8764.patch, SOLR-8764.patch
>
>
> Code marked as deprecated with {{@Deprecated}} and/or {{@deprecated}} should 
> be removed from master, unless it's being used internally, or the annotations 
> are there as markers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8764) Remove all deprecated methods and classes from master prior to the 6.0 release

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177129#comment-15177129
 ] 

ASF subversion and git services commented on SOLR-8764:
---

Commit 6dcb01ccb4ec77a1cf2fa89155335ead689535c1 in lucene-solr's branch 
refs/heads/branch_6x from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6dcb01c ]

SOLR-8764: test schema-latest.xml spatial dist units should be kilometers
(no test uses yet?)
(cherry picked from commit deb6a49)


> Remove all deprecated methods and classes from master prior to the 6.0 release
> --
>
> Key: SOLR-8764
> URL: https://issues.apache.org/jira/browse/SOLR-8764
> Project: Solr
>  Issue Type: Task
>Reporter: Steve Rowe
>Assignee: Steve Rowe
> Fix For: 6.0
>
> Attachments: SOLR-8764.patch, SOLR-8764.patch
>
>
> Code marked as deprecated with {{@Deprecated}} and/or {{@deprecated}} should 
> be removed from master, unless it's being used internally, or the annotations 
> are there as markers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8764) Remove all deprecated methods and classes from master prior to the 6.0 release

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15177124#comment-15177124
 ] 

ASF subversion and git services commented on SOLR-8764:
---

Commit deb6a4907b19b6756aa83cf149564e1fd05fc34a in lucene-solr's branch 
refs/heads/master from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=deb6a49 ]

SOLR-8764: test schema-latest.xml spatial dist units should be kilometers
(no test uses yet?)


> Remove all deprecated methods and classes from master prior to the 6.0 release
> --
>
> Key: SOLR-8764
> URL: https://issues.apache.org/jira/browse/SOLR-8764
> Project: Solr
>  Issue Type: Task
>Reporter: Steve Rowe
>Assignee: Steve Rowe
> Fix For: 6.0
>
> Attachments: SOLR-8764.patch, SOLR-8764.patch
>
>
> Code marked as deprecated with {{@Deprecated}} and/or {{@deprecated}} should 
> be removed from master, unless it's being used internally, or the annotations 
> are there as markers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8764) Remove all deprecated methods and classes from master prior to the 6.0 release

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175662#comment-15175662
 ] 

ASF subversion and git services commented on SOLR-8764:
---

Commit 8bbb1dcacea996f81b989e360578a64ba3b78bfb in lucene-solr's branch 
refs/heads/branch_6x from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=8bbb1dc ]

SOLR-8764: Remove deprecated methods and classes


> Remove all deprecated methods and classes from master prior to the 6.0 release
> --
>
> Key: SOLR-8764
> URL: https://issues.apache.org/jira/browse/SOLR-8764
> Project: Solr
>  Issue Type: Task
>Reporter: Steve Rowe
>Assignee: Steve Rowe
> Fix For: 6.0
>
> Attachments: SOLR-8764.patch, SOLR-8764.patch
>
>
> Code marked as deprecated with {{@Deprecated}} and/or {{@deprecated}} should 
> be removed from master, unless it's being used internally, or the annotations 
> are there as markers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8764) Remove all deprecated methods and classes from master prior to the 6.0 release

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15175654#comment-15175654
 ] 

ASF subversion and git services commented on SOLR-8764:
---

Commit 9427b7402da33cccff9692bb4d7146dad4bb16e1 in lucene-solr's branch 
refs/heads/master from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9427b74 ]

SOLR-8764: Remove deprecated methods and classes


> Remove all deprecated methods and classes from master prior to the 6.0 release
> --
>
> Key: SOLR-8764
> URL: https://issues.apache.org/jira/browse/SOLR-8764
> Project: Solr
>  Issue Type: Task
>Reporter: Steve Rowe
>Assignee: Steve Rowe
> Fix For: 6.0
>
> Attachments: SOLR-8764.patch, SOLR-8764.patch
>
>
> Code marked as deprecated with {{@Deprecated}} and/or {{@deprecated}} should 
> be removed from master, unless it's being used internally, or the annotations 
> are there as markers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8764) Remove all deprecated methods and classes from master prior to the 6.0 release

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe updated SOLR-8764:
-
Attachment: SOLR-8764.patch

Patch, fixes a few failing tests and adds a CHANGES.txt entry.

Tests and precommit pass.

I think it's ready to go.  If there are no objections I'll commit later today.

> Remove all deprecated methods and classes from master prior to the 6.0 release
> --
>
> Key: SOLR-8764
> URL: https://issues.apache.org/jira/browse/SOLR-8764
> Project: Solr
>  Issue Type: Task
>Reporter: Steve Rowe
>Assignee: Steve Rowe
> Fix For: 6.0
>
> Attachments: SOLR-8764.patch, SOLR-8764.patch
>
>
> Code marked as deprecated with {{@Deprecated}} and/or {{@deprecated}} should 
> be removed from master, unless it's being used internally, or the annotations 
> are there as markers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8764) Remove all deprecated methods and classes from master prior to the 6.0 release

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe updated SOLR-8764:
-
Fix Version/s: 6.0

> Remove all deprecated methods and classes from master prior to the 6.0 release
> --
>
> Key: SOLR-8764
> URL: https://issues.apache.org/jira/browse/SOLR-8764
> Project: Solr
>  Issue Type: Task
>Reporter: Steve Rowe
>Assignee: Steve Rowe
> Fix For: 6.0
>
> Attachments: SOLR-8764.patch
>
>
> Code marked as deprecated with {{@Deprecated}} and/or {{@deprecated}} should 
> be removed from master, unless it's being used internally, or the annotations 
> are there as markers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8764) Remove all deprecated methods and classes from master prior to the 6.0 release

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe updated SOLR-8764:
-
Attachment: SOLR-8764.patch

First cut at a patch.  Compiles and passes precommit but I haven't looked at 
test failures yet, so there may be problems.

I've excluded deprecations for SOLR-8668, and also facet.date deprecations, for 
which I'll put up a separate patch on SOLR-7010.

> Remove all deprecated methods and classes from master prior to the 6.0 release
> --
>
> Key: SOLR-8764
> URL: https://issues.apache.org/jira/browse/SOLR-8764
> Project: Solr
>  Issue Type: Task
>Reporter: Steve Rowe
>Assignee: Steve Rowe
> Fix For: 6.0
>
> Attachments: SOLR-8764.patch
>
>
> Code marked as deprecated with {{@Deprecated}} and/or {{@deprecated}} should 
> be removed from master, unless it's being used internally, or the annotations 
> are there as markers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-8764) Remove all deprecated methods and classes from master prior to the 6.0 release

[Steve Rowe (JIRA)]

Steve Rowe created SOLR-8764:


 Summary: Remove all deprecated methods and classes from master 
prior to the 6.0 release
 Key: SOLR-8764
 URL: https://issues.apache.org/jira/browse/SOLR-8764
 Project: Solr
  Issue Type: Task
Reporter: Steve Rowe
Assignee: Steve Rowe


Code marked as deprecated with {{@Deprecated}} and/or {{@deprecated}} should be 
removed from master, unless it's being used internally, or the annotations are 
there as markers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org