Re: [VOTE] Release Lucene/Solr 8.11.3 RC1

[Ishan Chattopadhyaya]

+1 (binding)

SUCCESS! [1:18:24.494917]

On Wed, 7 Feb 2024 at 18:24, Jan Høydahl  wrote:

> +1 (binding)
>
> SUCCESS! [1:18:11.930433]
>
> Only ran smoke tester. macOS, Temurin 1.8.0_402
>
> Jan
>
> 5. feb. 2024 kl. 23:23 skrev Houston Putman :
>
> Please vote for release candidate 1 for Lucene/Solr 8.11.3
>
> The artifacts can be downloaded from:
>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.11.3-RC1-revbaa7c80af4278cc8951a344d8e9320386588d12d
>
> You can run the smoke tester directly with this command:
>
> python3 -u dev-tools/scripts/smokeTestRelease.py \
>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.11.3-RC1-revbaa7c80af4278cc8951a344d8e9320386588d12d
>
> The vote will be open for at least 72 hours i.e. until 2024-02-08 23:00
> UTC.
>
> [ ] +1  approve
> [ ] +0  no opinion
> [ ] -1  disapprove (and reason why)
>
> Here is my +1
>
>
>

Re: Bugfix release Lucene/Solr 8.11.3

[Ishan Chattopadhyaya]

Shouldn't we wait for the 9.4.1 to go out first? That's what I was holding
out on.

On Sat, 13 Jan, 2024, 12:43 am Houston Putman,  wrote:

> NOTICE:
>
> I am now preparing for a bugfix release from branch branch_8_11
>
> Please observe the normal rules for committing to this branch:
>
> * Before committing to the branch, reply to this thread and argue
>   why the fix needs backporting and how long it will take.
> * All issues accepted for backporting should be marked with 8.11.3
>   in JIRA, and issues that should delay the release must be marked as
> Blocker
> * All patches that are intended for the branch should first be committed
>   to the unstable branch, merged into the stable branch, and then into
>   the current release branch.
> * Only Jira issues with Fix version 8.11.3 and priority "Blocker" will
> delay
>   a release candidate build.
>

Re: [VOTE] Release Lucene/Solr 8.11.3 RC1

[Ishan Chattopadhyaya]

Ah, damn, I missed those private issues. Also the backup/restore RCE.
Thanks, Jan! I'll respin.

On Tue, 31 Oct 2023 at 18:08, Jan Høydahl  wrote:

> Hi,
>
> Thanks for doing the release. I'm running smoke tester
>
> However, there seems to be still open blocker issues in JIRA:
> https://issues.apache.org/jira/issues/?filter=12352945
> I just resolved one other that was solved yesterday, but for the remaining
> three I cannot see that they are fixed.
> Not 100% sure if all three qualify as blockers either, but they all look
> security related.
>
> Jan
>
> 31. okt. 2023 kl. 10:24 skrev Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com>:
>
>
> Please vote for release candidate 1 for Lucene/Solr 8.11.3
>
> The artifacts can be downloaded from:
>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.11.3-RC1-rev10d925469f3d4e66b6339b9e1c7d4d4afa4cf206
>
> You can run the smoke tester directly with this command:
>
> python3 -u dev-tools/scripts/smokeTestRelease.py \
>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.11.3-RC1-rev10d925469f3d4e66b6339b9e1c7d4d4afa4cf206
>
> The vote will be open for at least 72 hours i.e. until 2023-11-03 10:00
> UTC.
>
> [ ] +1  approve
> [ ] +0  no opinion
> [ ] -1  disapprove (and reason why)
>
> Here is my +1
> 
>
>
>
>

[VOTE] Release Lucene/Solr 8.11.3 RC1

[Ishan Chattopadhyaya]

Please vote for release candidate 1 for Lucene/Solr 8.11.3

The artifacts can be downloaded from:
https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.11.3-RC1-rev10d925469f3d4e66b6339b9e1c7d4d4afa4cf206

You can run the smoke tester directly with this command:

python3 -u dev-tools/scripts/smokeTestRelease.py \
https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.11.3-RC1-rev10d925469f3d4e66b6339b9e1c7d4d4afa4cf206

The vote will be open for at least 72 hours i.e. until 2023-11-03 10:00 UTC.

[ ] +1  approve
[ ] +0  no opinion
[ ] -1  disapprove (and reason why)

Here is my +1

Re: 8.11.3 release

[Ishan Chattopadhyaya]

Thanks Jan. Some tests are failing; Noble and I are looking into them. If
all goes well, I'm planning to spin the RC1 by this weekend.

On Fri, 27 Oct, 2023, 2:45 pm Jan Høydahl,  wrote:

> Thanks Kevin, Ishan for doing the prep work here.
>
> I re-enabled the jenkins job
> https://ci-builds.apache.org/job/Lucene/job/Lucene-Solr-Tests-8.11/ to
> get a feel for the current status. We also have a smokeTest job that should
> be enabled when release is approaching.
>
> Jan
>
> > 12. okt. 2023 kl. 19:38 skrev Kevin Risden :
> >
> > I've pushed a few things to branch_8_11 recently
> > *
> >
> https://github.com/apache/lucene-solr/commit/e491400d961b771f146c6e12a7f8ed89eafe128f
> > *
> >
> https://github.com/apache/lucene-solr/commit/8e545e8f2154698e9bfeee92c40f8541fd7d0152
> > *
> >
> https://github.com/apache/lucene-solr/commit/e7560cd2ab024d4315933cd3965aab2961bba994
> >
> > https://github.com/apache/lucene-solr/pull/2681 is in draft and updated
> a
> > bunch of relatively low risk dependencies. There might be others we want
> to
> > upgrade, but figured it was a decent start.
> >
> > Kevin Risden
> >
> >
> > On Thu, Oct 5, 2023 at 4:27 AM Pierre Salagnac <
> pierre.salag...@gmail.com>
> > wrote:
> >
> >> I just opened a pull request.
> >> https://github.com/apache/lucene-solr/pull/2679
> >> Details are in the PR.
> >>
> >> Thanks !
> >>
> >> Le lun. 2 oct. 2023 à 20:25, Ishan Chattopadhyaya <
> >> ichattopadhy...@gmail.com>
> >> a écrit :
> >>
> >>> Thanks Pierre, I'll have it included in 8.11 once you are able to have
> a
> >> PR
> >>> for this. Thanks!
> >>>
> >>> On Mon, 2 Oct 2023 at 22:22, Pierre Salagnac <
> pierre.salag...@gmail.com>
> >>> wrote:
> >>>
> >>>> Hi Ishan,
> >>>> Sorry for the late chime in.
> >>>>
> >>>> Some time ago I filled a Jira for a Solr 8 specific bug:
> >>>> https://issues.apache.org/jira/browse/SOLR-16843
> >>>>
> >>>> At that time, I wasn't expecting more 8.x releases, so I did not open
> a
> >>> PR
> >>>> for it.
> >>>> I can work on a fix if we have a few days more before the release. I
> >>> think
> >>>> it is worth to have it in solr 8 (that's not a backport)
> >>>>
> >>>> Thanks
> >>>>
> >>>>
> >>>> Le ven. 29 sept. 2023 à 23:24, Ishan Chattopadhyaya <
> >>>> ichattopadhy...@gmail.com> a écrit :
> >>>>
> >>>>> I'm going to track items from 9x releases in the following sheet.
> >>>>>
> >>>>>
> >>>>
> >>>
> >>
> https://docs.google.com/spreadsheets/d/1FADkjDS0yfXpaUi3fbwsa7Izy5HOxE9AR_NKiss1sIo/edit#gid=0
> >>>>>
> >>>>> Please let me know if there's any item there that you think would be
> >>>> useful
> >>>>> to backport (if easy) to 8.11, and want me to take a look at
> >>> backporting
> >>>>> it.
> >>>>> Regards,
> >>>>> Ishan
> >>>>>
> >>>>> On Tue, 19 Sept 2023 at 01:15, Ishan Chattopadhyaya <
> >>>>> ichattopadhy...@gmail.com> wrote:
> >>>>>
> >>>>>> Just a reminder to backport any issues one sees fit for a 8.11.3
> >>>> release.
> >>>>>> I'll try to get an RC out by the end of September, so still more
> >>> than a
> >>>>>> week away.
> >>>>>>
> >>>>>> On Wed, 23 Aug 2023 at 17:09, Ishan Chattopadhyaya <
> >>>>>> ichattopadhy...@gmail.com> wrote:
> >>>>>>
> >>>>>>> Hi Jan,
> >>>>>>> Yes, still targeting September. But I will slip on my initial plan
> >>> of
> >>>>>>> doing it by first week of September. I'm foreseeing mid September
> >>>>> timeframe.
> >>>>>>> Thanks for checking in.
> >>>>>>> Regards,
> >>>>>>> Ishan
> >>>>>>>
> >>>>>>> On Wed, 23 Aug, 2023, 5:05 pm Jan Høydahl,  >>>
> >>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi,
> >>>>>>>

Re: Welcome Guo Feng to the Lucene PMC

[Ishan Chattopadhyaya]

Congratulations and welcome, Feng!

On Tue, 24 Oct 2023 at 22:35, Adrien Grand  wrote:

> I'm pleased to announce that Guo Feng has accepted an invitation to join
> the Lucene PMC!
>
> Congratulations Feng, and welcome aboard!
>
> --
> Adrien
>

Re: RAPIDS RAFT

[Ishan Chattopadhyaya]

+Solr list, Lucene list, FYI

On Wed, 25 Oct, 2023, 9:13 am Ishan Chattopadhyaya, 
wrote:

>
> Hi Nathan,
>
> I am a committer and PMC member of Apache Solr and Apache Lucene. I have
> integrated CUDA into Lucene in the past:
> https://twitter.com/ichattopadhyaya/status/1065136227500867585
>
> I am interested in pursuing this integration for Apache Solr. Would you be
> interested in discussing this further?
>
> Regards,
> Ishan Chattopadhyaya
> Search Consultant, SearchScale
>
>
> On Wed, 25 Oct, 2023, 5:10 am Nathan Stephens,
>  wrote:
>
>> Lucene,
>>
>> I have talked to OpenSearch, Datastax, Elastic, Lucidworks, and Atlas
>> about integrating NVIDIA RAFT for GPU accelerated VSS. They all use Lucene
>> at some level. Rather than integrating into each, we would like to
>> integrate foundationally into Lucene.
>>
>> Could we set up some time and make some introductions?
>>
>> You can read more about RAFT here:
>>
>> Accelerating Vector Search: Using GPU-Powered Indexes with RAPIDS RAFT |
>> NVIDIA Technical Blog
>> <https://developer.nvidia.com/blog/accelerating-vector-search-using-gpu-powered-indexes-with-rapids-raft/>
>>
>> Accelerating Vector Search: Fine-Tuning GPU Index Algorithms | NVIDIA
>> Technical Blog
>> <https://developer.nvidia.com/blog/accelerating-vector-search-fine-tuning-gpu-index-algorithms/>
>>
>> Thanks,
>>
>> Nathan
>> --
>> *From:* Nathan Stephens 
>> *Sent:* Monday, August 14, 2023 9:32 AM
>> *To:* dev@lucene.apache.org 
>> *Subject:* RAPIDS RAFT
>>
>> NVIDIA has created accelerated vector search algorithms for IVF-Flat,
>> IVF-PQ, and CAGRA (a new graph based algorithm specifically designed for
>> GPU's). These algorithms are part of RAPIDS RAFT.  RAFT contains
>> fundamental widely-used algorithms and primitives for machine learning and
>> information retrieval. The algorithms are CUDA-accelerated and form
>> building blocks for more easily writing high performance applications.
>> API's exist for Python and C++ (we do not currently have a Java API).
>>
>> RAFT has been integrated into Milvus, Redis, and Faiss. We are looking
>> for other integration partners. I wanted to see if there was an interest in
>> the Lucene community.
>>
>> https://github.com/rapidsai/raft
>>
>> Nathan
>>
>>
>>
>> *Nathan Stephens* (He/Him/His)
>> Enterprise Products: Data Science, AI/ML
>>
>> NVIDIA <http://www.nvidia.com/>
>>
>

Re: 8.11.3 release

[Ishan Chattopadhyaya]

Thanks Pierre, I'll have it included in 8.11 once you are able to have a PR
for this. Thanks!

On Mon, 2 Oct 2023 at 22:22, Pierre Salagnac 
wrote:

> Hi Ishan,
> Sorry for the late chime in.
>
> Some time ago I filled a Jira for a Solr 8 specific bug:
> https://issues.apache.org/jira/browse/SOLR-16843
>
> At that time, I wasn't expecting more 8.x releases, so I did not open a PR
> for it.
> I can work on a fix if we have a few days more before the release. I think
> it is worth to have it in solr 8 (that's not a backport)
>
> Thanks
>
>
> Le ven. 29 sept. 2023 à 23:24, Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> a écrit :
>
> > I'm going to track items from 9x releases in the following sheet.
> >
> >
> https://docs.google.com/spreadsheets/d/1FADkjDS0yfXpaUi3fbwsa7Izy5HOxE9AR_NKiss1sIo/edit#gid=0
> >
> > Please let me know if there's any item there that you think would be
> useful
> > to backport (if easy) to 8.11, and want me to take a look at backporting
> > it.
> > Regards,
> > Ishan
> >
> > On Tue, 19 Sept 2023 at 01:15, Ishan Chattopadhyaya <
> > ichattopadhy...@gmail.com> wrote:
> >
> > > Just a reminder to backport any issues one sees fit for a 8.11.3
> release.
> > > I'll try to get an RC out by the end of September, so still more than a
> > > week away.
> > >
> > > On Wed, 23 Aug 2023 at 17:09, Ishan Chattopadhyaya <
> > > ichattopadhy...@gmail.com> wrote:
> > >
> > >> Hi Jan,
> > >> Yes, still targeting September. But I will slip on my initial plan of
> > >> doing it by first week of September. I'm foreseeing mid September
> > timeframe.
> > >> Thanks for checking in.
> > >> Regards,
> > >> Ishan
> > >>
> > >> On Wed, 23 Aug, 2023, 5:05 pm Jan Høydahl, 
> > wrote:
> > >>
> > >>> Hi,
> > >>>
> > >>> Following up on Ishan's proposed 8.11.3 release (
> > >>> https://lists.apache.org/thread/3xjtv1sxqx8f9nvhkc0cb90b2p76nfx2)
> > >>>
> > >>> Does the Lucene project have any bugfix candidates for backporting?
> > >>>
> > >>> Ishan, are you still targeting September?
> > >>>
> > >>> Jan
> > >>>
> > >>>
> > >>> > 1. aug. 2023 kl. 14:57 skrev Ishan Chattopadhyaya <
> > >>> ichattopadhy...@gmail.com>:
> > >>> >
> > >>> > Oh yes, good idea. Forgot about the split!
> > >>> >
> > >>> > +Lucene Dev 
> > >>> >
> > >>> > On Tue, 1 Aug, 2023, 6:17 pm Uwe Schindler, 
> wrote:
> > >>> >
> > >>> >> Maybe ask on Lucene list, too, if there are some bug people like
> to
> > >>> have
> > >>> >> fixed in Lucene.
> > >>> >>
> > >>> >> Uwe
> > >>> >>
> > >>> >> Am 01.08.2023 um 11:10 schrieb Ishan Chattopadhyaya:
> > >>> >>> Hi all,
> > >>> >>> There have been lots of bug fixes that have gone into 9x that
> > should
> > >>> >>> benefit all 8x users as well.
> > >>> >>> I thought of volunteering for such a 8.x release based on this
> > >>> comment
> > >>> >> [0].
> > >>> >>>
> > >>> >>> Unless someone has any objections or concerns, can we tentatively
> > >>> plan
> > >>> >> 1st
> > >>> >>> September 2023 (1 month from now) as a tentative release for
> > 8.11.3?
> > >>> I
> > >>> >>> think we will get ample time to backport relevant fixes to 8x by
> > >>> then.
> > >>> >>>
> > >>> >>> Best regards,
> > >>> >>> Ishan
> > >>> >>>
> > >>> >>> [0] -
> > >>> >>>
> > >>> >>
> > >>>
> >
> https://issues.apache.org/jira/browse/SOLR-16777?focusedCommentId=17742854=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17742854
> > >>> >>>
> > >>> >> --
> > >>> >> Uwe Schindler
> > >>> >> Achterdiek 19, D-28357 Bremen
> > >>> >> https://www.thetaphi.de
> > >>> >> eMail: u...@thetaphi.de
> > >>> >>
> > >>> >>
> > >>> >>
> > -
> > >>> >> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > >>> >> For additional commands, e-mail: dev-h...@solr.apache.org
> > >>> >>
> > >>> >>
> > >>>
> > >>>
> > >>> -
> > >>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> > >>> For additional commands, e-mail: dev-h...@solr.apache.org
> > >>>
> > >>>
> >
>

Re: 8.11.3 release

[Ishan Chattopadhyaya]

I'm going to track items from 9x releases in the following sheet.
https://docs.google.com/spreadsheets/d/1FADkjDS0yfXpaUi3fbwsa7Izy5HOxE9AR_NKiss1sIo/edit#gid=0

Please let me know if there's any item there that you think would be useful
to backport (if easy) to 8.11, and want me to take a look at backporting it.
Regards,
Ishan

On Tue, 19 Sept 2023 at 01:15, Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Just a reminder to backport any issues one sees fit for a 8.11.3 release.
> I'll try to get an RC out by the end of September, so still more than a
> week away.
>
> On Wed, 23 Aug 2023 at 17:09, Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
>> Hi Jan,
>> Yes, still targeting September. But I will slip on my initial plan of
>> doing it by first week of September. I'm foreseeing mid September timeframe.
>> Thanks for checking in.
>> Regards,
>> Ishan
>>
>> On Wed, 23 Aug, 2023, 5:05 pm Jan Høydahl,  wrote:
>>
>>> Hi,
>>>
>>> Following up on Ishan's proposed 8.11.3 release (
>>> https://lists.apache.org/thread/3xjtv1sxqx8f9nvhkc0cb90b2p76nfx2)
>>>
>>> Does the Lucene project have any bugfix candidates for backporting?
>>>
>>> Ishan, are you still targeting September?
>>>
>>> Jan
>>>
>>>
>>> > 1. aug. 2023 kl. 14:57 skrev Ishan Chattopadhyaya <
>>> ichattopadhy...@gmail.com>:
>>> >
>>> > Oh yes, good idea. Forgot about the split!
>>> >
>>> > +Lucene Dev 
>>> >
>>> > On Tue, 1 Aug, 2023, 6:17 pm Uwe Schindler,  wrote:
>>> >
>>> >> Maybe ask on Lucene list, too, if there are some bug people like to
>>> have
>>> >> fixed in Lucene.
>>> >>
>>> >> Uwe
>>> >>
>>> >> Am 01.08.2023 um 11:10 schrieb Ishan Chattopadhyaya:
>>> >>> Hi all,
>>> >>> There have been lots of bug fixes that have gone into 9x that should
>>> >>> benefit all 8x users as well.
>>> >>> I thought of volunteering for such a 8.x release based on this
>>> comment
>>> >> [0].
>>> >>>
>>> >>> Unless someone has any objections or concerns, can we tentatively
>>> plan
>>> >> 1st
>>> >>> September 2023 (1 month from now) as a tentative release for 8.11.3?
>>> I
>>> >>> think we will get ample time to backport relevant fixes to 8x by
>>> then.
>>> >>>
>>> >>> Best regards,
>>> >>> Ishan
>>> >>>
>>> >>> [0] -
>>> >>>
>>> >>
>>> https://issues.apache.org/jira/browse/SOLR-16777?focusedCommentId=17742854=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17742854
>>> >>>
>>> >> --
>>> >> Uwe Schindler
>>> >> Achterdiek 19, D-28357 Bremen
>>> >> https://www.thetaphi.de
>>> >> eMail: u...@thetaphi.de
>>> >>
>>> >>
>>> >> -
>>> >> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
>>> >> For additional commands, e-mail: dev-h...@solr.apache.org
>>> >>
>>> >>
>>>
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
>>> For additional commands, e-mail: dev-h...@solr.apache.org
>>>
>>>

Re: 8.11.3 release

[Ishan Chattopadhyaya]

Just a reminder to backport any issues one sees fit for a 8.11.3 release.
I'll try to get an RC out by the end of September, so still more than a
week away.

On Wed, 23 Aug 2023 at 17:09, Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Hi Jan,
> Yes, still targeting September. But I will slip on my initial plan of
> doing it by first week of September. I'm foreseeing mid September timeframe.
> Thanks for checking in.
> Regards,
> Ishan
>
> On Wed, 23 Aug, 2023, 5:05 pm Jan Høydahl,  wrote:
>
>> Hi,
>>
>> Following up on Ishan's proposed 8.11.3 release (
>> https://lists.apache.org/thread/3xjtv1sxqx8f9nvhkc0cb90b2p76nfx2)
>>
>> Does the Lucene project have any bugfix candidates for backporting?
>>
>> Ishan, are you still targeting September?
>>
>> Jan
>>
>>
>> > 1. aug. 2023 kl. 14:57 skrev Ishan Chattopadhyaya <
>> ichattopadhy...@gmail.com>:
>> >
>> > Oh yes, good idea. Forgot about the split!
>> >
>> > +Lucene Dev 
>> >
>> > On Tue, 1 Aug, 2023, 6:17 pm Uwe Schindler,  wrote:
>> >
>> >> Maybe ask on Lucene list, too, if there are some bug people like to
>> have
>> >> fixed in Lucene.
>> >>
>> >> Uwe
>> >>
>> >> Am 01.08.2023 um 11:10 schrieb Ishan Chattopadhyaya:
>> >>> Hi all,
>> >>> There have been lots of bug fixes that have gone into 9x that should
>> >>> benefit all 8x users as well.
>> >>> I thought of volunteering for such a 8.x release based on this comment
>> >> [0].
>> >>>
>> >>> Unless someone has any objections or concerns, can we tentatively plan
>> >> 1st
>> >>> September 2023 (1 month from now) as a tentative release for 8.11.3? I
>> >>> think we will get ample time to backport relevant fixes to 8x by then.
>> >>>
>> >>> Best regards,
>> >>> Ishan
>> >>>
>> >>> [0] -
>> >>>
>> >>
>> https://issues.apache.org/jira/browse/SOLR-16777?focusedCommentId=17742854=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17742854
>> >>>
>> >> --
>> >> Uwe Schindler
>> >> Achterdiek 19, D-28357 Bremen
>> >> https://www.thetaphi.de
>> >> eMail: u...@thetaphi.de
>> >>
>> >>
>> >> -
>> >> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
>> >> For additional commands, e-mail: dev-h...@solr.apache.org
>> >>
>> >>
>>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
>> For additional commands, e-mail: dev-h...@solr.apache.org
>>
>>

Re: 8.11.3 release

[Ishan Chattopadhyaya]

Hi Jan,
Yes, still targeting September. But I will slip on my initial plan of doing
it by first week of September. I'm foreseeing mid September timeframe.
Thanks for checking in.
Regards,
Ishan

On Wed, 23 Aug, 2023, 5:05 pm Jan Høydahl,  wrote:

> Hi,
>
> Following up on Ishan's proposed 8.11.3 release (
> https://lists.apache.org/thread/3xjtv1sxqx8f9nvhkc0cb90b2p76nfx2)
>
> Does the Lucene project have any bugfix candidates for backporting?
>
> Ishan, are you still targeting September?
>
> Jan
>
>
> > 1. aug. 2023 kl. 14:57 skrev Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com>:
> >
> > Oh yes, good idea. Forgot about the split!
> >
> > +Lucene Dev 
> >
> > On Tue, 1 Aug, 2023, 6:17 pm Uwe Schindler,  wrote:
> >
> >> Maybe ask on Lucene list, too, if there are some bug people like to have
> >> fixed in Lucene.
> >>
> >> Uwe
> >>
> >> Am 01.08.2023 um 11:10 schrieb Ishan Chattopadhyaya:
> >>> Hi all,
> >>> There have been lots of bug fixes that have gone into 9x that should
> >>> benefit all 8x users as well.
> >>> I thought of volunteering for such a 8.x release based on this comment
> >> [0].
> >>>
> >>> Unless someone has any objections or concerns, can we tentatively plan
> >> 1st
> >>> September 2023 (1 month from now) as a tentative release for 8.11.3? I
> >>> think we will get ample time to backport relevant fixes to 8x by then.
> >>>
> >>> Best regards,
> >>> Ishan
> >>>
> >>> [0] -
> >>>
> >>
> https://issues.apache.org/jira/browse/SOLR-16777?focusedCommentId=17742854=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17742854
> >>>
> >> --
> >> Uwe Schindler
> >> Achterdiek 19, D-28357 Bremen
> >> https://www.thetaphi.de
> >> eMail: u...@thetaphi.de
> >>
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> >> For additional commands, e-mail: dev-h...@solr.apache.org
> >>
> >>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>

Re: 8.11.3 release

[Ishan Chattopadhyaya]

Oh yes, good idea. Forgot about the split!

+Lucene Dev 

On Tue, 1 Aug, 2023, 6:17 pm Uwe Schindler,  wrote:

> Maybe ask on Lucene list, too, if there are some bug people like to have
> fixed in Lucene.
>
> Uwe
>
> Am 01.08.2023 um 11:10 schrieb Ishan Chattopadhyaya:
> > Hi all,
> > There have been lots of bug fixes that have gone into 9x that should
> > benefit all 8x users as well.
> > I thought of volunteering for such a 8.x release based on this comment
> [0].
> >
> > Unless someone has any objections or concerns, can we tentatively plan
> 1st
> > September 2023 (1 month from now) as a tentative release for 8.11.3? I
> > think we will get ample time to backport relevant fixes to 8x by then.
> >
> > Best regards,
> > Ishan
> >
> > [0] -
> >
> https://issues.apache.org/jira/browse/SOLR-16777?focusedCommentId=17742854=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17742854
> >
> --
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail: u...@thetaphi.de
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>

Re: JavaDoc generated with -noindex

[Ishan Chattopadhyaya]

+1 to include this in release. Thanks for noticing!

On Sat, 8 Jul, 2023, 12:33 am Mike Drob,  wrote:

> Why is our javadoc currently generated with -noindex? I did some digging
> and found that we set that back in LUCENE-3977 to save 10MB, and then added
> a property to re-enable it in LUCENE-4237, but I think that got lost in the
> gradle migration.
>
> While the index might have been useless at the time, it now powers the
> javadoc search box, see a demo at https://youtu.be/VrI6rJNO2x4?t=925 --
> The full spec is described at
> https://docs.oracle.com/en/java/javase/17/docs/specs/javadoc/javadoc-search-spec.html
>
>
> I think this would be a useful thing to include, at least for releases.
> WDYT?
>
> Mike
>

Re: Welcome Chris Hegarty to the Lucene PMC

[Ishan Chattopadhyaya]

Congratulations Chris!

On Mon, 19 Jun, 2023, 3:23 pm Adrien Grand,  wrote:

> I'm pleased to announce that Chris Hegarty has accepted an invitation to
> join the Lucene PMC!
>
> Congratulations Chris, and welcome aboard!
>
> --
> Adrien
>

Re: [VOTE] Dimension Limit for KNN Vectors

[Ishan Chattopadhyaya]

That sounds promising, Michael. Can you share scripts/steps/code to
reproduce this?

On Thu, 18 May, 2023, 1:16 pm Michael Wechner, 
wrote:

> I just implemented it and tested it with OpenAI's text-embedding-ada-002,
> which is using 1536 dimensions and it works very fine :-)
>
> Thanks
>
> Michael
>
>
>
> Am 18.05.23 um 00:29 schrieb Michael Wechner:
>
> IIUC KnnVectorField is deprecated and one is supposed to use
> KnnFloatVectorField when using float as vector values, right?
>
> Am 17.05.23 um 16:41 schrieb Michael Sokolov:
>
> see https://markmail.org/message/kf4nzoqyhwacb7ri
>
> On Wed, May 17, 2023 at 10:09 AM David Smiley  wrote:
>
>> > easily be circumvented by a user
>>
>> This is a revelation to me and others, if true.  Michael, please then
>> point to a test or code snippet that shows the Lucene user community what
>> they want to see so they are unblocked from their explorations of vector
>> search.
>>
>> ~ David Smiley
>> Apache Lucene/Solr Search Developer
>> http://www.linkedin.com/in/davidwsmiley
>>
>>
>> On Wed, May 17, 2023 at 7:51 AM Michael Sokolov 
>> wrote:
>>
>>> I think I've said before on this list we don't actually enforce the
>>> limit in any way that can't easily be circumvented by a user. The codec
>>> already supports any size vector - it doesn't impose any limit. The way the
>>> API is written you can *already today* create an index with max-int sized
>>> vectors and we are committed to supporting that going forward by our
>>> backwards compatibility policy as Robert points out. This wasn't
>>> intentional, I think, but it is the facts.
>>>
>>> Given that, I think this whole discussion is not really necessary.
>>>
>>> On Tue, May 16, 2023 at 4:50 AM Alessandro Benedetti <
>>> a.benede...@sease.io> wrote:
>>>
 Hi all,
 we have finalized all the options proposed by the community and we are
 ready to vote for the preferred one and then proceed with the
 implementation.

 *Option 1*
 Keep it as it is (dimension limit hardcoded to 1024)
 *Motivation*:
 We are close to improving on many fronts. Given the criticality of
 Lucene in computing infrastructure and the concerns raised by one of the
 most active stewards of the project, I think we should keep working toward
 improving the feature as is and move to up the limit after we can
 demonstrate improvement unambiguously.

 *Option 2*
 make the limit configurable, for example through a system property
 *Motivation*:
 The system administrator can enforce a limit its users need to respect
 that it's in line with whatever the admin decided to be acceptable for
 them.
 The default can stay the current one.
 This should open the doors for Apache Solr, Elasticsearch, OpenSearch,
 and any sort of plugin development

 *Option 3*
 Move the max dimension limit lower level to a HNSW specific
 implementation. Once there, this limit would not bind any other potential
 vector engine alternative/evolution.
 *Motivation:* There seem to be contradictory performance
 interpretations about the current HNSW implementation. Some consider its
 performance ok, some not, and it depends on the target data set and use
 case. Increasing the max dimension limit where it is currently (in top
 level FloatVectorValues) would not allow potential alternatives (e.g. for
 other use-cases) to be based on a lower limit.

 *Option 4*
 Make it configurable and move it to an appropriate place.
 In particular, a simple Integer.getInteger("lucene.hnsw.maxDimensions",
 1024) should be enough.
 *Motivation*:
 Both are good and not mutually exclusive and could happen in any order.
 Someone suggested to perfect what the _default_ limit should be, but
 I've not seen an argument _against_ configurability.  Especially in this
 way -- a toggle that doesn't bind Lucene's APIs in any way.

 I'll keep this [VOTE] open for a week and then proceed to the
 implementation.
 --
 *Alessandro Benedetti*
 Director @ Sease Ltd.
 *Apache Lucene/Solr Committer*
 *Apache Solr PMC Member*

 e-mail: a.benede...@sease.io


 *Sease* - Information Retrieval Applied
 Consulting | Training | Open Source

 Website: Sease.io 
 LinkedIn  | Twitter
  | Youtube
  | Github
 

>>>
>
>

Re: Running 10.0 build with a custom lucene 9.5

[Ishan Chattopadhyaya]

You propose to throw an exception containing this, right?

> Java does not throw SecurityException if this
is the case, it just ignores the jar!

Are you serious?

On Wed, 17 May, 2023, 8:02 am Gus Heck,  wrote:

> Blaming?
>
> On Tue, May 16, 2023 at 10:05 PM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
>> > Having that explicitly called out would have been SUPER helpful.
>>
>> Blaming Java in an exception thrown by Lucene is a ridiculous idea.
>>
>> On Wed, 17 May, 2023, 3:33 am Gus Heck,  wrote:
>>
>>> Found it.
>>>
>>> It's a solr thing made worse by the interaction of lucene testutils and
>>> jdk.internal.loader.URLClassPath's decision to hide anything gone wrong
>>> when checking a URL
>>> /*
>>>  * Checks whether the resource URL should be returned.
>>>  * Returns null on security check failure.
>>>  * Called by java.net.URLClassLoader.
>>>  */
>>> public static URL checkURL(URL url) {
>>> if (url != null) {
>>> try {
>>> check(url);
>>> } catch (Exception e) {
>>> return null;
>>> }
>>> }
>>> return url;
>>> }
>>>
>>> Yay. Fun. JDK classes swallowing exceptions silently.
>>>
>>> At the start of this it only took me a little while to discover that
>>> there
>>> was a security manager in play via debugging. Remembering that I saw
>>> emails
>>> about that, I went to jira, found the ticket enabling it by default in
>>> 9.x
>>> and eventually tracked down the name of the security policy file by
>>> reading
>>> solr.in.sh and /bin/solr...  The key issue that tripped me up is that
>>> the
>>> tests have a *separate* security policy file, and there was pretty much
>>> no
>>> way to know this without extensive reading of the build. Thus I got
>>> thrown
>>> off track when
>>>
>>>   permission java.io.FilePermission
>>> "${user.home}${/}.m2${/}repository${/}-", "read";
>>>
>>> To  solr/server/etc/security.policy had no effect. That and the fact that
>>> no security exception was reported, led me to start chasing increasingly
>>> improbable hypotheses. Many hours later when I went back to debugging
>>> deeply into class loading, I found that the code was actually reading the
>>> jar files in question, and then I finally caught it throwing a security
>>> exception during my debugging.
>>>
>>> It turns out that adding the above permission to
>>> gradle/testing/randomization/policies/solr-tests.policy allows the test
>>> to
>>> pass. [1]
>>>
>>> I think we need to document this somewhere (or someone needs to point me
>>> to
>>> the doc I missed, FWIW I hit this basically following the process in
>>> dev-docs/dependency-upgrades.adoc treating lucene like a dependency, and
>>> unaware that there is a "shortcut" mode for lucene specifically in
>>> gradle/lucene-dev/lucene-dev-repo-composite.gradle and I find reading
>>> that
>>> file none-to clear anyway)
>>>
>>> That's the solr part, the lucene part is that the security exception is
>>> hit
>>> when in org.apache.lucene.codecs.Codec$Holder.(Codec.java:58)
>>> when org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv#before
>>> does
>>>
>>>  savedCodec = Codec.getDefault();
>>>
>>> The error message "An SPI class of type org.apache.lucene.codecs.Codec
>>> with
>>> name 'Lucene95' does not exist." was moderately misleading because the
>>> file
>>> and the services files in the jar definitely did exist. This message
>>> should
>>> vary if there is an installed security manager, maybe saying something
>>> like:
>>>
>>> "An SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene95'
>>> does not exist. We have detected that a security manager is installed so
>>> it
>>> is also possible that the jar containing the codec is inaccessible under
>>> the current security policy. (Java does not throw SecurityException if
>>> this
>>> is the case, it just ignores the jar!)" [2]
>>>
>>> Having that explicitly called out would have been SUPER helpful.
>>>
>>> -Gus
>>>
>>> [1]: https://iss

Re: Running 10.0 build with a custom lucene 9.5

[Ishan Chattopadhyaya]

> Having that explicitly called out would have been SUPER helpful.

Blaming Java in an exception thrown by Lucene is a ridiculous idea.

On Wed, 17 May, 2023, 3:33 am Gus Heck,  wrote:

> Found it.
>
> It's a solr thing made worse by the interaction of lucene testutils and
> jdk.internal.loader.URLClassPath's decision to hide anything gone wrong
> when checking a URL
> /*
>  * Checks whether the resource URL should be returned.
>  * Returns null on security check failure.
>  * Called by java.net.URLClassLoader.
>  */
> public static URL checkURL(URL url) {
> if (url != null) {
> try {
> check(url);
> } catch (Exception e) {
> return null;
> }
> }
> return url;
> }
>
> Yay. Fun. JDK classes swallowing exceptions silently.
>
> At the start of this it only took me a little while to discover that there
> was a security manager in play via debugging. Remembering that I saw emails
> about that, I went to jira, found the ticket enabling it by default in 9.x
> and eventually tracked down the name of the security policy file by reading
> solr.in.sh and /bin/solr...  The key issue that tripped me up is that the
> tests have a *separate* security policy file, and there was pretty much no
> way to know this without extensive reading of the build. Thus I got thrown
> off track when
>
>   permission java.io.FilePermission
> "${user.home}${/}.m2${/}repository${/}-", "read";
>
> To  solr/server/etc/security.policy had no effect. That and the fact that
> no security exception was reported, led me to start chasing increasingly
> improbable hypotheses. Many hours later when I went back to debugging
> deeply into class loading, I found that the code was actually reading the
> jar files in question, and then I finally caught it throwing a security
> exception during my debugging.
>
> It turns out that adding the above permission to
> gradle/testing/randomization/policies/solr-tests.policy allows the test to
> pass. [1]
>
> I think we need to document this somewhere (or someone needs to point me to
> the doc I missed, FWIW I hit this basically following the process in
> dev-docs/dependency-upgrades.adoc treating lucene like a dependency, and
> unaware that there is a "shortcut" mode for lucene specifically in
> gradle/lucene-dev/lucene-dev-repo-composite.gradle and I find reading that
> file none-to clear anyway)
>
> That's the solr part, the lucene part is that the security exception is hit
> when in org.apache.lucene.codecs.Codec$Holder.(Codec.java:58)
> when org.apache.lucene.tests.util.TestRuleSetupAndRestoreClassEnv#before
> does
>
>  savedCodec = Codec.getDefault();
>
> The error message "An SPI class of type org.apache.lucene.codecs.Codec with
> name 'Lucene95' does not exist." was moderately misleading because the file
> and the services files in the jar definitely did exist. This message should
> vary if there is an installed security manager, maybe saying something
> like:
>
> "An SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene95'
> does not exist. We have detected that a security manager is installed so it
> is also possible that the jar containing the codec is inaccessible under
> the current security policy. (Java does not throw SecurityException if this
> is the case, it just ignores the jar!)" [2]
>
> Having that explicitly called out would have been SUPER helpful.
>
> -Gus
>
> [1]: https://issues.apache.org/jira/browse/SOLR-16804
> [2]: https://github.com/apache/lucene/issues/12300
>
>
> On Mon, May 15, 2023 at 3:17 PM Michael Sokolov 
> wrote:
>
> > random guess - does it have something to do with modules?
> >
> > On Mon, May 15, 2023 at 11:14 AM Gus Heck  wrote:
> > >
> > > I hadn't seen that one. Thanks, I'll look at it. It already looks a bit
> > confusing though since it seems to have options for pointing to a repo,
> but
> > I appear to be pulling the jars successfully from .m2/repository
> already...
> > (except then they don't work, so successful means I see them in the
> > classpath of the relevant classloader). And if we can't deploy a valid
> jar
> > to mavenLocal for some reason (tweaked the solr build so it sees
> > mavenLocal()), (or solr can't consume such a jar) that seems like an
> issue
> > for whichever one is breaking that.
> > >
> > > Debugging: The JDK appears to be attempting to load the services file
> > from modules, but not seeing the lucene module. (just the jdk ones) Also
> it
> > passes through a block that says:
> > >
> > > // not in a package of a module defined to this loader
> > > for (URL url : findMiscResource(name)) {
> > >
> > > (but then iterates
> > jdk.internal.loader.BuiltinClassLoader#nameToModule.values() to load
> things
> > anyway)
> > >
> > > -Gus
> > >
> > > On Mon, May 15, 2023 at 10:54 AM Houston Putman <
> houstonput...@gmail.com>
> > wrote:
> > >>
> > >> Gus, I haven't done this myself, but are 

Re: Seeking Tools and Methods to Measure Lucene's Indexing Performance

[Ishan Chattopadhyaya]

Check Lucene bench: https://home.apache.org/~mikemccand/lucenebench/

On Sat, 6 May, 2023, 9:30 am donghai tang,  wrote:

> Hello Lucene Community,
>
> I am in the process of learning about Lucene's indexing capabilities, and
> I'm keen on conducting experiments to evaluate its performance. However, I
> haven't come across any official tools specifically designed for measuring
> Lucene's indexing performance.
>
> I would be extremely grateful if any of you could share your experiences
> with tools you've used in the past or suggest alternative methods for
> evaluating Lucene's indexing performance.
>

Re: New branch and feature freeze for Lucene 9.6.0

[Ishan Chattopadhyaya]

Don't remember the specifics, but I ran into GPG issues during Solr 9.1.0
release. The fix for me was https://github.com/apache/solr/pull/1125, but I
don't know if this is the same problem or if it is applicable in Lucene's
case.

On Tue, 2 May 2023 at 18:27, Alan Woodward  wrote:

> I am fighting with gradle and GPG yet again… Gradle fails when trying to
> sign artefacts with the message "Cannot perform signing task
> ':lucene:distribution:signReleaseArchives' because it has no configured
> signatory”.  I have GPG configured in ~/.gradle/gradle.properties as
> follows:
>
> org.gradle.caching=true
> signing.keyId=
> signing.secretKeyRingFile=/Users/romseygeek/.gnupg/secring.gpg
> signing.gnupg.executable=gpg
>
> This worked last time I did a release.  Does anybody know if anything has
> changed in gradle that means I need to change the properties file, or have
> any other ideas?
>
> > On 27 Apr 2023, at 10:54, Alan Woodward  wrote:
> >
> > I have started a release note here:
> https://cwiki.apache.org/confluence/display/LUCENE/Release+Notes+9.6
> >
> >> On 27 Apr 2023, at 09:45, Alan Woodward  wrote:
> >>
> >> I have successfully wrestled Jenkins into submission, and there are now
> 9.6 jobs for Artifacts, Check and NightlyTests.
> >>
> >>> On 26 Apr 2023, at 16:53, Alan Woodward  wrote:
> >>>
> >>> NOTICE:
> >>>
> >>> Branch branch_9_6 has been cut and versions updated to 9.7 on stable
> branch.
> >>>
> >>> Please observe the normal rules:
> >>>
> >>> * No new features may be committed to the branch.
> >>> * Documentation patches, build patches and serious bug fixes may be
> >>> committed to the branch. However, you should submit all patches you
> >>> want to commit as pull requests first to give others the chance to
> review
> >>> and possibly vote against them. Keep in mind that it is our
> >>> main intention to keep the branch as stable as possible.
> >>> * All patches that are intended for the branch should first be
> committed
> >>> to the unstable branch, merged into the stable branch, and then into
> >>> the current release branch.
> >>> * Normal unstable and stable branch development may continue as usual.
> >>> However, if you plan to commit a big change to the unstable branch
> >>> while the branch feature freeze is in effect, think twice: can't the
> >>> addition wait a couple more days? Merges of bug fixes into the branch
> >>> may become more difficult.
> >>> * Only Github issues with Milestone 9.6
> >>> and priority "Blocker" will delay a release candidate build.
> >>>
> >>>
> >>> I am struggling to find the lucene Jenkins jobs on the new apache
> build server at https://jenkins-ccos.apache.org/ - if anybody has any
> hints as to how to navigate the helpful new interface with a non-functional
> search box, I would be very grateful…
> >>>
> >>> It’s a holiday weekend coming up in the UK, so my plan is to give
> Jenkins a few days to chew things over (once I actually get the jobs
> running) and then build a RC on Tuesday 2nd May.
> >>
> >
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Concurrent HNSW index

[Ishan Chattopadhyaya]

+1, please contribute to Lucene. Thanks!

On Thu, 27 Apr, 2023, 10:59 pm Jonathan Ellis,  wrote:

> Hi all,
>
> I've created an HNSW index implementation that allows for concurrent build
> and querying.  On my i9-12900 (8 performance cores and 8 efficiency) I get
> a bit less than 10x speedup of wall clock time for building and querying
> the "siftsmall" and "sift" datasets from http://corpus-texmex.irisa.fr/.
> The small dataset is 10k vectors while the large is 1M.  This speedup feels
> pretty good for a data structure that isn't completely parallelizable, and
> it's good to see that it's consistent as the dataset gets larger.
>
> The concurrent classes achieve identical recall compared to the
> non-concurrent versions within my ability to test it, and are drop-in
> replacements for OnHeapHnswGraph and HnswGraphBuilder; I use threadlocals
> to work around the places where the existing API assumes no concurrency.
>
> The concurrent classes also pass the existing test suite with the
> exception of the ram usage ones; the estimator doesn't know about
> AtomicReference etc.  (Big thanks to Michael Sokolov for testAknnDiverse
> which made it much easier to track down subtle problems!)
>
> My motivation is
>
> 1. It is faster to query a single on-heap hnsw index, than to query
> multiple such indexes and combine the result.
> 2. Even with some contention necessarily occurring during building of the
> index, we still come out way ahead in terms of total efficiency vs creating
> per-thread indexes and combining them, since combining such indexes boils
> down to "pick the largest and then add all the other nodes normally," you
> don't really benefit from having computed the others previously.
>
> I am currently adding this to Cassandra as code in our repo, but my
> preference would be to upstream it.  Is Lucene open to a pull request?
>
> --
> Jonathan Ellis
> co-founder, http://www.datastax.com
> @spyced
>

Re: Should IndexWriter.flush return seqNo?

[Ishan Chattopadhyaya]

I think Apache Solr could explore leveraging the returned sequence number
for its transaction logs.

On Tue, 25 Apr 2023 at 18:36, Michael McCandless 
wrote:

> On Sun, Apr 23, 2023 at 6:19 AM Uwe Schindler  wrote:
>
> Having the sequence number public in API does not bring any benefit, as
>> you cannot use it for anything.
>>
>
> Actually there are some interesting use cases for sequence numbers:
>
> They enable the caller to know the effective order of operations of
> concurrent indexing events.  This can be useful for applications that might
> sometimes update the same document at the same time across threads to
> implement optimistic concurrency to re-index the same document if the order
> was not correct according to the applications external version tracking for
> out-of-order updates.  OpenSearch has an array of locks to implement
> pessimistic concurrency (ensuring the that same id is never updated
> concurrently) but for cases where the conflicts are rare, the optimistic
> implementation based on Lucene's sequence numbers is likely more efficient.
>
> Another use case is precise indexing operation replay (e.g. from a Kinesis
> queue or transaction log or whatever) on recovering from a commit point:
> upon commit, you know which precise indexing event was captured in the
> commit, and on recovering you can resume indexing from precisely the next
> indexing event.  This doesn't matter for idempotent updates, but, for other
> cases like append only, it is useful and performant.
>
> I also don't see why flush should return a sequence number -- it is not an
> externally visible event.  Patrick maybe you had an interesting use case in
> mind?  Note that commit also writes (and fsyncs) the next segments_N file,
> to light all the newly written/fsync'd segments for the next reader to open.
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>

Re: [Proposal] Remove max number of dimensions for KNN vectors

[Ishan Chattopadhyaya]

Seems like they were all 768 dimensions.

On Fri, 21 Apr, 2023, 11:48 am Michael Wechner, 
wrote:

> Hi Together
>
> Cohere just published approx. 100Mio embeddings based on Wikipedia content
>
> https://txt.cohere.com/embedding-archives-wikipedia/
>
> resp.
>
> https://huggingface.co/datasets/Cohere/wikipedia-22-12-en-embeddings
> https://huggingface.co/datasets/Cohere/wikipedia-22-12-de-embeddings
> 
>
> HTH
>
> Michael
>
>
>
> Am 13.04.23 um 07:58 schrieb Michael Wechner:
>
> Hi Kent
>
> Great, thank you very much!
>
> Will download it later today :-)
>
> All the best
>
> Michael
>
> Am 13.04.23 um 01:35 schrieb Kent Fitch:
>
> Hi Michael (and anyone else who wants just over 240K "real world" ada-002
> vectors of dimension 1536),
> you are welcome to retrieve a tar.gz file which contains:
> - 47K embeddings of Canberra Times news article text from 1994
> - 38K embeddings of the first paragraphs of wikipedia articles about
> organisations
> - 156.6K embeddings of the first paragraphs of wikipedia articles about
> people
>
>
> https://drive.google.com/file/d/13JP_5u7E8oZO6vRg0ekaTgBDQOaj-W00/view?usp=sharing
>
> The file is about 1.7GB and will expand to about 4.4GB. This file will be
> accessible for at least a week, and I hope you dont hit any google drive
> download limits trying to retrieve it.
>
> The embeddings were generated using my openAI account and you are welcome
> to use them for any purpose you like.
>
> best wishes,
>
> Kent Fitch
>
> On Wed, Apr 12, 2023 at 4:37 PM Michael Wechner 
> wrote:
>
>> thank you very much for your feedback!
>>
>> In a previous post (April 7) you wrote you could make availlable the 47K
>> ada-002 vectors, which would be great!
>>
>> Would it make sense to setup a public gitub repo, such that others could
>> use or also contribute vectors?
>>
>> Thanks
>>
>> Michael Wechner
>>
>>
>> Am 12.04.23 um 04:51 schrieb Kent Fitch:
>>
>> I only know some characteristics of the openAI ada-002 vectors, although
>> they are a very popular as embeddings/text-characterisations as they allow
>> more accurate/"human meaningful" semantic search results with fewer
>> dimensions than their predecessors - I've evaluated a few different
>> embedding models, including some BERT variants, CLIP ViT-L-14 (with 768
>> dims, which was quite good), openAI's ada-001 (1024 dims) and babbage-001
>> (2048 dims), and ada-002 are qualitatively the best, although that will
>> certainly change!
>>
>> In any case, ada-002 vectors have interesting characteristics that I
>> think mean you could confidently create synthetic vectors which would be
>> hard to distinguish from "real" vectors.  I found this from looking at 47K
>> ada-002 vectors generated across a full year (1994) of newspaper articles
>> from the Canberra Times and 200K wikipedia articles:
>> - there is no discernible/significant correlation between values in any
>> pair of dimensions
>> - all but 5 of the 1536 dimensions have an almost identical distribution
>> of values shown in the central blob on these graphs (that just show a few
>> of these 1531 dimensions with clumped values and the 5 "outlier"
>> dimensions, but all 1531 non-outlier dims are in there, which makes for
>> some easy quantisation from float to byte if you dont want to go the full
>> kmeans/clustering/Lloyds-algorithm approach):
>>
>> https://docs.google.com/spreadsheets/d/1DyyBCbirETZSUAEGcMK__mfbUNzsU_L48V9E0SyJYGg/edit?usp=sharing
>>
>> https://docs.google.com/spreadsheets/d/1czEAlzYdyKa6xraRLesXjNZvEzlj27TcDGiEFS1-MPs/edit?usp=sharing
>>
>> https://docs.google.com/spreadsheets/d/1RxTjV7Sj14etCNLk1GB-m44CXJVKdXaFlg2Y6yvj3z4/edit?usp=sharing
>> - the variance of the value of each dimension is characteristic:
>>
>> https://docs.google.com/spreadsheets/d/1w5LnRUXt1cRzI9Qwm07LZ6UfszjMOgPaJot9cOGLHok/edit#gid=472178228
>>
>> This probably represents something significant about how the ada-002
>> embeddings are created, but I think it also means creating "realistic"
>> values is possible.  I did not use this information when testing recall &
>> performance on Lucene's HNSW implementation on 192m documents, as I
>> slightly dithered the values of a "real" set on 47K docs and stored other
>> fields in the doc that referenced the "base" document that the dithers were
>> made from, and used different dithering magnitudes so that I could test
>> recall with different neighbour sizes ("M"), construction-beamwidth and
>> search-beamwidths.
>>
>> best regards
>>
>> Kent Fitch
>>
>>
>>
>>
>> On Wed, Apr 12, 2023 at 5:08 AM Michael Wechner <
>> michael.wech...@wyona.com> wrote:
>>
>>> I understand what you mean that it seems to be artificial, but I don't
>>> understand why this matters to test performance and scalability of the
>>> indexing?
>>>
>>> Let's assume the limit of Lucene would be 4 instead of 1024 and there
>>> are only open source models generating vectors with 4 dimensions, for
>>> example
>>>
>>>
>>> 

Re: [Proposal] Remove max number of dimensions for KNN vectors

[Ishan Chattopadhyaya]

Can the limit be raised using Java reflection at run time? Or is there more
to it that needs to be changed?

On Sun, 9 Apr, 2023, 12:58 am Alessandro Benedetti, 
wrote:

> I am very attentive to listen opinions but I am un-convinced here and I an
> not sure that a single person opinion should be allowed to be detrimental
> for such an important project.
>
> The limit as far as I know is literally just raising an exception.
> Removing it won't alter in any way the current performance for users in
> low dimensional space.
> Removing it will just enable more users to use Lucene.
>
> If new users in certain situations will be unhappy with the performance,
> they may contribute improvements.
> This is how you make progress.
>
> If it's a reputation thing, trust me that not allowing users to play with
> high dimensional space will equally damage it.
>
> To me it's really a no brainer.
> Removing the limit and enable people to use high dimensional vectors will
> take minutes.
> Improving the hnsw implementation can take months.
> Pick one to begin with...
>
> And there's no-one paying me here, no company interest whatsoever,
> actually I pay people to contribute, I am just convinced it's a good idea.
>
>
> On Sat, 8 Apr 2023, 18:57 Robert Muir,  wrote:
>
>> I disagree with your categorization. I put in plenty of work and
>> experienced plenty of pain myself, writing tests and fighting these
>> issues, after i saw that, two releases in a row, vector indexing fell
>> over and hit integer overflows etc on small datasets:
>>
>> https://github.com/apache/lucene/pull/11905
>>
>> Attacking me isn't helping the situation.
>>
>> PS: when i said the "one guy who wrote the code" I didn't mean it in
>> any kind of demeaning fashion really. I meant to describe the current
>> state of usability with respect to indexing a few million docs with
>> high dimensions. You can scroll up the thread and see that at least
>> one other committer on the project experienced similar pain as me.
>> Then, think about users who aren't committers trying to use the
>> functionality!
>>
>> On Sat, Apr 8, 2023 at 12:51 PM Michael Sokolov 
>> wrote:
>> >
>> > What you said about increasing dimensions requiring a bigger ram buffer
>> on merge is wrong. That's the point I was trying to make. Your concerns
>> about merge costs are not wrong, but your conclusion that we need to limit
>> dimensions is not justified.
>> >
>> > You complain that hnsw sucks it doesn't scale, but when I show it
>> scales linearly with dimension you just ignore that and complain about
>> something entirely different.
>> >
>> > You demand that people run all kinds of tests to prove you wrong but
>> when they do, you don't listen and you won't put in the work yourself or
>> complain that it's too hard.
>> >
>> > Then you complain about people not meeting you half way. Wow
>> >
>> > On Sat, Apr 8, 2023, 12:40 PM Robert Muir  wrote:
>> >>
>> >> On Sat, Apr 8, 2023 at 8:33 AM Michael Wechner
>> >>  wrote:
>> >> >
>> >> > What exactly do you consider reasonable?
>> >>
>> >> Let's begin a real discussion by being HONEST about the current
>> >> status. Please put politically correct or your own company's wishes
>> >> aside, we know it's not in a good state.
>> >>
>> >> Current status is the one guy who wrote the code can set a
>> >> multi-gigabyte ram buffer and index a small dataset with 1024
>> >> dimensions in HOURS (i didn't ask what hardware).
>> >>
>> >> My concerns are everyone else except the one guy, I want it to be
>> >> usable. Increasing dimensions just means even bigger multi-gigabyte
>> >> ram buffer and bigger heap to avoid OOM on merge.
>> >> It is also a permanent backwards compatibility decision, we have to
>> >> support it once we do this and we can't just say "oops" and flip it
>> >> back.
>> >>
>> >> It is unclear to me, if the multi-gigabyte ram buffer is really to
>> >> avoid merges because they are so slow and it would be DAYS otherwise,
>> >> or if its to avoid merges so it doesn't hit OOM.
>> >> Also from personal experience, it takes trial and error (means
>> >> experiencing OOM on merge!!!) before you get those heap values correct
>> >> for your dataset. This usually means starting over which is
>> >> frustrating and wastes more time.
>> >>
>> >> Jim mentioned some ideas about the memory usage in IndexWriter, seems
>> >> to me like its a good idea. maybe the multigigabyte ram buffer can be
>> >> avoided in this way and performance improved by writing bigger
>> >> segments with lucene's defaults. But this doesn't mean we can simply
>> >> ignore the horrors of what happens on merge. merging needs to scale so
>> >> that indexing really scales.
>> >>
>> >> At least it shouldnt spike RAM on trivial data amounts and cause OOM,
>> >> and definitely it shouldnt burn hours and hours of CPU in O(n^2)
>> >> fashion when indexing.
>> >>
>> >> -
>> >> To unsubscribe, e-mail: 

Re: [Proposal] Remove max number of dimensions for KNN vectors

[Ishan Chattopadhyaya]

ke something that
>>> > would be so bad that we need to prevent users from using numbers of
>>> > dimensions in the low thousands, e.g. top-k KNN searches would still
>>> > look at a very small subset of the full dataset.
>>> >
>>> > So overall, my vote would be to bump the limit to 2048 as suggested by
>>> > Mayya on the issue that you linked.
>>> >
>>> > On Fri, Mar 31, 2023 at 2:38 PM Michael Wechner
>>> >  wrote:
>>> >> Thanks Alessandro for summarizing the discussion below!
>>> >>
>>> >> I understand that there is no clear reasoning re what is the best
>>> embedding size, whereas I think heuristic approaches like described by the
>>> following link can be helpful
>>> >>
>>> >>
>>> https://datascience.stackexchange.com/questions/51404/word2vec-how-to-choose-the-embedding-size-parameter
>>> >>
>>> >> Having said this, we see various embedding services providing higher
>>> dimensions than 1024, like for example OpenAI, Cohere and Aleph Alpha.
>>> >>
>>> >> And it would be great if we could run benchmarks without having to
>>> recompile Lucene ourselves.
>>> >>
>>> >> Therefore I would to suggest to either increase the limit or even
>>> better to remove the limit and add a disclaimer, that people should be
>>> aware of possible crashes etc.
>>> >>
>>> >> Thanks
>>> >>
>>> >> Michael
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> Am 31.03.23 um 11:43 schrieb Alessandro Benedetti:
>>> >>
>>> >>
>>> >> I've been monitoring various discussions on Pull Requests about
>>> changing the max number of dimensions allowed for Lucene HNSW vectors:
>>> >>
>>> >> https://github.com/apache/lucene/pull/12191
>>> >>
>>> >> https://github.com/apache/lucene/issues/11507
>>> >>
>>> >>
>>> >> I would like to set up a discussion and potentially a vote about this.
>>> >>
>>> >> I have seen some strong opposition from a few people but a majority
>>> of favor in this direction.
>>> >>
>>> >>
>>> >> Motivation
>>> >>
>>> >> We were discussing in the Solr slack channel with Ishan
>>> Chattopadhyaya, Marcus Eagan, and David Smiley about some neural search
>>> integrations in Solr: https://github.com/openai/chatgpt-retrieval-plugin
>>> >>
>>> >>
>>> >> Proposal
>>> >>
>>> >> No hard limit at all.
>>> >>
>>> >> As for many other Lucene areas, users will be allowed to push the
>>> system to the limit of their resources and get terrible performances or
>>> crashes if they want.
>>> >>
>>> >>
>>> >> What we are NOT discussing
>>> >>
>>> >> - Quality and scalability of the HNSW algorithm
>>> >>
>>> >> - dimensionality reduction
>>> >>
>>> >> - strategies to fit in an arbitrary self-imposed limit
>>> >>
>>> >>
>>> >> Benefits
>>> >>
>>> >> - users can use the models they want to generate vectors
>>> >>
>>> >> - removal of an arbitrary limit that blocks some integrations
>>> >>
>>> >>
>>> >> Cons
>>> >>
>>> >>   - if you go for vectors with high dimensions, there's no guarantee
>>> you get acceptable performance for your use case
>>> >>
>>> >>
>>> >>
>>> >> I want to keep it simple, right now in many Lucene areas, you can
>>> push the system to not acceptable performance/ crashes.
>>> >>
>>> >> For example, we don't limit the number of docs per index to an
>>> arbitrary maximum of N, you push how many docs you like and if they are too
>>> much for your system, you get terrible performance/crashes/whatever.
>>> >>
>>> >>
>>> >> Limits caused by primitive java types will stay there behind the
>>> scene, and that's acceptable, but I would prefer to not have arbitrary
>>> hard-coded ones that may limit the software usability and integration which
>>> is extremely important for a library.
>>> >>
>>> >>
>>> >> I strongly encourage people to add benefits and cons, that I missed
>>> (I am sure I missed some of them, but wanted to keep it simple)
>>> >>
>>> >>
>>> >> Cheers
>>> >>
>>> >> --
>>> >> Alessandro Benedetti
>>> >> Director @ Sease Ltd.
>>> >> Apache Lucene/Solr Committer
>>> >> Apache Solr PMC Member
>>> >>
>>> >> e-mail: a.benede...@sease.io
>>> >>
>>> >>
>>> >> Sease - Information Retrieval Applied
>>> >> Consulting | Training | Open Source
>>> >>
>>> >> Website: Sease.io
>>> >> LinkedIn | Twitter | Youtube | Github
>>> >>
>>> >>
>>> >
>>>
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>>

Re: Welcome Ben Trent as Lucene committer

[Ishan Chattopadhyaya]

Welcome and congratulations, Ben!

On Fri, Jan 27, 2023 at 8:48 PM Adrien Grand  wrote:
>
> I'm pleased to announce that Ben Trent has accepted the PMC's
> invitation to become a committer.
>
> Ben, the tradition is that new committers introduce themselves with a
> brief bio.
>
> Congratulations and welcome!
>
> --
> Adrien
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Lucene 9.5.0 release

[Ishan Chattopadhyaya]

I think it would be too close, and the API would also change (so it is not
just a bugfix). I'll hold this off for the next release.
Thanks!

On Mon, Jan 23, 2023 at 6:59 PM Luca Cavanna 
wrote:

> Hi Ishan,
> thanks for asking, I am currently working on moving the byte vectors API
> away from BytesRef like Adrien suggested. I would aim for cutting the
> branch tomorrow, would that work for you too?
>
> Cheers
> Luca
>
> On Mon, Jan 23, 2023 at 2:23 PM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
>> Hi Luca,
>> Thanks for volunteering.
>> Approximately, when do you plan to cut the release branch? If there's
>> time, can I include https://issues.apache.org/jira/browse/LUCENE-9302
>> for 9.5 still? It is not merged yet.
>> Thanks,
>> Ishan
>>
>> On Mon, Jan 23, 2023 at 5:31 PM Alessandro Benedetti <
>> a.benede...@sease.io> wrote:
>>
>>> Done on main and cherry-picked on 9.x, thanks Luca for your patience!
>>> --
>>> *Alessandro Benedetti*
>>> Director @ Sease Ltd.
>>> *Apache Lucene/Solr Committer*
>>> *Apache Solr PMC Member*
>>>
>>> e-mail: a.benede...@sease.io
>>>
>>>
>>> *Sease* - Information Retrieval Applied
>>> Consulting | Training | Open Source
>>>
>>> Website: Sease.io <http://sease.io/>
>>> LinkedIn <https://linkedin.com/company/sease-ltd> | Twitter
>>> <https://twitter.com/seaseltd> | Youtube
>>> <https://www.youtube.com/channel/UCDx86ZKLYNpI3gzMercM7BQ> | Github
>>> <https://github.com/seaseltd>
>>>
>>>
>>> On Mon, 23 Jan 2023 at 12:31, Alessandro Benedetti 
>>> wrote:
>>>
>>>> Yes Luca, doing it right now!
>>>>
>>>> For Michael, it's just few getters.
>>>>
>>>> Cheers
>>>> --
>>>> *Alessandro Benedetti*
>>>> Director @ Sease Ltd.
>>>> *Apache Lucene/Solr Committer*
>>>> *Apache Solr PMC Member*
>>>>
>>>> e-mail: a.benede...@sease.io
>>>>
>>>>
>>>> *Sease* - Information Retrieval Applied
>>>> Consulting | Training | Open Source
>>>>
>>>> Website: Sease.io <http://sease.io/>
>>>> LinkedIn <https://linkedin.com/company/sease-ltd> | Twitter
>>>> <https://twitter.com/seaseltd> | Youtube
>>>> <https://www.youtube.com/channel/UCDx86ZKLYNpI3gzMercM7BQ> | Github
>>>> <https://github.com/seaseltd>
>>>>
>>>>
>>>> On Mon, 23 Jan 2023 at 11:21, Luca Cavanna 
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>> I meant to start the release today and I see this PR is not merged
>>>>> yet: https://github.com/apache/lucene/pull/12029 . Alessandro, do you
>>>>> still plan on merging it shortly?
>>>>>
>>>>> Thanks
>>>>> Luca
>>>>>
>>>>> On Sat, Jan 21, 2023 at 11:41 AM Michael Wechner <
>>>>> michael.wech...@wyona.com> wrote:
>>>>>
>>>>>> I tried to understand the issue described on github, but
>>>>>> unfortunately do not really understand it.
>>>>>>
>>>>>> Can you explain a little more?
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> Michael
>>>>>>
>>>>>>
>>>>>>
>>>>>> Am 21.01.23 um 11:00 schrieb Alessandro Benedetti:
>>>>>>
>>>>>> Hi,
>>>>>> this would be nice to have in 9.5 :
>>>>>> https://github.com/apache/lucene/issues/12099
>>>>>>
>>>>>> It's a minor (adding getters to KnnQuery) but can be beneficial in
>>>>>> Apache Solr as soon as possible.
>>>>>> Planning to merge in a few hours if no objections.
>>>>>> --
>>>>>> *Alessandro Benedetti*
>>>>>> Director @ Sease Ltd.
>>>>>> *Apache Lucene/Solr Committer*
>>>>>> *Apache Solr PMC Member*
>>>>>>
>>>>>> e-mail: a.benede...@sease.io
>>>>>>
>>>>>>
>>>>>> *Sease* - Information Retrieval Applied
>>>>>> Consulting | Training | Open Source
>>>>>>
>>>>>> Website: 

Re: Lucene 9.5.0 release

[Ishan Chattopadhyaya]

Hi Luca,
Thanks for volunteering.
Approximately, when do you plan to cut the release branch? If there's time,
can I include https://issues.apache.org/jira/browse/LUCENE-9302 for 9.5
still? It is not merged yet.
Thanks,
Ishan

On Mon, Jan 23, 2023 at 5:31 PM Alessandro Benedetti 
wrote:

> Done on main and cherry-picked on 9.x, thanks Luca for your patience!
> --
> *Alessandro Benedetti*
> Director @ Sease Ltd.
> *Apache Lucene/Solr Committer*
> *Apache Solr PMC Member*
>
> e-mail: a.benede...@sease.io
>
>
> *Sease* - Information Retrieval Applied
> Consulting | Training | Open Source
>
> Website: Sease.io 
> LinkedIn  | Twitter
>  | Youtube
>  | Github
> 
>
>
> On Mon, 23 Jan 2023 at 12:31, Alessandro Benedetti 
> wrote:
>
>> Yes Luca, doing it right now!
>>
>> For Michael, it's just few getters.
>>
>> Cheers
>> --
>> *Alessandro Benedetti*
>> Director @ Sease Ltd.
>> *Apache Lucene/Solr Committer*
>> *Apache Solr PMC Member*
>>
>> e-mail: a.benede...@sease.io
>>
>>
>> *Sease* - Information Retrieval Applied
>> Consulting | Training | Open Source
>>
>> Website: Sease.io 
>> LinkedIn  | Twitter
>>  | Youtube
>>  | Github
>> 
>>
>>
>> On Mon, 23 Jan 2023 at 11:21, Luca Cavanna 
>> wrote:
>>
>>> Hi all,
>>> I meant to start the release today and I see this PR is not merged yet:
>>> https://github.com/apache/lucene/pull/12029 . Alessandro, do you still
>>> plan on merging it shortly?
>>>
>>> Thanks
>>> Luca
>>>
>>> On Sat, Jan 21, 2023 at 11:41 AM Michael Wechner <
>>> michael.wech...@wyona.com> wrote:
>>>
 I tried to understand the issue described on github, but unfortunately
 do not really understand it.

 Can you explain a little more?

 Thanks

 Michael



 Am 21.01.23 um 11:00 schrieb Alessandro Benedetti:

 Hi,
 this would be nice to have in 9.5 :
 https://github.com/apache/lucene/issues/12099

 It's a minor (adding getters to KnnQuery) but can be beneficial in
 Apache Solr as soon as possible.
 Planning to merge in a few hours if no objections.
 --
 *Alessandro Benedetti*
 Director @ Sease Ltd.
 *Apache Lucene/Solr Committer*
 *Apache Solr PMC Member*

 e-mail: a.benede...@sease.io


 *Sease* - Information Retrieval Applied
 Consulting | Training | Open Source

 Website: Sease.io 
 LinkedIn  | Twitter
  | Youtube
  | Github
 


 On Thu, 19 Jan 2023 at 14:38, Luca Cavanna 
  wrote:

> Thanks Robert for the help with the github milestone.
>
> I am planning on cutting the release branch on Monday if there are no
> objections.
>
> Cheers
> Luca
>
> On Tue, Jan 17, 2023 at 7:08 PM Robert Muir  wrote:
>
>> +1 to release, thank you for volunteering to be RM!
>>
>> I went thru 9.5 section of CHANGES.txt and tagged all the GH issues in
>> there with milestone too, if they didn't already have it. It looks
>> even bigger now.
>>
>> On Fri, Jan 13, 2023 at 4:54 AM Luca Cavanna 
>> wrote:
>> >
>> > Hi all,
>> > I'd like to propose that we release Lucene 9.5.0. There is a decent
>> amount of changes that would go into it looking at the github milestone:
>> https://github.com/apache/lucene/milestone/4 . I'd volunteer to be
>> the release manager. There is one PR open listed for the 9.5 milestone:
>> https://github.com/apache/lucene/pull/11873 . Is this something that
>> we do want to address before we release? Is anybody aware of outstanding
>> work that we would like to include or known blocker issues that are not
>> listed in the 9.5 milestone?
>> >
>> > Cheers
>> > Luca
>> >
>> >
>> >
>> >
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>

Re: [VOTE] Migration to GitHub issue from Jira (LUCENE-10557)

[Ishan Chattopadhyaya]

-1

On Tue, 31 May, 2022, 4:06 am Xi Chen, 
wrote:

> +1 from me (committer, non-PMC)
>
> Thanks Tomoko for starting the discussion and organizing / leading this
> effort!
>
> Best,
> Zach
>
> On May 30, 2022, at 2:56 PM, Houston Putman  wrote:
>
> 
> +1 Approve (PMC)
>
> Thanks so much for doing all of the work for this Tomoko!
>
> - Houston
>
> On Mon, May 30, 2022 at 5:38 PM David Smiley  wrote:
>
>> +1 Approve (PMC)
>>
>> ~ David Smiley
>> Apache Lucene/Solr Search Developer
>> http://www.linkedin.com/in/davidwsmiley
>>
>>
>> On Mon, May 30, 2022 at 11:40 AM Tomoko Uchida <
>> tomoko.uchida.1...@gmail.com> wrote:
>>
>>> Hi everyone!
>>>
>>> As we had previous discussion thread [1], I propose migration to GitHub
>>> issue from Jira.
>>> It'd be technically possible (see [2] for details) and I think it'd be
>>> good for the project - not only for welcoming new developers who are not
>>> familiar with Jira, but also for improving the experiences of long-term
>>> committers/contributors by consolidating the conversation platform.
>>>
>>> You can see a short summary of the discussion, some stats on current
>>> Jira issues, and a draft migration plan in [2].
>>> Please review [2] if you haven't seen it and vote for this proposal.
>>>
>>> The vote will be open until 2022-06-06 16:00 UTC.
>>>
>>> [ ] +1  approve
>>> [ ] +0  no opinion
>>> [ ] -1  disapprove (and reason why)
>>>
>>> Here is my +1
>>>
>>> *IMPORTANT NOTE*
>>> I set a local protocol for this vote.
>>> There are 95 committers on this project [3] - the vote will be effective
>>> if it successfully gains more than 15% of voters (>= 15) from committers
>>> (including PMC members). This means, that although only PMC member votes
>>> are counted for the final result, the votes from all committers are
>>> important to make the vote result effective.
>>>
>>> If there are less than 15 votes at 2022-06-06 16:00 UTC, I will expand
>>> the term to 2022-06-13 16:00 UTC. If this fails to get sufficient voters
>>> after the expanded time limit, I'll cancel this vote regardless of the
>>> result.
>>> But why do I set such an extra bar? My fear is that if such things are
>>> decided by the opinions of a few members, the result shouldn't yield a good
>>> outcome for the future. It isn't my goal to just pass the vote [4].
>>>
>>> [1] https://lists.apache.org/thread/78wj0vll73sct065m5jjm4z8gqb5yffk
>>> [2] https://issues.apache.org/jira/browse/LUCENE-10557
>>> [3] https://projects.apache.org/committee.html?lucene
>>> [4] I'm sorry for being overly cautious, but I have never met in person
>>> or virtually any of the committers (with a very few exceptions), therefore
>>> cannot assess if the vote result is reliable or not unless there is certain
>>> explicit feedback.
>>>
>>> Tomoko
>>>
>>

Re: [DISCUSS] A proposal for migration to GitHub issue (LUCENE-10557)

[Ishan Chattopadhyaya]

(Repeating in public what I mentioned in private)


I'm generally opposed to this idea because GitHub has been known to take
political decisions to cut off access to developers just because of their
nationality/region etc. As a community, we should stay politically neutral
and not rely on GitHub to decide on our behalf who to exclude from our
community.

On Thu, 5 May, 2022, 8:45 pm Jan Høydahl,  wrote:

> Given how JIRA has become a monster of a product with different markup
> syntax for each version, and bugs everywhere (does not even work on
> mobile), I'm no longer the JIRA fan I once was.
>
> In Solr we already use github issues for the Solr-Operator sub project
> https://github.com/apache/solr-operator/issues and I believe it has
> worked well. Of course excellent integration with PRs :)
> In earlier discussions on this topic, the idea has been shot down with the
> argument of split bug history and migration challenges. But I think you are
> wise to delay the HOW discussion for now.
> This discussion should also not be about politics. Some may be opposed to
> Microsoft and GitHub, but as long as the ASF has officially blessed github
> as an official option, i'ts not a very constructive discussion.
> The most important decision point on my part is perception by new / young
> users. Look at OpenOffice, they have remained on Bugzilla - are you
> compelled to contribute? :)
>
> Jan
>
> 5. mai 2022 kl. 04:23 skrev Tomoko Uchida :
>
> Hello everyone!
>
> Recently, we relaxed the requirement for creating a Jira issue when
> opening a pull request (LUCENE-10545
> ).
>
> As the next and bigger (perhaps ambitious) step, I opened a rough proposal
> for migration to GitHub issue from Jira.
> https://issues.apache.org/jira/browse/LUCENE-10557
>
> According to the INFRA issue for the RocketMQ project (Michael McCandless
> gave the pointer in a comment on the issue; thanks!), a PMC agreement or
> Vote result is needed for the decision.
> https://issues.apache.org/jira/browse/INFRA-15702
>
> Eventually, we'd need a formal vote, but before that, I'd like to hear
> general opinions/thoughts (or feelings) on this topic from developers.
>
> In brief, I think it'd be technically possible and also be good for the
> project - not only for welcoming new developers who are not familiar with
> Jira, but also for improving the experiences of long-term contributors by
> consolidating the conversation platform.
> It'll need some administrative work though, I'm willing to work for it if
> we reach an agreement.
>
> Please note that:
> * This is not a VOTE. Simple vote-style feedback (+/- 1) is welcome, but
> we don't aim to reach a conclusion in this thread.
> * Let's not discuss "how to migrate existing Jira issues" for now. Once we
> decide the migration will be good for us, then we can try to figure out a
> reasonable solution for technical/administrative matters.
>
> I may be too optimistic about it; but - a bit of stupidness will be needed
> to start such a move, and I'm serious about this proposal :)
>
> Thanks and regards,
> Tomoko
>
>
>

Re: Welcome Guo Feng as Lucene committer

[Ishan Chattopadhyaya]

Welcome and congratulations, Feng!

On Tue, Jan 25, 2022 at 3:40 PM Koji Sekiguchi 
wrote:

> Welcome, Feng!
>
> Koji
>
>
> On 2022/01/25 18:09, Adrien Grand wrote:
> > I'm pleased to announce that Guo Feng has accepted the PMC's
> > invitation to become a committer.
> >
> > Feng, the tradition is that new committers introduce themselves with a
> > brief bio.
> >
> > Congratulations and welcome!
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Welcome Haoyu (Patrick) Zhai as Lucene Committer

[Ishan Chattopadhyaya]

Welcome Haoyu!

On Sun, 19 Dec, 2021, 7:16 pm Michael McCandless, 
wrote:

> Welcome Patrick!
>
> Mike
>
> On Sun, Dec 19, 2021 at 8:44 AM Robert Muir  wrote:
>
>> Congratulations!
>>
>> On Sun, Dec 19, 2021 at 4:12 AM Dawid Weiss 
>> wrote:
>> >
>> > Hello everyone!
>> >
>> > Please welcome Haoyu Zhai as the latest Lucene committer. You may also
>> > know Haoyu as Patrick - this is perhaps his kind gesture to those of
>> > us whose tongues are less flexible in pronouncing difficult first
>> > names. :)
>> >
>> > It's a tradition to briefly introduce yourself to the group, Patrick.
>> > Welcome and thank you!
>> >
>> > Dawid
>> >
>> > -
>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> > For additional commands, e-mail: dev-h...@lucene.apache.org
>> >
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>> --
> Mike McCandless
>
> http://blog.mikemccandless.com
>

Re: [VOTE] Release Lucene/Solr 8.11.1 RC1

[Ishan Chattopadhyaya]

I think we should publish, release and announce asap, not waiting for 72h
or the MVN propogation.

On Thu, 16 Dec, 2021, 2:40 am Anshum Gupta,  wrote:

> +1 (binding)
>
> Smoke tester is happy.
>
> SUCCESS! [1:03:13.162577]
>
> Also tested out a sample search/indexing app.
>
> On Tue, Dec 14, 2021 at 6:36 AM Jan Høydahl  wrote:
>
>> Please vote for release candidate 1 for Lucene/Solr 8.11.1
>>
>> The artifacts can be downloaded from:
>>
>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.11.1-RC1-rev0b002b11819df70783e83ef36b42ed1223c14b50
>>
>> You can run the smoke tester directly (from a fresh branch_8_11
>> checkout), with this command:
>>
>> python3 -u dev-tools/scripts/smokeTestRelease.py \
>>
>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.11.1-RC1-rev0b002b11819df70783e83ef36b42ed1223c14b50
>>
>> The vote will be open for at least 72 hours i.e. until 2021-12-17 15:00
>> UTC.
>>
>> [ ] +1  approve
>> [ ] +0  no opinion
>> [ ] -1  disapprove (and reason why)
>>
>> Here is my +1
>>
>> SUCCESS! [0:54:56.979538]
>>
>> NOTE: You must run the smoke tester from latest commit on branch_8_11,
>> since my surname contains a unicode-character, needing a fix in the gpg
>> command ran by the smoketester.
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>
>
> --
> Anshum Gupta
>

Re: [VOTE] Release Lucene/Solr 8.11.1 RC1

[Ishan Chattopadhyaya]

Ran the smoke tested, and it passed.

SUCCESS! [1:21:47.769507]

+1

On Wed, Dec 15, 2021 at 9:28 AM Mike Drob  wrote:

> +1 (binding)
>
> ran smoke tester - unit tests passed the first time but timed out
> downloading artifacts from maven. reran a second time, modifying the smoke
> test script to not run solr tests (again) and the script passed.
>
> started up a solr server from the unpacked download and verified it
> against a few log4j injections, server responded appropriately each time
>
> manually verified a few of the other bugs having been fixed going into
> 8.11.1
>
> SUCCESS! [0:35:37.290016]
>
> On Tue, Dec 14, 2021 at 7:24 PM Timothy Potter 
> wrote:
>
>> +1 (binding) ~ just ran smoke tester this time
>>
>> SUCCESS! [1:16:20.247006]
>>
>> On Tue, Dec 14, 2021 at 7:36 AM Jan Høydahl 
>> wrote:
>> >
>> > Please vote for release candidate 1 for Lucene/Solr 8.11.1
>> >
>> > The artifacts can be downloaded from:
>> >
>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.11.1-RC1-rev0b002b11819df70783e83ef36b42ed1223c14b50
>> >
>> > You can run the smoke tester directly (from a fresh branch_8_11
>> checkout), with this command:
>> >
>> > python3 -u dev-tools/scripts/smokeTestRelease.py \
>> >
>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.11.1-RC1-rev0b002b11819df70783e83ef36b42ed1223c14b50
>> >
>> > The vote will be open for at least 72 hours i.e. until 2021-12-17 15:00
>> UTC.
>> >
>> > [ ] +1  approve
>> > [ ] +0  no opinion
>> > [ ] -1  disapprove (and reason why)
>> >
>> > Here is my +1
>> >
>> > SUCCESS! [0:54:56.979538]
>> >
>> > NOTE: You must run the smoke tester from latest commit on branch_8_11,
>> since my surname contains a unicode-character, needing a fix in the gpg
>> command ran by the smoketester.
>> > -
>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> > For additional commands, e-mail: dev-h...@lucene.apache.org
>> >
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>

Re: What should we do of branch_8x?

[Ishan Chattopadhyaya]

> Makes me feel it would be OK to handle this cleanup asynchronously to the
9.0 release.
+1. It is unreasonable to hold up the 9.0 release for this.

On Tue, Nov 23, 2021 at 9:03 PM Michael Sokolov  wrote:

> +1 to remove all content and leave behind a README in 8.x and 7.x, and
> it sounds like adding the .asf..yaml file could even prevent further
> commits?
>
> I hope there weren't any consequences of having a few unintended
> commits in the 7x branch. Makes me feel it would be OK to handle this
> cleanup asynchronously to the 9.0 release.
>
> On Tue, Nov 23, 2021 at 10:14 AM Uwe Schindler  wrote:
> >
> > Hi,
> >
> > I checked a bit: branch_7x is also still alive and has some accidental
> commits in it. So maybe we should do the same there.
> >
> > In general if we change this, don't forget to change github workflows:
> https://github.com/apache/lucene-solr/blob/master/.github/workflows/ant.yml
> >
> > Side note: I am missing the .asf.yaml file in the master branch of old
> repo. Where is this information stored? This file was there also to protect
> branches from writing (at least in github):
> https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-BranchProtection
> >
> > Uwe
> >
> > -
> > Uwe Schindler
> > Achterdiek 19, D-28357 Bremen
> > https://www.thetaphi.de
> > eMail: u...@thetaphi.de
> >
> > > -Original Message-
> > > From: Adrien Grand 
> > > Sent: Tuesday, November 23, 2021 2:02 PM
> > > To: Lucene Dev 
> > > Subject: Re: What should we do of branch_8x?
> > >
> > > It looks like there is now general agreement on removing branch_8x?
> > >
> > > I wonder if we should actually remove it, which is prone to
> > > re-creating the branch by mistake, vs. replacing the content of the
> > > repository with a README that says that this branch is no longer under
> > > development like we did for the `master` branch.
> > >
> > > On Mon, Nov 22, 2021 at 5:09 PM Jan Høydahl 
> > > wrote:
> > > >
> > > > +1 to remove / lock branch_8x in the lucene-solr repo, i.e. there
> will not be an
> > > 8.12 release by Lucene PMC.
> > > >
> > > > Whether Solr needs to release an 8.12 from own repos or not can be
> > > discussed in dev@solr if/when needed. So far there is only loose
> talk, and I
> > > think Solr PMC's energy should be devoted to the Solr 9.0 release.
> > > >
> > > > Jan
> > > >
> > > > > 22. nov. 2021 kl. 08:28 skrev Atri Sharma :
> > > > >
> > > > > +1, agree with Uwe.
> > > > >
> > > > > On Mon, Nov 22, 2021 at 12:39 PM Ishan Chattopadhyaya
> > > > >  wrote:
> > > > >>
> > > > >> +1 to Uwe's suggestion
> > > > >>
> > > > >> On Mon, 22 Nov, 2021, 11:13 am Gus Heck, 
> > > wrote:
> > > > >>>
> > > > >>> +1 to uwe's suggestion
> > > > >>>
> > > > >>> On Sun, Nov 21, 2021 at 10:42 PM Noble Paul <
> noble.p...@gmail.com>
> > > wrote:
> > > > >>>>
> > > > >>>> I think this is a reasonable suggestion Uwe.
> > > > >>>>
> > > > >>>> - We don't need to bring Gradle to 8.x
> > > > >>>> - We can release 8.12 from a fork of 8.11.
> > > > >>>> - we don't need to keep the Lucene source files in that branch.
> We can
> > > nuke it and just keep the Lucene binaries
> > > > >>>>
> > > > >>>> On Mon, Nov 22, 2021, 8:49 AM Uwe Schindler 
> > > wrote:
> > > > >>>>>
> > > > >>>>> Hi,
> > > > >>>>>
> > > > >>>>> If this is really needed, I'd propose the following:
> > > > >>>>>
> > > > >>>>> - fork the branch_8_11 to solr's repo
> > > > >>>>> - delete all subdirectories below lucene, keep common-build
> and other
> > > stuff.
> > > > >>>>> - add a single ivy.xml there that refers to all lucene jars of
> 8.11.x
> > > (latest)
> > > > >>>>> - adapt solr's "copy-lucene-jars" ant task to copy the ivy
> output dir
> > > > >>>>> - delete the lucene stuff from release wizard.
> > > > >>>&

Re: What should we do of branch_8x?

[Ishan Chattopadhyaya]

gt;>>>> Uwe
>>>>>>
>>>>>>
>>>>>>
>>>>>> -
>>>>>>
>>>>>> Uwe Schindler
>>>>>>
>>>>>> Achterdiek 19, D-28357 Bremen
>>>>>>
>>>>>> https://www.thetaphi.de
>>>>>>
>>>>>> eMail: u...@thetaphi.de
>>>>>>
>>>>>>
>>>>>>
>>>>>> *From:* Gus Heck 
>>>>>> *Sent:* Sunday, November 21, 2021 5:05 PM
>>>>>> *To:* dev 
>>>>>> *Subject:* Re: What should we do of branch_8x?
>>>>>>
>>>>>>
>>>>>>
>>>>>> Release of Solr 8.12 It should require the current lucene-solr 8.x
>>>>>> branch to remove the lucene bits and declare a dependency on lucene 8.11
>>>>>> lucene, that bit shouldn't be too hard if done soon... and the release
>>>>>> process for 8.x would not publish a lucene artifact which is likely the
>>>>>> harder bit. I think the option should be open assuming someone is willing
>>>>>> to do that work.What should not be an option is any further lucene 
>>>>>> releases
>>>>>> on 8.x  and I'd be very leery of any attempt to consume lucene 9.0 on 
>>>>>> Solr
>>>>>> 8.x
>>>>>>
>>>>>>
>>>>>>
>>>>>> The Lucene guarantees are irrelevant unless someone contemplates
>>>>>> releasing an 8.12 lucene, and I really think that would require a 
>>>>>> positive
>>>>>> vote from the Lucene PMC (which sounds very unlikely since I see fingers
>>>>>> twitching over the -1 holsters there :) )
>>>>>>
>>>>>>
>>>>>>
>>>>>> So while I don't favor deleting the entire solr 8.x branch I think
>>>>>> it's now fine to remove lucene from it.
>>>>>>
>>>>>>
>>>>>>
>>>>>> To make things pretty, one could push the 8.x branch to the solr repo
>>>>>> AFTER lucene is removed, but that sounds like busy work unless there is
>>>>>> some formal or financial need to close the old repo. They are now fully
>>>>>> separate projects and what solr does with the non-lucene bits is not a
>>>>>> concern to lucene pmc (though almost all of us are on both committees of
>>>>>> course, but hat wearing etc..)
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sun, Nov 21, 2021 at 8:43 AM Robert Muir  wrote:
>>>>>>
>>>>>> I dunno, this seems really crazy to me. Splitting out solr into its
>>>>>> own repository and allowing it to be released independently from
>>>>>> lucene has already been done, lots of work :) Why not just move
>>>>>> forwards?
>>>>>>
>>>>>> On Sun, Nov 21, 2021 at 8:16 AM Ishan Chattopadhyaya
>>>>>>  wrote:
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > On Sun, 21 Nov, 2021, 6:31 pm Robert Muir, 
>>>>>> wrote:
>>>>>> >>
>>>>>> >> Sorry, I just don't understand the implications of what you are
>>>>>> suggesting.
>>>>>> >>
>>>>>> >> The code in question is lucene+solr combined, and the build system
>>>>>> and
>>>>>> >> packaging and everything only knows how to do that. So are you
>>>>>> forking
>>>>>> >> all the lucene code into the solr repo too?
>>>>>> >
>>>>>> >
>>>>>> > Need to split it up and remove the Lucene code from there in order
>>>>>> to be able to release Solr independently. We can do so later (I'm 
>>>>>> currently
>>>>>> on travel), if/when needed.
>>>>>> >>
>>>>>> >>
>>>>>> >> I don't really understand your need to have a branch_8x. we can
>>>>>> nuke
>>>>>> >> it, and you can do any of this from a branch_8_11 some other day,
>>>>>> no?
>>>>>> >
>>>>>> >
>>>>>> > I guess we can, just don't know t

Re: What should we do of branch_8x?

[Ishan Chattopadhyaya]

On Sun, 21 Nov, 2021, 6:31 pm Robert Muir,  wrote:

> Sorry, I just don't understand the implications of what you are suggesting.
>
> The code in question is lucene+solr combined, and the build system and
> packaging and everything only knows how to do that. So are you forking
> all the lucene code into the solr repo too?
>

Need to split it up and remove the Lucene code from there in order to be
able to release Solr independently. We can do so later (I'm currently on
travel), if/when needed.

>
> I don't really understand your need to have a branch_8x. we can nuke
> it, and you can do any of this from a branch_8_11 some other day, no?
>

I guess we can, just don't know the divergence. Just to be on the safer
side, don't want to lose access to the branch_8x over a weekend before I or
persons more knowledgeable (on the differences between the branches) than I
get a chance to review the situation. Hence, I just copied the branch there
for the moment.

>
> On Sun, Nov 21, 2021 at 7:57 AM Ishan Chattopadhyaya
>  wrote:
> >
> > > I don't think the solr PMC should issue Lucene 8.12 either.
> > I never expressed any intention of doing so. Besides, is it even
> possible (ASF policies wise)?
> >
> > This is a weekend, and I feel bad holding up the 9.0 release (since this
> is a blocker). Solr PMC can decide later on Solr's releases, and hence I'm
> going to copy this branch_8x over to Solr repo's "lucene-solr/branch_8x"
> branch.
> >
> >
> > On Sun, Nov 21, 2021 at 6:14 PM Robert Muir  wrote:
> >>
> >> I don't think the solr PMC should issue Lucene 8.12 either.
> >>
> >> On Sun, Nov 21, 2021 at 7:42 AM Ishan Chattopadhyaya
> >>  wrote:
> >> >
> >> > Sounds good, Rob. Should I copy over the branch_8x to the solr repo
> until we have further clarity on the course of action to be taken with Solr
> releases?
> >> >
> >> > On Sun, 21 Nov, 2021, 6:10 pm Robert Muir,  wrote:
> >> >>
> >> >> Nope, it isn't crazy. I am trying to ensure the backwards
> >> >> compatibility that we have is on solid, sustainable footing before we
> >> >> release a new version promising double the back compat.
> >> >>
> >> >> On Sun, Nov 21, 2021 at 7:37 AM Ishan Chattopadhyaya
> >> >>  wrote:
> >> >> >
> >> >> > Solr doesn't have backward compatability tests, only Lucene has.
> >> >> >
> >> >> > That's why I proposed leaving the door open for a Solr 8.12
> release based on already released 8.11 Lucene and not releasing any further
> 8.x minor version release of Lucene.
> >> >> >
> >> >> > As I said, if that's problematic to do on branch_8x of
> lucene-solr, then we can do so in the solr repo. If some urgent action to
> nuke the branch is to be taken, please give some time to explore
> alternatives that affect Solr's developement.
> >> >> >
> >> >> > Holding up Lucene 9.0 release for removal of branch_8x is lunacy,
> not the continued existence of this branch in the shared repo, since a
> future course of action should be deliberated upon before nuking the branch.
> >> >> >
> >> >> > On Sun, 21 Nov, 2021, 5:34 pm Uwe Schindler, 
> wrote:
> >> >> >>
> >> >> >> Hi,
> >> >> >>
> >> >> >> I fully agree with Robert here.
> >> >> >>
> >> >> >> I originally sent the question about branch_8x because of this.
> Once we released Lucene 9.0 wen can't release 8.12, because the index file
> format will be brand marked as originating from 8.12 then, which 9.0 will
> refuse to read.
> >> >> >>
> >> >> >> We can only release 8.11.x which is not allowed to have index
> format changes and minor version numbers are not persisted.
> >> >> >>
> >> >> >> So -1 to release a 8.12 an time in future. If you still want one,
> hold 9.0 release and add precautions for this.
> >> >> >>
> >> >> >> Imho. Let's stop releasing 8.12 or later for Lucene/Solr and just
> add Bugfixes. This also applies to Solr. Later this is decoupled, so Solr
> 9.1234 may use Lucene 10.4711.
> >> >> >>
> >> >> >> As said before: let's close branch 8.x and add protection to it
> in GitHub. Anybox may merge Bugfixes directly from Solr or Lucene main I to
> branch_8_11. I see no problem. Just no index changes!
> >> >> >>
> >> >> >> U

Re: What should we do of branch_8x?

[Ishan Chattopadhyaya]

> I don't think the solr PMC should issue Lucene 8.12 either.
I never expressed any intention of doing so. Besides, is it even possible
(ASF policies wise)?

This is a weekend, and I feel bad holding up the 9.0 release (since this is
a blocker). Solr PMC can decide later on Solr's releases, and hence I'm
going to copy this branch_8x over to Solr repo's "lucene-solr/branch_8x"
branch.


On Sun, Nov 21, 2021 at 6:14 PM Robert Muir  wrote:

> I don't think the solr PMC should issue Lucene 8.12 either.
>
> On Sun, Nov 21, 2021 at 7:42 AM Ishan Chattopadhyaya
>  wrote:
> >
> > Sounds good, Rob. Should I copy over the branch_8x to the solr repo
> until we have further clarity on the course of action to be taken with Solr
> releases?
> >
> > On Sun, 21 Nov, 2021, 6:10 pm Robert Muir,  wrote:
> >>
> >> Nope, it isn't crazy. I am trying to ensure the backwards
> >> compatibility that we have is on solid, sustainable footing before we
> >> release a new version promising double the back compat.
> >>
> >> On Sun, Nov 21, 2021 at 7:37 AM Ishan Chattopadhyaya
> >>  wrote:
> >> >
> >> > Solr doesn't have backward compatability tests, only Lucene has.
> >> >
> >> > That's why I proposed leaving the door open for a Solr 8.12 release
> based on already released 8.11 Lucene and not releasing any further 8.x
> minor version release of Lucene.
> >> >
> >> > As I said, if that's problematic to do on branch_8x of lucene-solr,
> then we can do so in the solr repo. If some urgent action to nuke the
> branch is to be taken, please give some time to explore alternatives that
> affect Solr's developement.
> >> >
> >> > Holding up Lucene 9.0 release for removal of branch_8x is lunacy, not
> the continued existence of this branch in the shared repo, since a future
> course of action should be deliberated upon before nuking the branch.
> >> >
> >> > On Sun, 21 Nov, 2021, 5:34 pm Uwe Schindler,  wrote:
> >> >>
> >> >> Hi,
> >> >>
> >> >> I fully agree with Robert here.
> >> >>
> >> >> I originally sent the question about branch_8x because of this. Once
> we released Lucene 9.0 wen can't release 8.12, because the index file
> format will be brand marked as originating from 8.12 then, which 9.0 will
> refuse to read.
> >> >>
> >> >> We can only release 8.11.x which is not allowed to have index format
> changes and minor version numbers are not persisted.
> >> >>
> >> >> So -1 to release a 8.12 an time in future. If you still want one,
> hold 9.0 release and add precautions for this.
> >> >>
> >> >> Imho. Let's stop releasing 8.12 or later for Lucene/Solr and just
> add Bugfixes. This also applies to Solr. Later this is decoupled, so Solr
> 9.1234 may use Lucene 10.4711.
> >> >>
> >> >> As said before: let's close branch 8.x and add protection to it in
> GitHub. Anybox may merge Bugfixes directly from Solr or Lucene main I to
> branch_8_11. I see no problem. Just no index changes!
> >> >>
> >> >> Uwe
> >> >>
> >> >> Am 21. November 2021 11:51:34 UTC schrieb Robert Muir <
> rcm...@gmail.com>:
> >> >>>
> >> >>> I gave my technical justification: our backwards compatibility
> testing
> >> >>> doesnt work this way. 9.0 can't have guaranteed back compat with
> >> >>> versions coming in the future. This is lunacy.
> >> >>>
> >> >>> On Sun, Nov 21, 2021 at 6:30 AM Ishan Chattopadhyaya
> >> >>>  wrote:
> >> >>>>
> >> >>>>
> >> >>>>  https://www.apache.org/foundation/voting.html#Veto
> >> >>>>
> >> >>>>  "To prevent vetoes from being used capriciously, the voter must
> provide with the veto a *technical justification* showing why the change is
> bad (opens a security exposure, negatively affects performance, etc. ). A
> veto without a justification is invalid and has no weight."
> >> >>>>
> >> >>>>  On Sun, 21 Nov, 2021, 3:30 pm Robert Muir, 
> wrote:
> >> >>>>>
> >> >>>>>
> >> >>>>>  I think we should remove this branch.
> >> >>>>>
> >> >>>>>  personally, i'll probably -1 any commit to it. I'll see if i can
> >> >>>>>  automate such an email response with a gmail rule.
> >> >>>>>
> >> >>>>>  we already released lucene 9.0, we can't change backwards
> >> >>>>>  compatibility for some 8.12, same old story, lets move on people.
> >> >>>>>
> >> >>>>>  On Sat, Nov 20, 2021 at 9:29 AM Adrien Grand 
> wrote:
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>  Uwe brought up the question on a the vote thread: we are not
> going to do a 8.12 release, so what should we do of branch_8x?
> >> >>>>>
> >> >>>>> 
> >> >>>>>  To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> >> >>>>>  For additional commands, e-mail: dev-h...@lucene.apache.org
> >> >>>>>
> >> >>> 
> >> >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> >> >>> For additional commands, e-mail: dev-h...@lucene.apache.org
> >> >>>
> >> >> --
> >> >> Uwe Schindler
> >> >> Achterdiek 19, 28357 Bremen
> >> >> https://www.thetaphi.de
>

Re: What should we do of branch_8x?

[Ishan Chattopadhyaya]

Sounds good, Rob. Should I copy over the branch_8x to the solr repo until
we have further clarity on the course of action to be taken with Solr
releases?

On Sun, 21 Nov, 2021, 6:10 pm Robert Muir,  wrote:

> Nope, it isn't crazy. I am trying to ensure the backwards
> compatibility that we have is on solid, sustainable footing before we
> release a new version promising double the back compat.
>
> On Sun, Nov 21, 2021 at 7:37 AM Ishan Chattopadhyaya
>  wrote:
> >
> > Solr doesn't have backward compatability tests, only Lucene has.
> >
> > That's why I proposed leaving the door open for a Solr 8.12 release
> based on already released 8.11 Lucene and not releasing any further 8.x
> minor version release of Lucene.
> >
> > As I said, if that's problematic to do on branch_8x of lucene-solr, then
> we can do so in the solr repo. If some urgent action to nuke the branch is
> to be taken, please give some time to explore alternatives that affect
> Solr's developement.
> >
> > Holding up Lucene 9.0 release for removal of branch_8x is lunacy, not
> the continued existence of this branch in the shared repo, since a future
> course of action should be deliberated upon before nuking the branch.
> >
> > On Sun, 21 Nov, 2021, 5:34 pm Uwe Schindler,  wrote:
> >>
> >> Hi,
> >>
> >> I fully agree with Robert here.
> >>
> >> I originally sent the question about branch_8x because of this. Once we
> released Lucene 9.0 wen can't release 8.12, because the index file format
> will be brand marked as originating from 8.12 then, which 9.0 will refuse
> to read.
> >>
> >> We can only release 8.11.x which is not allowed to have index format
> changes and minor version numbers are not persisted.
> >>
> >> So -1 to release a 8.12 an time in future. If you still want one, hold
> 9.0 release and add precautions for this.
> >>
> >> Imho. Let's stop releasing 8.12 or later for Lucene/Solr and just add
> Bugfixes. This also applies to Solr. Later this is decoupled, so Solr
> 9.1234 may use Lucene 10.4711.
> >>
> >> As said before: let's close branch 8.x and add protection to it in
> GitHub. Anybox may merge Bugfixes directly from Solr or Lucene main I to
> branch_8_11. I see no problem. Just no index changes!
> >>
> >> Uwe
> >>
> >> Am 21. November 2021 11:51:34 UTC schrieb Robert Muir  >:
> >>>
> >>> I gave my technical justification: our backwards compatibility testing
> >>> doesnt work this way. 9.0 can't have guaranteed back compat with
> >>> versions coming in the future. This is lunacy.
> >>>
> >>> On Sun, Nov 21, 2021 at 6:30 AM Ishan Chattopadhyaya
> >>>  wrote:
> >>>>
> >>>>
> >>>>  https://www.apache.org/foundation/voting.html#Veto
> >>>>
> >>>>  "To prevent vetoes from being used capriciously, the voter must
> provide with the veto a *technical justification* showing why the change is
> bad (opens a security exposure, negatively affects performance, etc. ). A
> veto without a justification is invalid and has no weight."
> >>>>
> >>>>  On Sun, 21 Nov, 2021, 3:30 pm Robert Muir,  wrote:
> >>>>>
> >>>>>
> >>>>>  I think we should remove this branch.
> >>>>>
> >>>>>  personally, i'll probably -1 any commit to it. I'll see if i can
> >>>>>  automate such an email response with a gmail rule.
> >>>>>
> >>>>>  we already released lucene 9.0, we can't change backwards
> >>>>>  compatibility for some 8.12, same old story, lets move on people.
> >>>>>
> >>>>>  On Sat, Nov 20, 2021 at 9:29 AM Adrien Grand 
> wrote:
> >>>>>>
> >>>>>>
> >>>>>>  Uwe brought up the question on a the vote thread: we are not going
> to do a 8.12 release, so what should we do of branch_8x?
> >>>>>
> >>>>> 
> >>>>>  To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> >>>>>  For additional commands, e-mail: dev-h...@lucene.apache.org
> >>>>>
> >>> 
> >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> >>> For additional commands, e-mail: dev-h...@lucene.apache.org
> >>>
> >> --
> >> Uwe Schindler
> >> Achterdiek 19, 28357 Bremen
> >> https://www.thetaphi.de
>

Re: What should we do of branch_8x?

[Ishan Chattopadhyaya]

Solr doesn't have backward compatability tests, only Lucene has.

That's why I proposed leaving the door open for a Solr 8.12 release based
on already released 8.11 Lucene and not releasing any further 8.x minor
version release of Lucene.

As I said, if that's problematic to do on branch_8x of lucene-solr, then we
can do so in the solr repo. If some urgent action to nuke the branch is to
be taken, please give some time to explore alternatives that affect Solr's
developement.

Holding up Lucene 9.0 release for removal of branch_8x is lunacy, not the
continued existence of this branch in the shared repo, since a future
course of action should be deliberated upon before nuking the branch.

On Sun, 21 Nov, 2021, 5:34 pm Uwe Schindler,  wrote:

> Hi,
>
> I fully agree with Robert here.
>
> I originally sent the question about branch_8x because of this. Once we
> released Lucene 9.0 wen can't release 8.12, because the index file format
> will be brand marked as originating from 8.12 then, which 9.0 will refuse
> to read.
>
> We can only release 8.11.x which is not allowed to have index format
> changes and minor version numbers are not persisted.
>
> So -1 to release a 8.12 an time in future. If you still want one, hold 9.0
> release and add precautions for this.
>
> Imho. Let's stop releasing 8.12 or later for Lucene/Solr and just add
> Bugfixes. This also applies to Solr. Later this is decoupled, so Solr
> 9.1234 may use Lucene 10.4711.
>
> As said before: let's close branch 8.x and add protection to it in GitHub.
> Anybox may merge Bugfixes directly from Solr or Lucene main I to
> branch_8_11. I see no problem. Just no index changes!
>
> Uwe
>
> Am 21. November 2021 11:51:34 UTC schrieb Robert Muir :
>>
>> I gave my technical justification: our backwards compatibility testing
>> doesnt work this way. 9.0 can't have guaranteed back compat with
>> versions coming in the future. This is lunacy.
>>
>> On Sun, Nov 21, 2021 at 6:30 AM Ishan Chattopadhyaya
>>  wrote:
>>
>>>
>>>  https://www.apache.org/foundation/voting.html#Veto
>>>
>>>  "To prevent vetoes from being used capriciously, the voter must provide 
>>> with the veto a *technical justification* showing why the change is bad 
>>> (opens a security exposure, negatively affects performance, etc. ). A veto 
>>> without a justification is invalid and has no weight."
>>>
>>>  On Sun, 21 Nov, 2021, 3:30 pm Robert Muir,  wrote:
>>>
>>>>
>>>>  I think we should remove this branch.
>>>>
>>>>  personally, i'll probably -1 any commit to it. I'll see if i can
>>>>  automate such an email response with a gmail rule.
>>>>
>>>>  we already released lucene 9.0, we can't change backwards
>>>>  compatibility for some 8.12, same old story, lets move on people.
>>>>
>>>>  On Sat, Nov 20, 2021 at 9:29 AM Adrien Grand  wrote:
>>>>
>>>>>
>>>>>  Uwe brought up the question on a the vote thread: we are not going to do 
>>>>> a 8.12 release, so what should we do of branch_8x?
>>>>>
>>>> --
>>>>  To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>>>  For additional commands, e-mail: dev-h...@lucene.apache.org
>>>>
>>>> --
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>> --
> Uwe Schindler
> Achterdiek 19, 28357 Bremen
> https://www.thetaphi.de
>

Re: What should we do of branch_8x?

[Ishan Chattopadhyaya]

https://www.apache.org/foundation/voting.html#Veto

"To prevent vetoes from being used capriciously, the voter must provide
with the veto a *technical justification* showing why the change is bad
(opens a security exposure, negatively affects performance, *etc.* ). A
veto without a justification is invalid and has no weight."

On Sun, 21 Nov, 2021, 3:30 pm Robert Muir,  wrote:

> I think we should remove this branch.
>
> personally, i'll probably -1 any commit to it. I'll see if i can
> automate such an email response with a gmail rule.
>
> we already released lucene 9.0, we can't change backwards
> compatibility for some 8.12, same old story, lets move on people.
>
> On Sat, Nov 20, 2021 at 9:29 AM Adrien Grand  wrote:
> >
> > Uwe brought up the question on a the vote thread: we are not going to do
> a 8.12 release, so what should we do of branch_8x?
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: What should we do of branch_8x?

[Ishan Chattopadhyaya]

Since there is no concrete design available for that as of today, that's
why I mentioned about "keeping the door open" for 8.12. I'm not proposing a
8.12 today, nor am I saying 8.12 is needed. But, in case we need one, we
should have the ability to release it. Anyway, this discussion should
rather happen on the Solr list.

On Sat, 20 Nov, 2021, 10:10 pm Timothy Potter,  wrote:

> A Solr 8.12 with Lucene 8.11? Not sure of the details on that but
> sounds like a giant mess waiting to happen (at the very least, would
> require a bunch of complicated changes to the release process). We
> need to stop adding features to 8x and focus on 9. I can foresee an
> 8.11.2 with bug fixes only (8.11.1 is already planned to drop
> soon'ish). Why would Solr need an 8.12? I suspect it's related to
> upgrading plugins, but that's been an open issue for a long while and
> seems to keep getting pushed out. We can't just keep planning new
> feature releases because of this plugin upgrade problem. If we really
> need an 8.12, then we need to see a concrete design on how the upgrade
> process will work in an 8.12. Perhaps there's a better approach that
> only relies on code changes to the 9x line? Tough to say, we have no
> designs or descriptions of the upgrade problem at this point.
>
> As of today, I'd be strongly against a Solr 8.12 release.
>
> Tim
>
> On Sat, Nov 20, 2021 at 8:32 AM Ishan Chattopadhyaya
>  wrote:
> >
> > I think we should keep the door open for a 8.12 release of Solr (based
> on 8.11 Lucene). This might mean some split in the codebase, and this can
> either happen in the lucene-solr repo or the solr repo (I'm okay with
> either).
> >
> > On Sat, Nov 20, 2021 at 7:59 PM Adrien Grand  wrote:
> >>
> >> Uwe brought up the question on a the vote thread: we are not going to
> do a 8.12 release, so what should we do of branch_8x?
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: What should we do of branch_8x?

[Ishan Chattopadhyaya]

I think we should keep the door open for a 8.12 release of Solr (based on
8.11 Lucene). This might mean some split in the codebase, and this can
either happen in the lucene-solr repo or the solr repo (I'm okay with
either).

On Sat, Nov 20, 2021 at 7:59 PM Adrien Grand  wrote:

> Uwe brought up the question on a the vote thread: we are not going to do a
> 8.12 release, so what should we do of branch_8x?
>

Re: Time to write an open-source book?

[Ishan Chattopadhyaya]

+1, great idea.

On Wed, 17 Nov, 2021, 2:19 pm Michael Wechner, 
wrote:

> I think this would be great and I would be very happy to contribute.
>
> For example I am currently trying to understand how the autocomplete /
> auto suggest functionality of Lucene works and I could contribute my
> learnings.
>
> All the best
>
> Michael
>
> Am 16.11.21 um 20:49 schrieb Dongyu Xu:
>
> Hi Devs,
>
> I'm finally motivated enough to start this thread as I believe this is a
> great thing to do for the Lucene community to continuously thrive as the
> library has become so feature-rich but as the same time much more complex.
>
> "What do you recommend to read for learning more Lucene?" -- A question I
> was often asked by my friends and coworkers at Amazon Product Search . I'm
> sure many of you have experienced the same. I always recommend Lucene In
> Action 2nd Edition[1] which is a great book. However, it features *Lucene
> 3.0*
> and we are at *Lucene 9.0* now! There is a huge gap.
>
> Inspired by my recent experience with the Rust Book[2] and the Solr ref
> guide[3], I believe it is possible for the Lucene community to collaborate
> on writing a book/user guide just like how the software is built in the
> open-source way!
>
> Concretely, it will require to first draw the outline of the book with
> clear
> intentions for all sections. Then the effort should be able to scale,
> allow-
> ing individuals to work on different sections in parallel. Once built, the
> book should be a live artifact and evolve together with Lucene.
>
> Thoughts?
>
> [1]
> https://www.amazon.com/Lucene-Action-Second-Covers-Apache/dp/1933988177
> [2] https://github.com/rust-lang/book
> [3] https://github.com/apache/solr/tree/main/solr/solr-ref-guide
>
>
>
> Thanks,
> Tony
>
>
>

Re: Is it Time to Deprecate the Legacy Facets API

[Ishan Chattopadhyaya]

> Personally I'd love to see us stop maintaining the duplicated code of
> the underlying implementations.  I wouldn't mind losing the legacy
> syntax as well - I'll take a clear, verbose API over a less-clear,
> concise one any day.  But I'm probably a minority there.

+1, agree with Jason here, fully.

On Wed, Oct 27, 2021 at 8:37 PM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Should we deprecate classic faceting in 9x now?
>
> > It's worth investigating deprecating the stats component also. I believe
> JSON facets covers that functionality as well. It will be painful for users
> though to switch over unfortunately.
>
> +1, lets deprecate stats component too.
>
>
> On Thu, Jan 28, 2021 at 5:22 AM Joel Bernstein  wrote:
>
>> It's worth investigating deprecating the stats component also. I believe
>> JSON facets covers that functionality as well. It will be painful for users
>> though to switch over unfortunately.
>>
>>
>> Joel Bernstein
>> http://joelsolr.blogspot.com/
>>
>>
>> On Fri, Jan 22, 2021 at 1:14 PM Jason Gerlowski 
>> wrote:
>>
>>> Personally I'd love to see us stop maintaining the duplicated code of
>>> the underlying implementations.  I wouldn't mind losing the legacy
>>> syntax as well - I'll take a clear, verbose API over a less-clear,
>>> concise one any day.  But I'm probably a minority there.
>>>
>>> Either way I agree with Michael when he said above that the first step
>>> would have to be a parity investigation for features and performance.
>>>
>>> Best,
>>>
>>> Jason
>>>
>>> On Fri, Jan 22, 2021 at 10:05 AM Michael Gibney
>>>  wrote:
>>> >
>>> > I agree it would make long-term sense to consolidate the backend
>>> implementation. I think leaving the "classic" user-facing facet API (with
>>> JSON Facet module as a backend) would be a good idea. Either way, I think a
>>> first step would be checking for parity between existing backend
>>> implementations -- possibly in terms of features [1], but certainly in
>>> terms of performance for common use cases [2].
>>> >
>>> > I think removal of the "classic" user-facing API would cause a lot of
>>> consternation in the user community. I can even see a
>>> non-backward-compatibility argument for preserving the "classic"
>>> user-facing API: it's simpler for simple use cases. _If_ the ultimate goal
>>> is removal of the "classic" user-facing API (not presuming that it is),
>>> that approach could be facilitated in the short term by enticing users
>>> towards "JSON Facet" API ... basically with a "feature freeze" on the
>>> legacy implementation. No new features [3], no new optimizations [4] for
>>> "classic"; concentrate such efforts on JSON Facet. This seems to already be
>>> the de facto case, but it could be a more intentional decision -- e.g. in
>>> [3] it's straightforward to extend the the proposed "facet cache" to the
>>> "classic" impl ... but I could see an argument for intentionally not doing
>>> so.
>>> >
>>> > Robert, I think your concerns about UninvertedField could be addressed
>>> by the `uninvertible="false"` property (currently defaults to "true" for
>>> backward compatibility iiuc; but could default to "false", or at least
>>> provide the ability to set the default for all fields to "false" at node
>>> level solr.xml? -- I know I've wished for the latter!). Also fwiw I'm not
>>> aware of any JSON Facet processors that work with string values in RAM ...
>>> I do think all JSON Facet processors use OrdinalMap now, where relevant.
>>> >
>>> > [1] https://issues.apache.org/jira/browse/SOLR-14921
>>> > [2] https://issues.apache.org/jira/browse/SOLR-14764
>>> > [3] https://issues.apache.org/jira/browse/SOLR-13807
>>> > [4] https://issues.apache.org/jira/browse/SOLR-10732
>>> >
>>> > On Fri, Jan 22, 2021 at 12:46 AM Robert Muir  wrote:
>>> >>
>>> >> Do these two options conflate concerns of input format vs. actual
>>> >> algorithm? That was always my disappointment.
>>> >>
>>> >> I feel like the java apis are off here at the lower level, and it
>>> >> hurts the user.
>>> >> I don't talk about the input format from the user, instead I mean the
>>> >> execution of

Re: Is it Time to Deprecate the Legacy Facets API

[Ishan Chattopadhyaya]

Should we deprecate classic faceting in 9x now?

> It's worth investigating deprecating the stats component also. I believe
JSON facets covers that functionality as well. It will be painful for users
though to switch over unfortunately.

+1, lets deprecate stats component too.


On Thu, Jan 28, 2021 at 5:22 AM Joel Bernstein  wrote:

> It's worth investigating deprecating the stats component also. I believe
> JSON facets covers that functionality as well. It will be painful for users
> though to switch over unfortunately.
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
>
> On Fri, Jan 22, 2021 at 1:14 PM Jason Gerlowski 
> wrote:
>
>> Personally I'd love to see us stop maintaining the duplicated code of
>> the underlying implementations.  I wouldn't mind losing the legacy
>> syntax as well - I'll take a clear, verbose API over a less-clear,
>> concise one any day.  But I'm probably a minority there.
>>
>> Either way I agree with Michael when he said above that the first step
>> would have to be a parity investigation for features and performance.
>>
>> Best,
>>
>> Jason
>>
>> On Fri, Jan 22, 2021 at 10:05 AM Michael Gibney
>>  wrote:
>> >
>> > I agree it would make long-term sense to consolidate the backend
>> implementation. I think leaving the "classic" user-facing facet API (with
>> JSON Facet module as a backend) would be a good idea. Either way, I think a
>> first step would be checking for parity between existing backend
>> implementations -- possibly in terms of features [1], but certainly in
>> terms of performance for common use cases [2].
>> >
>> > I think removal of the "classic" user-facing API would cause a lot of
>> consternation in the user community. I can even see a
>> non-backward-compatibility argument for preserving the "classic"
>> user-facing API: it's simpler for simple use cases. _If_ the ultimate goal
>> is removal of the "classic" user-facing API (not presuming that it is),
>> that approach could be facilitated in the short term by enticing users
>> towards "JSON Facet" API ... basically with a "feature freeze" on the
>> legacy implementation. No new features [3], no new optimizations [4] for
>> "classic"; concentrate such efforts on JSON Facet. This seems to already be
>> the de facto case, but it could be a more intentional decision -- e.g. in
>> [3] it's straightforward to extend the the proposed "facet cache" to the
>> "classic" impl ... but I could see an argument for intentionally not doing
>> so.
>> >
>> > Robert, I think your concerns about UninvertedField could be addressed
>> by the `uninvertible="false"` property (currently defaults to "true" for
>> backward compatibility iiuc; but could default to "false", or at least
>> provide the ability to set the default for all fields to "false" at node
>> level solr.xml? -- I know I've wished for the latter!). Also fwiw I'm not
>> aware of any JSON Facet processors that work with string values in RAM ...
>> I do think all JSON Facet processors use OrdinalMap now, where relevant.
>> >
>> > [1] https://issues.apache.org/jira/browse/SOLR-14921
>> > [2] https://issues.apache.org/jira/browse/SOLR-14764
>> > [3] https://issues.apache.org/jira/browse/SOLR-13807
>> > [4] https://issues.apache.org/jira/browse/SOLR-10732
>> >
>> > On Fri, Jan 22, 2021 at 12:46 AM Robert Muir  wrote:
>> >>
>> >> Do these two options conflate concerns of input format vs. actual
>> >> algorithm? That was always my disappointment.
>> >>
>> >> I feel like the java apis are off here at the lower level, and it
>> >> hurts the user.
>> >> I don't talk about the input format from the user, instead I mean the
>> >> execution of the faceting query.
>> >>
>> >> IMO: building top-level caches (e.g. uninvertedfield) or
>> >> on-the-fly-caches (e.g. fieldcache) is totally trappy already.
>> >> But with the uninvertedfield of json facets it does its own thing,
>> >> even if you went thru the trouble to enable docvalues at index time:
>> >> that's sad.
>> >>
>> >> the code by default should not give the user jvm
>> >> heap/garbage-collector hell. If you want to do that to yourself, for a
>> >> totally static index, IMO that should be opt-in.
>> >>
>> >> But for the record, it is no longer just two shitty choices like
>> >> "top-level vs per-segment". There are different field types, e.g.
>> >> numeric types where the per-segment approach works efficiently.
>> >> Then you have the strings, but there is a newish middle ground for
>> >> Strings: OrdinalMap (lucene Multi* interfaces do it) which builds
>> >> top-level integers structures to speed up string-faceting, but doesnt
>> >> need *string values* in ram.
>> >> It is just integers and mostly compresses as deltas. Adrien compresses
>> >> the shit out of it.
>> >>
>> >> So I'd hate for the user to lose the option here of using docvalues to
>> >> keep faceting out of heap memory, which should not be hassling them
>> >> already in 2021.
>> >> Maybe better to refactor the code such that all these concerns aren't
>> 

Re: Soften Jira's note when opening new issues?

[Ishan Chattopadhyaya]

+1

On Mon, 20 Sep, 2021, 12:33 pm Adrien Grand,  wrote:

> Hello,
>
> Jira gives the following note when opening an issue:
>
> ```
> This project has a user mailing list and an IRC channel for support.
> Please ensure that you have discussed your problem using one of those
> resources BEFORE creating this ticket.
> ```
>
> This can be quite intimidating for someone who has never worked with us
> before, and we don't apply this logic for ourselves, for instance I feel
> free to open JIRAs without discussing them first on IRC or dev@l.a.o.
> Given that we are not seeing much irrelevant traffic on JIRA, I'd like to
> soften the message to something like below:
>
> ```
> If you are looking for support, or if you are not sure whether the
> behavior that you are observing is expected or not, please discuss your
> problem on the user mailing-list instead before creating a ticket.
> ```
>
> What do you think?
>
> --
> Adrien
>

Re: New branch and feature freeze for Lucene/Solr 8.10.0

[Ishan Chattopadhyaya]

gt;
>>> >> : > > >> and it’s looking for Implementation-Version: 8.10.0
>>> 75a5061d3715cc5d93c4cbe4f1fa62bf035eea11 on one line.
>>> >> : > > >>
>>> >> : > > >> Because 8.10 is a character longer than 8.9, we happen to
>>> wrap the last character of the git commit sha. From the manifest spec:
>>> >> : > > >>
>>> >> : > > >> No line may be longer than 72 bytes (not characters), in its
>>> UTF8-encoded form.
>>> >> : > > >> If a value would make the initial line longer than this, it
>>> should be continued
>>> >> : > > >> on extra lines (each starting with a single SPACE).
>>> >> : > > >>
>>> >> : > > >> And we were already teetering on the edge of that limit.
>>> We'll run into this problem again in a few years when we try to release
>>> version 10.0.0, so solving it now has practical benefits down the line.
>>> >> : > > >>
>>> >> : > > >> There's a few options that I can come up with -
>>> >> : > > >> 1. Use the short-hash when we generate the jar
>>> >> : > > >> 2. Use the short-hash when we check the contents in the
>>> smoke test
>>> >> : > > >> 3. Do some line join magic in the smoke test.
>>> >> : > > >>
>>> >> : > > >> I'm leaning towards number 1 as I feel that would still be
>>> unique enough for our needs, but would like to hear from others as well.
>>> >> : > > >>
>>> >> : > > >> On Wed, Sep 15, 2021 at 9:46 AM Timothy potter <
>>> thelabd...@gmail.com> wrote:
>>> >> : > > >>>
>>> >> : > > >>> can someone also please look into that benchmark jar issue?
>>> >> : > > >>>
>>> >> : > > >>> Sent from my iPhone
>>> >> : > > >>>
>>> >> : > > >>> On Sep 15, 2021, at 9:44 AM, Nhat Nguyen <
>>> nhat.ngu...@elastic.co.invalid> wrote:
>>> >> : > > >>>
>>> >> : > > >>> 
>>> >> : > > >>> Thanks Mayya and Mike! I will backport it to the 8.10
>>> branch.
>>> >> : > > >>>
>>> >> : > > >>> On Wed, Sep 15, 2021 at 10:12 AM Mike Drob 
>>> wrote:
>>> >> : > > >>>>
>>> >> : > > >>>> I think since Tim is out on vacation, it's probably not
>>> too late. That looks like a good fix to have, do we know how long the bug
>>> has been present?
>>> >> : > > >>>>
>>> >> : > > >>>> On Wed, Sep 15, 2021 at 7:56 AM Mayya Sharipova <
>>> mayya.sharip...@elastic.co.invalid> wrote:
>>> >> : > > >>>>>
>>> >> : > > >>>>> Hello everyone,
>>> >> : > > >>>>> We have discovered a bug and fixed a bug in Lucene sort
>>> optimization (LUCENE-10106) and would like to merge it to Lucene 8.10 if it
>>> is not too late.
>>> >> : > > >>>>> I apologize for the inconvenience, the bug was discovered
>>> just yesterday.
>>> >> : > > >>>>>
>>> >> : > > >>>>> On Tue, Sep 14, 2021 at 9:26 PM Timothy Potter <
>>> thelabd...@apache.org> wrote:
>>> >> : > > >>>>>>
>>> >> : > > >>>>>> Ahem ... unfortunately there will not be an 8.10 RC this
>>> week. I'm
>>> >> : > > >>>>>> headed out on vacation tomorrow, back at keys on Monday,
>>> Sept 20
>>> >> : > > >>>>>> unless someone else wants to pick up the RM duties
>>> before then?
>>> >> : > > >>>>>>
>>> >> : > > >>>>>> After failing the test suite at various places and other
>>> weirdness
>>> >> : > > >>>>>> like .asc files not getting created, I finally got to
>>> the smoke test
>>> >> : > > >>>>>> part, which is now failing with:
>>> >> : > > >>>>>>
>>> >> : > > >>>>

Re: New branch and feature freeze for Lucene/Solr 8.10.0

[Ishan Chattopadhyaya]

All the best, this is the worst step.

On Tue, 14 Sep, 2021, 10:47 pm Timothy Potter,  wrote:

> Building RC1 now ... stay tuned.
>
> On Thu, Sep 9, 2021 at 2:30 PM Timothy Potter 
> wrote:
> >
> > Thanks for the update Mike!
> >
> > I'm backporting SOLR-15620 right now and am cooking up a quick PR for
> > SOLR-15621, which looks like an easy win for the issue Cassandra
> > reported on Slack earlier today.
> >
> > Cheers,
> > Tim
> >
> > On Thu, Sep 9, 2021 at 11:32 AM Mike Drob  wrote:
> > >
> > > Hi Tim, I'm still working on SOLR-1, the code and benchmarking
> > > both look pretty good, but I've got a few last unit tests that I need
> > > to chase down. Hopefully taken care of by today or tomorrow, I'll be
> > > sure to keep you updated though.
> > >
> > >
> > > On Thu, Sep 9, 2021 at 11:39 AM Timothy Potter 
> wrote:
> > > >
> > > > I found https://issues.apache.org/jira/browse/SOLR-15620 while
> testing
> > > > the schema designer. I haven't built the RC yet, so going to see if I
> > > > can get this in today.
> > > >
> > > > On Tue, Sep 7, 2021 at 12:36 PM Timothy Potter <
> thelabd...@apache.org> wrote:
> > > > >
> > > > > NOTICE:
> > > > >
> > > > > Branch branch_8_10 has been cut and versions updated to 8.11 on
> stable branch.
> > > > >
> > > > > Please observe the normal rules:
> > > > >
> > > > > * No new features may be committed to the branch.
> > > > >
> > > > > * Documentation patches, build patches and serious bug fixes may be
> > > > >   committed to the branch. However, you should submit all patches
> you
> > > > >   want to commit to Jira first to give others the chance to review
> > > > >   and possibly vote against the patch. Keep in mind that it is our
> > > > >   main intention to keep the branch as stable as possible.
> > > > >
> > > > > * All patches that are intended for the branch should first be
> committed
> > > > >   to the unstable branch, merged into the stable branch, and then
> into
> > > > >   the current release branch.
> > > > >
> > > > > * Normal unstable and stable branch development may continue as
> usual.
> > > > >   However, if you plan to commit a big change to the unstable
> branch
> > > > >   while the branch feature freeze is in effect, think twice: can't
> the
> > > > >   addition wait a couple more days? Merges of bug fixes into the
> branch
> > > > >   may become more difficult.
> > > > >
> > > > > * Only Jira issues with Fix version 8.10 and priority "Blocker"
> will delay
> > > > >   a release candidate build.
> > > > > 
> > > >
> > > > -
> > > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > > > For additional commands, e-mail: dev-h...@lucene.apache.org
> > > >
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > > For additional commands, e-mail: dev-h...@lucene.apache.org
> > >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org
> For additional commands, e-mail: dev-h...@solr.apache.org
>
>

Re: Welcome Mayya Sharipova to the Lucene PMC

[Ishan Chattopadhyaya]

Congratulations and welcome!

On Mon, 28 Jun, 2021, 7:55 pm Mike Drob,  wrote:

> Welcome!
>
> On Mon, Jun 28, 2021 at 7:21 AM Erik Hatcher 
> wrote:
>
>> Ditto - congrats and a huge welcome!
>>
>> Erik
>>
>>
>> > On Jun 28, 2021, at 9:16 AM, Robert Muir  wrote:
>> >
>> > I am pleased to announce that Mayya has accepted an invitation to join
>> > the Lucene PMC!
>> >
>> > Congratulations, and welcome aboard!
>> >
>> > -
>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> > For additional commands, e-mail: dev-h...@lucene.apache.org
>> >
>>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>

Re: [VOTE] Release Lucene/Solr 8.9.0 RC1

[Ishan Chattopadhyaya]

Hi Rob, could it be possible that the tests are timing out on your machine
due to lack of resources? Can you try running them with just just one JVM
at a time?

On Sat, 12 Jun, 2021, 8:20 pm Robert Muir,  wrote:

> I ran smoketester yet one more time and again numerous tests fail:
>
>[junit4] Tests with failures [seed: A3FDDCE09965D7AE] (first 10 out of
> 18):
>[junit4]   - org.apache.solr.update.TestHdfsUpdateLog.testFSThreadSafety
>[junit4]   - org.apache.solr.update.TestHdfsUpdateLog (suite)
>[junit4]   -
> org.apache.solr.cloud.hdfs.HDFSCollectionsAPITest.testDataDirIsNotReused
>[junit4]   - org.apache.solr.cloud.hdfs.HdfsRecoverLeaseTest.testBasic
>[junit4]   -
> org.apache.solr.cloud.hdfs.HdfsRecoverLeaseTest.testMultiThreaded
>[junit4]   - org.apache.solr.cloud.hdfs.HdfsRecoverLeaseTest (suite)
>[junit4]   -
>
> org.apache.solr.core.backup.repository.HdfsBackupRepositoryIntegrationTest.testCanDistinguishBetweenFilesAndDirectories
>[junit4]   -
>
> org.apache.solr.core.backup.repository.HdfsBackupRepositoryIntegrationTest.testCanDeleteEmptyOrFullDirectories
>[junit4]   -
>
> org.apache.solr.core.backup.repository.HdfsBackupRepositoryIntegrationTest.testCanDeleteIndividualFiles
>[junit4]   -
>
> org.apache.solr.core.backup.repository.HdfsBackupRepositoryIntegrationTest.testArbitraryFileDataCanBeStoredAndRetrieved
>[junit4]
>[junit4]
>[junit4] JVM J0: 0.72 ..  1241.80 =  1241.08s
>[junit4] JVM J1: 0.67 ..  1198.72 =  1198.05s
>[junit4] JVM J2: 0.91 ..  1198.75 =  1197.84s
>[junit4] Execution time total: 20 minutes 41 seconds
>[junit4] Tests summary: 939 suites (5 ignored), 4884 tests, 3
> suite-level errors, 15 errors, 1 failure, 2581 ignored (506
> assumptions)
>
> On Fri, Jun 11, 2021 at 11:52 AM Robert Muir  wrote:
> >
> > After nuking all settings (i simply removed the whole
> > lucene.build.properties in my homedir), it still fails. Seems maybe
> > like less failures though?
> >
> > I will upload logs to the JIRA issue.
> >
> >[junit4] Completed [939/939 (4!)] on J2 in 383.02s, 2 tests, 1
> > failure <<< FAILURES!
> >[junit4]
> >[junit4]
> >[junit4] Tests with failures [seed: AC205159663D0461]:
> >[junit4]   -
> org.apache.solr.update.TestHdfsUpdateLog.testFSThreadSafety
> >[junit4]   - org.apache.solr.update.TestHdfsUpdateLog (suite)
> >[junit4]   -
> > org.apache.solr.core.HdfsDirectoryFactoryTest.testLocalityReporter
> >[junit4]   - org.apache.solr.cloud.hdfs.HdfsRecoverLeaseTest.testBasic
> >[junit4]   -
> > org.apache.solr.cloud.hdfs.HdfsRecoverLeaseTest.testMultiThreaded
> >[junit4]   - org.apache.solr.cloud.hdfs.HdfsRecoverLeaseTest (suite)
> >[junit4]   -
> > org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test
> >[junit4]
> >[junit4]
> >[junit4] JVM J0: 0.68 ..  1197.30 =  1196.62s
> >[junit4] JVM J1: 0.71 ..  1113.59 =  1112.89s
> >[junit4] JVM J2: 0.68 ..  1406.83 =  1406.15s
> >[junit4] Execution time total: 23 minutes 26 seconds
> >[junit4] Tests summary: 939 suites (5 ignored), 4884 tests, 3
> > suite-level errors, 4 errors, 1 failure, 2457 ignored (517
> > assumptions)
> >
> > BUILD FAILED
> >
> /tmp/smoke_lucene_8.9.0_05c8a6f0163fe4c330e93775e8e91f3ab66a3f80/unpack/solr-8.9.0/solr/build.xml:231:
> > The following error occurred while executing this line:
> >
> /tmp/smoke_lucene_8.9.0_05c8a6f0163fe4c330e93775e8e91f3ab66a3f80/unpack/solr-8.9.0/solr/common-build.xml:550:
> > The following error occurred while executing this line:
> >
> /tmp/smoke_lucene_8.9.0_05c8a6f0163fe4c330e93775e8e91f3ab66a3f80/unpack/solr-8.9.0/lucene/common-build.xml:1608:
> > The following error occurred while executing this line:
> >
> /tmp/smoke_lucene_8.9.0_05c8a6f0163fe4c330e93775e8e91f3ab66a3f80/unpack/solr-8.9.0/lucene/common-build.xml:1135:
> > There were test failures: 939 suites (5 ignored), 4884 tests, 3
> > suite-level errors, 4 errors, 1 failure, 2457 ignored (517
> > assumptions) [seed: AC205159663D0461]
> >
> > Total time: 24 minutes 16 seconds
> >
> >
> > Traceback (most recent call last):
> >   File
> "/home/rmuir/workspace/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
> > line 1495, in 
> > main()
> >   File
> "/home/rmuir/workspace/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
> > line 1417, in main
> > smokeTest(c.java, c.url, c.revision, c.version, c.tmp_dir,
> > c.is_signed, c.local_keys, ' '.join(c.test_args),
> >   File
> "/home/rmuir/workspace/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
> > line 1483, in smokeTest
> > solrSrcUnpackPath = unpackAndVerify(java, 'solr', tmpDir,
> > 'solr-%s-src.tgz' % version,
> >   File
> "/home/rmuir/workspace/lucene-solr/dev-tools/scripts/smokeTestRelease.py",
> > line 566, in unpackAndVerify
> > verifyUnpacked(java, project, artifact, unpackPath, gitRevision,
> > version, testArgs, tmpDir, baseURL)
> >   File
> 

Re: Statement from the Solr and Lucene PMC regarding recent Code of Conduct violations

[Ishan Chattopadhyaya]

Thanks Jan and both the PMCs.

On Tue, 1 Jun, 2021, 1:03 pm Jan Høydahl,  wrote:

> This is a joint statement on behalf of the Apache Solr and Lucene Project
> Management Committees (PMCs).
>
> Some of the language used in May 2021 on the SOLR-14245 JIRA issue [1] and
> the concurrent dev@solr [2] and dev@lucene [3] mailing list threads
> violates our project community's Code of Conduct [4]. Personal attacks,
> blaming, and negative characterizations of others are harmful in open
> source development and will not be tolerated in our community.
>
> The Apache Solr and Lucene PMCs [5][6] have privately discussed the matter
> and wish to publicly and officially censure the referenced behavior of
> Ishan Chattopadhyaya and Noble Paul. Future infractions will have stronger
> consequences. Of course we hope it won't come to that, and we hope this
> addresses any concern in our communities about participating in open-source
> development with civility and respect. Any additional questions or concerns
> regarding the code of conduct can be sent to priv...@solr.apache.org or
> priv...@lucene.apache.org.
>
> [1] https://issues.apache.org/jira/browse/SOLR-14245
> [2]
> https://lists.apache.org/thread.html/rced539c2708284bf94a70c514c181ae643f0d7327a2d603e08422115%40%3Cdev.solr.apache.org%3E
> <https://lists.apache.org/thread.html/rced539c2708284bf94a70c514c181ae643f0d7327a2d603e08422115@%3Cdev.solr.apache.org%3E>
> [3]
> https://lists.apache.org/thread.html/rced539c2708284bf94a70c514c181ae643f0d7327a2d603e08422115%40%3Cdev.lucene.apache.org%3E
> [4] https://solr.apache.org/community.html#code-of-conduct
> [5] https://solr.apache.org/whoweare.html#project-management-committee
> [6]
> https://lucene.apache.org/whoweare.html#lucene-project-management-committee-pmc
>

Re: Doubling down on our mistakes?

[Ishan Chattopadhyaya]

I apologize for the harsh words, and personally to Andrzej for hurting your
feelings. I had no such intentions.

> You conveniently don’t mention that I WITHDREW my objection, and instead
proposed a lenient validation (but validation nonetheless!).
Yes, let me mention that you agreed in principal to reduce the impact of
the change (even though not completely revert it). I welcome that and thank
you for that. By the time you replied on JIRA, I had already sent this mail.

> I see no urgency at all in this matter. This can be handled as day-to-day
bug fixing as usual.
I think this requires an immediate notification to all users to be aware of
this situation before upgrading. Also, an immediate breakfix should be
helpful for them.

> My feelings are hurt, and I'm greatly disappointed in your words, quick
attacking off the cuff regularly rude (IMO) because you happened to have a
bad day.
I apologize.

How I saw things is that we have a commitment to our users to give them
good quality software that they can rely on. My intention was not to attack
Andrzej personally, but to bring about collective awareness regarding this
problem: that we, as a community, don't care enough for our users. We need
to get better at testing, get better at reviews, better at benchmarks, etc.
Individually, we all have the best of intentions, and obviously so does
Andrzej. However, we need to get better, and I wanted this to be a starting
point in that conversation. Clearly, I was carried over and I apologize for
that.

On Tue, May 18, 2021 at 5:52 PM Andrzej Białecki  wrote:

> Ishan, as I pointed out in Jira I don’t care for you implying that I have
> evil intentions, I resent also your implication that I’m behaving
> irrationally or don’t care for the users. Those of you who are interested
> may read the comments in Jira and judge for themselves.
>
> You conveniently don’t mention that I WITHDREW my objection, and instead
> proposed a lenient validation (but validation nonetheless!). It’s easy to
> scream “revert! revert!” but it actually takes some consideration to
> properly address the original purpose of this change - that is, detecting
> and avoiding the corruption of replica state. Let’s focus on this and not
> on pointing fingers.
>
> As for the production outage - I’m sorry this happened to you. As I hope
> you and Noble and others are sorry for other inadvertently introduced bugs,
> which I’m sure brought down many clusters at inconvenient hours...
>
>
> On 18 May 2021, at 13:26, Ishan Chattopadhyaya 
> wrote:
>
> https://issues.apache.org/jira/browse/SOLR-14245
>
> There was a *production outage* at *odd hours* at my (and Noble's)
> client, due to this above change in Solr 8.5 onwards by *Andrzej Bialecki*
> .
>
> In short, there is some bug in Solr where a replica gets "null" as the
> node_name (upon invocation of a collection API command). On the rare
> occasions where we encountered such situations in the past, the replica
> would be unavailable and the system would work fine overall. However, this
> change (which introduces strict validation of errors while *reading*
> Replica objects) now means that if such a situation arises (where some
> Solr's APIs itself results in node_name being null in a state.json), all
> SolrJ clients and all Solr nodes will go for a toss (possibly crash, and
> not start back up).
>
> This change was rushed in, *without any discussions or review*, without
> extensive testing for the failures it will cause on existing systems where
> cluster state is messed up but system is running, and *without any
> consideration for the impact on users*.
>
> Noble and I are of the opinion that this change should be *reverted
> immediately*, considering the impact to users. However, there is *strong
> disagreement on Andrzej's part*.
>
> *Mistakes* happen, but *doubling down on them irrationally* [1] will
> destroy the reputation of the project, let alone the peace of mind of those
> who are running Solr in production.
>
> Does someone have any thoughts or opinions?
>
> [1] -
> https://issues.apache.org/jira/browse/SOLR-14245?focusedCommentId=17346758=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17346758
>
>
>

Doubling down on our mistakes?

[Ishan Chattopadhyaya]

https://issues.apache.org/jira/browse/SOLR-14245

There was a *production outage* at *odd hours* at my (and Noble's) client,
due to this above change in Solr 8.5 onwards by *Andrzej Bialecki*.

In short, there is some bug in Solr where a replica gets "null" as the
node_name (upon invocation of a collection API command). On the rare
occasions where we encountered such situations in the past, the replica
would be unavailable and the system would work fine overall. However, this
change (which introduces strict validation of errors while *reading*
Replica objects) now means that if such a situation arises (where some
Solr's APIs itself results in node_name being null in a state.json), all
SolrJ clients and all Solr nodes will go for a toss (possibly crash, and
not start back up).

This change was rushed in, *without any discussions or review*, without
extensive testing for the failures it will cause on existing systems where
cluster state is messed up but system is running, and *without any
consideration for the impact on users*.

Noble and I are of the opinion that this change should be *reverted
immediately*, considering the impact to users. However, there is *strong
disagreement on Andrzej's part*.

*Mistakes* happen, but *doubling down on them irrationally* [1] will
destroy the reputation of the project, let alone the peace of mind of those
who are running Solr in production.

Does someone have any thoughts or opinions?

[1] -
https://issues.apache.org/jira/browse/SOLR-14245?focusedCommentId=17346758=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17346758

Re: Release Lucene/Solr 8.9.0 should we have it soon

[Ishan Chattopadhyaya]

Hi Mayya,
How much time would you give us for this release (until feature freeze)? I
have some package manager related changes (that might take a week to fully
build and test) that need to get into 8x so that users can seamlessly
upgrade to 9x. Based on your proposed timeline, I'll work on it immediately
(if there's time), or later (for a potential 8.10 or 8.9.1). A bit caught
up in the covid situation, and AFK (so can't find the JIRA).
Thanks for volunteering,
Ishan

On Tue, May 11, 2021 at 10:59 PM Mayya Sharipova
 wrote:

> Thanks everyone,
>
> Adrien, I  am happy to try to be a release manager for this release.
>
> Adrien, and Gus, please let me know when your changes are merged to 8.x
>
>
>
> On Tue, May 11, 2021 at 10:38 AM Gus Heck  wrote:
>
>> I'm also looking to find time to get
>> https://issues.apache.org/jira/browse/SOLR-14597 into some sort of 8x.
>> I've recently completed the back port of 2/3 of the lucene tickets that are
>> related, and hope to work on the third tomorrow
>>
>> I had some feedback there, but I think folks were waiting for the version
>> integrated with the final form of the Lucene tickets before delving
>> further. Hopefully this week I can start on a patch that does that.
>>
>> On Tue, May 11, 2021 at 10:25 AM Adrien Grand  wrote:
>>
>>> I would like to backport LUCENE-9827
>>> <https://issues.apache.org/jira/browse/LUCENE-9827> before we release
>>> 8.9, a performance regression to stored fields merges. I'll work on this as
>>> soon as possible.
>>>
>>> On Thu, May 6, 2021 at 10:28 PM Adrien Grand  wrote:
>>>
>>>> +1
>>>>
>>>> Mayya, are you volunteering to be the release manager?
>>>>
>>>> Le jeu. 6 mai 2021 à 18:06, Ishan Chattopadhyaya <
>>>> ichattopadhy...@gmail.com> a écrit :
>>>>
>>>>> +1
>>>>>
>>>>> On Thu, May 6, 2021 at 7:50 PM Mayya Sharipova
>>>>>  wrote:
>>>>>
>>>>>> Hello everyone,
>>>>>> I was wondering if we can have a 8.9.0 release. It has been more than
>>>>>> 3 months since 8.8.0 was released.
>>>>>> 8.9.0 doesn't need to be the last release in the 8.x series.
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>
>>>
>>> --
>>> Adrien
>>>
>>
>>
>> --
>> http://www.needhamsoftware.com (work)
>> http://www.the111shift.com (play)
>>
>

Re: Release Lucene/Solr 8.9.0 should we have it soon

[Ishan Chattopadhyaya]

+1

On Thu, May 6, 2021 at 7:50 PM Mayya Sharipova
 wrote:

> Hello everyone,
> I was wondering if we can have a 8.9.0 release. It has been more than 3
> months since 8.8.0 was released.
> 8.9.0 doesn't need to be the last release in the 8.x series.
>
> Thanks.
>

Re: Welcome Zach Chen as Lucene committer

[Ishan Chattopadhyaya]

Congrats, Zach! Thanks for your contributions, looking forward to more!

On Tue, 20 Apr, 2021, 2:26 pm Alan Woodward,  wrote:

> Congratulations and welcome!
>
> > On 19 Apr 2021, at 15:13, Adrien Grand  wrote:
> >
> > I'm pleased to announce that Zach Chen has accepted the PMC's invitation
> to become a committer.
> >
> > Zach, the tradition is that new committers introduce themselves with a
> brief bio.
> >
> > Congratulations and welcome!
> >
> > --
> > Adrien
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Welcome Peter Gromov as Lucene committer

[Ishan Chattopadhyaya]

Congrats and welcome, Peter!

On Tue, 6 Apr, 2021, 11:18 pm Robert Muir,  wrote:

> I'm pleased to announce that Peter Gromov has accepted the PMC's
> invitation to become a committer.
>
> Peter, the tradition is that new committers introduce themselves with a
> brief bio.
>
> Congratulations and welcome!
>
>

Re: Bugfix release Lucene/Solr 8.8.2

[Ishan Chattopadhyaya]

Hi Mike,

I wish to get https://issues.apache.org/jira/browse/SOLR-15288 in, but will
likely be able to wrap up by 2 April or so (on vacation right now due to
the festival of Holi)

Regards,
Ishan

On Sat, 27 Mar, 2021, 7:41 am Mike Drob,  wrote:

> I am now preparing for a bugfix release from branch branch_8_8
>
> I plan to have the RC built and vote started on Tuesday, Mar 30. If you
> have small, low risk bug fixes to backport before then, please do so using
> your best judgement.
>
> Please observe the normal rules for committing to this branch:
>
> * Before committing to the branch, reply to this thread and argue
>   why the fix needs backporting and how long it will take.
> * All issues accepted for backporting should be marked with 8.8.2
>   in JIRA, and issues that should delay the release must be marked as
> Blocker
> * All patches that are intended for the branch should first be committed
>   to the unstable branch, merged into the stable branch, and then into
>   the current release branch.
> * Only Jira issues with Fix version 8.8.2 and priority "Blocker" will delay
>   a release candidate build.
>
> Thanks,
> Mike
>

Re: SOLR-15056 circuit breaker bugfix?

[Ishan Chattopadhyaya]

I don't think asking contributors to do more work than required (as per
documentation) is a good idea.

On Wed, 24 Mar, 2021, 10:13 am Atri Sharma,  wrote:

> I did share steps for your reference the last time we talked about this:
>
>
> https://markmail.org/message/2buvbboyoy7ip4v7?q=list:org%2Eapache%2Elucene%2Esolr-dev
>
> On Wed, Mar 24, 2021 at 9:57 AM Walter Underwood 
> wrote:
> >
> > If patches are hard for people, then the How to Contribute page needs to
> be rewritten to tell people to create PRs. I followed that page when doing
> this work.
> >
> > It should have specific instructions on how to make PRs, maybe branch
> naming, etc. I make PRs all day at work, but I never use github and I don’t
> know what Solr wants.
> >
> > wunder
> > Walter Underwood
> > wun...@wunderwood.org
> > http://observer.wunderwood.org/  (my blog)
> >
> > On Mar 23, 2021, at 8:10 PM, Atri Sharma  wrote:
> >
> > +1
> >
> > I tried reviewing the patch and echoed AB's point of needing a PR. Happy
> to do the review as soon as we have that.
> >
> > On Wed, 24 Mar 2021, 03:04 Anshum Gupta,  wrote:
> >>
> >> Hi Walter,
> >>
> >> Can you please create a PR, as Ab mentioned in the JIRA. That would
> make it much easier to review considering the size of the patch. It will
> also be easier to comment and iterate that way.
> >>
> >> -Anshum
> >>
> >> On Tue, Mar 23, 2021 at 2:24 PM Walter Underwood 
> wrote:
> >>>
> >>> The patch for SOLR-15056 was submitted over two months ago. It fixes a
> bug, adds a new kind of circuit breaker, improves the documentation, and
> has unit tests.
> >>>
> >>> I’ll make it into a PR if that is required, bit it is discouraging to
> put a bunch of work into a fix and have it ignored. This is a much better
> submission than the bar in Yonik’s Law of Patches.
> >>>
> >>> https://issues.apache.org/jira/browse/SOLR-15056
> >>>
> >>> wunder
> >>> Walter Underwood
> >>> wun...@wunderwood.org
> >>> http://observer.wunderwood.org/  (my blog)
> >>>
> >>
> >>
> >> --
> >> Anshum Gupta
> >
> >
>
>
> --
> Regards,
>
> Atri
> Apache Concerted
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Repository fork (master) about to happen (Wednesday)

[Ishan Chattopadhyaya]

I'd like to commit SOLR-14381 and LUCENE-9302 together before the fork,
please. I'll hopefully wrap up by Tuesday.

On Mon, 8 Mar, 2021, 9:51 pm Uwe Schindler,  wrote:

> Hi again,
>
> we can maybe "improve" the situation a bit: On Github you can (with
> Admin/Ownership rights) rename a project. So my suggestion:
>
> - Check pull requests and count how many affect solr and how many affect
> Lucene.
> - In cooperation with Infra rename the Github project
> ("apache/lucene-solr.git") to "apache/lucene.git" (if more pull requests
> affect Lucene) or "apache/solr.git" (if more are Solr). The PRs will
> survive the rename. Also the old GitHub URL will redirect to the renamed
> one. The other project should be created as a fork - of course without PRs.
>
> We can only do this in cooperation with Apache Infra stuff, because we
> can' change the Github repo settings or rename them using the Github UI.
>
> Uwe
>
> -
> Uwe Schindler
> Achterdiek 19, D-28357 Bremen
> https://www.thetaphi.de
> eMail: u...@thetaphi.de
>
> > -Original Message-
> > From: Uwe Schindler 
> > Sent: Monday, March 8, 2021 5:16 PM
> > To: dev@lucene.apache.org
> > Subject: RE: Repository fork (master) about to happen (Wednesday)
> >
> > I think the problem was what happens with the PR in Githubs user
> interface.
> >
> > This question was asked many times, answer is simple: NO you can't move
> over
> > Pull requests to different repositories on Github! It's also impossible
> to export
> > and reimport them. People have to recreate them.
> >
> > Dawid is correct: You can merge the pull request also into another rlocal
> > repository, but this is impossible with the Github UI. So basically, you
> have to
> > read the email that comes in with the Pull Request that lists the link
> to the
> > branch and patch. Then use git command line (or Tortoise) and copypaste
> the
> > URL there as "source branch" for the merge. Then you execute the merge,
> > squash and commit/push.
> >
> > Uwe
> >
> > -
> > Uwe Schindler
> > Achterdiek 19, D-28357 Bremen
> > https://www.thetaphi.de
> > eMail: u...@thetaphi.de
> >
> > > -Original Message-
> > > From: Dawid Weiss 
> > > Sent: Monday, March 8, 2021 3:25 PM
> > > To: Lucene Dev 
> > > Subject: Re: Repository fork (master) about to happen (Wednesday)
> > >
> > > > What happens to open PRs?
> > >
> > > A pull request on github is essentially a diff between two commits.
> > > Existing PRs have to be placed on top of the new development branch.
> > > Note these repositories are (initially) identical with lucene-solr so
> > > if somebody clones the solr repo and the solr-lucene repo, they can
> > > create the same PR over at the new project (pointing at the new main
> > > branch as a reference).
> > >
> > > Dawid
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > > For additional commands, e-mail: dev-h...@lucene.apache.org
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: dev-h...@lucene.apache.org
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Solr webpage SEO

[Ishan Chattopadhyaya]

We can add robots.txt to stop Google from indexing/showing in results.

On Thu, 4 Mar, 2021, 2:34 pm Jan Høydahl,  wrote:

> Hi, sending to this list since dev@solr list is not yet announced
> properly.
>
> We have a few days of traffic to the new site and can see the most visited
> pages at https://uls.apache.org/exports/solr.apache.org.yaml (see copy
> below).
> When I search google for "solr query parser", I get the 6.6 guide on top,
> which is probably why /guide/6_6/the-standard-query-parser.html shows up,
> and the same for the other /guide/6_6/ links.
> Some questions:
>
>
>- How can we make Google forget about version 6.6? I know we had a
>bunch of redirects from Confluence to the 6.6 guide, are they still in
>place?
>- Why is /docs/6_6_0/solr-core/index.html the 2nd most visited page?
>Anywhere that links to it?
>- Why is /docs/4_8_1/solr-solrj/index.html so high? Ahywhere that
>links to it?
>- The /mirrors-solr-latest-redir.html redirect was not working. I just
>pushed a fix
>
>
>
> Sheet3:
>   Name: Most visited pages, past month
>   Values:
> /index.html: 443
> /docs/6_6_0/solr-core/index.html: 281
> /guide/8_8/solr-tutorial.html: 104
> /news.html: 92
> /guide/solr-tutorial.html: 91
> /resources.html: 75
> /features.html: 69
> /docs/8_7_0/solr-core/index.html: 68
> /downloads.html: 68
> /guide/6_6/the-standard-query-parser.html: 65
> /docs/4_8_1/solr-solrj/index.html: 62
> /guide/6_6/common-query-parameters.html: 50
> /guide/index.html: 46
> /docs/8_8_1/solr-solrj/index.html: 44
> /docs/8_7_0/solr-solrj/index.html: 38
> /docs/8_8_1/solr-core/index.html: 37
> /community.html: 24
> /guide/8_8/: 23
> /guide/6_6/uploading-data-with-index-handlers.html: 22
> /docs/8_6_3/solr-core/index.html: 21
> /guide/6_6/filter-descriptions.html: 21
> /guide/6_6/collections-api.html: 18
> /docs/8_6_2/solr-solrj/overview-summary.html: 16
> /guide/6_6/faceting.html: 16
> /mirrors-solr-latest-redir.html: 15
> /whoweare.html: 15
> /guide/6_6/solrcloud.html: 14
> /guide/6_6/tokenizers.html: 13
> /guide/7_0/solr-configuration-files.html: 13
> /guide/8_8/query-syntax-and-parsing.html: 13
> /security.html: 13
> /guide/6_6/introduction-to-solr-indexing.html: 12
> /guide/8_8/solr-upgrade-notes.html: 12
> /docs/8_0_0/solr-solrj/allclasses-frame.html: 11
> /docs/8_6_3/solr-solrj/index.html: 11
> /guide/6_6/the-dismax-query-parser.html: 11
> /guide/8_8/getting-started.html: 11
> /guide/solr-upgrade-notes.html: 11
> /docs/8_0_0/solr-solrj/overview-summary.html: 10
> /docs/8_6_2/solr-solrj/index.html: 10
> /guide/6_6/running-solr.html: 10
> /docs/7_2_1/solr-solrj/overview-summary.html: 9
> /docs/8_0_0/solr-solrj/overview-frame.html: 9
> /guide/6_6/format-of-solr-xml.html: 9
> /guide/6_6/index.html: 9
> /guide/6_6/learning-to-rank.html: 9
> /guide/6_6/making-and-restoring-backups.html: 9
> /guide/6_6/working-with-dates.html: 9
> /guide/8_0/reindexing.html: 9
> /docs/7_2_1/solr-solrj/allclasses-frame.html: 8
>
>
>

Re: Who has access to Google Analytics for Lucene site?

[Ishan Chattopadhyaya]

I am opposed, in principle, to letting a private company have access to the
IP addresses and other personally identifiable information of our users.

On Wed, Mar 3, 2021 at 8:11 PM Robert Muir  wrote:

> I'm not trying to come across as anti-analytics, i'm not. But I feel a lot
> of those questions can be answered by the aggregate stats already provided
> by apache (presumably from httpd access_log), without adding
> privacy-invading-google-tracker javascripts and cookies. So, while your
> answers are good, they don't justify google analytics in my eyes.
>
> As an example, lets look at
> https://uls.apache.org/exports/lucene.apache.org.yaml and consider your
> list
> 1. You can see breakdown of pageviews and "visitors" by day. I don't know
> how they determine unique "visitor" since it isn't cookie tracking: maybe
> some combo of (IP address, TLS session ID, user agent), but whatever they
> have is good enough for me.
> 2. I can see most popular pages and your 6.6 ref guide stuff
> 3. Top referrers gives you a rough idea of where people are coming from
> (including internal referrers). So people are clicking links on those
> pages.
> 4. see #1.
> 5. see #3. Google provides no additional magic here, this is referer (sic)
> header either way.
> 6. i think the download process is actually hacked up/convoluted just to
> force some GA tracking. At least i know if i disable javascript, the
> download buttons still work.
> 7. what is missing?
>
>
> On Wed, Mar 3, 2021 at 9:15 AM Alexandre Rafalovitch 
> wrote:
>
>> I block any analytics I can find. I am with you on the overall
>> positioning. And yes, the absolute numbers lie.
>>
>> At the same time, we can get a lot of relative numbers and trends that
>> are valuable in other ways.
>>
>> For example:
>> 1) Are the social media announcements of new releases drive people to
>> download Solr?
>> 2) Which Ref Guide pages (if we had GA there) are most popular and why
>> can't we convince users to use the latest version instead of 6.6 (looking
>> at referrals). My specific peeve is that I think URPs page should be a lot
>> more visible, I would love to see if my assumptions are true by seeing if
>> people discover that page, relative to other pages.
>> 3) What is the page flow on the website? Are there any pages that are
>> complete invisible because of how we linked to them? Are there super
>> popular pages that are completely out of date?
>> 4) Do we have increase or decrease in traffic matching specific events
>> 5) Is there a specific partner/agency site that is driving a lot of
>> attention to Solr; can we replicate that with others?
>> 6) Do we even count downloads in GA? Because GA is for HTML pages only by
>> default
>> 7) If any of this is valuable, but we want to pull out GA anyway, this
>> would help to know what tracking information we would like from Apache
>> Infra?
>>
>> In general, these kinds of questions are the domain of Developer
>> Relationships role. Lucene/Solr project does not have one as such, which
>> may explain why not many people understand the values of modern analytics
>> solutions. I am offering my time to make the value of analytics concrete,
>> so we are making the next decision based on  reality rather than our
>> collective imagination of what analytics actually does or does not.
>>
>> Regards,
>>Alex.
>>
>>
>>
>>
>> On Wed., Mar. 3, 2021, 8:40 a.m. Robert Muir,  wrote:
>>
>>>
>>>
>>> On Wed, Mar 3, 2021 at 8:35 AM Michael Sokolov 
>>> wrote:
>>>
 Before you look, should we have a betting pool on the number of
 downloads/day? I will arrange for a bottle of some excellent liquid to
 be sent to the closest guess at the number of redirects to the mirror
 sites, as determined by Alexandre. Also, has it been increasing over
 the last year? Finally, if we can predict these trends using activity
 on the main apache site, maybe we don't need to track independently.

>>>
>>> Why do we even care?
>>>
>>> How many users are downloading lucene tgz from the site versus using an
>>> artifact in maven repositories (via maven, gradle, etc)? How many users are
>>> downloading solr tgz from the site versus using solr official image from
>>> docker hub?
>>>
>>> I'm just asking these questions to try to understand the need for the
>>> google tracking.
>>>
>>>

Re: Who has access to Google Analytics for Lucene site?

[Ishan Chattopadhyaya]

+1 Rob

On Wed, 3 Mar, 2021, 5:55 pm Uwe Schindler,  wrote:

> Hi,
>
>
>
> sorry, I just noticed that the account disappeared from my google
> analytics profile.
>
>
>
> It was setup by Grant Ingersoll, maybe he can give us access again. If it
> is no longer there, we lost the data, but we can recreate one.
>
>
>
> Uwe
>
>
>
> -
>
> Uwe Schindler
>
> Achterdiek 19, D-28357 Bremen
>
> https://www.thetaphi.de
>
> eMail: u...@thetaphi.de
>
>
>
> *From:* Uwe Schindler 
> *Sent:* Wednesday, March 3, 2021 1:10 PM
> *To:* dev@lucene.apache.org
> *Subject:* Re: Who has access to Google Analytics for Lucene site?
>
>
>
> Hi,
> I have access.
> Uwe
>
> Am March 3, 2021 8:10:29 AM UTC schrieb "Jan Høydahl" <
> jan@cominvent.com>:
>
> Hi,
>
> Who has access to the Lucene site GA account? If it is dead in the waters, 
> I'd like to setup a new one also for Lucene.
>
> I plan to publish the new web sites today, would be nice to track and graph 
> the traffic ramp-up.
>
> Jan
>
> --
>
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>
> --
> Uwe Schindler
> Achterdiek 19, 28357 Bremen
> https://www.thetaphi.de
>

Re: Who has access to Google Analytics for Lucene site?

[Ishan Chattopadhyaya]

Lets avoid GA for our website. I don't think it is a good idea to give a
private company (Google Inc) data about our website traffic and their IP
addresses etc.

On Wed, Mar 3, 2021 at 1:40 PM Jan Høydahl  wrote:

> Hi,
>
> Who has access to the Lucene site GA account? If it is dead in the waters,
> I'd like to setup a new one also for Lucene.
>
> I plan to publish the new web sites today, would be nice to track and
> graph the traffic ramp-up.
>
> Jan
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Proposal for the Lucene Dependency after git repo split

[Ishan Chattopadhyaya]

I would prefer Lucene as a git submodule inside Solr. That way, I can work
with solr and Lucene together easily. The commit to which Lucene is pegged
at could be from a release branch or a tag.

On Sat, 27 Feb, 2021, 3:03 am Adrien Grand,  wrote:

> FYI Elasticsearch has been regularly depending on builds of specific
> commits of Lucene for this case of features that need changes both in
> Lucene and Elasticsearch.
>
> The workflow usually looks like this:
>  - Do work in Lucene.
>  - When it becomes clear that the next release of Lucene should happen
> before the next feature freeze of Elasticsearch, we do a new build of
> Lucene and upgrade Elasticsearch to it.
>  - Do work in Elasticsearch.
>  - When a new Lucene release is out, upgrade Elasticsearch to this Lucene
> release.
>
> We have done this dozens of times and it has worked well for us. We do the
> same when a vote for a new Lucene release is about to start in order to
> check whether it breaks anything in Elasticsearch.
>
> The rest of the time (which is most of the time) Elasticsearch depends on
> an actual Lucene release.
>
> Le ven. 26 févr. 2021 à 19:29, Eric Pugh 
> a écrit :
>
>> Plus, isn’t this a reason for folks in the Solr side to continue to be
>> involved in the Lucene project?   It’s the inverse of the days when folks
>> wanted to cut releases of Lucene, and were waiting for Solr to be ready!
>>
>>
>> On Feb 26, 2021, at 1:26 PM, Houston Putman 
>> wrote:
>>
>> I don't think Jan's workflow blocks Solr releases on Lucene releases. It
>> just blocks this one feature branch merge on a Lucene release. Multiple
>> Solr releases can happen between step 6 and step 7.
>>
>> I completely agree with that being the workflow going forward with
>> separate repos, Jan. It will unfortunately be a pain to integrate changes
>> that affect both Lucene and Solr, but I think that's just a consequence of
>> splitting the projects.
>>
>> Neither option gives us everything we want, so here are the pros and cons
>> in my opinion.
>>
>> Using a snapshot lucene version
>> - Easier to make changes to lucene and solr concurrently
>> - Solr releases are blocked until the snapshot version being depended on
>> is released.
>> - Builds may break at any time, and possibly for different sets of users
>> depending on dependency caches.
>>
>> Using a released lucene version
>> - Harder to update lucene and solr concurrently
>> - Solr can make releases independent of Lucene's release schedule
>> - Builds are stable and consistent.
>>
>> Personally I think stability and the ability to own our own release
>> schedule outweigh the benefits of being able to iterate on both projects
>> concurrently. But it's definitely something that we should decide on as a
>> community.
>>
>> On Fri, Feb 26, 2021 at 12:43 PM Mike Drob  wrote:
>>
>>> The part of this process that I really don't like is that Solr then
>>> still becomes beholden to Lucene's release schedule. We don't know how long
>>> step 7 takes, and will be effectively blocked from making our own releases
>>> until that happens.
>>>
>>> On Fri, Feb 26, 2021 at 8:51 AM Jan Høydahl 
>>> wrote:
>>>
 The developer workflow for adding something to both Lucene and Solr
 would be as any other dependency, right?
 So say we are on Luene 9.0. This is the process in my head:

1. Adapt Lucene as needed, and "mvn install" lucene-9.1-SNAPSHOT to
your local laptop (whatever command that is in gradle)
2. Make your Solr feature branch depend on lucene-9.1-SNAPSHOT
instead of lucene-9.0.0 -hopefully Gradle will pick the local version 
 over
Apache Nexus version
3. Iterate 1-2 until happy
4. Make a Lucene PR and eventually commit the Lucene change
5. After next Jenkins build the feature is in Apache Nexus snapshot
as lucene-9.1-SNAPSHOT
6. Now the Solr Pull Request will compile and can be tested by
others
7. Wait until Lucene 9.1 release
8. Upgrade Solr's lucene dependency on 'main'
9. Merge Solr PR
10. Backporting will be a similar process, i.e. first backport and
release in Lucene, then backport in Sol

 Hmm, as I wrote this list I can understand why so many features were
 added only to Solr and not to Lucene in the early days :)

 Jan

 26. feb. 2021 kl. 14:22 skrev Gus Heck :

 Except I just finished helping a contributor with a feature that
 touches both and I know for a fact  that it was developed for his customer
 who was using solr (payload inequalities)... and have another in the works
 (the AQP) Not being able to enhance lucene to support a feature in solr
 is an issue IMHO.

 On Thu, Feb 25, 2021 at 6:05 PM Mike Drob  wrote:

> It is possible to publish snapshots into the Apache Nexus repository.
> That said, I think it is a bad idea for Solr to depend on Lucene snapshots
> because that constrains the 

Re: JIRA issues to close?

[Ishan Chattopadhyaya]

What is the difference between resolving an issue vs closing the issue? Do
we need that extra step of closing an issue?

On Fri, Feb 19, 2021 at 2:05 AM Alexandre Rafalovitch 
wrote:

> You may enjoy using JiraSearch for this kinds of things, it has a very
> nice "Updated Ago >x", and "Last comment user" facet and 'shift-click'
> multi-select on status:
> http://jirasearch.mikemccandless.com/
>
> For the bulk changes, from the screen you are at, you click on Tools
> near top-right area of the screen and select "screen" or "1000" and
> that's the beginning of the process. Makes sure to unselect "notify
> everybody" box on the last (2nd last?) screen or a lot of people will
> be upset.
>
> But also, we had less than fantastic history with bulk closing (had to
> reopen/revert). So, it may be better to focus on a very small, clearly
> "done" subset and/or leave quick notes "can this be closed" with a
> follow-up to close.
>
> Regards,
>Alex.
>
> On Thu, 18 Feb 2021 at 14:43, Eric Pugh 
> wrote:
> >
> > I’m still learning my way around JIRA, and I was looking for a Solr JIRA
> that I swear I had seen open recently.   In trying to query for open/active
> JIRAs, I crafted this query, which returned a lot of tickets that I *think*
> should be CLOSED at this point:
> >
> >
> https://issues.apache.org/jira/browse/SOLR-13394?jql=project%20%3D%20SOLR%20AND%20status%20in%20(Open%2C%20Reopened%2C%20Resolved%2C%20%22Patch%20Available%22)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20in%20releasedVersions()
> >
> > There are 1102 tickets that appear to be old dealt with tickets and
> should be CLOSED.
> >
> > I wanted to point that out, and see if someone could move them to
> CLOSED, or point me to how to do the Bulk Close that I’ve seen happen
> periodically over the years.
> >
> > Eric
> >
> > ___
> > Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com | My Free/Busy
> > Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> > This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless of
> whether attachments are marked as such.
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: [VOTE] Release Lucene/Solr 8.8.1 RC1

[Ishan Chattopadhyaya]

Sounds good, Tim. I've ported the fix to the release branch. Just ran the
tests to make sure it works fine.
Thanks for the extra work you'll have to do (RC2) in order to save me
future work (8.8.2). Really owe you one!

> Are there other fixes you're aware of that are slated for 8.8.2 @Ishan
Chattopadhyaya ?
I am not aware of anything else.

On Tue, Feb 16, 2021 at 9:19 PM Timothy Potter  wrote:

> I'm beasting AutoscalingHistoryHandlerTest locally now, I haven't seen
> that one fail on my side yet.
>
> As far as respin 8.8.1 RC, it's not a problem for me and I prefer that to
> doing an 8.8.2 soon after 8.8.1 comes out. Are there other fixes you're
> aware of that are slated for 8.8.2 @Ishan Chattopadhyaya
> ? In other words, if the fix for 15138 is all
> that will be in 8.8.2, let's just include it in 8.8.1 and hopefully we
> won't need an 8.8.2 ;-)
>
> Tim
>
> On Tue, Feb 16, 2021 at 7:01 AM Michael Sokolov 
> wrote:
>
>> Hmm, I got a failure on
>> org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory,
>> but it did not reproduce (tried twice). Would that possibly also be
>> addressed by those fixes?
>>
>> On Tue, Feb 16, 2021 at 7:38 AM Ishan Chattopadhyaya
>>  wrote:
>> >
>> > > The failure seems to be because of a timeout during collection
>> > > creation
>> >
>> > Thanks for digging in. Seems like that is the exact class of fix that
>> we did for SOLR-15138 and are planning for 8.8.2. Shall we backport that
>> fix to the release branch now (for RC2 or 8.8.2)?
>> >
>> > > My h/w is really fast and beefy and may be that's why it doesn't get
>> reproduced.
>> > Same here, Ryzen 9 5950X (fastest mainstream CPU out there).
>> >
>> > On Tue, Feb 16, 2021 at 5:36 PM Michael McCandless <
>> luc...@mikemccandless.com> wrote:
>> >>
>> >> Curious, the smoke tester passed for me on the first try:
>> >>
>> >> SUCCESS! [0:44:29.979512]
>> >>
>> >>
>> >> Mike McCandless
>> >>
>> >> http://blog.mikemccandless.com
>> >>
>> >>
>> >> On Sun, Feb 14, 2021 at 11:26 AM Timothy Potter 
>> wrote:
>> >>>
>> >>> Please vote for release candidate 1 for Lucene/Solr 8.8.1
>> >>>
>> >>>
>> >>> The artifacts can be downloaded from:
>> >>>
>> >>>
>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.1-RC1-rev6a50a0315ac7e4979abb0b530857c7795bb3b928
>> >>>
>> >>>
>> >>> You can run the smoke tester directly with this command:
>> >>>
>> >>>
>> >>> python3 -u dev-tools/scripts/smokeTestRelease.py \
>> >>>
>> >>>
>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.1-RC1-rev6a50a0315ac7e4979abb0b530857c7795bb3b928
>> >>>
>> >>>
>> >>> The vote will be open for at least 72 hours i.e. until 2021-02-17
>> 17:00 UTC.
>> >>>
>> >>>
>> >>> Here is my +1 ~ SUCCESS! [0:50:06.728441]
>> >>>
>> >>>
>> >>> In addition to the smoke test, I built a Docker image from
>> solr-8.8.1.tgz locally and verified:
>> >>>
>> >>>
>> >>> a. A rolling upgrade of a 3-node 8.7.0 cluster to the 8.8.1 RC
>> completes successfully w/o any NPEs or weirdness with leader election /
>> recoveries.
>> >>>
>> >>>
>> >>> b. The base_url property is stored in replica state after the upgrade
>> >>>
>> >>>
>> >>> c. A basic client application built with SolrJ 8.7.0 can load cluster
>> state info directly from ZK and query the 8.8.1 RC1 servers.
>> >>>
>> >>>
>> >>> d. Same client app built with SolrJ 8.8.0 works as well.
>> >>>
>> >>>
>> >>> As this bug-fix release is primarily needed to address a SolrJ
>> back-compat break (SOLR-15145) and unfortunately our smoke tester framework
>> does not test for backcompat of older SolrJ against the RC, I ask others to
>> please test rolling upgrades of servers (ideally multi-node clusters)
>> running pre-8.8.0 to this RC if possible. Also, please try client
>> applications that are using an older SolrJ, esp. those that load cluster
>> state directly from ZK.
>> >>>
>> >>>
>> >>> Best regards,
>> >>>
>> >>> Tim
>> >>>
>> >>>
>> >>>
>> >>>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>

Re: Separate git repo(s) for Solr modules

[Ishan Chattopadhyaya]

Imagine a component that wants to have its own release cadence. Say, a 1.x
line of the component that supports certain features (across Solr 6x to 9x)
and 2.x line that supports certain other features (which are a big upgrade
from previous 1.x). A Solr UI can be a good example, where say 1.x uses
React and 2.x uses AngularJS.
Shoving such a component into lucene-solr repo makes no sense, given its
branching strategy is based on master / branch_8x (and later it will become
branch_9x and master).

On Tue, Feb 16, 2021 at 5:19 PM Jason Gerlowski 
wrote:

> > separate repo can be used to test against many Solr versions. Imagine a
> component that works across Solr versions 6x through 9x from day one. I
> can't imagine such a component being part of the lucene-solr repo itself.
>
> It'll be great to test contrib's on multiple Solr versions, but I
> can't see any blockers for this that are same-repo specific.  Can you
> elaborate on what makes this tricky in a same-repo setup?  AFAICT you
> can set up multiple gradle test tasks that rely on different
> solr-test-framework versions and run tests that way - regardless of
> where the code lives.   (Or alternatively, docker images for various
> Solr versions as David suggested.)
>
> I think Jan's got a good point about how do-able it'll be in practice
> to have plugins support multiple major release versions.  That said
> though, a lot of these questions (docker tests vs otherwise, cross
> major-version plugins?) will likely sort themselves out when it comes
> time to implement.
>
> On Tue, Feb 16, 2021 at 12:21 AM David Smiley  wrote:
> >
> > I think embracing Docker is how we can identify if a plugin works in
> multiple versions of Solr.  The setup of such a test would start Solr via
> Docker and install the plugin into a Solr server using the package
> manager.  The test code itself would interact with Solr purely via
> SolrClient.  This would be an integration / smoke level test for the
> plugin.  I test my plugins at work similar to this way at a "smoke" level,
> which follows "integration".  What's cool is that we can take the very same
> test and either have it run via an embedded Solr (which we call an
> integration test) or a remote'ed Solr (run via Docker, which we call our
> "smoke" test).  Naturally, if a test makes assumptions that only work in
> embedded, then it won't run in smoke mode, so we have annotations to
> categorize the tests.  I'd like to work on some JUnit "Rule" for Solr,
> similar to what I have at work (and have done in the past) --
> https://issues.apache.org/jira/browse/SOLR-11872
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> > http://www.linkedin.com/in/davidwsmiley
> >
> >
> > On Mon, Feb 15, 2021 at 9:17 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
> >>
> >> Hoss,
> >> One important reason why separate repo for solr-extras is a good idea,
> as opposed to conrib modules, is that separate repo can be used to test
> against many Solr versions. Imagine a component that works across Solr
> versions 6x through 9x from day one. I can't imagine such a component being
> part of the lucene-solr repo itself.
> >>
> >> Also, today, contrib modules are shipped with Solr, which we don't want
> for new things we might want to build.
> >>
> >> On Mon, Jan 25, 2021 at 4:25 PM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
> >>>
> >>> I haven't been able to follow up on creation of the extras repo, but
> more importantly I wanted to respond to Hoss. I'm out on an emergency for a
> week or so, shall resume then. If there's a decision on this until then, I
> shall accept it.
> >>>
> >>> On Mon, 25 Jan, 2021, 9:04 am Jason Gerlowski, 
> wrote:
> >>>>
> >>>> Tentative +1 to Hoss' questions.  I agree with his summary of the
> >>>> potential risks here, and I share his ignorance of the perceived
> >>>> benefits.
> >>>>
> >>>> (I thought for a time that this was driven by interest in having
> >>>> release cadences independent from Solr-core releases.  I'm all for
> >>>> that goal, but if that's the motivation I'm not sure what the obstacle
> >>>> is to doing that with a single repo - all build systems these days
> >>>> support versioning and releasing modules independent of one another.
> >>>> But maybe that was never a driving factor here.)
> >>>>
> >>>> I know there have been a lot of discussions about this, and I know the
> >>>> repo has already b

Re: [VOTE] Release Lucene/Solr 8.8.1 RC1

[Ishan Chattopadhyaya]

> The failure seems to be because of a timeout during collection
> creation

Thanks for digging in. Seems like that is the exact class of fix that we
did for SOLR-15138 and are planning for 8.8.2. Shall we backport that fix
to the release branch now (for RC2 or 8.8.2)?

> My h/w is really fast and beefy and may be that's why it doesn't get
reproduced.
Same here, Ryzen 9 5950X (fastest mainstream CPU out there).

On Tue, Feb 16, 2021 at 5:36 PM Michael McCandless <
luc...@mikemccandless.com> wrote:

> Curious, the smoke tester passed for me on the first try:
>
> SUCCESS! [0:44:29.979512]
>
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Sun, Feb 14, 2021 at 11:26 AM Timothy Potter 
> wrote:
>
>> Please vote for release candidate 1 for Lucene/Solr 8.8.1
>>
>>
>> The artifacts can be downloaded from:
>>
>>
>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.1-RC1-rev6a50a0315ac7e4979abb0b530857c7795bb3b928
>>
>>
>> You can run the smoke tester directly with this command:
>>
>>
>> python3 -u dev-tools/scripts/smokeTestRelease.py \
>>
>>
>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.1-RC1-rev6a50a0315ac7e4979abb0b530857c7795bb3b928
>>
>>
>> The vote will be open for at least 72 hours i.e. until 2021-02-17 17:00
>> UTC.
>>
>>
>> Here is my +1 ~ SUCCESS! [0:50:06.728441]
>>
>>
>> In addition to the smoke test, I built a Docker image from
>> solr-8.8.1.tgz locally and verified:
>>
>>
>> a. A rolling upgrade of a 3-node 8.7.0 cluster to the 8.8.1 RC completes
>> successfully w/o any NPEs or weirdness with leader election / recoveries.
>>
>>
>> b. The base_url property is stored in replica state after the upgrade
>>
>>
>> c. A basic client application built with SolrJ 8.7.0 can load cluster
>> state info directly from ZK and query the 8.8.1 RC1 servers.
>>
>>
>> d. Same client app built with SolrJ 8.8.0 works as well.
>>
>>
>> As this bug-fix release is primarily needed to address a SolrJ
>> back-compat break (SOLR-15145) and unfortunately our smoke tester framework 
>> does
>> not test for backcompat of older SolrJ against the RC, I ask others to
>> please test rolling upgrades of servers (ideally multi-node clusters)
>> running pre-8.8.0 to this RC if possible. Also, please try client
>> applications that are using an older SolrJ, esp. those that load cluster
>> state directly from ZK.
>>
>>
>> Best regards,
>>
>> Tim
>>
>>
>>
>>
>>

Re: [VOTE] Release Lucene/Solr 8.8.1 RC1

[Ishan Chattopadhyaya]

I did another round of beast, this time setting that flag to true (as
suggested to me by Noble).

ant -Duse.perreplica=true -Dtests.dups=1 -Dtests.iters=5 -Dbeast.iters=5
-Dtestcase=SolrCloudReportersTest beast

-beast:
  [beaster] Beast round 1 results:
/home/ishan/code/lucene-solr/solr/build/solr-core/test/1
  [beaster] Beast round 2 results:
/home/ishan/code/lucene-solr/solr/build/solr-core/test/2
  [beaster] Beast round 3 results:
/home/ishan/code/lucene-solr/solr/build/solr-core/test/3
  [beaster] Beast round 4 results:
/home/ishan/code/lucene-solr/solr/build/solr-core/test/4
  [beaster] Beast round 5 results:
/home/ishan/code/lucene-solr/solr/build/solr-core/test/5
  [beaster] Beasting finished Successfully.


I'm not sure what's going on :-(

On Tue, Feb 16, 2021 at 11:13 AM Anshum Gupta 
wrote:

> Ishan/Noble, thanks for taking a look at this.
>
> I only just started to look at the cause, so I'm sure you have better
> context on why this is failing and if it makes sense to still release with
> this issue.
>
> FYI, I was able to get a successful smoke test run finally, but the fact
> that it took me over 7 runs.
>
> Also, can you confirm how did you run the test? you might be getting lucky
> with the randomization here. Both me and Tim just commented out the
> randomization for USE_PER_REPLICA_STATE and hardcoding this value to true
> consistently got the test to fail. The default (false) did get the test to
> pass 100% of the times.
>
> If you think we can have this fix before the release, it might make more
> sense to have a single release for users as it wouldn't involve tracking
> the complexity of what's broken in a released version. I still would like
> to spend some more time tomorrow before voting on this one, but at least
> the smoke test is out of the way. I'll try and debug this tomorrow.
>
>
> On Mon, Feb 15, 2021 at 8:40 PM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
>> I tried light beasting the test on branch_8_8:
>> ant -Dtests.dups=1 -Dtests.iters=5 -Dbeast.iters=5
>> -Dtestcase=SolrCloudReportersTest beast
>>
>> No failures.
>>
>>   [beaster] Beast round 1 results:
>> /home/ishan/code/lucene-solr/solr/build/solr-core/test/1
>>   [beaster] Beast round 2 results:
>> /home/ishan/code/lucene-solr/solr/build/solr-core/test/2
>>   [beaster] Beast round 3 results:
>> /home/ishan/code/lucene-solr/solr/build/solr-core/test/3
>>   [beaster] Beast round 4 results:
>> /home/ishan/code/lucene-solr/solr/build/solr-core/test/4
>>   [beaster] Beast round 5 results:
>> /home/ishan/code/lucene-solr/solr/build/solr-core/test/5
>>   [beaster] Beasting finished Successfully.
>>
>> On Tue, Feb 16, 2021 at 10:07 AM Noble Paul  wrote:
>>
>>> @Anshum Gupta
>>>
>>> I think we should not hold up the release of RC1 because of that failure.
>>>
>>> This is a new feature and new features take time to get hardened.
>>>
>>> However, We can investigate and fix this anyway.
>>>
>>> If required, we can do a 8.8.3
>>>
>>> On Tue, Feb 16, 2021 at 3:10 PM Ishan Chattopadhyaya
>>>  wrote:
>>> >
>>> > Here's my +1 for the RC1.
>>> >
>>> > SUCCESS! [0:42:38.936787]
>>> >
>>> > On Tue, Feb 16, 2021 at 9:02 AM Ishan Chattopadhyaya <
>>> ichattopadhy...@gmail.com> wrote:
>>> >>
>>> >> Per Replica States is a new feature introduced in 8.8.0. It will
>>> require a critical bugfix (SOLR-15138) immediately after 8.8.1 (in a 8.8.2
>>> release). If this issue is confirmed to be PRS related, then I think we
>>> should continue with this release and fix PRS in 8.8.2.
>>> >>
>>> >> However, if you still want us to investigate and fix this issue now,
>>> we can take a look. If you have a failing seed handy, please let me know.
>>> >>
>>> >> On Tue, Feb 16, 2021 at 8:33 AM Ishan Chattopadhyaya <
>>> ichattopadhy...@gmail.com> wrote:
>>> >>>
>>> >>> Surprising. I'll take a look.
>>> >>>
>>> >>> On Tue, 16 Feb, 2021, 7:29 am Anshum Gupta, 
>>> wrote:
>>> >>>>
>>> >>>> I've unsuccessfully tried getting the smoketester to pass and have
>>> had 6 fails so far.
>>> >>>>
>>> >>>> At this point it seems like SolrCloudReporterTest and
>>> AutoscalingHistoryTest tests are failing pretty consistently for me.
>>> >>>>
>>> >>>> The former is a new failure, and seems to be caused by t

Re: [VOTE] Release Lucene/Solr 8.8.1 RC1

[Ishan Chattopadhyaya]

I tried light beasting the test on branch_8_8:
ant -Dtests.dups=1 -Dtests.iters=5 -Dbeast.iters=5
-Dtestcase=SolrCloudReportersTest beast

No failures.

  [beaster] Beast round 1 results:
/home/ishan/code/lucene-solr/solr/build/solr-core/test/1
  [beaster] Beast round 2 results:
/home/ishan/code/lucene-solr/solr/build/solr-core/test/2
  [beaster] Beast round 3 results:
/home/ishan/code/lucene-solr/solr/build/solr-core/test/3
  [beaster] Beast round 4 results:
/home/ishan/code/lucene-solr/solr/build/solr-core/test/4
  [beaster] Beast round 5 results:
/home/ishan/code/lucene-solr/solr/build/solr-core/test/5
  [beaster] Beasting finished Successfully.

On Tue, Feb 16, 2021 at 10:07 AM Noble Paul  wrote:

> @Anshum Gupta
>
> I think we should not hold up the release of RC1 because of that failure.
>
> This is a new feature and new features take time to get hardened.
>
> However, We can investigate and fix this anyway.
>
> If required, we can do a 8.8.3
>
> On Tue, Feb 16, 2021 at 3:10 PM Ishan Chattopadhyaya
>  wrote:
> >
> > Here's my +1 for the RC1.
> >
> > SUCCESS! [0:42:38.936787]
> >
> > On Tue, Feb 16, 2021 at 9:02 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
> >>
> >> Per Replica States is a new feature introduced in 8.8.0. It will
> require a critical bugfix (SOLR-15138) immediately after 8.8.1 (in a 8.8.2
> release). If this issue is confirmed to be PRS related, then I think we
> should continue with this release and fix PRS in 8.8.2.
> >>
> >> However, if you still want us to investigate and fix this issue now, we
> can take a look. If you have a failing seed handy, please let me know.
> >>
> >> On Tue, Feb 16, 2021 at 8:33 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
> >>>
> >>> Surprising. I'll take a look.
> >>>
> >>> On Tue, 16 Feb, 2021, 7:29 am Anshum Gupta, 
> wrote:
> >>>>
> >>>> I've unsuccessfully tried getting the smoketester to pass and have
> had 6 fails so far.
> >>>>
> >>>> At this point it seems like SolrCloudReporterTest and
> AutoscalingHistoryTest tests are failing pretty consistently for me.
> >>>>
> >>>> The former is a new failure, and seems to be caused by the
> USE_PER_REPLICA_STATE randomization.
> >>>>
> >>>> Both Tim and me tried running the tests without the randomization and
> defaulting that property to false gets the tests to pass, however it seems
> to be failing every time the value for USE_PER_REPLICA_STATE is set to true.
> >>>>
> >>>> I'm not voting -1 yet, as I'm not sure how much this affects the
> build vs the test, but once we have a clearer picture, we might need a fix
> and have to respin this.
> >>>>
> >>>> -Anshum
> >>>>
> >>>> On Sun, Feb 14, 2021 at 8:31 AM Timothy Potter 
> wrote:
> >>>>>
> >>>>> Looks like an extra space got added on the end of the python3
> command, try this one:
> >>>>>
> >>>>> python3 -u dev-tools/scripts/smokeTestRelease.py
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.1-RC1-rev6a50a0315ac7e4979abb0b530857c7795bb3b928
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Sun, Feb 14, 2021 at 9:26 AM Timothy Potter <
> thelabd...@apache.org> wrote:
> >>>>>>
> >>>>>> Please vote for release candidate 1 for Lucene/Solr 8.8.1
> >>>>>>
> >>>>>>
> >>>>>> The artifacts can be downloaded from:
> >>>>>>
> >>>>>>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.1-RC1-rev6a50a0315ac7e4979abb0b530857c7795bb3b928
> >>>>>>
> >>>>>>
> >>>>>> You can run the smoke tester directly with this command:
> >>>>>>
> >>>>>>
> >>>>>> python3 -u dev-tools/scripts/smokeTestRelease.py \
> >>>>>>
> >>>>>>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.1-RC1-rev6a50a0315ac7e4979abb0b530857c7795bb3b928
> >>>>>>
> >>>>>>
> >>>>>> The vote will be open for at least 72 hours i.e. until 2021-02-17
> 17:00 UTC.
> >>>>>>
> >>>>>>
> >>>>>> Here is my +1 ~ SUCCESS! [0:50:06.728441]
> >>>>>>
> >>>>>>
>

Re: [VOTE] Release Lucene/Solr 8.8.1 RC1

[Ishan Chattopadhyaya]

Here's my +1 for the RC1.

SUCCESS! [0:42:38.936787]

On Tue, Feb 16, 2021 at 9:02 AM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Per Replica States is a new feature introduced in 8.8.0. It will require a
> critical bugfix (SOLR-15138) immediately after 8.8.1 (in a 8.8.2 release).
> If this issue is confirmed to be PRS related, then I think we should
> continue with this release and fix PRS in 8.8.2.
>
> However, if you still want us to investigate and fix this issue now, we
> can take a look. If you have a failing seed handy, please let me know.
>
> On Tue, Feb 16, 2021 at 8:33 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
>> Surprising. I'll take a look.
>>
>> On Tue, 16 Feb, 2021, 7:29 am Anshum Gupta, 
>> wrote:
>>
>>> I've unsuccessfully tried getting the smoketester to pass and have had 6
>>> fails so far.
>>>
>>> At this point it seems like SolrCloudReporterTest and
>>> AutoscalingHistoryTest tests are failing pretty consistently for me.
>>>
>>> The former is a new failure, and seems to be caused by the
>>> USE_PER_REPLICA_STATE randomization.
>>>
>>> Both Tim and me tried running the tests without the randomization and
>>> defaulting that property to false gets the tests to pass, however it seems
>>> to be failing every time the value for USE_PER_REPLICA_STATE is set to true.
>>>
>>> I'm not voting -1 yet, as I'm not sure how much this affects the build
>>> vs the test, but once we have a clearer picture, we might need a fix and
>>> have to respin this.
>>>
>>> -Anshum
>>>
>>> On Sun, Feb 14, 2021 at 8:31 AM Timothy Potter 
>>> wrote:
>>>
>>>> Looks like an extra space got added on the end of the python3 command,
>>>> try this one:
>>>>
>>>> python3 -u dev-tools/scripts/smokeTestRelease.py
>>>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.1-RC1-rev6a50a0315ac7e4979abb0b530857c7795bb3b928
>>>>
>>>>
>>>>
>>>>
>>>> On Sun, Feb 14, 2021 at 9:26 AM Timothy Potter 
>>>> wrote:
>>>>
>>>>> Please vote for release candidate 1 for Lucene/Solr 8.8.1
>>>>>
>>>>>
>>>>> The artifacts can be downloaded from:
>>>>>
>>>>>
>>>>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.1-RC1-rev6a50a0315ac7e4979abb0b530857c7795bb3b928
>>>>>
>>>>>
>>>>> You can run the smoke tester directly with this command:
>>>>>
>>>>>
>>>>> python3 -u dev-tools/scripts/smokeTestRelease.py \
>>>>>
>>>>>
>>>>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.1-RC1-rev6a50a0315ac7e4979abb0b530857c7795bb3b928
>>>>>
>>>>>
>>>>> The vote will be open for at least 72 hours i.e. until 2021-02-17
>>>>> 17:00 UTC.
>>>>>
>>>>>
>>>>> Here is my +1 ~ SUCCESS! [0:50:06.728441]
>>>>>
>>>>>
>>>>> In addition to the smoke test, I built a Docker image from
>>>>> solr-8.8.1.tgz locally and verified:
>>>>>
>>>>>
>>>>> a. A rolling upgrade of a 3-node 8.7.0 cluster to the 8.8.1 RC
>>>>> completes successfully w/o any NPEs or weirdness with leader election /
>>>>> recoveries.
>>>>>
>>>>>
>>>>> b. The base_url property is stored in replica state after the upgrade
>>>>>
>>>>>
>>>>> c. A basic client application built with SolrJ 8.7.0 can load cluster
>>>>> state info directly from ZK and query the 8.8.1 RC1 servers.
>>>>>
>>>>>
>>>>> d. Same client app built with SolrJ 8.8.0 works as well.
>>>>>
>>>>>
>>>>> As this bug-fix release is primarily needed to address a SolrJ
>>>>> back-compat break (SOLR-15145) and unfortunately our smoke tester 
>>>>> framework does
>>>>> not test for backcompat of older SolrJ against the RC, I ask others
>>>>> to please test rolling upgrades of servers (ideally multi-node clusters)
>>>>> running pre-8.8.0 to this RC if possible. Also, please try client
>>>>> applications that are using an older SolrJ, esp. those that load cluster
>>>>> state directly from ZK.
>>>>>
>>>>>
>>>>> Best regards,
>>>>>
>>>>> Tim
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>
>>> --
>>> Anshum Gupta
>>>
>>

Re: [VOTE] Release Lucene/Solr 8.8.1 RC1

[Ishan Chattopadhyaya]

Per Replica States is a new feature introduced in 8.8.0. It will require a
critical bugfix (SOLR-15138) immediately after 8.8.1 (in a 8.8.2 release).
If this issue is confirmed to be PRS related, then I think we should
continue with this release and fix PRS in 8.8.2.

However, if you still want us to investigate and fix this issue now, we can
take a look. If you have a failing seed handy, please let me know.

On Tue, Feb 16, 2021 at 8:33 AM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Surprising. I'll take a look.
>
> On Tue, 16 Feb, 2021, 7:29 am Anshum Gupta, 
> wrote:
>
>> I've unsuccessfully tried getting the smoketester to pass and have had 6
>> fails so far.
>>
>> At this point it seems like SolrCloudReporterTest and
>> AutoscalingHistoryTest tests are failing pretty consistently for me.
>>
>> The former is a new failure, and seems to be caused by the
>> USE_PER_REPLICA_STATE randomization.
>>
>> Both Tim and me tried running the tests without the randomization and
>> defaulting that property to false gets the tests to pass, however it seems
>> to be failing every time the value for USE_PER_REPLICA_STATE is set to true.
>>
>> I'm not voting -1 yet, as I'm not sure how much this affects the build vs
>> the test, but once we have a clearer picture, we might need a fix and have
>> to respin this.
>>
>> -Anshum
>>
>> On Sun, Feb 14, 2021 at 8:31 AM Timothy Potter 
>> wrote:
>>
>>> Looks like an extra space got added on the end of the python3 command,
>>> try this one:
>>>
>>> python3 -u dev-tools/scripts/smokeTestRelease.py
>>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.1-RC1-rev6a50a0315ac7e4979abb0b530857c7795bb3b928
>>>
>>>
>>>
>>>
>>> On Sun, Feb 14, 2021 at 9:26 AM Timothy Potter 
>>> wrote:
>>>
>>>> Please vote for release candidate 1 for Lucene/Solr 8.8.1
>>>>
>>>>
>>>> The artifacts can be downloaded from:
>>>>
>>>>
>>>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.1-RC1-rev6a50a0315ac7e4979abb0b530857c7795bb3b928
>>>>
>>>>
>>>> You can run the smoke tester directly with this command:
>>>>
>>>>
>>>> python3 -u dev-tools/scripts/smokeTestRelease.py \
>>>>
>>>>
>>>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.1-RC1-rev6a50a0315ac7e4979abb0b530857c7795bb3b928
>>>>
>>>>
>>>> The vote will be open for at least 72 hours i.e. until 2021-02-17 17:00
>>>> UTC.
>>>>
>>>>
>>>> Here is my +1 ~ SUCCESS! [0:50:06.728441]
>>>>
>>>>
>>>> In addition to the smoke test, I built a Docker image from
>>>> solr-8.8.1.tgz locally and verified:
>>>>
>>>>
>>>> a. A rolling upgrade of a 3-node 8.7.0 cluster to the 8.8.1 RC
>>>> completes successfully w/o any NPEs or weirdness with leader election /
>>>> recoveries.
>>>>
>>>>
>>>> b. The base_url property is stored in replica state after the upgrade
>>>>
>>>>
>>>> c. A basic client application built with SolrJ 8.7.0 can load cluster
>>>> state info directly from ZK and query the 8.8.1 RC1 servers.
>>>>
>>>>
>>>> d. Same client app built with SolrJ 8.8.0 works as well.
>>>>
>>>>
>>>> As this bug-fix release is primarily needed to address a SolrJ
>>>> back-compat break (SOLR-15145) and unfortunately our smoke tester 
>>>> framework does
>>>> not test for backcompat of older SolrJ against the RC, I ask others to
>>>> please test rolling upgrades of servers (ideally multi-node clusters)
>>>> running pre-8.8.0 to this RC if possible. Also, please try client
>>>> applications that are using an older SolrJ, esp. those that load cluster
>>>> state directly from ZK.
>>>>
>>>>
>>>> Best regards,
>>>>
>>>> Tim
>>>>
>>>>
>>>>
>>>>
>>>>
>>
>> --
>> Anshum Gupta
>>
>

Re: [VOTE] Release Lucene/Solr 8.8.1 RC1

[Ishan Chattopadhyaya]

Surprising. I'll take a look.

On Tue, 16 Feb, 2021, 7:29 am Anshum Gupta,  wrote:

> I've unsuccessfully tried getting the smoketester to pass and have had 6
> fails so far.
>
> At this point it seems like SolrCloudReporterTest and
> AutoscalingHistoryTest tests are failing pretty consistently for me.
>
> The former is a new failure, and seems to be caused by the
> USE_PER_REPLICA_STATE randomization.
>
> Both Tim and me tried running the tests without the randomization and
> defaulting that property to false gets the tests to pass, however it seems
> to be failing every time the value for USE_PER_REPLICA_STATE is set to true.
>
> I'm not voting -1 yet, as I'm not sure how much this affects the build vs
> the test, but once we have a clearer picture, we might need a fix and have
> to respin this.
>
> -Anshum
>
> On Sun, Feb 14, 2021 at 8:31 AM Timothy Potter 
> wrote:
>
>> Looks like an extra space got added on the end of the python3 command,
>> try this one:
>>
>> python3 -u dev-tools/scripts/smokeTestRelease.py
>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.1-RC1-rev6a50a0315ac7e4979abb0b530857c7795bb3b928
>>
>>
>>
>>
>> On Sun, Feb 14, 2021 at 9:26 AM Timothy Potter 
>> wrote:
>>
>>> Please vote for release candidate 1 for Lucene/Solr 8.8.1
>>>
>>>
>>> The artifacts can be downloaded from:
>>>
>>>
>>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.1-RC1-rev6a50a0315ac7e4979abb0b530857c7795bb3b928
>>>
>>>
>>> You can run the smoke tester directly with this command:
>>>
>>>
>>> python3 -u dev-tools/scripts/smokeTestRelease.py \
>>>
>>>
>>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.1-RC1-rev6a50a0315ac7e4979abb0b530857c7795bb3b928
>>>
>>>
>>> The vote will be open for at least 72 hours i.e. until 2021-02-17 17:00
>>> UTC.
>>>
>>>
>>> Here is my +1 ~ SUCCESS! [0:50:06.728441]
>>>
>>>
>>> In addition to the smoke test, I built a Docker image from
>>> solr-8.8.1.tgz locally and verified:
>>>
>>>
>>> a. A rolling upgrade of a 3-node 8.7.0 cluster to the 8.8.1 RC completes
>>> successfully w/o any NPEs or weirdness with leader election / recoveries.
>>>
>>>
>>> b. The base_url property is stored in replica state after the upgrade
>>>
>>>
>>> c. A basic client application built with SolrJ 8.7.0 can load cluster
>>> state info directly from ZK and query the 8.8.1 RC1 servers.
>>>
>>>
>>> d. Same client app built with SolrJ 8.8.0 works as well.
>>>
>>>
>>> As this bug-fix release is primarily needed to address a SolrJ
>>> back-compat break (SOLR-15145) and unfortunately our smoke tester framework 
>>> does
>>> not test for backcompat of older SolrJ against the RC, I ask others to
>>> please test rolling upgrades of servers (ideally multi-node clusters)
>>> running pre-8.8.0 to this RC if possible. Also, please try client
>>> applications that are using an older SolrJ, esp. those that load cluster
>>> state directly from ZK.
>>>
>>>
>>> Best regards,
>>>
>>> Tim
>>>
>>>
>>>
>>>
>>>
>
> --
> Anshum Gupta
>

Re: Separate git repo(s) for Solr modules

[Ishan Chattopadhyaya]

Hoss,
One important reason why separate repo for solr-extras is a good idea, as
opposed to conrib modules, is that separate repo can be used to test
against many Solr versions. Imagine a component that works across Solr
versions 6x through 9x from day one. I can't imagine such a component being
part of the lucene-solr repo itself.

Also, today, contrib modules are shipped with Solr, which we don't want for
new things we might want to build.

On Mon, Jan 25, 2021 at 4:25 PM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> I haven't been able to follow up on creation of the extras repo, but more
> importantly I wanted to respond to Hoss. I'm out on an emergency for a week
> or so, shall resume then. If there's a decision on this until then, I shall
> accept it.
>
> On Mon, 25 Jan, 2021, 9:04 am Jason Gerlowski, 
> wrote:
>
>> Tentative +1 to Hoss' questions.  I agree with his summary of the
>> potential risks here, and I share his ignorance of the perceived
>> benefits.
>>
>> (I thought for a time that this was driven by interest in having
>> release cadences independent from Solr-core releases.  I'm all for
>> that goal, but if that's the motivation I'm not sure what the obstacle
>> is to doing that with a single repo - all build systems these days
>> support versioning and releasing modules independent of one another.
>> But maybe that was never a driving factor here.)
>>
>> I know there have been a lot of discussions about this, and I know the
>> repo has already been created.  So maybe it's too late to object even
>> if I wanted to, which I'm not sure I do.  But can someone that
>> understands the motivation please summarize what multiple-repos gets
>> us over a single repo that outweighs the "cons" that Hoss mentioned?
>>
>> Jason
>>
>> On Thu, Jan 14, 2021 at 12:34 PM Chris Hostetter
>>  wrote:
>> >
>> >
>> > : As we discussed over the last few months, there seems a need to move
>> > : non-core pieces away from the Solr core module. The contribs are
>> presently
>> > : a good place, but it makes sense to have a separate git repository
>> hosting
>> > : such modules. Some candidates that come to mind are the present day
>> contrib
>> >
>> > can you explain why it makes sense to have a separate git repo for these
>> > things?
>> >
>> > I can think of lots of reasons why it makes sense to have a single
>> > repo for all things solr (unified CI that quickly identifies if core
>> > changes break "first order" plugins, shared feature branches & monotomic
>> > commits of code that affects APIs and impls of those APIs, etc...) but
>> > I've yet to see any concrete specifics of why multiple git repos are
>> > "better" then just having distinct sub-projects (with distinct
>> artifacts)
>> > in the same repo other then "it makes sense"
>> >
>> > why does it make sense?
>> >
>> > why can't the ideas of "solr-sandbox" and "solr-extras" just be
>> > directories in the "solr repo" ? ... what value is gained by making them
>> > new repos?
>> >
>> >
>> > -Hoss
>> > http://www.lucidworks.com/
>> >
>> > -
>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> > For additional commands, e-mail: dev-h...@lucene.apache.org
>> >
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>>

Re: [ANNOUNCE] Apache Solr 8.8.0 released

[Ishan Chattopadhyaya]

Hi all,
This release contains a critical bug, that should be fixed in 8.8.1
shortly. Please avoid upgrading to this release for the moment.
https://twitter.com/ichattopadhyaya/status/1360163382171586562
Apologies for the inconvenience.
Thanks,
Ishan

On Mon, Feb 1, 2021 at 6:01 PM Noble Paul  wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA512
>
> Solr is the popular, blazing fast, open source NoSQL search platform from
> the Apache Lucene project. Its major features include powerful full-text
> search, hit highlighting, faceted search and analytics, rich document
> parsing, geospatial search, extensive REST APIs as well as parallel SQL.
> Solr is enterprise grade, secure and highly scalable, providing fault
> tolerant distributed search and indexing, and powers the search and
> navigation features of many of the world's largest internet sites.
>
> The release is available for immediate download at:
>
>  https://lucene.apache.org/solr/downloads.html
>
> Please read CHANGES.txt for a detailed list of changes:
>
>  https://lucene.apache.org/solr/8_8_0/changes/Changes.html Solr 8.8.0
>
> Release Highlights:
>
> Reducing overseer bottlenecks using per-replica states. More stability and
> lesser load on large cluster that use this feature.
>
> Better restart and collection creation performance.
>
> Interleaving support in Learning To Rank
>
> A summary of important changes is published in the Solr Reference Guide at:
>
>  https://lucene.apache.org/solr/guide/8_8/solr-upgrade-notes.html.
>
> For the most exhaustive list, see the full release notes at
> https://lucene.apache.org/solr/8_8_0/changes/Changes.html
>
> or
>
> by viewing the CHANGES.txt file accompanying the distribution. Solr's
> release notes usually don't include Lucene layer changes. Lucene's release
> notes are at
>
> https://lucene.apache.org/core/8_8_0/changes/Changes.html
>
> - -
> Noble Paul
> -BEGIN PGP SIGNATURE-
> Version: FlowCrypt Email Encryption 8.0.0
> Comment: Seamlessly send and receive encrypted email
>
> wsFzBAEBCgAGBQJgF/RnACEJEMOP9ew/z9s+FiEEz85fu5IMPHRc7uCEw4/1
> 7D/P2z6fzRAAm4AKbeIGWfPK+0nsrZCAPaDucGZYVL0lPQr3eF4jnmhi60dF
> Sv9rD5Mq5ZSTTuJlpwoaxowxVp4M1tV1vmCdfBRkgoUD3dwS/snryr/AK69R
> zdjjV/BABtcMNA7cMYIrkolGl37g4kI1alLfU36Uf/3M0NfUcw0keW1XuMOr
> uV7AzXhZGw4eL4LJt7I7gXJs1kgE6/sPSmoKBVckKisrruiUSYmH9r/EhXXU
> YB8cxd5tenMrchbjcOquC9X2JJjB++/LyJw3mFNIO5W3UpjqwtI8IGDo1Sxl
> fM32FuAWVVDZsiBKXuRzsIO/iEPfgZFfTcoSJkD0Rt/Q6gJPZIuBmiUFaYfs
> 9fzufNDuXdPKFEndSHfwdPMJwvk3XA5+xYzhkcQH+3FKOPmYXkvLolOC3j+r
> ZtbgI421jDIahpVPbFtgUPB2dM3mw34B73wP5MIOHHxz22tVKe6PBOeihccK
> mOr0r1tZHR+11aijYf+Nlhv3hpbpRoDbQ7pRkRyu53Od47p6itZAi60TFFIJ
> bDw26wZRNRrEuYhriJUeM7ahvJNlcE6VaO0szUDL5g/x2Oa9jKMHPpsUF9pS
> 9HbJWcnflxq0iU+sfdv7Eoxzv6zkXMTUsbpT2XjKcZZN5jd2rWV3JfiU6FiZ
> jpqJBHzwGan9qKKswNKyDKhoa2jPdSYIbqQ=
> =NbSI
> -END PGP SIGNATURE-
>

Re: 8.8.1 release soon

[Ishan Chattopadhyaya]

I'd like for us to include SOLR-15138 please, but the fix is still under
review and development. Please let us know if it should be possible for us
to wait until that one is done (hopefully quickly), otherwise we can
release it later (if you want to proceed with the release before this is
ready). Thanks for volunteering!

On Wed, 10 Feb, 2021, 9:07 pm Timothy Potter,  wrote:

> I was a tad bit ambitious with backporting SOLR-12182 to 8.8.0 and it
> seems we have no automated SolrJ back-compat tests in our RC vetting
> process, so unfortunately older SolrJ clients don't work with Solr 8.8
> server, see SOLR-15145.
>
> I'd like to release 8.8.1 ASAP to address this problem and will be the RM.
>
> Let me know if you have any other issues you think need to go into 8.8.1,
> otherwise I'd like to build an RC tomorrow AM US time. It looks like there
> are already a number of updates going in for 8.9 so let's keep the updates
> for 8.8.1 to a minimum please.
>
> Cheers,
> Tim
>

Re: [VOTE] Release Lucene/Solr 8.8.0 RC2

[Ishan Chattopadhyaya]

Thanks Noble!

On Thu, 28 Jan, 2021, 4:24 pm Noble Paul,  wrote:

> [+1]  9  (4 binding)
>
>  [0]  0
>
> [-1]  0
>
>
> This vote has PASSED.
>
> I shall proceed with the rest of the release process
>
> On Thu, Jan 28, 2021 at 6:24 PM Anshum Gupta 
> wrote:
> >
> > Thanks for handing the release, Noble!
> >
> > +1 (binding)
> >
> > SUCCESS! [0:56:12.016387]
> >
> > Ran the smoke tester, a demo app, and checked the change log. All of
> that looks good.
> >
> > On Mon, Jan 25, 2021 at 2:22 AM Noble Paul  wrote:
> >>
> >> Please vote for release candidate 2 for Lucene/Solr 8.8.0
> >>
> >> The artifacts can be downloaded from:
> >>
> >>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.0-RC2-revb10659f0fc18b58b90929cfdadde94544d202c4a/
> >>
> >> python3 -u dev-tools/scripts/smokeTestRelease.py \
> >>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.0-RC2-revb10659f0fc18b58b90929cfdadde94544d202c4a/
> >>
> >>
> >>
> >> The vote will be open for at least 72 hours
> >>
> >> [ ] +1  approve
> >> [ ] +0  no opinion
> >> [ ] -1  disapprove (and reason why)
> >>
> >> Here is my +1
> >> --
> >> -
> >> Noble Paul
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> >> For additional commands, e-mail: dev-h...@lucene.apache.org
> >>
> >
> >
> > --
> > Anshum Gupta
>
>
>
> --
> -
> Noble Paul
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Separate git repo(s) for Solr modules

[Ishan Chattopadhyaya]

I haven't been able to follow up on creation of the extras repo, but more
importantly I wanted to respond to Hoss. I'm out on an emergency for a week
or so, shall resume then. If there's a decision on this until then, I shall
accept it.

On Mon, 25 Jan, 2021, 9:04 am Jason Gerlowski, 
wrote:

> Tentative +1 to Hoss' questions.  I agree with his summary of the
> potential risks here, and I share his ignorance of the perceived
> benefits.
>
> (I thought for a time that this was driven by interest in having
> release cadences independent from Solr-core releases.  I'm all for
> that goal, but if that's the motivation I'm not sure what the obstacle
> is to doing that with a single repo - all build systems these days
> support versioning and releasing modules independent of one another.
> But maybe that was never a driving factor here.)
>
> I know there have been a lot of discussions about this, and I know the
> repo has already been created.  So maybe it's too late to object even
> if I wanted to, which I'm not sure I do.  But can someone that
> understands the motivation please summarize what multiple-repos gets
> us over a single repo that outweighs the "cons" that Hoss mentioned?
>
> Jason
>
> On Thu, Jan 14, 2021 at 12:34 PM Chris Hostetter
>  wrote:
> >
> >
> > : As we discussed over the last few months, there seems a need to move
> > : non-core pieces away from the Solr core module. The contribs are
> presently
> > : a good place, but it makes sense to have a separate git repository
> hosting
> > : such modules. Some candidates that come to mind are the present day
> contrib
> >
> > can you explain why it makes sense to have a separate git repo for these
> > things?
> >
> > I can think of lots of reasons why it makes sense to have a single
> > repo for all things solr (unified CI that quickly identifies if core
> > changes break "first order" plugins, shared feature branches & monotomic
> > commits of code that affects APIs and impls of those APIs, etc...) but
> > I've yet to see any concrete specifics of why multiple git repos are
> > "better" then just having distinct sub-projects (with distinct artifacts)
> > in the same repo other then "it makes sense"
> >
> > why does it make sense?
> >
> > why can't the ideas of "solr-sandbox" and "solr-extras" just be
> > directories in the "solr repo" ? ... what value is gained by making them
> > new repos?
> >
> >
> > -Hoss
> > http://www.lucidworks.com/
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: dev-h...@lucene.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: [JENKINS] Lucene » Lucene-Solr-Tests-8.8 - Build # 137 - Unstable!

[Ishan Chattopadhyaya]

Fyi @Noble Paul നോബിള്‍ नोब्ळ् 

On Sun, 24 Jan, 2021, 5:58 am Michael McCandless, 
wrote:

> Hmm, is anyone looking into this one / is there an issue open already?  A
> quick search on http://jirasearch.mikemccandless.com did not seem to find
> an existing issue ...
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Sat, Jan 23, 2021 at 5:34 AM Apache Jenkins Server <
> jenk...@builds.apache.org> wrote:
>
>> Build:
>> https://ci-builds.apache.org/job/Lucene/job/Lucene-Solr-Tests-8.8/137/
>>
>> 1 tests failed.
>> FAILED:  org.apache.lucene.monitor.TestCachePurging.testBackgroundPurges
>>
>> Error Message:
>> expected:<-1> but was:<12440479308859038>
>>
>> Stack Trace:
>> java.lang.AssertionError: expected:<-1> but was:<12440479308859038>
>> at
>> __randomizedtesting.SeedInfo.seed([3C1014B0DC241B26:2CAB527B8C038666]:0)
>> at org.junit.Assert.fail(Assert.java:89)
>> at org.junit.Assert.failNotEquals(Assert.java:835)
>> at org.junit.Assert.assertEquals(Assert.java:647)
>> at org.junit.Assert.assertEquals(Assert.java:633)
>> at
>> org.apache.lucene.monitor.TestCachePurging.testBackgroundPurges(TestCachePurging.java:138)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at
>> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
>> at
>> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
>> at
>> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
>> at
>> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
>> at
>> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
>> at
>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>> at
>> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>> at
>> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>> at
>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>> at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> at
>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
>> at
>> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
>> at
>> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
>> at
>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
>> at
>> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
>> at
>> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
>> at
>> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
>> at
>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>> at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> at
>> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
>> at
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> at
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> at
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> at
>> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
>> at
>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>> at
>> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>> at
>> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
>> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>> at
>> 

Re: 8.8 Release

[Ishan Chattopadhyaya]

Please take over, I'll be out for a week now. Thanks

On Sat, 23 Jan, 2021, 9:47 am Noble Paul,  wrote:

> I've pushed the changes.
>
> @Ishan Chattopadhyaya  if you are unavailable just ping me. I'll put up RC2
>
> On Sat, Jan 23, 2021 at 2:48 PM Noble Paul  wrote:
> >
> > please hold on there is a concurrency issue, I'm fixing it
> >
> > On Sat, Jan 23, 2021 at 12:54 PM Ishan Chattopadhyaya
> >  wrote:
> > >
> > > My attempt to spin RC2 failed at one of the LTR tests,
> https://gist.github.com/chatman/567afa09cc3e233d151f8e8b0f6b63e7.
> > > This is a rare failure, so wondering if there's a bug somewhere
> (either in LTR or schema loading). Ping Christine, Noble, FYI.
> > >
> > > I'm trying another run, I have only an hour before I'm gone for a
> while. If it fails again, I would request Noble to pick it up.
> > >
> > > On Fri, Jan 22, 2021 at 8:37 PM Christine Poerschke (BLOOMBERG/
> LONDON)  wrote:
> > >>
> > >> No worries, thanks for the review input!
> > >>
> > >> The SOLR-15071 commits are now complete.
> > >>
> > >> From: dev@lucene.apache.org At: 01/22/21 13:33:19
> > >> To: Christine Poerschke (BLOOMBERG/ LONDON ) , dev@lucene.apache.org
> > >> Subject: Re: 8.8 Release
> > >>
> > >> Sorry, I should have explicitly approved that PR; you had addressed
> my concerns and I forgot about it, thinking it was in 8.8.
> > >> ~ David Smiley
> > >> Apache Lucene/Solr Search Developer
> > >> http://www.linkedin.com/in/davidwsmiley
> > >>
> > >>
> > >> On Fri, Jan 22, 2021 at 4:29 AM Christine Poerschke (BLOOMBERG/
> LONDON)  wrote:
> > >>>
> > >>> Hello Ishan. Is there still time for me to merge and backport
> https://github.com/apache/lucene-solr/pull/2196 for the fix this London
> morning for inclusion in RC2? The PR is essentially reviewed but not yet
> approved and with other things also going on I'd held off on it then so
> far. What do you think? Thanks!
> > >>>
> > >>> From: dev@lucene.apache.org At: 01/12/21 02:38:20
> > >>> To: dev@lucene.apache.org
> > >>> Subject: Re: 8.8 Release
> > >>>
> > >>> Thanks Tim. I've now cut the branch for 8.8.
> > >>> @David, let's get the fix in for the release.
> > >>>
> > >>>
> > >>> On Tue, Jan 12, 2021 at 2:13 AM David Smiley 
> wrote:
> > >>>>
> > >>>> FYI https://issues.apache.org/jira/browse/SOLR-15071 for LTR a
> regression
> > >>>> Christine hasn't posted the fix yet but I'd guess there's a user
> work-around so the issue isn't pressing I suppose.
> > >>>>
> > >>>> ~ David Smiley
> > >>>> Apache Lucene/Solr Search Developer
> > >>>> http://www.linkedin.com/in/davidwsmiley
> > >>>>
> > >>>>
> > >>>> On Mon, Jan 11, 2021 at 1:32 PM Timothy Potter <
> thelabd...@gmail.com> wrote:
> > >>>>>
> > >>>>> Fix is in branch_8x and master now, sorry for the delay ;-)
> > >>>>>
> > >>>>> Cheers,
> > >>>>> Tim
> > >>>>>
> > >>>>> On Mon, Jan 11, 2021 at 10:50 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
> > >>>>>>
> > >>>>>> Thanks Tim!
> > >>>>>>
> > >>>>>> On Mon, 11 Jan, 2021, 11:00 pm Timothy Potter, <
> thelabd...@gmail.com> wrote:
> > >>>>>>>
> > >>>>>>> 15036 will be in later today, so you can plan to cut this
> evening US time or tomorrow.
> > >>>>>>>
> > >>>>>>> Cheers,
> > >>>>>>> Tim
> > >>>>>>>
> > >>>>>>> On Mon, Jan 11, 2021 at 9:54 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
> > >>>>>>>>
> > >>>>>>>> I think all the issues mentioned above are in the branch,
> except SOLR-15036. Tim, I'll cut a branch once that is in, latest by
> Wednesday AM (USA time).
> > >>>>>>>> Thanks,
> > >>>>>>>> Ishan
> > >>>>>>>>
> > >>>>>>>> On Thu, Jan 7, 2021 at 5:07 AM Timothy Potter <
> thelabd

Re: 8.8 Release

[Ishan Chattopadhyaya]

My attempt to spin RC2 failed at one of the LTR tests,
https://gist.github.com/chatman/567afa09cc3e233d151f8e8b0f6b63e7.
This is a rare failure, so wondering if there's a bug somewhere (either in
LTR or schema loading). Ping Christine, Noble, FYI.

I'm trying another run, I have only an hour before I'm gone for a while. If
it fails again, I would request Noble to pick it up.

On Fri, Jan 22, 2021 at 8:37 PM Christine Poerschke (BLOOMBERG/ LONDON) <
cpoersc...@bloomberg.net> wrote:

> No worries, thanks for the review input!
>
> The SOLR-15071 commits are now complete.
>
> From: dev@lucene.apache.org At: 01/22/21 13:33:19
> To: Christine Poerschke (BLOOMBERG/ LONDON ) ,
> dev@lucene.apache.org
> Subject: Re: 8.8 Release
>
> Sorry, I should have explicitly approved that PR; you had addressed my
> concerns and I forgot about it, thinking it was in 8.8.
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Fri, Jan 22, 2021 at 4:29 AM Christine Poerschke (BLOOMBERG/ LONDON) <
> cpoersc...@bloomberg.net> wrote:
>
>> Hello Ishan. Is there still time for me to merge and backport
>> https://github.com/apache/lucene-solr/pull/2196 for the fix this London
>> morning for inclusion in RC2? The PR is essentially reviewed but not yet
>> approved and with other things also going on I'd held off on it then so
>> far. What do you think? Thanks!
>>
>> From: dev@lucene.apache.org At: 01/12/21 02:38:20
>> To: dev@lucene.apache.org
>> Subject: Re: 8.8 Release
>>
>> Thanks Tim. I've now cut the branch for 8.8.
>> @David, let's get the fix in for the release.
>>
>>
>> On Tue, Jan 12, 2021 at 2:13 AM David Smiley  wrote:
>>
>>> FYI https://issues.apache.org/jira/browse/SOLR-15071 for LTR a
>>> regression
>>> Christine hasn't posted the fix yet but I'd guess there's a user
>>> work-around so the issue isn't pressing I suppose.
>>>
>>> ~ David Smiley
>>> Apache Lucene/Solr Search Developer
>>> http://www.linkedin.com/in/davidwsmiley
>>>
>>>
>>> On Mon, Jan 11, 2021 at 1:32 PM Timothy Potter 
>>> wrote:
>>>
>>>> Fix is in branch_8x and master now, sorry for the delay ;-)
>>>>
>>>> Cheers,
>>>> Tim
>>>>
>>>> On Mon, Jan 11, 2021 at 10:50 AM Ishan Chattopadhyaya <
>>>> ichattopadhy...@gmail.com> wrote:
>>>>
>>>>> Thanks Tim!
>>>>>
>>>>> On Mon, 11 Jan, 2021, 11:00 pm Timothy Potter, 
>>>>> wrote:
>>>>>
>>>>>> 15036 will be in later today, so you can plan to cut this evening US
>>>>>> time or tomorrow.
>>>>>>
>>>>>> Cheers,
>>>>>> Tim
>>>>>>
>>>>>> On Mon, Jan 11, 2021 at 9:54 AM Ishan Chattopadhyaya <
>>>>>> ichattopadhy...@gmail.com> wrote:
>>>>>>
>>>>>>> I think all the issues mentioned above are in the branch, except
>>>>>>> SOLR-15036. Tim, I'll cut a branch once that is in, latest by Wednesday 
>>>>>>> AM
>>>>>>> (USA time).
>>>>>>> Thanks,
>>>>>>> Ishan
>>>>>>>
>>>>>>> On Thu, Jan 7, 2021 at 5:07 AM Timothy Potter 
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Thanks for following up on this Ishan ... I intend to get
>>>>>>>> SOLR-15059 and -15036 into 8.8 as well. I should have a proper PR up 
>>>>>>>> for
>>>>>>>> SOLR-15036 by Friday sometime, which seems to align with other's 
>>>>>>>> timeframes
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Tim
>>>>>>>>
>>>>>>>> On Wed, Jan 6, 2021 at 6:54 AM David Smiley 
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Happy New Year!
>>>>>>>>> I would much prefer that ensure 8.8 includes SOLR-14923 (a bad
>>>>>>>>> nested docs performance issue)
>>>>>>>>>
>>>>>>>>> ~ David Smiley
>>>>>>>>> Apache Lucene/Solr Search Developer
>>>>>>>>> http://www.linkedin.com/in/davidwsmiley
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 

Re: [VOTE] Release Lucene/Solr 8.8.0 RC1

[Ishan Chattopadhyaya]

and *upload (not told) the artifacts.

On Fri, 22 Jan, 2021, 3:57 pm Ishan Chattopadhyaya, <
ichattopadhy...@gmail.com> wrote:

> Hi Christine,
> I've already built the artifacts for rc2. But I'm fairly busy at the
> moment to smoke test and told the artifact's right now.
>
> I took a look at the pr and I approve it, thanks for fixing it (it was my
> bad). Please merge it to the release branch in case it makes it to rc3.
>
> Regards,
> Ishan
>
> On Fri, 22 Jan, 2021, 3:03 pm Christine Poerschke (BLOOMBERG/ LONDON), <
> cpoersc...@bloomberg.net> wrote:
>
>> Hello. If it's not too late for RC2 and if someone could review and
>> approve https://github.com/apache/lucene-solr/pull/2210 then
>> https://issues.apache.org/jira/browse/SOLR-15073 bug fix could be a
>> candidate for inclusion in RC2. What do you think? Thank you.
>>
>> From: dev@lucene.apache.org At: 01/22/21 04:00:00
>> To: dev@lucene.apache.org
>> Subject: Re: [VOTE] Release Lucene/Solr 8.8.0 RC1
>>
>> Thanks all, I'll respin. I'll abort this RC1.
>>
>> On Fri, Jan 22, 2021 at 5:08 AM Anshum Gupta 
>> wrote:
>>
>>> -1 for RC1.
>>>
>>> The fix has been merged into 8x and 8.8 branches so we should be able to
>>> respin and start another vote.
>>>
>>> Thanks for the fix, Tim!
>>>
>>> On Thu, Jan 21, 2021 at 2:42 PM Mike Drob  wrote:
>>>
>>>> -1
>>>>
>>>> I have been able to reproduce the regression that Anshum found, and
>>>> would like to change my vote accordingly.
>>>>
>>>> On Thu, Jan 21, 2021 at 4:31 PM Anshum Gupta 
>>>> wrote:
>>>>
>>>>> https://issues.apache.org/jira/browse/SOLR-15097
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Jan 21, 2021 at 2:28 PM Anshum Gupta 
>>>>> wrote:
>>>>>
>>>>>> While testing the artifacts some more, I realized that the cluster
>>>>>> graph doesn't show up on the admin UI any more.
>>>>>>
>>>>>> I started the cluster with -e cloud -noprompt, and the getting
>>>>>> started collection was created successfully. The clusterstate, as well as
>>>>>> the admin UI confirm that.
>>>>>>
>>>>>> I also tried creating a collection using the admin API, which was
>>>>>> successful, but the graph didn't reflect that either.
>>>>>>
>>>>>> I've tested this using both Safari and Firefox, and it's clearly not
>>>>>> a browser issue.
>>>>>>
>>>>>> Considering that the users rely on the graph, I think we might need
>>>>>> to fix and respin this.
>>>>>>
>>>>>>
>>>>>> On Tue, Jan 19, 2021 at 8:51 AM Ishan Chattopadhyaya <
>>>>>> ichattopadhy...@gmail.com> wrote:
>>>>>>
>>>>>>> Please vote for release candidate 1 for Lucene/Solr 8.8.0
>>>>>>>
>>>>>>> The artifacts can be downloaded from:
>>>>>>>
>>>>>>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.0-RC1-rev737cb9c49b08f6e3964c1e8a80132da3c764e027
>>>>>>>
>>>>>>> You can run the smoke tester directly with this command:
>>>>>>>
>>>>>>> python3 -u dev-tools/scripts/smokeTestRelease.py \
>>>>>>>
>>>>>>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.0-RC1-rev737cb9c49b08f6e3964c1e8a80132da3c764e027
>>>>>>>
>>>>>>> The vote will be open for at least 72 hours i.e. until 2021-01-22
>>>>>>> 17:00 UTC.
>>>>>>>
>>>>>>> [ ] +1  approve
>>>>>>> [ ] +0  no opinion
>>>>>>> [ ] -1  disapprove (and reason why)
>>>>>>>
>>>>>>> Here is my +1
>>>>>>> 
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Anshum Gupta
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Anshum Gupta
>>>>>
>>>>
>>>
>>> --
>>> Anshum Gupta
>>>
>>
>>

Re: [VOTE] Release Lucene/Solr 8.8.0 RC1

[Ishan Chattopadhyaya]

Hi Christine,
I've already built the artifacts for rc2. But I'm fairly busy at the moment
to smoke test and told the artifact's right now.

I took a look at the pr and I approve it, thanks for fixing it (it was my
bad). Please merge it to the release branch in case it makes it to rc3.

Regards,
Ishan

On Fri, 22 Jan, 2021, 3:03 pm Christine Poerschke (BLOOMBERG/ LONDON), <
cpoersc...@bloomberg.net> wrote:

> Hello. If it's not too late for RC2 and if someone could review and
> approve https://github.com/apache/lucene-solr/pull/2210 then
> https://issues.apache.org/jira/browse/SOLR-15073 bug fix could be a
> candidate for inclusion in RC2. What do you think? Thank you.
>
> From: dev@lucene.apache.org At: 01/22/21 04:00:00
> To: dev@lucene.apache.org
> Subject: Re: [VOTE] Release Lucene/Solr 8.8.0 RC1
>
> Thanks all, I'll respin. I'll abort this RC1.
>
> On Fri, Jan 22, 2021 at 5:08 AM Anshum Gupta 
> wrote:
>
>> -1 for RC1.
>>
>> The fix has been merged into 8x and 8.8 branches so we should be able to
>> respin and start another vote.
>>
>> Thanks for the fix, Tim!
>>
>> On Thu, Jan 21, 2021 at 2:42 PM Mike Drob  wrote:
>>
>>> -1
>>>
>>> I have been able to reproduce the regression that Anshum found, and
>>> would like to change my vote accordingly.
>>>
>>> On Thu, Jan 21, 2021 at 4:31 PM Anshum Gupta 
>>> wrote:
>>>
>>>> https://issues.apache.org/jira/browse/SOLR-15097
>>>>
>>>>
>>>>
>>>> On Thu, Jan 21, 2021 at 2:28 PM Anshum Gupta 
>>>> wrote:
>>>>
>>>>> While testing the artifacts some more, I realized that the cluster
>>>>> graph doesn't show up on the admin UI any more.
>>>>>
>>>>> I started the cluster with -e cloud -noprompt, and the getting started
>>>>> collection was created successfully. The clusterstate, as well as the 
>>>>> admin
>>>>> UI confirm that.
>>>>>
>>>>> I also tried creating a collection using the admin API, which was
>>>>> successful, but the graph didn't reflect that either.
>>>>>
>>>>> I've tested this using both Safari and Firefox, and it's clearly not a
>>>>> browser issue.
>>>>>
>>>>> Considering that the users rely on the graph, I think we might need to
>>>>> fix and respin this.
>>>>>
>>>>>
>>>>> On Tue, Jan 19, 2021 at 8:51 AM Ishan Chattopadhyaya <
>>>>> ichattopadhy...@gmail.com> wrote:
>>>>>
>>>>>> Please vote for release candidate 1 for Lucene/Solr 8.8.0
>>>>>>
>>>>>> The artifacts can be downloaded from:
>>>>>>
>>>>>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.0-RC1-rev737cb9c49b08f6e3964c1e8a80132da3c764e027
>>>>>>
>>>>>> You can run the smoke tester directly with this command:
>>>>>>
>>>>>> python3 -u dev-tools/scripts/smokeTestRelease.py \
>>>>>>
>>>>>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.0-RC1-rev737cb9c49b08f6e3964c1e8a80132da3c764e027
>>>>>>
>>>>>> The vote will be open for at least 72 hours i.e. until 2021-01-22
>>>>>> 17:00 UTC.
>>>>>>
>>>>>> [ ] +1  approve
>>>>>> [ ] +0  no opinion
>>>>>> [ ] -1  disapprove (and reason why)
>>>>>>
>>>>>> Here is my +1
>>>>>> 
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Anshum Gupta
>>>>>
>>>>
>>>>
>>>> --
>>>> Anshum Gupta
>>>>
>>>
>>
>> --
>> Anshum Gupta
>>
>
>

Re: [VOTE] Release Lucene/Solr 8.8.0 RC1

[Ishan Chattopadhyaya]

Thanks all, I'll respin. I'll abort this RC1.

On Fri, Jan 22, 2021 at 5:08 AM Anshum Gupta  wrote:

> -1 for RC1.
>
> The fix has been merged into 8x and 8.8 branches so we should be able to
> respin and start another vote.
>
> Thanks for the fix, Tim!
>
> On Thu, Jan 21, 2021 at 2:42 PM Mike Drob  wrote:
>
>> -1
>>
>> I have been able to reproduce the regression that Anshum found, and would
>> like to change my vote accordingly.
>>
>> On Thu, Jan 21, 2021 at 4:31 PM Anshum Gupta 
>> wrote:
>>
>>> https://issues.apache.org/jira/browse/SOLR-15097
>>>
>>>
>>>
>>> On Thu, Jan 21, 2021 at 2:28 PM Anshum Gupta 
>>> wrote:
>>>
>>>> While testing the artifacts some more, I realized that the cluster
>>>> graph doesn't show up on the admin UI any more.
>>>>
>>>> I started the cluster with -e cloud -noprompt, and the getting started
>>>> collection was created successfully. The clusterstate, as well as the admin
>>>> UI confirm that.
>>>>
>>>> I also tried creating a collection using the admin API, which was
>>>> successful, but the graph didn't reflect that either.
>>>>
>>>> I've tested this using both Safari and Firefox, and it's clearly not a
>>>> browser issue.
>>>>
>>>> Considering that the users rely on the graph, I think we might need to
>>>> fix and respin this.
>>>>
>>>>
>>>> On Tue, Jan 19, 2021 at 8:51 AM Ishan Chattopadhyaya <
>>>> ichattopadhy...@gmail.com> wrote:
>>>>
>>>>> Please vote for release candidate 1 for Lucene/Solr 8.8.0
>>>>>
>>>>> The artifacts can be downloaded from:
>>>>>
>>>>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.0-RC1-rev737cb9c49b08f6e3964c1e8a80132da3c764e027
>>>>>
>>>>> You can run the smoke tester directly with this command:
>>>>>
>>>>> python3 -u dev-tools/scripts/smokeTestRelease.py \
>>>>>
>>>>> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.0-RC1-rev737cb9c49b08f6e3964c1e8a80132da3c764e027
>>>>>
>>>>> The vote will be open for at least 72 hours i.e. until 2021-01-22
>>>>> 17:00 UTC.
>>>>>
>>>>> [ ] +1  approve
>>>>> [ ] +0  no opinion
>>>>> [ ] -1  disapprove (and reason why)
>>>>>
>>>>> Here is my +1
>>>>> 
>>>>>
>>>>
>>>>
>>>> --
>>>> Anshum Gupta
>>>>
>>>
>>>
>>> --
>>> Anshum Gupta
>>>
>>
>
> --
> Anshum Gupta
>

[VOTE] Release Lucene/Solr 8.8.0 RC1

[Ishan Chattopadhyaya]

Please vote for release candidate 1 for Lucene/Solr 8.8.0

The artifacts can be downloaded from:
https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.0-RC1-rev737cb9c49b08f6e3964c1e8a80132da3c764e027

You can run the smoke tester directly with this command:

python3 -u dev-tools/scripts/smokeTestRelease.py \
https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.8.0-RC1-rev737cb9c49b08f6e3964c1e8a80132da3c764e027

The vote will be open for at least 72 hours i.e. until 2021-01-22 17:00 UTC.

[ ] +1  approve
[ ] +0  no opinion
[ ] -1  disapprove (and reason why)

Here is my +1

Re: 8.8 Release

[Ishan Chattopadhyaya]

Thanks for the issue, Jim. Seemed important to get it in. I'll build the
RC1 today, but I have a very short time of availability. If I'm unable to
wrap it up, Noble will take over tomorrow.

On Tue, 19 Jan, 2021, 3:44 pm jim ferenczi,  wrote:

> Thanks Ishan, the change is merged in the 8.8 branch.
> I should have said that it's not a blocker per say so if you've already
> built RC1 and want to proceed with it please go ahead and I'll adapt the
> change.
>
>
> Le lun. 18 janv. 2021 à 16:30, Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> a écrit :
>
>> Oh, I just started the RC1.. I'm barely half way through the Solr tests,
>> though :-)
>> Jim, Please go ahead and backport it to the release branch, and I'll pick
>> it up in another attempt.
>> Thanks,
>> Ishan
>>
>> p.s. I got a window of opportunity to get some work done, so I took over
>> from Noble tonight.
>>
>> On Mon, Jan 18, 2021 at 7:24 PM jim ferenczi 
>> wrote:
>>
>>> Sorry, I forgot to add the link to the issue:
>>> https://issues.apache.org/jira/browse/LUCENE-9675
>>>
>>>
>>> Le lun. 18 janv. 2021 à 14:33, jim ferenczi  a
>>> écrit :
>>>
>>>> Hi Noble,
>>>> I opened an issue to expose the compression mode that is used in binary
>>>> doc values. The configurable compression is a new feature in 8.8 so we'd
>>>> like to expose the compression mode that was used to write the segment in
>>>> the attributes of the field.
>>>> I'd like to backport to the 8.8 branch when it's ready so that we don't
>>>> need to add a new internal version in the doc values format. Is it
>>>> acceptable ?
>>>>
>>>> Cheers,
>>>> Jim
>>>>
>>>> Le sam. 16 janv. 2021 à 00:15, Namgyu Kim  a écrit :
>>>>
>>>>> Take care, Ishan!
>>>>> Family is the most important thing.
>>>>>
>>>>> Hi Noble,
>>>>> Thank you for your support :D
>>>>> The commit for the blocker issue(LUCENE-9661) that I mentioned before
>>>>> is submitted on branch_8_8.
>>>>> (github :
>>>>> https://github.com/apache/lucene-solr/commit/d1eca6399131950c74de156a4fb564ba70bdfc86
>>>>> )
>>>>> (Apache gitbox :
>>>>> https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=d1eca63)
>>>>> Please let me know if there is a problem!
>>>>>
>>>>> On Fri, Jan 15, 2021 at 6:37 PM Noble Paul 
>>>>> wrote:
>>>>>
>>>>>> Family first.
>>>>>>
>>>>>> We got this covered Ishan
>>>>>>
>>>>>> On Fri, Jan 15, 2021 at 12:43 PM David Smiley 
>>>>>> wrote:
>>>>>> >
>>>>>> > Wow. Good call on handing off the release duties so you can focus
>>>>>> on family. Take care!
>>>>>> >
>>>>>> > On Thu, Jan 14, 2021 at 3:46 PM Ishan Chattopadhyaya <
>>>>>> ichattopadhy...@gmail.com> wrote:
>>>>>> >>
>>>>>> >> Hi Devs,
>>>>>> >>
>>>>>> >> Just admitted mom to a hospital, she has given us a sudden health
>>>>>> scare. I doubt I'll be around for this release or for some time now.
>>>>>> >>
>>>>>> >> I've spoken to Noble and requested him to take over the release
>>>>>> duties here.
>>>>>> >>
>>>>>> >> Thanks and regards,
>>>>>> >> Ishan
>>>>>> >>
>>>>>> >> On Thu, 14 Jan, 2021, 2:03 pm Noble Paul, 
>>>>>> wrote:
>>>>>> >>>
>>>>>> >>> I've merged it .
>>>>>> >>> Thanks
>>>>>> >>>
>>>>>> >>> On Thu, Jan 14, 2021 at 5:12 PM Ishan Chattopadhyaya
>>>>>> >>>  wrote:
>>>>>> >>> >
>>>>>> >>> > Sure Noble, if you are confident it doesn't disrupt the
>>>>>> stability of the release, please go ahead. In case there are any problems
>>>>>> with stability/performance discovered (due to this issue) at the time of
>>>>>> the release, I'll revert this. Thanks!
>>>>>> >>> >
>>>>>> >>> > On Thu, Jan 14, 2021 a

Re: 8.8 Release

[Ishan Chattopadhyaya]

Oh, I just started the RC1.. I'm barely half way through the Solr tests,
though :-)
Jim, Please go ahead and backport it to the release branch, and I'll pick
it up in another attempt.
Thanks,
Ishan

p.s. I got a window of opportunity to get some work done, so I took over
from Noble tonight.

On Mon, Jan 18, 2021 at 7:24 PM jim ferenczi  wrote:

> Sorry, I forgot to add the link to the issue:
> https://issues.apache.org/jira/browse/LUCENE-9675
>
>
> Le lun. 18 janv. 2021 à 14:33, jim ferenczi  a
> écrit :
>
>> Hi Noble,
>> I opened an issue to expose the compression mode that is used in binary
>> doc values. The configurable compression is a new feature in 8.8 so we'd
>> like to expose the compression mode that was used to write the segment in
>> the attributes of the field.
>> I'd like to backport to the 8.8 branch when it's ready so that we don't
>> need to add a new internal version in the doc values format. Is it
>> acceptable ?
>>
>> Cheers,
>> Jim
>>
>> Le sam. 16 janv. 2021 à 00:15, Namgyu Kim  a écrit :
>>
>>> Take care, Ishan!
>>> Family is the most important thing.
>>>
>>> Hi Noble,
>>> Thank you for your support :D
>>> The commit for the blocker issue(LUCENE-9661) that I mentioned before is
>>> submitted on branch_8_8.
>>> (github :
>>> https://github.com/apache/lucene-solr/commit/d1eca6399131950c74de156a4fb564ba70bdfc86
>>> )
>>> (Apache gitbox :
>>> https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=d1eca63)
>>> Please let me know if there is a problem!
>>>
>>> On Fri, Jan 15, 2021 at 6:37 PM Noble Paul  wrote:
>>>
>>>> Family first.
>>>>
>>>> We got this covered Ishan
>>>>
>>>> On Fri, Jan 15, 2021 at 12:43 PM David Smiley 
>>>> wrote:
>>>> >
>>>> > Wow. Good call on handing off the release duties so you can focus on
>>>> family. Take care!
>>>> >
>>>> > On Thu, Jan 14, 2021 at 3:46 PM Ishan Chattopadhyaya <
>>>> ichattopadhy...@gmail.com> wrote:
>>>> >>
>>>> >> Hi Devs,
>>>> >>
>>>> >> Just admitted mom to a hospital, she has given us a sudden health
>>>> scare. I doubt I'll be around for this release or for some time now.
>>>> >>
>>>> >> I've spoken to Noble and requested him to take over the release
>>>> duties here.
>>>> >>
>>>> >> Thanks and regards,
>>>> >> Ishan
>>>> >>
>>>> >> On Thu, 14 Jan, 2021, 2:03 pm Noble Paul, 
>>>> wrote:
>>>> >>>
>>>> >>> I've merged it .
>>>> >>> Thanks
>>>> >>>
>>>> >>> On Thu, Jan 14, 2021 at 5:12 PM Ishan Chattopadhyaya
>>>> >>>  wrote:
>>>> >>> >
>>>> >>> > Sure Noble, if you are confident it doesn't disrupt the stability
>>>> of the release, please go ahead. In case there are any problems with
>>>> stability/performance discovered (due to this issue) at the time of the
>>>> release, I'll revert this. Thanks!
>>>> >>> >
>>>> >>> > On Thu, Jan 14, 2021 at 11:33 AM Noble Paul 
>>>> wrote:
>>>> >>> >>
>>>> >>> >> @Ishan Chattopadhyaya
>>>> >>> >>
>>>> >>> >> I wish to port https://issues.apache.org/jira/browse/SOLR-14155
>>>> to
>>>> >>> >> 8.8, if it is OK
>>>> >>> >>
>>>> >>> >> On Wed, Jan 13, 2021 at 9:20 AM Ishan Chattopadhyaya
>>>> >>> >>  wrote:
>>>> >>> >> >
>>>> >>> >> > Wish you a very happy new year, Namgyu!
>>>> >>> >> > Please proceed, thanks!
>>>> >>> >> >
>>>> >>> >> > On Wed, 13 Jan, 2021, 1:01 am Namgyu Kim, 
>>>> wrote:
>>>> >>> >> >>
>>>> >>> >> >> Hi Ishan,
>>>> >>> >> >> It's late, but happy new year!
>>>> >>> >> >>
>>>> >>> >> >> There is a blocker issue for 8.x (including 8.8)
>>>> >>> >> >> https://issues.apache.org/jira/browse/LUCENE-9

Re: 8.8 Release

[Ishan Chattopadhyaya]

Hi Devs,

Just admitted mom to a hospital, she has given us a sudden health scare. I
doubt I'll be around for this release or for some time now.

I've spoken to Noble and requested him to take over the release duties here.

Thanks and regards,
Ishan

On Thu, 14 Jan, 2021, 2:03 pm Noble Paul,  wrote:

> I've merged it .
> Thanks
>
> On Thu, Jan 14, 2021 at 5:12 PM Ishan Chattopadhyaya
>  wrote:
> >
> > Sure Noble, if you are confident it doesn't disrupt the stability of the
> release, please go ahead. In case there are any problems with
> stability/performance discovered (due to this issue) at the time of the
> release, I'll revert this. Thanks!
> >
> > On Thu, Jan 14, 2021 at 11:33 AM Noble Paul 
> wrote:
> >>
> >> @Ishan Chattopadhyaya
> >>
> >> I wish to port https://issues.apache.org/jira/browse/SOLR-14155 to
> >> 8.8, if it is OK
> >>
> >> On Wed, Jan 13, 2021 at 9:20 AM Ishan Chattopadhyaya
> >>  wrote:
> >> >
> >> > Wish you a very happy new year, Namgyu!
> >> > Please proceed, thanks!
> >> >
> >> > On Wed, 13 Jan, 2021, 1:01 am Namgyu Kim,  wrote:
> >> >>
> >> >> Hi Ishan,
> >> >> It's late, but happy new year!
> >> >>
> >> >> There is a blocker issue for 8.x (including 8.8)
> >> >> https://issues.apache.org/jira/browse/LUCENE-9661
> >> >>
> >> >> Would it be okay to cherry-pick the commit for LUCENE-9661 to the
> 8.8 branch after merging?
> >> >>
> >> >> On Wed, Jan 13, 2021 at 1:48 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
> >> >>>
> >> >>> Please go for it if you think it won't disrupt the stability and
> can be wrapped up in 2-3 days.
> >> >>>
> >> >>> On Tue, 12 Jan, 2021, 10:08 pm Walter Underwood, <
> wun...@wunderwood.org> wrote:
> >> >>>>
> >> >>>> I’d love for SOLR-15056 to be in, but it is just a patch now and
> hasn’t had anything besides
> >> >>>> local testing, so that is a bit of a long shot.
> >> >>>>
> >> >>>> wunder
> >> >>>> Walter Underwood
> >> >>>> wun...@wunderwood.org
> >> >>>> http://observer.wunderwood.org/  (my blog)
> >> >>>>
> >> >>>> On Jan 11, 2021, at 9:29 AM, Timothy Potter 
> wrote:
> >> >>>>
> >> >>>> 15036 will be in later today, so you can plan to cut this evening
> US time or tomorrow.
> >> >>>>
> >> >>>> Cheers,
> >> >>>> Tim
> >> >>>>
> >> >>>> On Mon, Jan 11, 2021 at 9:54 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
> >> >>>>>
> >> >>>>> I think all the issues mentioned above are in the branch, except
> SOLR-15036. Tim, I'll cut a branch once that is in, latest by Wednesday AM
> (USA time).
> >> >>>>> Thanks,
> >> >>>>> Ishan
> >> >>>>>
> >> >>>>> On Thu, Jan 7, 2021 at 5:07 AM Timothy Potter <
> thelabd...@gmail.com> wrote:
> >> >>>>>>
> >> >>>>>> Thanks for following up on this Ishan ... I intend to get
> SOLR-15059 and -15036 into 8.8 as well. I should have a proper PR up for
> SOLR-15036 by Friday sometime, which seems to align with other's timeframes
> >> >>>>>>
> >> >>>>>> Cheers,
> >> >>>>>> Tim
> >> >>>>>>
> >> >>>>>> On Wed, Jan 6, 2021 at 6:54 AM David Smiley 
> wrote:
> >> >>>>>>>
> >> >>>>>>> Happy New Year!
> >> >>>>>>> I would much prefer that ensure 8.8 includes SOLR-14923 (a bad
> nested docs performance issue)
> >> >>>>>>>
> >> >>>>>>> ~ David Smiley
> >> >>>>>>> Apache Lucene/Solr Search Developer
> >> >>>>>>> http://www.linkedin.com/in/davidwsmiley
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> On Wed, Jan 6, 2021 at 6:59 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
> >> >>>>>>>>
> >> >>>>>&g

Re: https://lucene.apache.org/solr/guide/ needs to be updated on each release

[Ishan Chattopadhyaya]

@Atri Sharma  FYI

On Thu, Jan 14, 2021 at 2:58 PM Colvin Cowie 
wrote:

> I've just noticed https://lucene.apache.org/solr/guide/ says 8.6 is the
> latest release and doesn't mention 8.7
>
> *Latest release*:
>
>- Solr 8.6 
>
>

Re: 8.8 Release

[Ishan Chattopadhyaya]

Sure Noble, if you are confident it doesn't disrupt the stability of the
release, please go ahead. In case there are any problems with
stability/performance discovered (due to this issue) at the time of the
release, I'll revert this. Thanks!

On Thu, Jan 14, 2021 at 11:33 AM Noble Paul  wrote:

> @Ishan Chattopadhyaya
>
> I wish to port https://issues.apache.org/jira/browse/SOLR-14155 to
> 8.8, if it is OK
>
> On Wed, Jan 13, 2021 at 9:20 AM Ishan Chattopadhyaya
>  wrote:
> >
> > Wish you a very happy new year, Namgyu!
> > Please proceed, thanks!
> >
> > On Wed, 13 Jan, 2021, 1:01 am Namgyu Kim,  wrote:
> >>
> >> Hi Ishan,
> >> It's late, but happy new year!
> >>
> >> There is a blocker issue for 8.x (including 8.8)
> >> https://issues.apache.org/jira/browse/LUCENE-9661
> >>
> >> Would it be okay to cherry-pick the commit for LUCENE-9661 to the 8.8
> branch after merging?
> >>
> >> On Wed, Jan 13, 2021 at 1:48 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
> >>>
> >>> Please go for it if you think it won't disrupt the stability and can
> be wrapped up in 2-3 days.
> >>>
> >>> On Tue, 12 Jan, 2021, 10:08 pm Walter Underwood, <
> wun...@wunderwood.org> wrote:
> >>>>
> >>>> I’d love for SOLR-15056 to be in, but it is just a patch now and
> hasn’t had anything besides
> >>>> local testing, so that is a bit of a long shot.
> >>>>
> >>>> wunder
> >>>> Walter Underwood
> >>>> wun...@wunderwood.org
> >>>> http://observer.wunderwood.org/  (my blog)
> >>>>
> >>>> On Jan 11, 2021, at 9:29 AM, Timothy Potter 
> wrote:
> >>>>
> >>>> 15036 will be in later today, so you can plan to cut this evening US
> time or tomorrow.
> >>>>
> >>>> Cheers,
> >>>> Tim
> >>>>
> >>>> On Mon, Jan 11, 2021 at 9:54 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
> >>>>>
> >>>>> I think all the issues mentioned above are in the branch, except
> SOLR-15036. Tim, I'll cut a branch once that is in, latest by Wednesday AM
> (USA time).
> >>>>> Thanks,
> >>>>> Ishan
> >>>>>
> >>>>> On Thu, Jan 7, 2021 at 5:07 AM Timothy Potter 
> wrote:
> >>>>>>
> >>>>>> Thanks for following up on this Ishan ... I intend to get
> SOLR-15059 and -15036 into 8.8 as well. I should have a proper PR up for
> SOLR-15036 by Friday sometime, which seems to align with other's timeframes
> >>>>>>
> >>>>>> Cheers,
> >>>>>> Tim
> >>>>>>
> >>>>>> On Wed, Jan 6, 2021 at 6:54 AM David Smiley 
> wrote:
> >>>>>>>
> >>>>>>> Happy New Year!
> >>>>>>> I would much prefer that ensure 8.8 includes SOLR-14923 (a bad
> nested docs performance issue)
> >>>>>>>
> >>>>>>> ~ David Smiley
> >>>>>>> Apache Lucene/Solr Search Developer
> >>>>>>> http://www.linkedin.com/in/davidwsmiley
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Jan 6, 2021 at 6:59 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
> >>>>>>>>
> >>>>>>>> Happy New Year!
> >>>>>>>>
> >>>>>>>> I was supposed to start the process tomorrow, but I think we're
> not ready yet? I see SOLR-15052 still under review with intention of
> inclusion into 8.8.
> >>>>>>>> Would it be reasonable to cut the release branch end of this week
> and start the RC process around 13th January?
> >>>>>>>> If there are any issues someone would want me to wait on, please
> let me know.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Ishan
> >>>>>>>>
> >>>>>>>> On Fri, Dec 18, 2020 at 6:10 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
> >>>>>>>>>
> >>>>>>>>> Sure, Houston. I'll wait another week. Have a good new year and
> merry Christmas!
> >>>>>>>>>
> >>>>>>>>> On Fri, 18 Dec, 2020

Re: Separate git repo(s) for Solr modules

[Ishan Chattopadhyaya]

Sounds good, thanks Houston for bringing it up. I'll take a stab at a few
first party packages soon and then we can take a call where they should
belong (maybe on a case to case basis).

Anshum has applied for the sandbox repo, I'll request for the extras repo
soon.

Thanks!

On Thu, 14 Jan, 2021, 4:04 am David Smiley,  wrote:

> Both paths have an appeal -- do contribs stay or move?  I recommend not
> moving existing contribs there until we explore how we can maintain
> compatibility with new Solr releases. (what Houston & Tomas point to
> below).  For example, *perhaps* a "git sub-module" pointing to Solr might
> be a key mechanism?  Could CI automatically keep it up to date?  Someone
> needs to try before we really know.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Wed, Jan 13, 2021 at 10:30 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
>> Hi Devs,
>>
>> As we discussed over the last few months, there seems a need to move
>> non-core pieces away from the Solr core module. The contribs are presently
>> a good place, but it makes sense to have a separate git repository hosting
>> such modules. Some candidates that come to mind are the present day contrib
>> modules, upcoming HDFS support module (separated away from solr-core),
>> other first party packages. Along with that, there is also a need for a
>> repository for hosting WIP modules/sub-projects.
>>
>> I propose that we apply for the creation of two new git repositories:
>> 1. solr-extras (or lucene-solr-extras)
>> 2. solr-sandbox (or lucene-solr-sandbox)
>>
>> Well tested, well supported modules/sub-projects can be released straight
>> away from *solr-extras*. The first party packages can be built from this
>> location and shipped with Solr (or be available for install using the
>> package manager CLI).
>>
>> New, unproved, beta, unstable modules can be hosted on *solr-sandbox*
>> (and graduate to solr-extras once stable).
>>
>> Please let me know if there are any questions/concerns with this approach.
>>
>> Thanks and regards,
>> Ishan
>>
>

Separate git repo(s) for Solr modules

[Ishan Chattopadhyaya]

Hi Devs,

As we discussed over the last few months, there seems a need to move
non-core pieces away from the Solr core module. The contribs are presently
a good place, but it makes sense to have a separate git repository hosting
such modules. Some candidates that come to mind are the present day contrib
modules, upcoming HDFS support module (separated away from solr-core),
other first party packages. Along with that, there is also a need for a
repository for hosting WIP modules/sub-projects.

I propose that we apply for the creation of two new git repositories:
1. solr-extras (or lucene-solr-extras)
2. solr-sandbox (or lucene-solr-sandbox)

Well tested, well supported modules/sub-projects can be released straight
away from *solr-extras*. The first party packages can be built from this
location and shipped with Solr (or be available for install using the
package manager CLI).

New, unproved, beta, unstable modules can be hosted on *solr-sandbox* (and
graduate to solr-extras once stable).

Please let me know if there are any questions/concerns with this approach.

Thanks and regards,
Ishan

Re: Requesting a new GH repository for CrossDC modules

[Ishan Chattopadhyaya]

On Wed, 13 Jan, 2021, 6:00 am Anshum Gupta,  wrote:

> Building this as a branch is an option, but building it outside in a
> personal repo is exactly what's not the Apache Way.
>

Negative. Nothing of that is mentioned in
https://www.apache.org/theapacheway/. Private decision making isn't
allowed, but discussions happening around code in an Apache communication
channels (say ASF GitHub over a PR from a privately owned public
repository) definitely seems The Apache Way.


> Code should be designed and built in the Apache world, else it'd be a
> grant/donation and not really a PR.
>

An ICLA is just sufficient for that, a grant of copyright license of all
code written by you to the ASF. You already filed the ICLA when you became
a committer.

Also, you can't create a PR against a repo that doesn't exist upstream.
>

You should create the PR against an existing branch and demonstrate that
the code is at least functioning and we can consider a separate repository
at that point.

David's blobstore work is an example of this. Jason's incremental backup
work another example (base code is in lucene-solr repo of lucidworks' fork
as a PR by Shalin). Both know such code may or may not continue to reside
in lucene-solr repo, but are working on building consensus and building a
working prototype first, not discussing the release cycle and the final
location of the code.


> Do you have an objection against a mono-repo i.e. solr-sandbox too? That
> would open the door for us to use this for similar purposes in the future,
> until the code is ready to be released.
>
> Also, just to reiterate, creating a repo doesn't cost anything and we
> aren't releasing anything.
>

It costs the headache of dealing with stubborn actors who may oppose
deletion of the repo if it is decided that it is not the right solution.
Just look at deprecation of original CDCR from Solr as an example, where
you and some others came across as a shining stars of such opposition,
despite knowing it was deeply broken and not offering any help to fix it.

This is a placeholder to put the code in. If it works out well, we can
> release it or iterate on the code/implementation. In any case, it would
> have zero impact on the project itself.
>

I disagree on zero impact to project. It causes reputational damage if not
done properly, and in your case here, the work  hasn't even started yet.


> -Anshum
>
> On Tue, Jan 12, 2021 at 3:37 PM Noble Paul  wrote:
>
>> I feel this is placing the cart before the horse.
>>
>> We can always build this as a branch or a repo under your own account.
>> Once we reach a point where the project is reasonably mature, you can
>> create a repo and contribute it upstream.
>>
>>
>> On Wed, Jan 13, 2021 at 6:27 AM Anshum Gupta 
>> wrote:
>> >
>> > I understand what you are saying, which is also my reason to not have a
>> mono-repo. This way it's easier to manage and drop a repository when it's
>> not needed. It doesn't cause clutter and lives in isolation.
>> >
>> > I think we are on the same page in terms of the intention.
>> >
>> >
>> > On Tue, Jan 12, 2021 at 10:51 AM Ishan Chattopadhyaya <
>> ichattopadhy...@gmail.com> wrote:
>> >>
>> >> Look at the branches that are cluttering up our main repository, many
>> symbolic of unfinished work. If we start one repo each for everything we
>> hope to finish, we'll make Solr annoying in a new way.
>> >>
>> >> There is no reason multiple artifacts can't be released independently
>> from the same repo. Why are you opposed to that idea, Anshum?
>> >>
>> >> On Tue, 12 Jan, 2021, 11:53 pm Anshum Gupta, 
>> wrote:
>> >>>
>> >>> Thank you everyone!
>> >>>
>> >>> I'll move forward with the cross-dc repo creation then as mentioned
>> in the original email :)
>> >>>
>> >>> If we want to change the approach on the repo, we can always change
>> that before we release anything in the future.
>> >>>
>> >>> On Mon, Jan 11, 2021 at 10:32 AM Mike Drob  wrote:
>> >>>>
>> >>>> I'm seeing valid reasons to prefer one solr sandbox repo, or prefer
>> multiple many repos for future plugins or integrations. In this specific
>> case, I think the relevant deciding points are 1) we don't have multiple
>> things yet, so deciding between a "mono-repo" and a "multi-repo" is not
>> very consequential 2) we can always rename things later 3) in the absence
>> of a strong reason otherwise i'll defer to the people doing the work (in
>> this case, Anshum). We considered sandbox and can always c

Re: 8.8 Release

[Ishan Chattopadhyaya]

Wish you a very happy new year, Namgyu!
Please proceed, thanks!

On Wed, 13 Jan, 2021, 1:01 am Namgyu Kim,  wrote:

> Hi Ishan,
> It's late, but *happy new year*!
>
> There is a blocker issue for 8.x (including 8.8)
> https://issues.apache.org/jira/browse/LUCENE-9661
>
> Would it be okay to cherry-pick the commit for LUCENE-9661 to the 8.8
> branch after merging?
>
> On Wed, Jan 13, 2021 at 1:48 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
>> Please go for it if you think it won't disrupt the stability and can be
>> wrapped up in 2-3 days.
>>
>> On Tue, 12 Jan, 2021, 10:08 pm Walter Underwood, 
>> wrote:
>>
>>> I’d love for SOLR-15056 to be in, but it is just a patch now and hasn’t
>>> had anything besides
>>> local testing, so that is a bit of a long shot.
>>>
>>> wunder
>>> Walter Underwood
>>> wun...@wunderwood.org
>>> http://observer.wunderwood.org/  (my blog)
>>>
>>> On Jan 11, 2021, at 9:29 AM, Timothy Potter 
>>> wrote:
>>>
>>> 15036 will be in later today, so you can plan to cut this evening US
>>> time or tomorrow.
>>>
>>> Cheers,
>>> Tim
>>>
>>> On Mon, Jan 11, 2021 at 9:54 AM Ishan Chattopadhyaya <
>>> ichattopadhy...@gmail.com> wrote:
>>>
>>>> I think all the issues mentioned above are in the branch, except
>>>> SOLR-15036. Tim, I'll cut a branch once that is in, latest by Wednesday AM
>>>> (USA time).
>>>> Thanks,
>>>> Ishan
>>>>
>>>> On Thu, Jan 7, 2021 at 5:07 AM Timothy Potter 
>>>> wrote:
>>>>
>>>>> Thanks for following up on this Ishan ... I intend to get SOLR-15059
>>>>> and -15036 into 8.8 as well. I should have a proper PR up for SOLR-15036 
>>>>> by
>>>>> Friday sometime, which seems to align with other's timeframes
>>>>>
>>>>> Cheers,
>>>>> Tim
>>>>>
>>>>> On Wed, Jan 6, 2021 at 6:54 AM David Smiley 
>>>>> wrote:
>>>>>
>>>>>> Happy New Year!
>>>>>> I would much prefer that ensure 8.8 includes SOLR-14923 (a bad nested
>>>>>> docs performance issue)
>>>>>>
>>>>>> ~ David Smiley
>>>>>> Apache Lucene/Solr Search Developer
>>>>>> http://www.linkedin.com/in/davidwsmiley
>>>>>>
>>>>>>
>>>>>> On Wed, Jan 6, 2021 at 6:59 AM Ishan Chattopadhyaya <
>>>>>> ichattopadhy...@gmail.com> wrote:
>>>>>>
>>>>>>> Happy New Year!
>>>>>>>
>>>>>>> I was supposed to start the process tomorrow, but I think we're not
>>>>>>> ready yet? I see SOLR-15052 still under review with intention of 
>>>>>>> inclusion
>>>>>>> into 8.8.
>>>>>>> Would it be reasonable to cut the release branch end of this week
>>>>>>> and start the RC process around 13th January?
>>>>>>> If there are any issues someone would want me to wait on, please let
>>>>>>> me know.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Ishan
>>>>>>>
>>>>>>> On Fri, Dec 18, 2020 at 6:10 AM Ishan Chattopadhyaya <
>>>>>>> ichattopadhy...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Sure, Houston. I'll wait another week. Have a good new year and
>>>>>>>> merry Christmas!
>>>>>>>>
>>>>>>>> On Fri, 18 Dec, 2020, 5:58 am Timothy Potter, 
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Great point Houston! +1 on waiting until a week into January
>>>>>>>>>
>>>>>>>>> On Thu, Dec 17, 2020 at 4:46 PM Houston Putman <
>>>>>>>>> houstonput...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Thanks for volunteering Ishan.
>>>>>>>>>>
>>>>>>>>>> I think it might be a good idea to wait to cut and release 8.8 at
>>>>>>>>>> least a week into January. Many people are going to be away during 
>>>>>>>>>> the
>>>>>>>>>> holiday season, and

Re: Requesting a new GH repository for CrossDC modules

[Ishan Chattopadhyaya]

Look at the branches that are cluttering up our main repository, many
symbolic of unfinished work. If we start one repo each for everything we
hope to finish, we'll make Solr annoying in a new way.

There is no reason multiple artifacts can't be released independently from
the same repo. Why are you opposed to that idea, Anshum?

On Tue, 12 Jan, 2021, 11:53 pm Anshum Gupta,  wrote:

> Thank you everyone!
>
> I'll move forward with the cross-dc repo creation then as mentioned in the
> original email :)
>
> If we want to change the approach on the repo, we can always change that
> before we release anything in the future.
>
> On Mon, Jan 11, 2021 at 10:32 AM Mike Drob  wrote:
>
>> I'm seeing valid reasons to prefer one solr sandbox repo, or prefer
>> multiple many repos for future plugins or integrations. In this specific
>> case, I think the relevant deciding points are 1) we don't have multiple
>> things yet, so deciding between a "mono-repo" and a "multi-repo" is not
>> very consequential 2) we can always rename things later 3) in the absence
>> of a strong reason otherwise i'll defer to the people doing the work (in
>> this case, Anshum). We considered sandbox and can always create one in the
>> future. If Anshum feels that solr-cross-dc is better for now than I'm fine
>> with that too.
>>
>> On Sun, Jan 10, 2021 at 5:07 PM David Smiley  wrote:
>>
>>> (palm-to-face) -- LOL okay sorry.  I'm getting my threads crossed.
>>>
>>> A repo which holds multiple independent modules that can work with Solr
>>> need not release them all at once.
>>>
>>> ~ David Smiley
>>> Apache Lucene/Solr Search Developer
>>> http://www.linkedin.com/in/davidwsmiley
>>>
>>>
>>> On Sat, Jan 9, 2021 at 4:48 PM Anshum Gupta 
>>> wrote:
>>>
>>>> David, this is about the Cross DC work that was supposed to be done :-)
>>>>
>>>> The independent release cadence is primarily the reason why a new repo
>>>> makes sense to me in this case.
>>>>
>>>> On Sat, Jan 9, 2021 at 1:24 PM David Smiley  wrote:
>>>>
>>>>> While I like the idea of a single (Apache!) repo for multiple
>>>>> packages/plugins, that does not apply to the Solr Operator, which isn't
>>>>> even in Java.  It's too unique.  So I agree with Anshum & others about
>>>>> creating an Apache repo for the Solr Operator.
>>>>>
>>>>> I think the ship has sailed on the Solr Operator being an Apache
>>>>> project instead of some committer's pet project.
>>>>>
>>>>> ~ David Smiley
>>>>> Apache Lucene/Solr Search Developer
>>>>> http://www.linkedin.com/in/davidwsmiley
>>>>>
>>>>>
>>>>> On Fri, Jan 8, 2021 at 4:47 AM Ishan Chattopadhyaya <
>>>>> ichattopadhy...@gmail.com> wrote:
>>>>>
>>>>>> Not necessarily. Most people contribute to Apache Lucene/Solr using
>>>>>> external repositories (forks) and raise pull requests against Apache 
>>>>>> owned
>>>>>> repositories. There's no SGA needed on such occasions.
>>>>>>
>>>>>> I see two paths forward from here.
>>>>>>
>>>>>> a) Lets setup a single repository for all packages/plugins, say
>>>>>> lucene-solr-extras or lucene-solr-contribs or lucene-solr-sandbox etc., 
>>>>>> and
>>>>>> develop it there.
>>>>>> b) All development for this effort happens in an external repository (
>>>>>> https://github.com/apple/solr-dc or
>>>>>> https://github.com/anshumg/solr-dc) and we raise a PR against Apache
>>>>>> owned repository (which can be created if needed once we are all 
>>>>>> onboard).
>>>>>>
>>>>>> What does everyone else think?
>>>>>>
>>>>>> On Fri, Jan 8, 2021 at 10:23 AM Mike Drob  wrote:
>>>>>>
>>>>>>> An external repository probably ends up requiring a software grant?
>>>>>>> I know there is a material difference between code originating 
>>>>>>> externally
>>>>>>> and code originating within the umbrella of the ASF in terms of IP,
>>>>>>> copyright, or other legal status.
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jan 7, 2021 at 8:11 PM Ishan Ch

Re: 8.8 Release

[Ishan Chattopadhyaya]

Please go for it if you think it won't disrupt the stability and can be
wrapped up in 2-3 days.

On Tue, 12 Jan, 2021, 10:08 pm Walter Underwood, 
wrote:

> I’d love for SOLR-15056 to be in, but it is just a patch now and hasn’t
> had anything besides
> local testing, so that is a bit of a long shot.
>
> wunder
> Walter Underwood
> wun...@wunderwood.org
> http://observer.wunderwood.org/  (my blog)
>
> On Jan 11, 2021, at 9:29 AM, Timothy Potter  wrote:
>
> 15036 will be in later today, so you can plan to cut this evening US time
> or tomorrow.
>
> Cheers,
> Tim
>
> On Mon, Jan 11, 2021 at 9:54 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
>> I think all the issues mentioned above are in the branch, except
>> SOLR-15036. Tim, I'll cut a branch once that is in, latest by Wednesday AM
>> (USA time).
>> Thanks,
>> Ishan
>>
>> On Thu, Jan 7, 2021 at 5:07 AM Timothy Potter 
>> wrote:
>>
>>> Thanks for following up on this Ishan ... I intend to get SOLR-15059 and
>>> -15036 into 8.8 as well. I should have a proper PR up for SOLR-15036 by
>>> Friday sometime, which seems to align with other's timeframes
>>>
>>> Cheers,
>>> Tim
>>>
>>> On Wed, Jan 6, 2021 at 6:54 AM David Smiley  wrote:
>>>
>>>> Happy New Year!
>>>> I would much prefer that ensure 8.8 includes SOLR-14923 (a bad nested
>>>> docs performance issue)
>>>>
>>>> ~ David Smiley
>>>> Apache Lucene/Solr Search Developer
>>>> http://www.linkedin.com/in/davidwsmiley
>>>>
>>>>
>>>> On Wed, Jan 6, 2021 at 6:59 AM Ishan Chattopadhyaya <
>>>> ichattopadhy...@gmail.com> wrote:
>>>>
>>>>> Happy New Year!
>>>>>
>>>>> I was supposed to start the process tomorrow, but I think we're not
>>>>> ready yet? I see SOLR-15052 still under review with intention of inclusion
>>>>> into 8.8.
>>>>> Would it be reasonable to cut the release branch end of this week and
>>>>> start the RC process around 13th January?
>>>>> If there are any issues someone would want me to wait on, please let
>>>>> me know.
>>>>>
>>>>> Thanks,
>>>>> Ishan
>>>>>
>>>>> On Fri, Dec 18, 2020 at 6:10 AM Ishan Chattopadhyaya <
>>>>> ichattopadhy...@gmail.com> wrote:
>>>>>
>>>>>> Sure, Houston. I'll wait another week. Have a good new year and merry
>>>>>> Christmas!
>>>>>>
>>>>>> On Fri, 18 Dec, 2020, 5:58 am Timothy Potter, 
>>>>>> wrote:
>>>>>>
>>>>>>> Great point Houston! +1 on waiting until a week into January
>>>>>>>
>>>>>>> On Thu, Dec 17, 2020 at 4:46 PM Houston Putman <
>>>>>>> houstonput...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Thanks for volunteering Ishan.
>>>>>>>>
>>>>>>>> I think it might be a good idea to wait to cut and release 8.8 at
>>>>>>>> least a week into January. Many people are going to be away during the
>>>>>>>> holiday season, and particularly the last week of the year. Pushing 
>>>>>>>> into
>>>>>>>> January just gives more people a chance to look at the release and be
>>>>>>>> involved.
>>>>>>>>
>>>>>>>> - Houston
>>>>>>>>
>>>>>>>> On Fri, Dec 11, 2020 at 3:26 PM Noble Paul 
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Thanks Ishan for volunteering
>>>>>>>>>
>>>>>>>>> On Fri, Dec 11, 2020 at 5:07 AM Christine Poerschke (BLOOMBERG/
>>>>>>>>> LONDON)  wrote:
>>>>>>>>> >
>>>>>>>>> > With a view towards including it in the release, I'd appreciate
>>>>>>>>> code review input on
>>>>>>>>> >
>>>>>>>>> > https://github.com/apache/lucene-solr/pull/1992 for
>>>>>>>>> >
>>>>>>>>> > https://issues.apache.org/jira/browse/SOLR-14939 (JSON facets:
>>>>>>>>> range faceting to support cache=false parameter)
>>>>>

Re: New branch and feature freeze for Lucene/Solr 8.8.0

[Ishan Chattopadhyaya]

If one thinks that any new feature/change needs to go in, please discuss in
the 8.8 release thread. So long as it doesn't disrupt the stability of the
release, we can consider it on a case by case basis.

On Tue, 12 Jan, 2021, 8:04 am Ishan Chattopadhyaya, <
ichattopadhy...@gmail.com> wrote:

>
> Branch branch_8_8 has been cut and versions updated to 8.9 on stable
> branch.
>
> Please observe the normal rules:
>
> * No new features may be committed to the branch.
> * Documentation patches, build patches and serious bug fixes may be
>   committed to the branch. However, you should submit all patches you
>   want to commit to Jira first to give others the chance to review
>   and possibly vote against the patch. Keep in mind that it is our
>   main intention to keep the branch as stable as possible.
> * All patches that are intended for the branch should first be committed
>   to the unstable branch, merged into the stable branch, and then into
>   the current release branch.
> * Normal unstable and stable branch development may continue as usual.
>   However, if you plan to commit a big change to the unstable branch
>   while the branch feature freeze is in effect, think twice: can't the
>   addition wait a couple more days? Merges of bug fixes into the branch
>   may become more difficult.
> * Only Jira issues with Fix version 8.8 and priority "Blocker" will delay
>   a release candidate build.
>

Re: 8.8 Release

[Ishan Chattopadhyaya]

Thanks Tim. I've now cut the branch for 8.8.
@David, let's get the fix in for the release.


On Tue, Jan 12, 2021 at 2:13 AM David Smiley  wrote:

> FYI https://issues.apache.org/jira/browse/SOLR-15071 for LTR a regression
> Christine hasn't posted the fix yet but I'd guess there's a user
> work-around so the issue isn't pressing I suppose.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Mon, Jan 11, 2021 at 1:32 PM Timothy Potter 
> wrote:
>
>> Fix is in branch_8x and master now, sorry for the delay ;-)
>>
>> Cheers,
>> Tim
>>
>> On Mon, Jan 11, 2021 at 10:50 AM Ishan Chattopadhyaya <
>> ichattopadhy...@gmail.com> wrote:
>>
>>> Thanks Tim!
>>>
>>> On Mon, 11 Jan, 2021, 11:00 pm Timothy Potter, 
>>> wrote:
>>>
>>>> 15036 will be in later today, so you can plan to cut this evening US
>>>> time or tomorrow.
>>>>
>>>> Cheers,
>>>> Tim
>>>>
>>>> On Mon, Jan 11, 2021 at 9:54 AM Ishan Chattopadhyaya <
>>>> ichattopadhy...@gmail.com> wrote:
>>>>
>>>>> I think all the issues mentioned above are in the branch, except
>>>>> SOLR-15036. Tim, I'll cut a branch once that is in, latest by Wednesday AM
>>>>> (USA time).
>>>>> Thanks,
>>>>> Ishan
>>>>>
>>>>> On Thu, Jan 7, 2021 at 5:07 AM Timothy Potter 
>>>>> wrote:
>>>>>
>>>>>> Thanks for following up on this Ishan ... I intend to get SOLR-15059
>>>>>> and -15036 into 8.8 as well. I should have a proper PR up for SOLR-15036 
>>>>>> by
>>>>>> Friday sometime, which seems to align with other's timeframes
>>>>>>
>>>>>> Cheers,
>>>>>> Tim
>>>>>>
>>>>>> On Wed, Jan 6, 2021 at 6:54 AM David Smiley 
>>>>>> wrote:
>>>>>>
>>>>>>> Happy New Year!
>>>>>>> I would much prefer that ensure 8.8 includes SOLR-14923 (a bad
>>>>>>> nested docs performance issue)
>>>>>>>
>>>>>>> ~ David Smiley
>>>>>>> Apache Lucene/Solr Search Developer
>>>>>>> http://www.linkedin.com/in/davidwsmiley
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jan 6, 2021 at 6:59 AM Ishan Chattopadhyaya <
>>>>>>> ichattopadhy...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Happy New Year!
>>>>>>>>
>>>>>>>> I was supposed to start the process tomorrow, but I think we're not
>>>>>>>> ready yet? I see SOLR-15052 still under review with intention of 
>>>>>>>> inclusion
>>>>>>>> into 8.8.
>>>>>>>> Would it be reasonable to cut the release branch end of this week
>>>>>>>> and start the RC process around 13th January?
>>>>>>>> If there are any issues someone would want me to wait on, please
>>>>>>>> let me know.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Ishan
>>>>>>>>
>>>>>>>> On Fri, Dec 18, 2020 at 6:10 AM Ishan Chattopadhyaya <
>>>>>>>> ichattopadhy...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Sure, Houston. I'll wait another week. Have a good new year and
>>>>>>>>> merry Christmas!
>>>>>>>>>
>>>>>>>>> On Fri, 18 Dec, 2020, 5:58 am Timothy Potter, <
>>>>>>>>> thelabd...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Great point Houston! +1 on waiting until a week into January
>>>>>>>>>>
>>>>>>>>>> On Thu, Dec 17, 2020 at 4:46 PM Houston Putman <
>>>>>>>>>> houstonput...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Thanks for volunteering Ishan.
>>>>>>>>>>>
>>>>>>>>>>> I think it might be a good idea to wait to cut and release 8.8
>>>>>>>>>>> at least a week into January. Many people are going to be away 

New branch and feature freeze for Lucene/Solr 8.8.0

[Ishan Chattopadhyaya]

Branch branch_8_8 has been cut and versions updated to 8.9 on stable branch.

Please observe the normal rules:

* No new features may be committed to the branch.
* Documentation patches, build patches and serious bug fixes may be
  committed to the branch. However, you should submit all patches you
  want to commit to Jira first to give others the chance to review
  and possibly vote against the patch. Keep in mind that it is our
  main intention to keep the branch as stable as possible.
* All patches that are intended for the branch should first be committed
  to the unstable branch, merged into the stable branch, and then into
  the current release branch.
* Normal unstable and stable branch development may continue as usual.
  However, if you plan to commit a big change to the unstable branch
  while the branch feature freeze is in effect, think twice: can't the
  addition wait a couple more days? Merges of bug fixes into the branch
  may become more difficult.
* Only Jira issues with Fix version 8.8 and priority "Blocker" will delay
  a release candidate build.