Re: [DISCUSS] 8.5.2 Release?
Are you are talking about these? https://github.com/apache/lucene-solr/commit/ba0891415e9567bac96ace79013184aa039ac580 https://github.com/apache/lucene-solr/commit/ada8be1b45d97c45234b45ae00ddce3425631420 On Fri, May 15, 2020 at 3:41 AM Mike Drob wrote: > > Noble, > > We're still missing a few: > > https://cwiki.apache.org/confluence/display/LUCENE/ReleaseTodo#ReleaseTodo-UpdatetheprojectDOAPfiles > https://cwiki.apache.org/confluence/display/LUCENE/ReleaseTodo#ReleaseTodo-GenerateBackcompatIndexes > > Also, the release is not on the mirrors - > https://lucene.apache.org/core/downloads.html > The link to > https://www.apache.org/dyn/closer.lua/lucene/java/7.7.3/lucene-7.7.3-src.tgz > doesn't resolve to anything... > > Mike > > On Wed, May 13, 2020 at 9:18 PM Noble Paul wrote: >> >> I just finished all the steps given in >> https://cwiki.apache.org/confluence/display/LUCENE/ReleaseTodo >> >> Please let me know if anything is missing. >> I still don't see the solr release details in https://lucene.apache.org/ >> >> How do i do it? >> >> >> On Thu, May 14, 2020 at 10:47 AM Noble Paul wrote: >> > >> > I'll fix them today >> > >> > On Thu, May 14, 2020, 9:19 AM Mike Drob wrote: >> >> >> >> Thanks. I’ve found one small change for the release script so far, I plan >> >> to batch all my notes together and commit them at the end of the process. >> >> >> >> Right now I think I’m waiting on a few final steps from 7.7.3 to complete >> >> and then we’ll be ready to roll with 8.5.2 >> >> >> >> On Wed, May 13, 2020 at 4:59 PM Jan Høydahl wrote: >> >>> >> >>> Mike, I merged latest changes to releaseWizard to branch_8_5 as well. So >> >>> you’ll be first to test the new instructions for updating the web site. >> >>> So please be prepared for discovering quirks in that part of the >> >>> releaseWizard. >> >>> >> >>> Jan >> >>> >> >>> > 7. mai 2020 kl. 19:13 skrev Mike Drob : >> >>> > >> >>> > Devs, >> >>> > >> >>> > I know that we had 8.5.1 only a few weeks ago, but with the fix for >> >>> > LUCENE-9350 I think we should consider another bug-fix. I know that >> >>> > without it I will be explicitly recommending several users to stay off >> >>> > of 8.5.x on their upgrade plans. There are some pretty scary looking >> >>> > charts on SOLR-14428 that describe the impact of the bug in more >> >>> > detail. >> >>> > >> >>> > I'd be happy to volunteer as RM for this, would probably be looking at >> >>> > trying to get it a vote started sometime next week. >> >>> > >> >>> > Thanks, >> >>> > Mike >> >>> > >> >>> > >> >>> > https://issues.apache.org/jira/browse/SOLR-14428 >> >>> > https://issues.apache.org/jira/browse/LUCENE-9350 >> >>> > >> >>> > - >> >>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> >>> > For additional commands, e-mail: dev-h...@lucene.apache.org >> >>> > >> >>> >> >>> >> >>> - >> >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> >>> For additional commands, e-mail: dev-h...@lucene.apache.org >> >>> >> >> >> -- >> - >> Noble Paul >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> -- - Noble Paul - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: SortedDocValues.lookupOrd and BytesRef reuse
Ok thanks, this makes sense. It's safe to use for the same SortedDocValues
instance until the method is called again. I think changing the javadoc to
the following will help clear up the confusion:
/** Retrieves the value for the specified ordinal. The returned
* {@link BytesRef} may be re-used across calls on the same instance
to the {@link #lookupOrd(int)}
* so make sure to {@link BytesRef#deepCopyOf(BytesRef) copy it} if you want
* to keep it around.
* @param ord ordinal to lookup (must be = 0 and {@link
#getValueCount()})
* @see #ordValue()
*/
I can make this change if others agree.
Joel Bernstein
http://joelsolr.blogspot.com/
On Thu, May 14, 2020 at 4:37 PM Michael McCandless <
luc...@mikemccandless.com> wrote:
> Hi Joel,
>
> You should trust the javadocs.
>
> Looking at our default Codec on master (Lucene84Codec), and at its default
> doc values implementation (Lucene80DocValuesProducer), it is clearly
> reusing the private "BytesRef term" instance.
>
> If your code is fully consuming this BytesRef before calling any other
> methods on the same SortedDocValues instance, you can safely reuse it for
> that duration.
>
> But if you want to call methods on that same SortedDocValues and continue
> using the previous BytesRef, you'll need to make a copy.
>
> Maybe improve the javadocs?
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Thu, May 14, 2020 at 4:12 PM Joel Bernstein wrote:
>
>> In the SortedDocValues.lookupOrd documentation it says that a deep copy is
>> needed for the returned BytesRef. I wanted to verify that this was actually
>> true. I'm
>>
>> trying to see a way that this BytesRef could be safely reused by the API but
>> I don't see one. Is there actually an implementation of lookupOrd that
>> somehow reuses the
>>
>> same BytesRef between invocations. The java doc is copied below:
>>
>> Thanks!
>>
>> /** Retrieves the value for the specified ordinal. The returned
>> * {@link BytesRef} may be re-used across calls to {@link #lookupOrd(int)}
>> * so make sure to {@link BytesRef#deepCopyOf(BytesRef) copy it} if you want
>> * to keep it around.
>> * @param ord ordinal to lookup (must be = 0 and {@link
>> #getValueCount()})
>> * @see #ordValue()
>> */
>> public abstract BytesRef lookupOrd(int ord) throws IOException;
>>
>>
>>
>> Joel Bernstein
>> http://joelsolr.blogspot.com/
>>
>
Re: SortedDocValues.lookupOrd and BytesRef reuse
Hi Joel,
You should trust the javadocs.
Looking at our default Codec on master (Lucene84Codec), and at its default
doc values implementation (Lucene80DocValuesProducer), it is clearly
reusing the private "BytesRef term" instance.
If your code is fully consuming this BytesRef before calling any other
methods on the same SortedDocValues instance, you can safely reuse it for
that duration.
But if you want to call methods on that same SortedDocValues and continue
using the previous BytesRef, you'll need to make a copy.
Maybe improve the javadocs?
Mike McCandless
http://blog.mikemccandless.com
On Thu, May 14, 2020 at 4:12 PM Joel Bernstein wrote:
> In the SortedDocValues.lookupOrd documentation it says that a deep copy is
> needed for the returned BytesRef. I wanted to verify that this was actually
> true. I'm
>
> trying to see a way that this BytesRef could be safely reused by the API but
> I don't see one. Is there actually an implementation of lookupOrd that
> somehow reuses the
>
> same BytesRef between invocations. The java doc is copied below:
>
> Thanks!
>
> /** Retrieves the value for the specified ordinal. The returned
> * {@link BytesRef} may be re-used across calls to {@link #lookupOrd(int)}
> * so make sure to {@link BytesRef#deepCopyOf(BytesRef) copy it} if you want
> * to keep it around.
> * @param ord ordinal to lookup (must be = 0 and {@link
> #getValueCount()})
> * @see #ordValue()
> */
> public abstract BytesRef lookupOrd(int ord) throws IOException;
>
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
SortedDocValues.lookupOrd and BytesRef reuse
In the SortedDocValues.lookupOrd documentation it says that a deep
copy is needed for the returned BytesRef. I wanted to verify that this
was actually true. I'm
trying to see a way that this BytesRef could be safely reused by the
API but I don't see one. Is there actually an implementation of
lookupOrd that somehow reuses the
same BytesRef between invocations. The java doc is copied below:
Thanks!
/** Retrieves the value for the specified ordinal. The returned
* {@link BytesRef} may be re-used across calls to {@link #lookupOrd(int)}
* so make sure to {@link BytesRef#deepCopyOf(BytesRef) copy it} if you want
* to keep it around.
* @param ord ordinal to lookup (must be = 0 and {@link
#getValueCount()})
* @see #ordValue()
*/
public abstract BytesRef lookupOrd(int ord) throws IOException;
Joel Bernstein
http://joelsolr.blogspot.com/
Re: [DISCUSS] Lucene-Solr split (Solr promoted to TLP)
Perhaps Christine! That's a nice idea! On naming, it would have to be probably something snazzier than "Search" as you get at. It would probably not be a good trademark, and would imply that Lucene & Solr are the only things in ASF that could be "Search". Who knows, one day Vespa or something else search related could become an ASF project with different governance & committers. -Doug On Thu, May 14, 2020 at 2:16 PM Anshum Gupta wrote: > Thanks Christine! > > I genuinely like this idea. > > This actually gets us what we want without having to handle everything at > the same time, and also giving us time to see if the split is working or > not. This process also ensures that both, Lucene and Solr maintain the > symbiotic relationship at least at the beginning. > > > On Thu, May 14, 2020 at 9:35 AM Christine Poerschke > wrote: > >> Hello. >> >> The discussion subject here has two parts i.e. "Lucene-Solr split" and >> "Solr promoted to TLP" and I'd be curious what doing the former separately >> ahead of the latter might look like and/or if consensus around that would >> be different? >> >> Thinking aloud, as a hypothetical scenario like. >> * For the 8.x series Lucene and Solr release together as before. >> * With 9.0 the releases begin to split: Lucene has 9.0 release and Solr >> has a release that uses Lucene 9.0 (and which may be called Solr 9.0 or >> which may be called something else like Solr 2021.0 or something). Both >> releases happen at the same time and it being a 8-to-9 major release might >> help with user communications clarity. >> * Lucene and Solr now live in separate repos, development progresses, >> there's releases for one or other or both. We adapt to the split approach >> and still being one project and one dev mailing list and community helps, >> hopefully, with that adjustment. >> * After a while (perhaps with Lucene 10.0 or perhaps at some other >> natural point) we re-arrive at the "together or separate" question. If >> splitting worked well then Solr promotion to TLP could be a natural next >> step, or if getting back together might be better for both parties then >> from the next major release things would be combined again. >> >> Christine >> >> On 2020/05/04 09:10:35, Dawid Weiss wrote: >> > Dear Lucene and Solr developers! >> > >> > A few days ago, I initiated a discussion among PMC members about >> > potential pros and cons of splitting the project into separate Lucene >> > and Solr entities by promoting Solr to its own top-level Apache >> > project (TLP). Let me share with you the motivation for such an action >> > and some follow-up thoughts I heard from other PMC members so far. >> > >> > Please read this e-mail carefully. Both the PMC and I look forward to >> > hearing your opinion. This is a DISCUSS thread and it will be followed >> > next week by a VOTE thread. This is our shared project and we should >> > all shape its future responsibly. >> > >> > The big question is this: “Is this the right time to split Solr and >> > Lucene into two independent projects?”. >> > >> > Here are several technical considerations that drove me to ask the >> > question above (in no order of priorities): >> > >> > 1) Precommit/ test times. These are crazy high. If we split into two >> > projects we can pretty much cut all of Lucene testing out of Solr (and >> > likewise), making development a bit more fun again. >> > >> > 2) Build system itself and source release packaging. The current >> > combined codebase is a *beast* to maintain. Working with gradle on >> > both projects at once made me realise how little the two have in >> > common. The code layout, the dependencies, even the workflow of people >> > >> > working on these projects... The build (both ant and gradle) is full >> > of Solr and Lucene-specific exceptions and hooks that could be more >> > elegantly solved if moved to each project independently. >> > >> > 3) Packaging. There is no single source distribution package for >> > Solr+Lucene. They are already "independent" there. Why should Lucene >> > and Solr always be released at the same pace? Does it always make >> > sense? >> > >> > 4) Solr is essentially taking in Lucene and its dependencies as a >> > whole (so is Elasticsearch and many other projects). In my opinion >> > this makes Lucene eligible for refactoring and >> > >> > maintenance as a separate component. The learning curve for people >> > coming to each project separately is going to be gentler than trying >> > to dive into the combined codebase. >> > >> > 5) Mailing lists, build servers. Mailing lists for users are already >> > separated. I think this is yet another indication that Solr is >> > something more than a component within Lucene. It is perceived as an >> > independent entity and used as an independent product. I would really >> > like to have separate mailing lists for these two projects (this >> > includes build and test results) as it would make life easier: if your >> > focus is more on Lucene
Re: [DISCUSS] Lucene-Solr split (Solr promoted to TLP)
Thanks Christine! I genuinely like this idea. This actually gets us what we want without having to handle everything at the same time, and also giving us time to see if the split is working or not. This process also ensures that both, Lucene and Solr maintain the symbiotic relationship at least at the beginning. On Thu, May 14, 2020 at 9:35 AM Christine Poerschke wrote: > Hello. > > The discussion subject here has two parts i.e. "Lucene-Solr split" and > "Solr promoted to TLP" and I'd be curious what doing the former separately > ahead of the latter might look like and/or if consensus around that would > be different? > > Thinking aloud, as a hypothetical scenario like. > * For the 8.x series Lucene and Solr release together as before. > * With 9.0 the releases begin to split: Lucene has 9.0 release and Solr > has a release that uses Lucene 9.0 (and which may be called Solr 9.0 or > which may be called something else like Solr 2021.0 or something). Both > releases happen at the same time and it being a 8-to-9 major release might > help with user communications clarity. > * Lucene and Solr now live in separate repos, development progresses, > there's releases for one or other or both. We adapt to the split approach > and still being one project and one dev mailing list and community helps, > hopefully, with that adjustment. > * After a while (perhaps with Lucene 10.0 or perhaps at some other natural > point) we re-arrive at the "together or separate" question. If splitting > worked well then Solr promotion to TLP could be a natural next step, or if > getting back together might be better for both parties then from the next > major release things would be combined again. > > Christine > > On 2020/05/04 09:10:35, Dawid Weiss wrote: > > Dear Lucene and Solr developers! > > > > A few days ago, I initiated a discussion among PMC members about > > potential pros and cons of splitting the project into separate Lucene > > and Solr entities by promoting Solr to its own top-level Apache > > project (TLP). Let me share with you the motivation for such an action > > and some follow-up thoughts I heard from other PMC members so far. > > > > Please read this e-mail carefully. Both the PMC and I look forward to > > hearing your opinion. This is a DISCUSS thread and it will be followed > > next week by a VOTE thread. This is our shared project and we should > > all shape its future responsibly. > > > > The big question is this: “Is this the right time to split Solr and > > Lucene into two independent projects?”. > > > > Here are several technical considerations that drove me to ask the > > question above (in no order of priorities): > > > > 1) Precommit/ test times. These are crazy high. If we split into two > > projects we can pretty much cut all of Lucene testing out of Solr (and > > likewise), making development a bit more fun again. > > > > 2) Build system itself and source release packaging. The current > > combined codebase is a *beast* to maintain. Working with gradle on > > both projects at once made me realise how little the two have in > > common. The code layout, the dependencies, even the workflow of people > > > > working on these projects... The build (both ant and gradle) is full > > of Solr and Lucene-specific exceptions and hooks that could be more > > elegantly solved if moved to each project independently. > > > > 3) Packaging. There is no single source distribution package for > > Solr+Lucene. They are already "independent" there. Why should Lucene > > and Solr always be released at the same pace? Does it always make > > sense? > > > > 4) Solr is essentially taking in Lucene and its dependencies as a > > whole (so is Elasticsearch and many other projects). In my opinion > > this makes Lucene eligible for refactoring and > > > > maintenance as a separate component. The learning curve for people > > coming to each project separately is going to be gentler than trying > > to dive into the combined codebase. > > > > 5) Mailing lists, build servers. Mailing lists for users are already > > separated. I think this is yet another indication that Solr is > > something more than a component within Lucene. It is perceived as an > > independent entity and used as an independent product. I would really > > like to have separate mailing lists for these two projects (this > > includes build and test results) as it would make life easier: if your > > focus is more on Lucene (or Solr), you would only need to track half > > of the current traffic. > > > > > > As I already mentioned, the discussion among PMC members highlighted > > some initial concerns and reasons why the project should perhaps > > remain glued together. These are outlined below with some of the > > counter-arguments presented under each concern to avoid repetition of > > the same content from the PMC mailing list (they’re copied from the > > private discussion list). > > > > 1) Both projects may gradually split their ways after the separation >
Re: [DISCUSS] 8.5.2 Release?
Noble, We're still missing a few: https://cwiki.apache.org/confluence/display/LUCENE/ReleaseTodo#ReleaseTodo-UpdatetheprojectDOAPfiles https://cwiki.apache.org/confluence/display/LUCENE/ReleaseTodo#ReleaseTodo-GenerateBackcompatIndexes Also, the release is not on the mirrors - https://lucene.apache.org/core/downloads.html The link to https://www.apache.org/dyn/closer.lua/lucene/java/7.7.3/lucene-7.7.3-src.tgz doesn't resolve to anything... Mike On Wed, May 13, 2020 at 9:18 PM Noble Paul wrote: > I just finished all the steps given in > https://cwiki.apache.org/confluence/display/LUCENE/ReleaseTodo > > Please let me know if anything is missing. > I still don't see the solr release details in https://lucene.apache.org/ > > How do i do it? > > > On Thu, May 14, 2020 at 10:47 AM Noble Paul wrote: > > > > I'll fix them today > > > > On Thu, May 14, 2020, 9:19 AM Mike Drob wrote: > >> > >> Thanks. I’ve found one small change for the release script so far, I > plan to batch all my notes together and commit them at the end of the > process. > >> > >> Right now I think I’m waiting on a few final steps from 7.7.3 to > complete and then we’ll be ready to roll with 8.5.2 > >> > >> On Wed, May 13, 2020 at 4:59 PM Jan Høydahl > wrote: > >>> > >>> Mike, I merged latest changes to releaseWizard to branch_8_5 as well. > So you’ll be first to test the new instructions for updating the web site. > So please be prepared for discovering quirks in that part of the > releaseWizard. > >>> > >>> Jan > >>> > >>> > 7. mai 2020 kl. 19:13 skrev Mike Drob : > >>> > > >>> > Devs, > >>> > > >>> > I know that we had 8.5.1 only a few weeks ago, but with the fix for > LUCENE-9350 I think we should consider another bug-fix. I know that without > it I will be explicitly recommending several users to stay off of 8.5.x on > their upgrade plans. There are some pretty scary looking charts on > SOLR-14428 that describe the impact of the bug in more detail. > >>> > > >>> > I'd be happy to volunteer as RM for this, would probably be looking > at trying to get it a vote started sometime next week. > >>> > > >>> > Thanks, > >>> > Mike > >>> > > >>> > > >>> > https://issues.apache.org/jira/browse/SOLR-14428 > >>> > https://issues.apache.org/jira/browse/LUCENE-9350 > >>> > > >>> > - > >>> > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > >>> > For additional commands, e-mail: dev-h...@lucene.apache.org > >>> > > >>> > >>> > >>> - > >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > >>> For additional commands, e-mail: dev-h...@lucene.apache.org > >>> > > > -- > - > Noble Paul > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > >
Re: [VOTE] Solr to become a top-level Apache project (TLP)
+1 On Thu, 14 May 2020 at 22:42, Kevin Risden wrote: > +1 (binding) > > Kevin Risden > > > > On Thu, May 14, 2020 at 12:17 PM Nicholas Knize wrote: > >> +1 (binding) >> >> Nicholas Knize, Ph.D., GISP >> Geospatial Software Guy | Elasticsearch >> Apache Lucene PMC Member and Committer >> nkn...@apache.org >> >> >> On Thu, May 14, 2020 at 11:02 AM Namgyu Kim wrote: >> >>> +1 >>> >>> On Thu, May 14, 2020 at 10:33 PM Tommaso Teofili < >>> tommaso.teof...@gmail.com> wrote: >>> +1 (binding) Tommaso On Tue, 12 May 2020 at 09:37, Dawid Weiss wrote: > Dear Lucene and Solr developers! > > According to an earlier [DISCUSS] thread on the dev list [2], I am > calling for a vote on the proposal to make Solr a top-level Apache > project (TLP) and separate Lucene and Solr development into two > independent entities. > > To quickly recap the reasons and consequences of such a move: it seems > like the reasons for the initial merge of Lucene and Solr, around 10 > years ago, have been achieved. Both projects are in good shape and > exhibit signs of independence already (mailing lists, committers, > patch flow). There are many technical considerations that would make > development much easier if we move Solr out into its own TLP. > > We discussed this issue [2] and both PMC members and committers had a > chance to review all the pros and cons and express their views. The > discussion showed that there are clearly different opinions on the > matter - some people are in favor, some are neutral, others are > against or not seeing the point of additional labor. Realistically, I > don't think reaching 100% level consensus is going to be possible -- > we are a diverse bunch with different opinions and personalities. I > firmly believe this is the right direction hence the decision to put > it under the voting process. Should something take a wrong turn in the > future (as some folks worry it may), all blame is on me. > > Therefore, the proposal is to separate Solr from under Lucene TLP, and > make it a TLP on its own. The initial structure of the new PMC, > committer base, git repositories and other managerial aspects can be > worked out during the process if the decision passes. > > Please indicate one of the following (see [1] for guidelines): > > [ ] +1 - yes, I vote for the proposal > [ ] -1 - no, I vote against the proposal > > Please note that anyone in the Lucene+Solr community is invited to > express their opinion, though only Lucene+Solr committers cast binding > votes (indicate non-binding votes in your reply, please). > > The vote will be active for a week to give everyone a chance to read > and cast a vote. > > Dawid > > [1] https://www.apache.org/foundation/voting.html > [2] > https://lists.apache.org/thread.html/rfae2440264f6f874e91545b2030c98e7b7e3854ddf090f7747d338df%40%3Cdev.lucene.apache.org%3E > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > >
Re: [VOTE] Solr to become a top-level Apache project (TLP)
+1 (binding) Kevin Risden On Thu, May 14, 2020 at 12:17 PM Nicholas Knize wrote: > +1 (binding) > > Nicholas Knize, Ph.D., GISP > Geospatial Software Guy | Elasticsearch > Apache Lucene PMC Member and Committer > nkn...@apache.org > > > On Thu, May 14, 2020 at 11:02 AM Namgyu Kim wrote: > >> +1 >> >> On Thu, May 14, 2020 at 10:33 PM Tommaso Teofili < >> tommaso.teof...@gmail.com> wrote: >> >>> +1 (binding) >>> >>> Tommaso >>> >>> On Tue, 12 May 2020 at 09:37, Dawid Weiss wrote: >>> Dear Lucene and Solr developers! According to an earlier [DISCUSS] thread on the dev list [2], I am calling for a vote on the proposal to make Solr a top-level Apache project (TLP) and separate Lucene and Solr development into two independent entities. To quickly recap the reasons and consequences of such a move: it seems like the reasons for the initial merge of Lucene and Solr, around 10 years ago, have been achieved. Both projects are in good shape and exhibit signs of independence already (mailing lists, committers, patch flow). There are many technical considerations that would make development much easier if we move Solr out into its own TLP. We discussed this issue [2] and both PMC members and committers had a chance to review all the pros and cons and express their views. The discussion showed that there are clearly different opinions on the matter - some people are in favor, some are neutral, others are against or not seeing the point of additional labor. Realistically, I don't think reaching 100% level consensus is going to be possible -- we are a diverse bunch with different opinions and personalities. I firmly believe this is the right direction hence the decision to put it under the voting process. Should something take a wrong turn in the future (as some folks worry it may), all blame is on me. Therefore, the proposal is to separate Solr from under Lucene TLP, and make it a TLP on its own. The initial structure of the new PMC, committer base, git repositories and other managerial aspects can be worked out during the process if the decision passes. Please indicate one of the following (see [1] for guidelines): [ ] +1 - yes, I vote for the proposal [ ] -1 - no, I vote against the proposal Please note that anyone in the Lucene+Solr community is invited to express their opinion, though only Lucene+Solr committers cast binding votes (indicate non-binding votes in your reply, please). The vote will be active for a week to give everyone a chance to read and cast a vote. Dawid [1] https://www.apache.org/foundation/voting.html [2] https://lists.apache.org/thread.html/rfae2440264f6f874e91545b2030c98e7b7e3854ddf090f7747d338df%40%3Cdev.lucene.apache.org%3E - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [DISCUSS] Lucene-Solr split (Solr promoted to TLP)
Perhaps a bit of a wildcard question or thought ... would any split out top-level project necessarily be called "Apache Solr" or could the split out project be called "Apache " with "Apache Solr" as its initial sub-project and over time there may be other sub-projects added? No particular name in mind, "Apache Search" might be too obvious, just wondering in principle. Christine On 2020/05/04 09:10:35, Dawid Weiss wrote: > Dear Lucene and Solr developers! > > A few days ago, I initiated a discussion among PMC members about > potential pros and cons of splitting the project into separate Lucene > and Solr entities by promoting Solr to its own top-level Apache > project (TLP). Let me share with you the motivation for such an action > and some follow-up thoughts I heard from other PMC members so far. > > Please read this e-mail carefully. Both the PMC and I look forward to > hearing your opinion. This is a DISCUSS thread and it will be followed > next week by a VOTE thread. This is our shared project and we should > all shape its future responsibly. > > The big question is this: “Is this the right time to split Solr and > Lucene into two independent projects?”. > > Here are several technical considerations that drove me to ask the > question above (in no order of priorities): > > 1) Precommit/ test times. These are crazy high. If we split into two > projects we can pretty much cut all of Lucene testing out of Solr (and > likewise), making development a bit more fun again. > > 2) Build system itself and source release packaging. The current > combined codebase is a *beast* to maintain. Working with gradle on > both projects at once made me realise how little the two have in > common. The code layout, the dependencies, even the workflow of people > > working on these projects... The build (both ant and gradle) is full > of Solr and Lucene-specific exceptions and hooks that could be more > elegantly solved if moved to each project independently. > > 3) Packaging. There is no single source distribution package for > Solr+Lucene. They are already "independent" there. Why should Lucene > and Solr always be released at the same pace? Does it always make > sense? > > 4) Solr is essentially taking in Lucene and its dependencies as a > whole (so is Elasticsearch and many other projects). In my opinion > this makes Lucene eligible for refactoring and > > maintenance as a separate component. The learning curve for people > coming to each project separately is going to be gentler than trying > to dive into the combined codebase. > > 5) Mailing lists, build servers. Mailing lists for users are already > separated. I think this is yet another indication that Solr is > something more than a component within Lucene. It is perceived as an > independent entity and used as an independent product. I would really > like to have separate mailing lists for these two projects (this > includes build and test results) as it would make life easier: if your > focus is more on Lucene (or Solr), you would only need to track half > of the current traffic. > > > As I already mentioned, the discussion among PMC members highlighted > some initial concerns and reasons why the project should perhaps > remain glued together. These are outlined below with some of the > counter-arguments presented under each concern to avoid repetition of > the same content from the PMC mailing list (they’re copied from the > private discussion list). > > 1) Both projects may gradually split their ways after the separation > and even develop “against” each other like it used to be before the > merge. > > Whether this is a legitimate concern is hard to tell. If Solr goes TLP > then all existing Lucene committers will automatically become Solr > committers (unless they opt not to) so there will be both procedural > ways to prevent this from happening (vetoes) as well as common-sense > reasons to just cooperate. > > 2) Some people like parallel version numbering (concurrent Solr and > Lucene releases) as it gives instant clarity which Solr version uses > which version of Lucene. > > This can still be done on Solr side (it is Solr’s decision to adapt > any versioning scheme the project feels comfortable with). I > personally (DW) think this kind of versioning is actually more > confusing than helpful; Solr should have its own cadence of releases > driven by features, not sub-component changes. If the “backwards > compatibility” is a factor then a solution might be to sync on major > version releases only (e.g., this is how Elasticsearch is handling > this). > > 3) Solr tests are the first “battlefield” test zone for Lucene changes > - if it becomes TLP this part will be gone. > > Yes, true. But realistically Solr will have to adopt some kind of > snapshot-based dependency on Lucene anyway (whether as a git submodule > or a maven snapshot dependency). So if there are bugs in Lucene they > will still be detected by Solr tests (and fairly
Re: [DISCUSS] Lucene-Solr split (Solr promoted to TLP)
Hello. The discussion subject here has two parts i.e. "Lucene-Solr split" and "Solr promoted to TLP" and I'd be curious what doing the former separately ahead of the latter might look like and/or if consensus around that would be different? Thinking aloud, as a hypothetical scenario like. * For the 8.x series Lucene and Solr release together as before. * With 9.0 the releases begin to split: Lucene has 9.0 release and Solr has a release that uses Lucene 9.0 (and which may be called Solr 9.0 or which may be called something else like Solr 2021.0 or something). Both releases happen at the same time and it being a 8-to-9 major release might help with user communications clarity. * Lucene and Solr now live in separate repos, development progresses, there's releases for one or other or both. We adapt to the split approach and still being one project and one dev mailing list and community helps, hopefully, with that adjustment. * After a while (perhaps with Lucene 10.0 or perhaps at some other natural point) we re-arrive at the "together or separate" question. If splitting worked well then Solr promotion to TLP could be a natural next step, or if getting back together might be better for both parties then from the next major release things would be combined again. Christine On 2020/05/04 09:10:35, Dawid Weiss wrote: > Dear Lucene and Solr developers! > > A few days ago, I initiated a discussion among PMC members about > potential pros and cons of splitting the project into separate Lucene > and Solr entities by promoting Solr to its own top-level Apache > project (TLP). Let me share with you the motivation for such an action > and some follow-up thoughts I heard from other PMC members so far. > > Please read this e-mail carefully. Both the PMC and I look forward to > hearing your opinion. This is a DISCUSS thread and it will be followed > next week by a VOTE thread. This is our shared project and we should > all shape its future responsibly. > > The big question is this: “Is this the right time to split Solr and > Lucene into two independent projects?”. > > Here are several technical considerations that drove me to ask the > question above (in no order of priorities): > > 1) Precommit/ test times. These are crazy high. If we split into two > projects we can pretty much cut all of Lucene testing out of Solr (and > likewise), making development a bit more fun again. > > 2) Build system itself and source release packaging. The current > combined codebase is a *beast* to maintain. Working with gradle on > both projects at once made me realise how little the two have in > common. The code layout, the dependencies, even the workflow of people > > working on these projects... The build (both ant and gradle) is full > of Solr and Lucene-specific exceptions and hooks that could be more > elegantly solved if moved to each project independently. > > 3) Packaging. There is no single source distribution package for > Solr+Lucene. They are already "independent" there. Why should Lucene > and Solr always be released at the same pace? Does it always make > sense? > > 4) Solr is essentially taking in Lucene and its dependencies as a > whole (so is Elasticsearch and many other projects). In my opinion > this makes Lucene eligible for refactoring and > > maintenance as a separate component. The learning curve for people > coming to each project separately is going to be gentler than trying > to dive into the combined codebase. > > 5) Mailing lists, build servers. Mailing lists for users are already > separated. I think this is yet another indication that Solr is > something more than a component within Lucene. It is perceived as an > independent entity and used as an independent product. I would really > like to have separate mailing lists for these two projects (this > includes build and test results) as it would make life easier: if your > focus is more on Lucene (or Solr), you would only need to track half > of the current traffic. > > > As I already mentioned, the discussion among PMC members highlighted > some initial concerns and reasons why the project should perhaps > remain glued together. These are outlined below with some of the > counter-arguments presented under each concern to avoid repetition of > the same content from the PMC mailing list (they’re copied from the > private discussion list). > > 1) Both projects may gradually split their ways after the separation > and even develop “against” each other like it used to be before the > merge. > > Whether this is a legitimate concern is hard to tell. If Solr goes TLP > then all existing Lucene committers will automatically become Solr > committers (unless they opt not to) so there will be both procedural > ways to prevent this from happening (vetoes) as well as common-sense > reasons to just cooperate. > > 2) Some people like parallel version numbering (concurrent Solr and > Lucene releases) as it gives instant clarity which Solr
Re: [VOTE] Solr to become a top-level Apache project (TLP)
+1 (binding) Nicholas Knize, Ph.D., GISP Geospatial Software Guy | Elasticsearch Apache Lucene PMC Member and Committer nkn...@apache.org On Thu, May 14, 2020 at 11:02 AM Namgyu Kim wrote: > +1 > > On Thu, May 14, 2020 at 10:33 PM Tommaso Teofili < > tommaso.teof...@gmail.com> wrote: > >> +1 (binding) >> >> Tommaso >> >> On Tue, 12 May 2020 at 09:37, Dawid Weiss wrote: >> >>> Dear Lucene and Solr developers! >>> >>> According to an earlier [DISCUSS] thread on the dev list [2], I am >>> calling for a vote on the proposal to make Solr a top-level Apache >>> project (TLP) and separate Lucene and Solr development into two >>> independent entities. >>> >>> To quickly recap the reasons and consequences of such a move: it seems >>> like the reasons for the initial merge of Lucene and Solr, around 10 >>> years ago, have been achieved. Both projects are in good shape and >>> exhibit signs of independence already (mailing lists, committers, >>> patch flow). There are many technical considerations that would make >>> development much easier if we move Solr out into its own TLP. >>> >>> We discussed this issue [2] and both PMC members and committers had a >>> chance to review all the pros and cons and express their views. The >>> discussion showed that there are clearly different opinions on the >>> matter - some people are in favor, some are neutral, others are >>> against or not seeing the point of additional labor. Realistically, I >>> don't think reaching 100% level consensus is going to be possible -- >>> we are a diverse bunch with different opinions and personalities. I >>> firmly believe this is the right direction hence the decision to put >>> it under the voting process. Should something take a wrong turn in the >>> future (as some folks worry it may), all blame is on me. >>> >>> Therefore, the proposal is to separate Solr from under Lucene TLP, and >>> make it a TLP on its own. The initial structure of the new PMC, >>> committer base, git repositories and other managerial aspects can be >>> worked out during the process if the decision passes. >>> >>> Please indicate one of the following (see [1] for guidelines): >>> >>> [ ] +1 - yes, I vote for the proposal >>> [ ] -1 - no, I vote against the proposal >>> >>> Please note that anyone in the Lucene+Solr community is invited to >>> express their opinion, though only Lucene+Solr committers cast binding >>> votes (indicate non-binding votes in your reply, please). >>> >>> The vote will be active for a week to give everyone a chance to read >>> and cast a vote. >>> >>> Dawid >>> >>> [1] https://www.apache.org/foundation/voting.html >>> [2] >>> https://lists.apache.org/thread.html/rfae2440264f6f874e91545b2030c98e7b7e3854ddf090f7747d338df%40%3Cdev.lucene.apache.org%3E >>> >>> - >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: dev-h...@lucene.apache.org >>> >>>
Re: [VOTE] Solr to become a top-level Apache project (TLP)
+1 On Thu, May 14, 2020 at 10:33 PM Tommaso Teofili wrote: > +1 (binding) > > Tommaso > > On Tue, 12 May 2020 at 09:37, Dawid Weiss wrote: > >> Dear Lucene and Solr developers! >> >> According to an earlier [DISCUSS] thread on the dev list [2], I am >> calling for a vote on the proposal to make Solr a top-level Apache >> project (TLP) and separate Lucene and Solr development into two >> independent entities. >> >> To quickly recap the reasons and consequences of such a move: it seems >> like the reasons for the initial merge of Lucene and Solr, around 10 >> years ago, have been achieved. Both projects are in good shape and >> exhibit signs of independence already (mailing lists, committers, >> patch flow). There are many technical considerations that would make >> development much easier if we move Solr out into its own TLP. >> >> We discussed this issue [2] and both PMC members and committers had a >> chance to review all the pros and cons and express their views. The >> discussion showed that there are clearly different opinions on the >> matter - some people are in favor, some are neutral, others are >> against or not seeing the point of additional labor. Realistically, I >> don't think reaching 100% level consensus is going to be possible -- >> we are a diverse bunch with different opinions and personalities. I >> firmly believe this is the right direction hence the decision to put >> it under the voting process. Should something take a wrong turn in the >> future (as some folks worry it may), all blame is on me. >> >> Therefore, the proposal is to separate Solr from under Lucene TLP, and >> make it a TLP on its own. The initial structure of the new PMC, >> committer base, git repositories and other managerial aspects can be >> worked out during the process if the decision passes. >> >> Please indicate one of the following (see [1] for guidelines): >> >> [ ] +1 - yes, I vote for the proposal >> [ ] -1 - no, I vote against the proposal >> >> Please note that anyone in the Lucene+Solr community is invited to >> express their opinion, though only Lucene+Solr committers cast binding >> votes (indicate non-binding votes in your reply, please). >> >> The vote will be active for a week to give everyone a chance to read >> and cast a vote. >> >> Dawid >> >> [1] https://www.apache.org/foundation/voting.html >> [2] >> https://lists.apache.org/thread.html/rfae2440264f6f874e91545b2030c98e7b7e3854ddf090f7747d338df%40%3Cdev.lucene.apache.org%3E >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> >>
Re: GitHub JIRA PR linking broken...
Yeah; I'm glad it seems it's fixed. ~ David On Thu, May 14, 2020 at 8:32 AM Erick Erickson wrote: > Oh good, thanks for posting ‘cause I thought I’d screwed one up. Looks > like it’s fixed now though? > > > On May 14, 2020, at 1:33 AM, David Smiley wrote: > > > > FYI https://issues.apache.org/jira/browse/INFRA-20253 > > > > ~ David Smiley > > Apache Lucene/Solr Search Developer > > http://www.linkedin.com/in/davidwsmiley > > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > >
Re: [VOTE] Solr to become a top-level Apache project (TLP)
+1 (binding) Tommaso On Tue, 12 May 2020 at 09:37, Dawid Weiss wrote: > Dear Lucene and Solr developers! > > According to an earlier [DISCUSS] thread on the dev list [2], I am > calling for a vote on the proposal to make Solr a top-level Apache > project (TLP) and separate Lucene and Solr development into two > independent entities. > > To quickly recap the reasons and consequences of such a move: it seems > like the reasons for the initial merge of Lucene and Solr, around 10 > years ago, have been achieved. Both projects are in good shape and > exhibit signs of independence already (mailing lists, committers, > patch flow). There are many technical considerations that would make > development much easier if we move Solr out into its own TLP. > > We discussed this issue [2] and both PMC members and committers had a > chance to review all the pros and cons and express their views. The > discussion showed that there are clearly different opinions on the > matter - some people are in favor, some are neutral, others are > against or not seeing the point of additional labor. Realistically, I > don't think reaching 100% level consensus is going to be possible -- > we are a diverse bunch with different opinions and personalities. I > firmly believe this is the right direction hence the decision to put > it under the voting process. Should something take a wrong turn in the > future (as some folks worry it may), all blame is on me. > > Therefore, the proposal is to separate Solr from under Lucene TLP, and > make it a TLP on its own. The initial structure of the new PMC, > committer base, git repositories and other managerial aspects can be > worked out during the process if the decision passes. > > Please indicate one of the following (see [1] for guidelines): > > [ ] +1 - yes, I vote for the proposal > [ ] -1 - no, I vote against the proposal > > Please note that anyone in the Lucene+Solr community is invited to > express their opinion, though only Lucene+Solr committers cast binding > votes (indicate non-binding votes in your reply, please). > > The vote will be active for a week to give everyone a chance to read > and cast a vote. > > Dawid > > [1] https://www.apache.org/foundation/voting.html > [2] > https://lists.apache.org/thread.html/rfae2440264f6f874e91545b2030c98e7b7e3854ddf090f7747d338df%40%3Cdev.lucene.apache.org%3E > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > >
Re: [DISCUSS] 8.5.2 Release?
I've committed the file name change directly to prod . It's showing the correct version now thanks, On Thu, May 14, 2020 at 7:28 PM Jan Høydahl wrote: > > Perhaps because you named the news file "2020-04-dd-7-7-3-available.md» ? Not > sure. The staging site is down at the moment, so hard to cross-check… > > Jan > > > 14. mai 2020 kl. 04:18 skrev Noble Paul : > > > > I just finished all the steps given in > > https://cwiki.apache.org/confluence/display/LUCENE/ReleaseTodo > > > > Please let me know if anything is missing. > > I still don't see the solr release details in https://lucene.apache.org/ > > > > How do i do it? > > > > > > On Thu, May 14, 2020 at 10:47 AM Noble Paul wrote: > >> > >> I'll fix them today > >> > >> On Thu, May 14, 2020, 9:19 AM Mike Drob wrote: > >>> > >>> Thanks. I’ve found one small change for the release script so far, I plan > >>> to batch all my notes together and commit them at the end of the process. > >>> > >>> Right now I think I’m waiting on a few final steps from 7.7.3 to complete > >>> and then we’ll be ready to roll with 8.5.2 > >>> > >>> On Wed, May 13, 2020 at 4:59 PM Jan Høydahl wrote: > > Mike, I merged latest changes to releaseWizard to branch_8_5 as well. So > you’ll be first to test the new instructions for updating the web site. > So please be prepared for discovering quirks in that part of the > releaseWizard. > > Jan > > > 7. mai 2020 kl. 19:13 skrev Mike Drob : > > > > Devs, > > > > I know that we had 8.5.1 only a few weeks ago, but with the fix for > > LUCENE-9350 I think we should consider another bug-fix. I know that > > without it I will be explicitly recommending several users to stay off > > of 8.5.x on their upgrade plans. There are some pretty scary looking > > charts on SOLR-14428 that describe the impact of the bug in more detail. > > > > I'd be happy to volunteer as RM for this, would probably be looking at > > trying to get it a vote started sometime next week. > > > > Thanks, > > Mike > > > > > > https://issues.apache.org/jira/browse/SOLR-14428 > > https://issues.apache.org/jira/browse/LUCENE-9350 > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > For additional commands, e-mail: dev-h...@lucene.apache.org > > > > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > > > > > > > -- > > - > > Noble Paul > > > > - > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > > For additional commands, e-mail: dev-h...@lucene.apache.org > > > > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > -- - Noble Paul - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: GitHub JIRA PR linking broken...
Oh good, thanks for posting ‘cause I thought I’d screwed one up. Looks like it’s fixed now though? > On May 14, 2020, at 1:33 AM, David Smiley wrote: > > FYI https://issues.apache.org/jira/browse/INFRA-20253 > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.linkedin.com/in/davidwsmiley - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: pre-commit failing
If it's something that happens with gradle then please report the stack trace to when this is happening, otherwise it's hard to tell what's going on and where. Dawid On Wed, May 13, 2020 at 8:17 PM Joel Bernstein wrote: > > Is anybody seeing the following error on pre-commit: > > JAR resource does not exist: core/lib/commons-beanutils-1.9.3.jar > > > Thanks, > > Joel > > - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Gradle build question
> WDYT about making 3g the default (I volunteer)? I honestly don't know what consumes so much memory in the daemon - whether it's the caches or something else. It doesn't seem the build should need SO much memory... Feel free to bump but it'd be even better to figure out why it needs so much and perhaps clean it up! :) D. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [DISCUSS] 8.5.2 Release?
Perhaps because you named the news file "2020-04-dd-7-7-3-available.md» ? Not sure. The staging site is down at the moment, so hard to cross-check… Jan > 14. mai 2020 kl. 04:18 skrev Noble Paul : > > I just finished all the steps given in > https://cwiki.apache.org/confluence/display/LUCENE/ReleaseTodo > > Please let me know if anything is missing. > I still don't see the solr release details in https://lucene.apache.org/ > > How do i do it? > > > On Thu, May 14, 2020 at 10:47 AM Noble Paul wrote: >> >> I'll fix them today >> >> On Thu, May 14, 2020, 9:19 AM Mike Drob wrote: >>> >>> Thanks. I’ve found one small change for the release script so far, I plan >>> to batch all my notes together and commit them at the end of the process. >>> >>> Right now I think I’m waiting on a few final steps from 7.7.3 to complete >>> and then we’ll be ready to roll with 8.5.2 >>> >>> On Wed, May 13, 2020 at 4:59 PM Jan Høydahl wrote: Mike, I merged latest changes to releaseWizard to branch_8_5 as well. So you’ll be first to test the new instructions for updating the web site. So please be prepared for discovering quirks in that part of the releaseWizard. Jan > 7. mai 2020 kl. 19:13 skrev Mike Drob : > > Devs, > > I know that we had 8.5.1 only a few weeks ago, but with the fix for > LUCENE-9350 I think we should consider another bug-fix. I know that > without it I will be explicitly recommending several users to stay off of > 8.5.x on their upgrade plans. There are some pretty scary looking charts > on SOLR-14428 that describe the impact of the bug in more detail. > > I'd be happy to volunteer as RM for this, would probably be looking at > trying to get it a vote started sometime next week. > > Thanks, > Mike > > > https://issues.apache.org/jira/browse/SOLR-14428 > https://issues.apache.org/jira/browse/LUCENE-9350 > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org > > > -- > - > Noble Paul > > - > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
