Re: additional term meta data

[Simon Willnauer]

John, can you explain what the usecase for such a new API is? I don't
see a user of the API in your code. Is there a query you can optimize
with this or what is the reasoning behind this change? I personally
think it's quite invasive to add this information and there must be a
good reason to add this to the TermsEnum? I also don't think we should
have an option on the field for this if we add it but if we don't do
that it's quite a heavy change so I am on the fence if we should even
consider this?
I wonder if you can use the TermsEnum#getAttributeSource() API instead
and add this as a dedicated attribute which is present if the info is
stored. That way you can build your own PostingsFormat that does store
this information?

simon

On Wed, Jan 6, 2021 at 8:06 PM John Wang  wrote:
>
> Thank you, Martin!
>
> You can apply the patch to the 8.7 build by just ignoring the changes to 
> Lucene90xxx. Appreciate the help and guidance!
>
> -John
>
>
> On Wed, Jan 6, 2021 at 10:36 AM Martin Gainty  wrote:
>>
>> appears you are targeting 9.0 for your code
>> lucene/core/src/java/org/apache/lucene/codecs/lucene90/Lucene90FieldInfosFormat.java
>> (Lucene90FIeldInfosFormat.java is not contained in either 8.4 or 8.7 distros)
>>
>> 
>> someone had the bright idea to nuke ant 8.x build.xml without consulting 
>> anyone
>> not a fan of ant but the execution model of gradle is woefully inflexible in 
>> comparison to maven
>> 
>>
>> i will try with 90 distro to get the 
>> codecs/lucene90/Lucene90FieldInfosFormat and recompile and hopefully your 
>> TestLucene84PostingsFormat will run w/o fail or error
>>
>> Thx
>> martin-
>>
>> 
>> From: John Wang 
>> Sent: Wednesday, January 6, 2021 10:15 AM
>> To: dev@lucene.apache.org 
>> Subject: Re: additional term meta data
>>
>> Hey Martin:
>>
>> There is a test case in the PR we created on our own fork: 
>> https://github.com/dashbase/lucene-solr/pull/1, which also contains some 
>> example code on how to access in the PR description.
>>
>> Here is the link to the beginning of the tests: 
>> https://github.com/dashbase/lucene-solr/blob/posting-last-docid/lucene/core/src/test/org/apache/lucene/codecs/lucene84/TestLucene84PostingsFormat.java#L142
>>
>> I am not sure which version this should be applied to, currently, it was 
>> based on master as of a few days ago. We intend to patch 8.7 for our own 
>> environment.
>>
>> Any advice or feedback is much appreciated.
>>
>> Thank you!
>>
>> -John
>>
>> On Wed, Jan 6, 2021 at 3:28 AM Martin Gainty  wrote:
>>
>> how to access first and last?
>> which version will you be merging
>>
>> 
>> From: John Wang 
>> Sent: Tuesday, January 5, 2021 8:19 PM
>> To: dev@lucene.apache.org 
>> Subject: additional term meta data
>>
>> Hi folks:
>>
>> We like to propose a feature to add additional per-term metadata to the term 
>> diction.
>>
>> Currently, the TermsEnum API returns docFreq as its only meta-data. We 
>> needed a way to quickly get the first and last doc id in the postings 
>> without having to scan through the entire postings list.
>>
>> We have created a PR on our own fork and we would like to contribute this 
>> back to the community. Please let us know if this is something that's useful 
>> and/or fits Lucene's roadmap, we would be happy to submit a patch.
>>
>> https://github.com/dashbase/lucene-solr/pull/1
>>
>> Thank you
>>
>> -John

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: SOLR-15056 change circuit breaker metric

[Walter Underwood]

OK, I’ll do it against master. We’d love to see it in an 8.x release, though.
This would protect against the major Solr outages we’ve had in the past
few years.

A question for someone who knows the Java metrics stuff better than I do:

Is it OK to use com.sun.management.OperatingSystemMXBean instead
of the java.lang.management version? getSystemCpuLoad() is only in the
former. The current code uses java.lang.management.OperatingSystemMXBean.

wunder
Walter Underwood
wun...@wunderwood.org
http://observer.wunderwood.org/  (my blog)

> On Jan 8, 2021, at 10:16 AM, David Smiley  wrote:
> 
> Glad to see you contributing Walter!
> 
> Unless you know it only applies to 8x, you should branch against master.
> 
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley 
> 
> 
> On Thu, Jan 7, 2021 at 4:26 PM Walter Underwood  > wrote:
> Starting work on this change. Should that be against branch_8x?
> 
> https://issues.apache.org/jira/browse/SOLR-15056 
> 
> 
> wunder
> Walter Underwood
> wun...@wunderwood.org 
> http://observer.wunderwood.org/   (my blog)
> 

Re: [DISCUSS] SIP-12: Incremental Backup and Restore

[David Smiley]

Jason, I know you've seen SOLR-15051 (Shared storage -- BlobDirectory), but
perhaps not a part of the linked document:
https://docs.google.com/document/d/1kjQPK80sLiZJyRjek_Edhokfc5q9S3ISvFRM2_YeL8M/edit#
 "ZK Data: Listings" that discusses an aspect of the design that is very
similar to a part of SIP-12.  We both want to store checksums and file
lengths.  Maybe the approaches should/could be aligned on this?  I'll look
at SIP-12 more.  A brief browse shows that it seems kind of Git like in
that it has a store of all files with a UUID name.  Your proposal did not
discuss how these files are GC'ed.  In my BlobDirectory proposal, files are
stored where they are created by their natural name.  Files are shared via
ref-counting in the per-directory metadata files (I call "listings") so
that files can be deleted immediately if not shared.  To handle race
conditions of a metadata file, BlobDirectory uses ZK instead of requiring
anything of the backing blob store.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Fri, Jan 8, 2021 at 4:51 PM Mike Drob  wrote:

> Do the shard metadata files list all of the segments that make up the
> backup, or only the segments that were uploaded in this incremental update?
>
> On Fri, Jan 8, 2021 at 3:19 PM Jason Gerlowski 
> wrote:
>
>> Sure thing.  I put together a writeup on the file layout and formats
>> here:
>> https://cwiki.apache.org/confluence/display/SOLR/Incremental+Backup+File+Format
>> The details get a little verbose, so I made it a subpage that the
>> SIP-proper calls out to.
>>
>> Let me know what you think when you get a chance to read - hopefully
>> that's sufficient to fill the gap.
>>
>> Jason
>>
>> On Thu, Jan 7, 2021 at 8:34 PM Tomás Fernández Löbbe
>>  wrote:
>> >
>> > Thanks Jason! This is great, and a very much needed feature.
>> >
>> > > This helps to avoid confusion that would
>> > > otherwise arise between identically named files when e.g. a shard
>> > > leader changes between two incremental backups.  (I'll try to expand
>> > > on this in the SIP, as it's a bit hard to give the full context here.)
>> >
>> > Thanks, I was wondering the same thing. Maybe it would be good to put
>> an example of how the file structure of a backup looks like in the backup?
>> and how the manifest file looks like. As you said, a file with the same
>> name may refer to different segments created by different cores or the same
>> one (even if the leader changed, it may be a file from a previous
>> replication).
>> >
>> > On Thu, Jan 7, 2021 at 1:20 PM Jason Gerlowski 
>> wrote:
>> >>
>> >> Thanks for the feedback Mike.  I've gotta give any credit to Shalin
>> >> though, he wrote most of it before the holiday.  He and Dat wrote much
>> >> of the code involved as well.  I haven't done more than steward things
>> >> along so far.  As you suggested, I've updated the SIP to mention the
>> >> related SOLR-13608 (see the bottom of the "Motivation" section).
>> >>
>> >> As for your questions, I've tried to answer them below.
>> >>
>> >> 1. Good catch - it doesn't. The SIP should read that each backup
>> >> creates its own manifest files as needed for directories it creates
>> >> under the base "location".  This way, additional backups can be added
>> >> to the same location without needing to modify existing metadata
>> >> files.  I've updated the SIP to reflect this.
>> >>
>> >> 2. The proposed metadata file is a lot like segments_n (in spirit) in
>> >> that it has pointers to each index file that comprise an
>> >> index/replica.  But it differs in that it stores additional
>> >> information about each file (checksum, size) separate from the file
>> >> itself.  It also allows a layer of naming indirection between what
>> >> files are named in the storage repository and what name they should be
>> >> given upon restoration.  This helps to avoid confusion that would
>> >> otherwise arise between identically named files when e.g. a shard
>> >> leader changes between two incremental backups.  (I'll try to expand
>> >> on this in the SIP, as it's a bit hard to give the full context here.)
>> >>
>> >> 3. My intention was that the 'maxNumBackups' parameter would only
>> >> refer to the incremental backups in a given location.  This was mostly
>> >> informed by the fact that traditional backups today are required to be
>> >> 1-per-location.  (i.e. a backup in 8.6.3 will error out if the
>> >> specified directory has files in it.).  We could fix that aspect of
>> >> traditional backups and find semantics for 'maxNumBackups' that might
>> >> include traditional ones, but IMO it'd add complexity and work for a
>> >> format that the SIP is trying to replace more broadly anyways.
>> >>
>> >> 4. I definitely intended to update LocalFileSystemRepository.  I have
>> >> code to update HdfsBackupRepository as well, but wasn't quite sure
>> >> where that stood since it's currently deprecated.  I haven't seen
>> >> plans to make it a plugin, but might've 

Re: [DISCUSS] SIP-12: Incremental Backup and Restore

[Mike Drob]

Do the shard metadata files list all of the segments that make up the
backup, or only the segments that were uploaded in this incremental update?

On Fri, Jan 8, 2021 at 3:19 PM Jason Gerlowski 
wrote:

> Sure thing.  I put together a writeup on the file layout and formats
> here:
> https://cwiki.apache.org/confluence/display/SOLR/Incremental+Backup+File+Format
> The details get a little verbose, so I made it a subpage that the
> SIP-proper calls out to.
>
> Let me know what you think when you get a chance to read - hopefully
> that's sufficient to fill the gap.
>
> Jason
>
> On Thu, Jan 7, 2021 at 8:34 PM Tomás Fernández Löbbe
>  wrote:
> >
> > Thanks Jason! This is great, and a very much needed feature.
> >
> > > This helps to avoid confusion that would
> > > otherwise arise between identically named files when e.g. a shard
> > > leader changes between two incremental backups.  (I'll try to expand
> > > on this in the SIP, as it's a bit hard to give the full context here.)
> >
> > Thanks, I was wondering the same thing. Maybe it would be good to put an
> example of how the file structure of a backup looks like in the backup? and
> how the manifest file looks like. As you said, a file with the same name
> may refer to different segments created by different cores or the same one
> (even if the leader changed, it may be a file from a previous replication).
> >
> > On Thu, Jan 7, 2021 at 1:20 PM Jason Gerlowski 
> wrote:
> >>
> >> Thanks for the feedback Mike.  I've gotta give any credit to Shalin
> >> though, he wrote most of it before the holiday.  He and Dat wrote much
> >> of the code involved as well.  I haven't done more than steward things
> >> along so far.  As you suggested, I've updated the SIP to mention the
> >> related SOLR-13608 (see the bottom of the "Motivation" section).
> >>
> >> As for your questions, I've tried to answer them below.
> >>
> >> 1. Good catch - it doesn't. The SIP should read that each backup
> >> creates its own manifest files as needed for directories it creates
> >> under the base "location".  This way, additional backups can be added
> >> to the same location without needing to modify existing metadata
> >> files.  I've updated the SIP to reflect this.
> >>
> >> 2. The proposed metadata file is a lot like segments_n (in spirit) in
> >> that it has pointers to each index file that comprise an
> >> index/replica.  But it differs in that it stores additional
> >> information about each file (checksum, size) separate from the file
> >> itself.  It also allows a layer of naming indirection between what
> >> files are named in the storage repository and what name they should be
> >> given upon restoration.  This helps to avoid confusion that would
> >> otherwise arise between identically named files when e.g. a shard
> >> leader changes between two incremental backups.  (I'll try to expand
> >> on this in the SIP, as it's a bit hard to give the full context here.)
> >>
> >> 3. My intention was that the 'maxNumBackups' parameter would only
> >> refer to the incremental backups in a given location.  This was mostly
> >> informed by the fact that traditional backups today are required to be
> >> 1-per-location.  (i.e. a backup in 8.6.3 will error out if the
> >> specified directory has files in it.).  We could fix that aspect of
> >> traditional backups and find semantics for 'maxNumBackups' that might
> >> include traditional ones, but IMO it'd add complexity and work for a
> >> format that the SIP is trying to replace more broadly anyways.
> >>
> >> 4. I definitely intended to update LocalFileSystemRepository.  I have
> >> code to update HdfsBackupRepository as well, but wasn't quite sure
> >> where that stood since it's currently deprecated.  I haven't seen
> >> plans to make it a plugin, but might've just missed those discussions
> >> in other mail.  Anyway, I plan to update it but that assumes it's
> >> sticking around in one form or another.
> >>
> >> 5. Good idea - I didn't realize that was an option.  But it would be
> >> really nice if possible.  I don't have an estimate on resources.  I
> >> expect the need would be relatively small - you could restrict the
> >> tests to running on the nightly runs on ASF's Jenkins unless devs
> >> provide their own (e.g.) s3 creds.  But that's just a guess obviously,
> >> and not even in concrete terms.
> >>
> >> Thanks again for taking the time to wade through the SIP - really
> >> appreciate the feedback.  Hope the answers help!
> >>
> >> Best,
> >>
> >> Jason
> >>
> >> On Tue, Jan 5, 2021 at 11:52 AM Mike Drob  wrote:
> >> >
> >> > This is a very thorough SIP, thank you for spending the time on it,
> Jason!
> >> >
> >> > I have a few minor questions about points that are unclear to me.
> >> >
> >> > 1) If we assume that we cannot overwrite files, how does the manifest
> file stay current for incremental backup operations to the same directory?
> >> > 2) How is the manifest file functionally different from the
> segments_n and 

Re: Failing gradle precommits

[Dawid Weiss]

I don't remember, to be honest. I think javadocs, html linters - they
all contribute. We could remove some of them (via -x) on github if
they're too heavy.

On Fri, Jan 8, 2021 at 9:08 PM Michael Sokolov  wrote:
>
> Whoops! Well I guess we'll see. I wonder what other checks we are running now 
> that's were not part of precommit, aside from spotless?
>
> On Fri, Jan 8, 2021, 2:28 PM Dawid Weiss  wrote:
>>
>> The switch to full check (minus tests) makes the "precommit" really
>> heavy... it's around 14 minutes on my not-so-fancy machine. I'm
>> curious if it's going to pass on github or timeout somehow.
>>
>> On Fri, Jan 8, 2021 at 8:26 PM Houston Putman  
>> wrote:
>> >
>> > Yeah, that should work Dawid. I'll create a PR that tests it out.
>> >
>> > - Houston
>> >
>> > On Fri, Jan 8, 2021 at 2:02 PM Dawid Weiss  wrote:
>> >>
>> >> Can those jobs just run a sequence of two commands -
>> >>
>> >> ./gradlew localSettings
>> >> ./gradlew check -x test
>> >>
>> >> This would solve the problem as proper JVM settings (tuned for the
>> >> machine/ image it's running on) would be written and used on
>> >> subsequent run.
>> >>
>> >> Dawid
>> >>
>> >> On Fri, Jan 8, 2021 at 7:44 PM Uwe Schindler  wrote:
>> >> >
>> >> > The problembcomes from the fact that Gradle sets a Xmx for itself in 
>> >> > the settings file. Jenkins installs a settings file before.
>> >> >
>> >> > As every GitHub run is a clean checkout in new working dir, the 
>> >> > settings never persist.
>> >> >
>> >> > IMHO, we should change the command line and pass JVM options to set 
>> >> > heap size as it is written to the settings file.
>> >> >
>> >> > Uwe
>> >> >
>> >> > Am January 8, 2021 6:13:01 PM UTC schrieb David Smiley 
>> >> > :
>> >> >>
>> >> >> Perhaps the OOMs are because .github/workflows/gradle-precommit.yml 
>> >> >> yesterday switched from doing "gradlew check -x test" to "gradlew 
>> >> >> precomit" ?  CC Michael Sokolov
>> >> >>
>> >> >> ~ David Smiley
>> >> >> Apache Lucene/Solr Search Developer
>> >> >> http://www.linkedin.com/in/davidwsmiley
>> >> >>
>> >> >>
>> >> >> On Fri, Jan 8, 2021 at 1:06 PM Timothy Potter  
>> >> >> wrote:
>> >> >>>
>> >> >>> Same for my PR too ... OOMs about 14 minutes in ...
>> >> >>>
>> >> >>> On Fri, Jan 8, 2021 at 9:45 AM Houston Putman 
>> >> >>>  wrote:
>> >> 
>> >>  Weirdly enough, Github PR precommit actions have started to OOM. Not 
>> >>  sure if it's a github thing or something that changed on our end...
>> >> 
>> >>  On Fri, Jan 8, 2021 at 11:37 AM Joel Bernstein  
>> >>  wrote:
>> >> >
>> >> > It turned out to be this while I merged branches:
>> >> >
>> >> > warning: inexact rename detection was skipped due to too many files.
>> >> >
>> >> > warning: you may want to set your merge.renamelimit variable to at 
>> >> > least 1639 and retry the command.
>> >> >
>> >> >
>> >> > Joel Bernstein
>> >> > http://joelsolr.blogspot.com/
>> >> >
>> >> >
>> >> > On Fri, Jan 8, 2021 at 11:16 AM Joel Bernstein  
>> >> > wrote:
>> >> >>
>> >> >> Thanks Eric, I'll do a fresh clone, something must be out of wack 
>> >> >> with my local repo.
>> >> >>
>> >> >>
>> >> >> Joel Bernstein
>> >> >> http://joelsolr.blogspot.com/
>> >> >>
>> >> >>
>> >> >> On Fri, Jan 8, 2021 at 10:55 AM Eric Pugh 
>> >> >>  wrote:
>> >> >>>
>> >> >>> It ran for me just fine.   I *think* you may not be up to date, 
>> >> >>> as dataimporthandler/ is no longer in master!
>> >> >>>
>> >> >>>
>> >> >>> On Jan 8, 2021, at 10:08 AM, Joel Bernstein  
>> >> >>> wrote:
>> >> >>>
>> >> >>> I'm getting failing gradle precommits in master:
>> >> >>>
>> >> >>> > Task :solr:contrib:validateSourcePatterns FAILED
>> >> >>> tabs instead spaces: 
>> >> >>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder.xml
>> >> >>> tabs instead spaces: 
>> >> >>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.xml
>> >> >>> tabs instead spaces: 
>> >> >>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestErrorHandling.xml
>> >> >>> tabs instead spaces: 
>> >> >>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestScriptTransformer.xml
>> >> >>> tabs instead spaces: 
>> >> >>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSqlEntityProcessor.xml
>> >> >>> tabs instead spaces: 
>> >> >>> 

Re: [DISCUSS] SIP-12: Incremental Backup and Restore

[Jason Gerlowski]

Sure thing.  I put together a writeup on the file layout and formats
here: 
https://cwiki.apache.org/confluence/display/SOLR/Incremental+Backup+File+Format
The details get a little verbose, so I made it a subpage that the
SIP-proper calls out to.

Let me know what you think when you get a chance to read - hopefully
that's sufficient to fill the gap.

Jason

On Thu, Jan 7, 2021 at 8:34 PM Tomás Fernández Löbbe
 wrote:
>
> Thanks Jason! This is great, and a very much needed feature.
>
> > This helps to avoid confusion that would
> > otherwise arise between identically named files when e.g. a shard
> > leader changes between two incremental backups.  (I'll try to expand
> > on this in the SIP, as it's a bit hard to give the full context here.)
>
> Thanks, I was wondering the same thing. Maybe it would be good to put an 
> example of how the file structure of a backup looks like in the backup? and 
> how the manifest file looks like. As you said, a file with the same name may 
> refer to different segments created by different cores or the same one (even 
> if the leader changed, it may be a file from a previous replication).
>
> On Thu, Jan 7, 2021 at 1:20 PM Jason Gerlowski  wrote:
>>
>> Thanks for the feedback Mike.  I've gotta give any credit to Shalin
>> though, he wrote most of it before the holiday.  He and Dat wrote much
>> of the code involved as well.  I haven't done more than steward things
>> along so far.  As you suggested, I've updated the SIP to mention the
>> related SOLR-13608 (see the bottom of the "Motivation" section).
>>
>> As for your questions, I've tried to answer them below.
>>
>> 1. Good catch - it doesn't. The SIP should read that each backup
>> creates its own manifest files as needed for directories it creates
>> under the base "location".  This way, additional backups can be added
>> to the same location without needing to modify existing metadata
>> files.  I've updated the SIP to reflect this.
>>
>> 2. The proposed metadata file is a lot like segments_n (in spirit) in
>> that it has pointers to each index file that comprise an
>> index/replica.  But it differs in that it stores additional
>> information about each file (checksum, size) separate from the file
>> itself.  It also allows a layer of naming indirection between what
>> files are named in the storage repository and what name they should be
>> given upon restoration.  This helps to avoid confusion that would
>> otherwise arise between identically named files when e.g. a shard
>> leader changes between two incremental backups.  (I'll try to expand
>> on this in the SIP, as it's a bit hard to give the full context here.)
>>
>> 3. My intention was that the 'maxNumBackups' parameter would only
>> refer to the incremental backups in a given location.  This was mostly
>> informed by the fact that traditional backups today are required to be
>> 1-per-location.  (i.e. a backup in 8.6.3 will error out if the
>> specified directory has files in it.).  We could fix that aspect of
>> traditional backups and find semantics for 'maxNumBackups' that might
>> include traditional ones, but IMO it'd add complexity and work for a
>> format that the SIP is trying to replace more broadly anyways.
>>
>> 4. I definitely intended to update LocalFileSystemRepository.  I have
>> code to update HdfsBackupRepository as well, but wasn't quite sure
>> where that stood since it's currently deprecated.  I haven't seen
>> plans to make it a plugin, but might've just missed those discussions
>> in other mail.  Anyway, I plan to update it but that assumes it's
>> sticking around in one form or another.
>>
>> 5. Good idea - I didn't realize that was an option.  But it would be
>> really nice if possible.  I don't have an estimate on resources.  I
>> expect the need would be relatively small - you could restrict the
>> tests to running on the nightly runs on ASF's Jenkins unless devs
>> provide their own (e.g.) s3 creds.  But that's just a guess obviously,
>> and not even in concrete terms.
>>
>> Thanks again for taking the time to wade through the SIP - really
>> appreciate the feedback.  Hope the answers help!
>>
>> Best,
>>
>> Jason
>>
>> On Tue, Jan 5, 2021 at 11:52 AM Mike Drob  wrote:
>> >
>> > This is a very thorough SIP, thank you for spending the time on it, Jason!
>> >
>> > I have a few minor questions about points that are unclear to me.
>> >
>> > 1) If we assume that we cannot overwrite files, how does the manifest file 
>> > stay current for incremental backup operations to the same directory?
>> > 2) How is the manifest file functionally different from the segments_n and 
>> > segments.gen files?
>> > 3) Does the maxNumBackups parameter consider incremental backups or only 
>> > full backups? What happens if we have a full backup and then N incremental 
>> > ones? Do we delete the full backup and convert the oldest incremental one 
>> > into a full? I imagine this might be a metadata operation, but then the 
>> > concerns from question 1 

Re: Requesting a new GH repository for CrossDC modules

[Anshum Gupta]

The problem with a single repository is that it will/may conflict at times.
Also, I still don't see the problem with having the extra repo as long as
we aren't releasing anything.

The problem with (b) is that you can't create a PR from a random repository
to a repo it isn't a fork of. I also don't think people should own code and
build outside of the ASF umbrella until things are ready to be released,
it's completely against the Apache Way. Having the code in the ASF umbrella
comes at no cost to the project. If at any point it needs to be dropped,
it's easier and cleaner as it wouldn't touch anything else.

Does this make sense?


On Fri, Jan 8, 2021 at 1:46 AM Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> Not necessarily. Most people contribute to Apache Lucene/Solr using
> external repositories (forks) and raise pull requests against Apache owned
> repositories. There's no SGA needed on such occasions.
>
> I see two paths forward from here.
>
> a) Lets setup a single repository for all packages/plugins, say
> lucene-solr-extras or lucene-solr-contribs or lucene-solr-sandbox etc., and
> develop it there.
> b) All development for this effort happens in an external repository (
> https://github.com/apple/solr-dc or https://github.com/anshumg/solr-dc)
> and we raise a PR against Apache owned repository (which can be created if
> needed once we are all onboard).
>
> What does everyone else think?
>
> On Fri, Jan 8, 2021 at 10:23 AM Mike Drob  wrote:
>
>> An external repository probably ends up requiring a software grant? I
>> know there is a material difference between code originating externally and
>> code originating within the umbrella of the ASF in terms of IP, copyright,
>> or other legal status.
>>
>>
>> On Thu, Jan 7, 2021 at 8:11 PM Ishan Chattopadhyaya <
>> ichattopadhy...@gmail.com> wrote:
>>
>>> If all we need now is a place to commit a PoC for now (and something
>>> like sandbox repo or contribs won't suffice), why can't we have a separate
>>> repository in GitHub outside Apache and merge into an Apache repository
>>> only once the code takes reasonable shape?
>>>
>>> On Fri, 8 Jan, 2021, 2:31 am Anshum Gupta, 
>>> wrote:
>>>
 Thanks for the feedback, Mike.

 I like the idea of the sandbox, but that might be restricting when we
 want to work on more than one repos.

 I'm not sure if that would happen in the near future, but as we can
 always discard the repo and it doesn't really come at a cost, I don't see a
 problem with having a repo created for this specific reason.

 On Thu, Jan 7, 2021 at 12:45 PM Mike Drob  wrote:

> I'm not sure where I sit on this, going to start typing things and
> then hopefully I'll reach a conclusion by the end.
>
> This definitely needs to be outside of the core solr repo so that it
> can be versioned and released independently. And I disagree with Ishan
> about the consequence of abandoning the repository - if we realize that
> it's a bad direction then we can pivot, but we shouldn't let a fear of the
> unknown stop us from doing it in the first place.
>
> However, if all we need right now is a place to commit code that is
> WIP, then what we really want is a sandbox to play with, and not
> necessarily a strongly directed repo. Lucene has a sandbox in the main
> code. We could similarly start this under Solr contrib and move it out
> before an actual release of 9x happens. Or maybe we start with a
> [lucene-]solr-sandbox repository that we can throw all sorts of stuff into
> and then when components are mature enough they get to graduate into their
> own repo?
>
> Mike
>
> On Thu, Jan 7, 2021 at 2:32 PM Anshum Gupta 
> wrote:
>
>> I understand your concern, but this is the placeholder for where the
>> code would be, not what the code would look like.
>>
>> Considering we agreed to do this in a repository outside of the core,
>> I believe this is a good place to start. The idea that the release 
>> cadence
>> for the cross-dc effort should be different from that of core is an
>> argument in favor of this approach, but I'm happy to talk more about it.
>> I just thought that based on the original email, folks were on-board
>> with the idea of this being outside of core Solr artifact/release.
>>
>> On Thu, Jan 7, 2021 at 11:06 AM Ishan Chattopadhyaya <
>> ichattopadhy...@gmail.com> wrote:
>>
>>> -1 on this. Without finalizing on the shape of how the solution will
>>> look like, I don't think we should start a repository: it would be bad 
>>> if
>>> we have to abandon the repository of our approach changes (say we want 
>>> to
>>> keep it tightly integrated inside Solr).
>>>
>>> On Thu, 7 Jan, 2021, 11:45 pm Anshum Gupta, 
>>> wrote:
>>>
 Hi everyone,

 Inline with my earlier email, I'll be requesting a 

Re: [JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-16-ea+30) - Build # 29209 - Still Unstable!

[Michael Sokolov]

I ran 3000+ iterations with that seed (on JDK11) and it did not
reproduce. Then another 2000+ with JDK-16 runtime, and no fails there
either.

On Fri, Jan 8, 2021 at 10:32 AM David Smiley  wrote:
>
> Hm; this is spooky
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> http://www.linkedin.com/in/davidwsmiley
>
>
> On Fri, Jan 8, 2021 at 5:18 AM Policeman Jenkins Server  
> wrote:
>>
>> Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/29209/
>> Java: 64bit/jdk-16-ea+30 -XX:+UseCompressedOops -XX:+UseSerialGC
>>
>> 1 tests failed.
>> FAILED:  org.apache.lucene.index.TestIndexFileDeleter.testExcInDecRef
>>
>> Error Message:
>> org.apache.lucene.store.AlreadyClosedException: ReaderPool is already closed
>>
>> Stack Trace:
>> org.apache.lucene.store.AlreadyClosedException: ReaderPool is already closed
>> at 
>> __randomizedtesting.SeedInfo.seed([303C698E1694E4C0:D9A11EBC605D033D]:0)
>> at org.apache.lucene.index.ReaderPool.get(ReaderPool.java:400)
>> at 
>> org.apache.lucene.index.IndexWriter.writeReaderPool(IndexWriter.java:3733)
>> at 
>> org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:588)
>> at 
>> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:474)
>> at 
>> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:406)
>> at 
>> org.apache.lucene.index.TestIndexFileDeleter.testExcInDecRef(TestIndexFileDeleter.java:484)
>> at 
>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
>> Method)
>> at 
>> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
>> at 
>> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.base/java.lang.reflect.Method.invoke(Method.java:567)
>> at 
>> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
>> at 
>> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
>> at 
>> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
>> at 
>> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
>> at 
>> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:44)
>> at 
>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>> at 
>> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
>> at 
>> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
>> at 
>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>> at 
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> at 
>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
>> at 
>> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
>> at 
>> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
>> at 
>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
>> at 
>> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
>> at 
>> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
>> at 
>> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
>> at 
>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
>> at 
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> at 
>> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
>> at 
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> at 
>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>> at 
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> at 
>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>> at 
>> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:51)
>> at 
>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
>> at 
>> 

Re: Failing gradle precommits

[Michael Sokolov]

Whoops! Well I guess we'll see. I wonder what other checks we are running
now that's were not part of precommit, aside from spotless?

On Fri, Jan 8, 2021, 2:28 PM Dawid Weiss  wrote:

> The switch to full check (minus tests) makes the "precommit" really
> heavy... it's around 14 minutes on my not-so-fancy machine. I'm
> curious if it's going to pass on github or timeout somehow.
>
> On Fri, Jan 8, 2021 at 8:26 PM Houston Putman 
> wrote:
> >
> > Yeah, that should work Dawid. I'll create a PR that tests it out.
> >
> > - Houston
> >
> > On Fri, Jan 8, 2021 at 2:02 PM Dawid Weiss 
> wrote:
> >>
> >> Can those jobs just run a sequence of two commands -
> >>
> >> ./gradlew localSettings
> >> ./gradlew check -x test
> >>
> >> This would solve the problem as proper JVM settings (tuned for the
> >> machine/ image it's running on) would be written and used on
> >> subsequent run.
> >>
> >> Dawid
> >>
> >> On Fri, Jan 8, 2021 at 7:44 PM Uwe Schindler  wrote:
> >> >
> >> > The problembcomes from the fact that Gradle sets a Xmx for itself in
> the settings file. Jenkins installs a settings file before.
> >> >
> >> > As every GitHub run is a clean checkout in new working dir, the
> settings never persist.
> >> >
> >> > IMHO, we should change the command line and pass JVM options to set
> heap size as it is written to the settings file.
> >> >
> >> > Uwe
> >> >
> >> > Am January 8, 2021 6:13:01 PM UTC schrieb David Smiley <
> dsmi...@apache.org>:
> >> >>
> >> >> Perhaps the OOMs are because .github/workflows/gradle-precommit.yml
> yesterday switched from doing "gradlew check -x test" to "gradlew precomit"
> ?  CC Michael Sokolov
> >> >>
> >> >> ~ David Smiley
> >> >> Apache Lucene/Solr Search Developer
> >> >> http://www.linkedin.com/in/davidwsmiley
> >> >>
> >> >>
> >> >> On Fri, Jan 8, 2021 at 1:06 PM Timothy Potter 
> wrote:
> >> >>>
> >> >>> Same for my PR too ... OOMs about 14 minutes in ...
> >> >>>
> >> >>> On Fri, Jan 8, 2021 at 9:45 AM Houston Putman <
> houstonput...@gmail.com> wrote:
> >> 
> >>  Weirdly enough, Github PR precommit actions have started to OOM.
> Not sure if it's a github thing or something that changed on our end...
> >> 
> >>  On Fri, Jan 8, 2021 at 11:37 AM Joel Bernstein 
> wrote:
> >> >
> >> > It turned out to be this while I merged branches:
> >> >
> >> > warning: inexact rename detection was skipped due to too many
> files.
> >> >
> >> > warning: you may want to set your merge.renamelimit variable to
> at least 1639 and retry the command.
> >> >
> >> >
> >> > Joel Bernstein
> >> > http://joelsolr.blogspot.com/
> >> >
> >> >
> >> > On Fri, Jan 8, 2021 at 11:16 AM Joel Bernstein <
> joels...@gmail.com> wrote:
> >> >>
> >> >> Thanks Eric, I'll do a fresh clone, something must be out of
> wack with my local repo.
> >> >>
> >> >>
> >> >> Joel Bernstein
> >> >> http://joelsolr.blogspot.com/
> >> >>
> >> >>
> >> >> On Fri, Jan 8, 2021 at 10:55 AM Eric Pugh <
> ep...@opensourceconnections.com> wrote:
> >> >>>
> >> >>> It ran for me just fine.   I *think* you may not be up to date,
> as dataimporthandler/ is no longer in master!
> >> >>>
> >> >>>
> >> >>> On Jan 8, 2021, at 10:08 AM, Joel Bernstein 
> wrote:
> >> >>>
> >> >>> I'm getting failing gradle precommits in master:
> >> >>>
> >> >>> > Task :solr:contrib:validateSourcePatterns FAILED
> >> >>> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder.xml
> >> >>> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.xml
> >> >>> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestErrorHandling.xml
> >> >>> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestScriptTransformer.xml
> >> >>> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSqlEntityProcessor.xml
> >> >>> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder2.xml
> >> >>> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestZKPropertiesWriter.xml
> >> >>> tabs instead spaces:
> 

Re: Failing gradle precommits

[Dawid Weiss]

The switch to full check (minus tests) makes the "precommit" really
heavy... it's around 14 minutes on my not-so-fancy machine. I'm
curious if it's going to pass on github or timeout somehow.

On Fri, Jan 8, 2021 at 8:26 PM Houston Putman  wrote:
>
> Yeah, that should work Dawid. I'll create a PR that tests it out.
>
> - Houston
>
> On Fri, Jan 8, 2021 at 2:02 PM Dawid Weiss  wrote:
>>
>> Can those jobs just run a sequence of two commands -
>>
>> ./gradlew localSettings
>> ./gradlew check -x test
>>
>> This would solve the problem as proper JVM settings (tuned for the
>> machine/ image it's running on) would be written and used on
>> subsequent run.
>>
>> Dawid
>>
>> On Fri, Jan 8, 2021 at 7:44 PM Uwe Schindler  wrote:
>> >
>> > The problembcomes from the fact that Gradle sets a Xmx for itself in the 
>> > settings file. Jenkins installs a settings file before.
>> >
>> > As every GitHub run is a clean checkout in new working dir, the settings 
>> > never persist.
>> >
>> > IMHO, we should change the command line and pass JVM options to set heap 
>> > size as it is written to the settings file.
>> >
>> > Uwe
>> >
>> > Am January 8, 2021 6:13:01 PM UTC schrieb David Smiley 
>> > :
>> >>
>> >> Perhaps the OOMs are because .github/workflows/gradle-precommit.yml 
>> >> yesterday switched from doing "gradlew check -x test" to "gradlew 
>> >> precomit" ?  CC Michael Sokolov
>> >>
>> >> ~ David Smiley
>> >> Apache Lucene/Solr Search Developer
>> >> http://www.linkedin.com/in/davidwsmiley
>> >>
>> >>
>> >> On Fri, Jan 8, 2021 at 1:06 PM Timothy Potter  
>> >> wrote:
>> >>>
>> >>> Same for my PR too ... OOMs about 14 minutes in ...
>> >>>
>> >>> On Fri, Jan 8, 2021 at 9:45 AM Houston Putman  
>> >>> wrote:
>> 
>>  Weirdly enough, Github PR precommit actions have started to OOM. Not 
>>  sure if it's a github thing or something that changed on our end...
>> 
>>  On Fri, Jan 8, 2021 at 11:37 AM Joel Bernstein  
>>  wrote:
>> >
>> > It turned out to be this while I merged branches:
>> >
>> > warning: inexact rename detection was skipped due to too many files.
>> >
>> > warning: you may want to set your merge.renamelimit variable to at 
>> > least 1639 and retry the command.
>> >
>> >
>> > Joel Bernstein
>> > http://joelsolr.blogspot.com/
>> >
>> >
>> > On Fri, Jan 8, 2021 at 11:16 AM Joel Bernstein  
>> > wrote:
>> >>
>> >> Thanks Eric, I'll do a fresh clone, something must be out of wack 
>> >> with my local repo.
>> >>
>> >>
>> >> Joel Bernstein
>> >> http://joelsolr.blogspot.com/
>> >>
>> >>
>> >> On Fri, Jan 8, 2021 at 10:55 AM Eric Pugh 
>> >>  wrote:
>> >>>
>> >>> It ran for me just fine.   I *think* you may not be up to date, as 
>> >>> dataimporthandler/ is no longer in master!
>> >>>
>> >>>
>> >>> On Jan 8, 2021, at 10:08 AM, Joel Bernstein  
>> >>> wrote:
>> >>>
>> >>> I'm getting failing gradle precommits in master:
>> >>>
>> >>> > Task :solr:contrib:validateSourcePatterns FAILED
>> >>> tabs instead spaces: 
>> >>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder.xml
>> >>> tabs instead spaces: 
>> >>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.xml
>> >>> tabs instead spaces: 
>> >>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestErrorHandling.xml
>> >>> tabs instead spaces: 
>> >>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestScriptTransformer.xml
>> >>> tabs instead spaces: 
>> >>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSqlEntityProcessor.xml
>> >>> tabs instead spaces: 
>> >>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder2.xml
>> >>> tabs instead spaces: 
>> >>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestZKPropertiesWriter.xml
>> >>> tabs instead spaces: 
>> >>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler-extras/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestTikaEntityProcessor.xml
>> >>>
>> >>> FAILURE: Build failed with an exception.
>> >>>
>> >>> * Where:
>> >>> Script 
>> >>> '/Users/joelbernstein/committer/lucene-solr/gradle/validation/validate-source-patterns.gradle'
>> 

Re: Failing gradle precommits

[Houston Putman]

Yeah, that should work Dawid. I'll create a PR that tests it out.

- Houston

On Fri, Jan 8, 2021 at 2:02 PM Dawid Weiss  wrote:

> Can those jobs just run a sequence of two commands -
>
> ./gradlew localSettings
> ./gradlew check -x test
>
> This would solve the problem as proper JVM settings (tuned for the
> machine/ image it's running on) would be written and used on
> subsequent run.
>
> Dawid
>
> On Fri, Jan 8, 2021 at 7:44 PM Uwe Schindler  wrote:
> >
> > The problembcomes from the fact that Gradle sets a Xmx for itself in the
> settings file. Jenkins installs a settings file before.
> >
> > As every GitHub run is a clean checkout in new working dir, the settings
> never persist.
> >
> > IMHO, we should change the command line and pass JVM options to set heap
> size as it is written to the settings file.
> >
> > Uwe
> >
> > Am January 8, 2021 6:13:01 PM UTC schrieb David Smiley <
> dsmi...@apache.org>:
> >>
> >> Perhaps the OOMs are because .github/workflows/gradle-precommit.yml
> yesterday switched from doing "gradlew check -x test" to "gradlew precomit"
> ?  CC Michael Sokolov
> >>
> >> ~ David Smiley
> >> Apache Lucene/Solr Search Developer
> >> http://www.linkedin.com/in/davidwsmiley
> >>
> >>
> >> On Fri, Jan 8, 2021 at 1:06 PM Timothy Potter 
> wrote:
> >>>
> >>> Same for my PR too ... OOMs about 14 minutes in ...
> >>>
> >>> On Fri, Jan 8, 2021 at 9:45 AM Houston Putman 
> wrote:
> 
>  Weirdly enough, Github PR precommit actions have started to OOM. Not
> sure if it's a github thing or something that changed on our end...
> 
>  On Fri, Jan 8, 2021 at 11:37 AM Joel Bernstein 
> wrote:
> >
> > It turned out to be this while I merged branches:
> >
> > warning: inexact rename detection was skipped due to too many files.
> >
> > warning: you may want to set your merge.renamelimit variable to at
> least 1639 and retry the command.
> >
> >
> > Joel Bernstein
> > http://joelsolr.blogspot.com/
> >
> >
> > On Fri, Jan 8, 2021 at 11:16 AM Joel Bernstein 
> wrote:
> >>
> >> Thanks Eric, I'll do a fresh clone, something must be out of wack
> with my local repo.
> >>
> >>
> >> Joel Bernstein
> >> http://joelsolr.blogspot.com/
> >>
> >>
> >> On Fri, Jan 8, 2021 at 10:55 AM Eric Pugh <
> ep...@opensourceconnections.com> wrote:
> >>>
> >>> It ran for me just fine.   I *think* you may not be up to date, as
> dataimporthandler/ is no longer in master!
> >>>
> >>>
> >>> On Jan 8, 2021, at 10:08 AM, Joel Bernstein 
> wrote:
> >>>
> >>> I'm getting failing gradle precommits in master:
> >>>
> >>> > Task :solr:contrib:validateSourcePatterns FAILED
> >>> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder.xml
> >>> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.xml
> >>> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestErrorHandling.xml
> >>> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestScriptTransformer.xml
> >>> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSqlEntityProcessor.xml
> >>> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder2.xml
> >>> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestZKPropertiesWriter.xml
> >>> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler-extras/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestTikaEntityProcessor.xml
> >>>
> >>> FAILURE: Build failed with an exception.
> >>>
> >>> * Where:
> >>> Script
> '/Users/joelbernstein/committer/lucene-solr/gradle/validation/validate-source-patterns.gradle'
> line: 324
> >>>
> >>> * What went wrong:
> >>> Execution failed for task ':solr:contrib:validateSourcePatterns'.
> >>> > Found 8 violations in source files (tabs instead spaces).
> >>>
> >>>
> >>> Are others seeing this as well? I'm not seeing Jenkins emails
> about this.
> >>>
> >>>
> >>> Joel Bernstein
> >>> http://joelsolr.blogspot.com/
> >>>
> >>>
> >>> ___
> >>> Eric Pugh | Founder & CEO | OpenSource 

Re: Failing gradle precommits

[Dawid Weiss]

Can those jobs just run a sequence of two commands -

./gradlew localSettings
./gradlew check -x test

This would solve the problem as proper JVM settings (tuned for the
machine/ image it's running on) would be written and used on
subsequent run.

Dawid

On Fri, Jan 8, 2021 at 7:44 PM Uwe Schindler  wrote:
>
> The problembcomes from the fact that Gradle sets a Xmx for itself in the 
> settings file. Jenkins installs a settings file before.
>
> As every GitHub run is a clean checkout in new working dir, the settings 
> never persist.
>
> IMHO, we should change the command line and pass JVM options to set heap size 
> as it is written to the settings file.
>
> Uwe
>
> Am January 8, 2021 6:13:01 PM UTC schrieb David Smiley :
>>
>> Perhaps the OOMs are because .github/workflows/gradle-precommit.yml 
>> yesterday switched from doing "gradlew check -x test" to "gradlew precomit" 
>> ?  CC Michael Sokolov
>>
>> ~ David Smiley
>> Apache Lucene/Solr Search Developer
>> http://www.linkedin.com/in/davidwsmiley
>>
>>
>> On Fri, Jan 8, 2021 at 1:06 PM Timothy Potter  wrote:
>>>
>>> Same for my PR too ... OOMs about 14 minutes in ...
>>>
>>> On Fri, Jan 8, 2021 at 9:45 AM Houston Putman  
>>> wrote:

 Weirdly enough, Github PR precommit actions have started to OOM. Not sure 
 if it's a github thing or something that changed on our end...

 On Fri, Jan 8, 2021 at 11:37 AM Joel Bernstein  wrote:
>
> It turned out to be this while I merged branches:
>
> warning: inexact rename detection was skipped due to too many files.
>
> warning: you may want to set your merge.renamelimit variable to at least 
> 1639 and retry the command.
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
>
> On Fri, Jan 8, 2021 at 11:16 AM Joel Bernstein  wrote:
>>
>> Thanks Eric, I'll do a fresh clone, something must be out of wack with 
>> my local repo.
>>
>>
>> Joel Bernstein
>> http://joelsolr.blogspot.com/
>>
>>
>> On Fri, Jan 8, 2021 at 10:55 AM Eric Pugh 
>>  wrote:
>>>
>>> It ran for me just fine.   I *think* you may not be up to date, as 
>>> dataimporthandler/ is no longer in master!
>>>
>>>
>>> On Jan 8, 2021, at 10:08 AM, Joel Bernstein  wrote:
>>>
>>> I'm getting failing gradle precommits in master:
>>>
>>> > Task :solr:contrib:validateSourcePatterns FAILED
>>> tabs instead spaces: 
>>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder.xml
>>> tabs instead spaces: 
>>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.xml
>>> tabs instead spaces: 
>>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestErrorHandling.xml
>>> tabs instead spaces: 
>>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestScriptTransformer.xml
>>> tabs instead spaces: 
>>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSqlEntityProcessor.xml
>>> tabs instead spaces: 
>>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder2.xml
>>> tabs instead spaces: 
>>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestZKPropertiesWriter.xml
>>> tabs instead spaces: 
>>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler-extras/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestTikaEntityProcessor.xml
>>>
>>> FAILURE: Build failed with an exception.
>>>
>>> * Where:
>>> Script 
>>> '/Users/joelbernstein/committer/lucene-solr/gradle/validation/validate-source-patterns.gradle'
>>>  line: 324
>>>
>>> * What went wrong:
>>> Execution failed for task ':solr:contrib:validateSourcePatterns'.
>>> > Found 8 violations in source files (tabs instead spaces).
>>>
>>>
>>> Are others seeing this as well? I'm not seeing Jenkins emails about 
>>> this.
>>>
>>>
>>> Joel Bernstein
>>> http://joelsolr.blogspot.com/
>>>
>>>
>>> ___
>>> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 
>>> | http://www.opensourceconnections.com | My Free/Busy
>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
>>> This e-mail and all contents, including attachments, is considered to 
>>> be Company Confidential 

Re: Failing gradle precommits

[Uwe Schindler]

The problembcomes from the fact that Gradle sets a Xmx for itself in the 
settings file. Jenkins installs a settings file before.

As every GitHub run is a clean checkout in new working dir, the settings never 
persist.

IMHO, we should change the command line and pass JVM options to set heap size 
as it is written to the settings file.

Uwe

Am January 8, 2021 6:13:01 PM UTC schrieb David Smiley :
>Perhaps the OOMs are because .github/workflows/gradle-precommit.yml
>yesterday switched from doing "gradlew check -x test" to "gradlew
>precomit"
>?  CC Michael Sokolov
>
>~ David Smiley
>Apache Lucene/Solr Search Developer
>http://www.linkedin.com/in/davidwsmiley
>
>
>On Fri, Jan 8, 2021 at 1:06 PM Timothy Potter 
>wrote:
>
>> Same for my PR too ... OOMs about 14 minutes in ...
>>
>> On Fri, Jan 8, 2021 at 9:45 AM Houston Putman
>
>> wrote:
>>
>>> Weirdly enough, Github PR precommit actions have started to OOM. Not
>sure
>>> if it's a github thing or something that changed on our end...
>>>
>>> On Fri, Jan 8, 2021 at 11:37 AM Joel Bernstein 
>>> wrote:
>>>
 It turned out to be this while I merged branches:

 warning: inexact rename detection was skipped due to too many
>files.

 warning: you may want to set your merge.renamelimit variable to at
>least
 1639 and retry the command.

 Joel Bernstein
 http://joelsolr.blogspot.com/


 On Fri, Jan 8, 2021 at 11:16 AM Joel Bernstein 
 wrote:

> Thanks Eric, I'll do a fresh clone, something must be out of wack
>with
> my local repo.
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
>
> On Fri, Jan 8, 2021 at 10:55 AM Eric Pugh <
> ep...@opensourceconnections.com> wrote:
>
>> It ran for me just fine.   I *think* you may not be up to date,
>as
>> dataimporthandler/ is no longer in master!
>>
>>
>> On Jan 8, 2021, at 10:08 AM, Joel Bernstein 
>> wrote:
>>
>> I'm getting failing gradle precommits in master:
>>
>> *> Task :solr:contrib:validateSourcePatterns* FAILED
>> tabs instead spaces:
>>
>/Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder.xml
>> tabs instead spaces:
>>
>/Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.xml
>> tabs instead spaces:
>>
>/Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestErrorHandling.xml
>> tabs instead spaces:
>>
>/Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestScriptTransformer.xml
>> tabs instead spaces:
>>
>/Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSqlEntityProcessor.xml
>> tabs instead spaces:
>>
>/Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder2.xml
>> tabs instead spaces:
>>
>/Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestZKPropertiesWriter.xml
>> tabs instead spaces:
>>
>/Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler-extras/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestTikaEntityProcessor.xml
>>
>> FAILURE: Build failed with an exception.
>>
>> * Where:
>> Script
>>
>'/Users/joelbernstein/committer/lucene-solr/gradle/validation/validate-source-patterns.gradle'
>> line: 324
>>
>> * What went wrong:
>> Execution failed for task ':solr:contrib:validateSourcePatterns'.
>> > Found 8 violations in source files (tabs instead spaces).
>>
>>
>> Are others seeing this as well? I'm not seeing Jenkins emails
>about
>> this.
>>
>>
>> Joel Bernstein
>> http://joelsolr.blogspot.com/
>>
>>
>> ___
>> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC |
>434.466.1467
>> | http://www.opensourceconnections.com | My Free/Busy
>> 
>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
>>
>
>> This e-mail and all contents, including attachments, is
>considered to
>> be Company Confidential unless explicitly stated otherwise,
>regardless
>> of whether attachments are marked as such.
>>
>>

--
Uwe Schindler
Achterdiek 19, 28357 Bremen
https://www.thetaphi.de

Re: questions of FST

[Michael McCandless]

Hello!

A few quick answers:

In theory, an FST node can have inCount > 1 (when it is in the "suffix"
part of the FST), but it sounds like maybe you found an exciting bug if
indeed FST Builder fails to ever increment it.  E.g. try building some FSTs
here: http://examples.mikemccandless.com/fst.py the default FST there
already shows 2 nodes with inCount = 2.

Note that the pack method has been deleted in newer Lucene versions,
because it was sizable complexity, not very well tested and not so big
gains in practice.

I'm not sure about question 2)

Question 3) does sound like a bug, because as the code now stands it means
we cannot reference an address below nodeRefToAddress.size() which seems
like it might be problematic.

These might be real bugs!  I would strongly suggest ensuring pack() really
is working for your use case before trusting it!

Mike McCandless

http://blog.mikemccandless.com


On Thu, Jan 7, 2021 at 10:14 PM guohuawu227  wrote:

> I am a software developer from China. Recently I am reading the source of
> lucene of 6.6.0. I can not understand some code of FST. List them below.
>
> 1 . There is a field called 'inCounts' in the class of FST. But in my
> opinion every node of FST has only one arc pointing to it. If a node is not
> shared, it is certain it has only one arc pointing to it since it is like a
> tree. If a node is shared,it is returned directly from NodeHash so the
> number of it's coming in arc is not updated. So I just can't understand
> this attribute though it is used is 'pach' method.
>
> 2. Is about the method of 'pach' of class FST. This line 'long delta =
> newNodeAddress.get((int) arc.target) + addressError - writer.getPosition()
> - 2;' . I know it is comparing which one is smaller but I just can't
> understand why 2 is minused.
>
> 3. If a FST is packed,then there are 3 ways to save the target of an arc,
> first is the difference of the target and this arc ,second is the ordinal
> number from a hashmap if the target is among those having top N incoming
> arcs, third is absolute position of the node. But in the method
> 'readNextRealArc' , I can not understand the process of the absolute
> position while reading. Below is the code.
>
> if (packed) {
> final long pos = in.getPosition();
> final long code = in.readVLong();
> if (arc.flag(BIT_TARGET_DELTA)) {
> // record the difference
> } else if (code < nodeRefToAddress.size()) {
> // record the ordinal number from a hashmap
> } else {// Absolute position
> arc.target = code;//I can not understa this.In opinion,
> nodeRefToAddress.size() should be minused because it is summed into the
> absolute value in 'pack' method.
> }
> } else {
>
> ---
> Please pardon me if I use some wrong words ,my English is not good.Thanks
> for reading the email. Looking forward to the reply.
> - To
> unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional
> commands, e-mail: dev-h...@lucene.apache.org

Re: SOLR-15056 change circuit breaker metric

[David Smiley]

Glad to see you contributing Walter!

Unless you know it only applies to 8x, you should branch against master.

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Thu, Jan 7, 2021 at 4:26 PM Walter Underwood 
wrote:

> Starting work on this change. Should that be against branch_8x?
>
> https://issues.apache.org/jira/browse/SOLR-15056
>
> wunder
> Walter Underwood
> wun...@wunderwood.org
> http://observer.wunderwood.org/  (my blog)
>
>

Re: Failing gradle precommits

[David Smiley]

Perhaps the OOMs are because .github/workflows/gradle-precommit.yml
yesterday switched from doing "gradlew check -x test" to "gradlew precomit"
?  CC Michael Sokolov

~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Fri, Jan 8, 2021 at 1:06 PM Timothy Potter  wrote:

> Same for my PR too ... OOMs about 14 minutes in ...
>
> On Fri, Jan 8, 2021 at 9:45 AM Houston Putman 
> wrote:
>
>> Weirdly enough, Github PR precommit actions have started to OOM. Not sure
>> if it's a github thing or something that changed on our end...
>>
>> On Fri, Jan 8, 2021 at 11:37 AM Joel Bernstein 
>> wrote:
>>
>>> It turned out to be this while I merged branches:
>>>
>>> warning: inexact rename detection was skipped due to too many files.
>>>
>>> warning: you may want to set your merge.renamelimit variable to at least
>>> 1639 and retry the command.
>>>
>>> Joel Bernstein
>>> http://joelsolr.blogspot.com/
>>>
>>>
>>> On Fri, Jan 8, 2021 at 11:16 AM Joel Bernstein 
>>> wrote:
>>>
 Thanks Eric, I'll do a fresh clone, something must be out of wack with
 my local repo.


 Joel Bernstein
 http://joelsolr.blogspot.com/


 On Fri, Jan 8, 2021 at 10:55 AM Eric Pugh <
 ep...@opensourceconnections.com> wrote:

> It ran for me just fine.   I *think* you may not be up to date, as
> dataimporthandler/ is no longer in master!
>
>
> On Jan 8, 2021, at 10:08 AM, Joel Bernstein 
> wrote:
>
> I'm getting failing gradle precommits in master:
>
> *> Task :solr:contrib:validateSourcePatterns* FAILED
> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder.xml
> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.xml
> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestErrorHandling.xml
> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestScriptTransformer.xml
> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSqlEntityProcessor.xml
> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder2.xml
> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestZKPropertiesWriter.xml
> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler-extras/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestTikaEntityProcessor.xml
>
> FAILURE: Build failed with an exception.
>
> * Where:
> Script
> '/Users/joelbernstein/committer/lucene-solr/gradle/validation/validate-source-patterns.gradle'
> line: 324
>
> * What went wrong:
> Execution failed for task ':solr:contrib:validateSourcePatterns'.
> > Found 8 violations in source files (tabs instead spaces).
>
>
> Are others seeing this as well? I'm not seeing Jenkins emails about
> this.
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
>
> ___
> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467
> | http://www.opensourceconnections.com | My Free/Busy
> 
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> 
> This e-mail and all contents, including attachments, is considered to
> be Company Confidential unless explicitly stated otherwise, regardless
> of whether attachments are marked as such.
>
>

Re: Failing gradle precommits

[Timothy Potter]

Same for my PR too ... OOMs about 14 minutes in ...

On Fri, Jan 8, 2021 at 9:45 AM Houston Putman 
wrote:

> Weirdly enough, Github PR precommit actions have started to OOM. Not sure
> if it's a github thing or something that changed on our end...
>
> On Fri, Jan 8, 2021 at 11:37 AM Joel Bernstein  wrote:
>
>> It turned out to be this while I merged branches:
>>
>> warning: inexact rename detection was skipped due to too many files.
>>
>> warning: you may want to set your merge.renamelimit variable to at least
>> 1639 and retry the command.
>>
>> Joel Bernstein
>> http://joelsolr.blogspot.com/
>>
>>
>> On Fri, Jan 8, 2021 at 11:16 AM Joel Bernstein 
>> wrote:
>>
>>> Thanks Eric, I'll do a fresh clone, something must be out of wack with
>>> my local repo.
>>>
>>>
>>> Joel Bernstein
>>> http://joelsolr.blogspot.com/
>>>
>>>
>>> On Fri, Jan 8, 2021 at 10:55 AM Eric Pugh <
>>> ep...@opensourceconnections.com> wrote:
>>>
 It ran for me just fine.   I *think* you may not be up to date, as
 dataimporthandler/ is no longer in master!


 On Jan 8, 2021, at 10:08 AM, Joel Bernstein  wrote:

 I'm getting failing gradle precommits in master:

 *> Task :solr:contrib:validateSourcePatterns* FAILED
 tabs instead spaces:
 /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder.xml
 tabs instead spaces:
 /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.xml
 tabs instead spaces:
 /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestErrorHandling.xml
 tabs instead spaces:
 /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestScriptTransformer.xml
 tabs instead spaces:
 /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSqlEntityProcessor.xml
 tabs instead spaces:
 /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder2.xml
 tabs instead spaces:
 /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestZKPropertiesWriter.xml
 tabs instead spaces:
 /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler-extras/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestTikaEntityProcessor.xml

 FAILURE: Build failed with an exception.

 * Where:
 Script
 '/Users/joelbernstein/committer/lucene-solr/gradle/validation/validate-source-patterns.gradle'
 line: 324

 * What went wrong:
 Execution failed for task ':solr:contrib:validateSourcePatterns'.
 > Found 8 violations in source files (tabs instead spaces).


 Are others seeing this as well? I'm not seeing Jenkins emails about
 this.


 Joel Bernstein
 http://joelsolr.blogspot.com/


 ___
 *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467
 | http://www.opensourceconnections.com | My Free/Busy
 
 Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
 
 This e-mail and all contents, including attachments, is considered to
 be Company Confidential unless explicitly stated otherwise, regardless
 of whether attachments are marked as such.

Re: Failing gradle precommits

[Houston Putman]

Weirdly enough, Github PR precommit actions have started to OOM. Not sure
if it's a github thing or something that changed on our end...

On Fri, Jan 8, 2021 at 11:37 AM Joel Bernstein  wrote:

> It turned out to be this while I merged branches:
>
> warning: inexact rename detection was skipped due to too many files.
>
> warning: you may want to set your merge.renamelimit variable to at least
> 1639 and retry the command.
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
>
> On Fri, Jan 8, 2021 at 11:16 AM Joel Bernstein  wrote:
>
>> Thanks Eric, I'll do a fresh clone, something must be out of wack with my
>> local repo.
>>
>>
>> Joel Bernstein
>> http://joelsolr.blogspot.com/
>>
>>
>> On Fri, Jan 8, 2021 at 10:55 AM Eric Pugh <
>> ep...@opensourceconnections.com> wrote:
>>
>>> It ran for me just fine.   I *think* you may not be up to date, as
>>> dataimporthandler/ is no longer in master!
>>>
>>>
>>> On Jan 8, 2021, at 10:08 AM, Joel Bernstein  wrote:
>>>
>>> I'm getting failing gradle precommits in master:
>>>
>>> *> Task :solr:contrib:validateSourcePatterns* FAILED
>>> tabs instead spaces:
>>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder.xml
>>> tabs instead spaces:
>>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.xml
>>> tabs instead spaces:
>>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestErrorHandling.xml
>>> tabs instead spaces:
>>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestScriptTransformer.xml
>>> tabs instead spaces:
>>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSqlEntityProcessor.xml
>>> tabs instead spaces:
>>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder2.xml
>>> tabs instead spaces:
>>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestZKPropertiesWriter.xml
>>> tabs instead spaces:
>>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler-extras/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestTikaEntityProcessor.xml
>>>
>>> FAILURE: Build failed with an exception.
>>>
>>> * Where:
>>> Script
>>> '/Users/joelbernstein/committer/lucene-solr/gradle/validation/validate-source-patterns.gradle'
>>> line: 324
>>>
>>> * What went wrong:
>>> Execution failed for task ':solr:contrib:validateSourcePatterns'.
>>> > Found 8 violations in source files (tabs instead spaces).
>>>
>>>
>>> Are others seeing this as well? I'm not seeing Jenkins emails about this.
>>>
>>>
>>> Joel Bernstein
>>> http://joelsolr.blogspot.com/
>>>
>>>
>>> ___
>>> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467
>>> | http://www.opensourceconnections.com | My Free/Busy
>>> 
>>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
>>> 
>>> This e-mail and all contents, including attachments, is considered to be
>>> Company Confidential unless explicitly stated otherwise, regardless
>>> of whether attachments are marked as such.
>>>
>>>

Re: Failing gradle precommits

[Joel Bernstein]

It turned out to be this while I merged branches:

warning: inexact rename detection was skipped due to too many files.

warning: you may want to set your merge.renamelimit variable to at least
1639 and retry the command.

Joel Bernstein
http://joelsolr.blogspot.com/


On Fri, Jan 8, 2021 at 11:16 AM Joel Bernstein  wrote:

> Thanks Eric, I'll do a fresh clone, something must be out of wack with my
> local repo.
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
>
> On Fri, Jan 8, 2021 at 10:55 AM Eric Pugh 
> wrote:
>
>> It ran for me just fine.   I *think* you may not be up to date, as
>> dataimporthandler/ is no longer in master!
>>
>>
>> On Jan 8, 2021, at 10:08 AM, Joel Bernstein  wrote:
>>
>> I'm getting failing gradle precommits in master:
>>
>> *> Task :solr:contrib:validateSourcePatterns* FAILED
>> tabs instead spaces:
>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder.xml
>> tabs instead spaces:
>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.xml
>> tabs instead spaces:
>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestErrorHandling.xml
>> tabs instead spaces:
>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestScriptTransformer.xml
>> tabs instead spaces:
>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSqlEntityProcessor.xml
>> tabs instead spaces:
>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder2.xml
>> tabs instead spaces:
>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestZKPropertiesWriter.xml
>> tabs instead spaces:
>> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler-extras/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestTikaEntityProcessor.xml
>>
>> FAILURE: Build failed with an exception.
>>
>> * Where:
>> Script
>> '/Users/joelbernstein/committer/lucene-solr/gradle/validation/validate-source-patterns.gradle'
>> line: 324
>>
>> * What went wrong:
>> Execution failed for task ':solr:contrib:validateSourcePatterns'.
>> > Found 8 violations in source files (tabs instead spaces).
>>
>>
>> Are others seeing this as well? I'm not seeing Jenkins emails about this.
>>
>>
>> Joel Bernstein
>> http://joelsolr.blogspot.com/
>>
>>
>> ___
>> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467
>> | http://www.opensourceconnections.com | My Free/Busy
>> 
>> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
>> 
>> This e-mail and all contents, including attachments, is considered to be
>> Company Confidential unless explicitly stated otherwise, regardless
>> of whether attachments are marked as such.
>>
>>

Re: Failing gradle precommits

[Joel Bernstein]

Thanks Eric, I'll do a fresh clone, something must be out of wack with my
local repo.


Joel Bernstein
http://joelsolr.blogspot.com/


On Fri, Jan 8, 2021 at 10:55 AM Eric Pugh 
wrote:

> It ran for me just fine.   I *think* you may not be up to date, as
> dataimporthandler/ is no longer in master!
>
>
> On Jan 8, 2021, at 10:08 AM, Joel Bernstein  wrote:
>
> I'm getting failing gradle precommits in master:
>
> *> Task :solr:contrib:validateSourcePatterns* FAILED
> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder.xml
> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.xml
> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestErrorHandling.xml
> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestScriptTransformer.xml
> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSqlEntityProcessor.xml
> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder2.xml
> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestZKPropertiesWriter.xml
> tabs instead spaces:
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler-extras/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestTikaEntityProcessor.xml
>
> FAILURE: Build failed with an exception.
>
> * Where:
> Script
> '/Users/joelbernstein/committer/lucene-solr/gradle/validation/validate-source-patterns.gradle'
> line: 324
>
> * What went wrong:
> Execution failed for task ':solr:contrib:validateSourcePatterns'.
> > Found 8 violations in source files (tabs instead spaces).
>
>
> Are others seeing this as well? I'm not seeing Jenkins emails about this.
>
>
> Joel Bernstein
> http://joelsolr.blogspot.com/
>
>
> ___
> *Eric Pugh **| *Founder & CEO | OpenSource Connections, LLC | 434.466.1467
> | http://www.opensourceconnections.com | My Free/Busy
> 
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> 
> This e-mail and all contents, including attachments, is considered to be
> Company Confidential unless explicitly stated otherwise, regardless
> of whether attachments are marked as such.
>
>

Re: Failing gradle precommits

[Eric Pugh]

It ran for me just fine.   I *think* you may not be up to date, as 
dataimporthandler/ is no longer in master!


> On Jan 8, 2021, at 10:08 AM, Joel Bernstein  wrote:
> 
> I'm getting failing gradle precommits in master:
> 
> > Task :solr:contrib:validateSourcePatterns FAILED
> tabs instead spaces: 
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder.xml
> tabs instead spaces: 
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.xml
> tabs instead spaces: 
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestErrorHandling.xml
> tabs instead spaces: 
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestScriptTransformer.xml
> tabs instead spaces: 
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSqlEntityProcessor.xml
> tabs instead spaces: 
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder2.xml
> tabs instead spaces: 
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestZKPropertiesWriter.xml
> tabs instead spaces: 
> /Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler-extras/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestTikaEntityProcessor.xml
> 
> FAILURE: Build failed with an exception.
> 
> * Where:
> Script 
> '/Users/joelbernstein/committer/lucene-solr/gradle/validation/validate-source-patterns.gradle'
>  line: 324
> 
> * What went wrong:
> Execution failed for task ':solr:contrib:validateSourcePatterns'.
> > Found 8 violations in source files (tabs instead spaces).
> 
> 
> Are others seeing this as well? I'm not seeing Jenkins emails about this.
> 
> 
> Joel Bernstein
> http://joelsolr.blogspot.com/ 

___
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | 
http://www.opensourceconnections.com  | 
My Free/Busy   
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed 


This e-mail and all contents, including attachments, is considered to be 
Company Confidential unless explicitly stated otherwise, regardless of whether 
attachments are marked as such.

Re: [JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-16-ea+30) - Build # 29209 - Still Unstable!

[David Smiley]

Hm; this is spooky
~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley


On Fri, Jan 8, 2021 at 5:18 AM Policeman Jenkins Server 
wrote:

> Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/29209/
> Java: 64bit/jdk-16-ea+30 -XX:+UseCompressedOops -XX:+UseSerialGC
>
> 1 tests failed.
> FAILED:  org.apache.lucene.index.TestIndexFileDeleter.testExcInDecRef
>
> Error Message:
> org.apache.lucene.store.AlreadyClosedException: ReaderPool is already
> closed
>
> Stack Trace:
> org.apache.lucene.store.AlreadyClosedException: ReaderPool is already
> closed
> at
> __randomizedtesting.SeedInfo.seed([303C698E1694E4C0:D9A11EBC605D033D]:0)
> at org.apache.lucene.index.ReaderPool.get(ReaderPool.java:400)
> at
> org.apache.lucene.index.IndexWriter.writeReaderPool(IndexWriter.java:3733)
> at
> org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:588)
> at
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:474)
> at
> org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:406)
> at
> org.apache.lucene.index.TestIndexFileDeleter.testExcInDecRef(TestIndexFileDeleter.java:484)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:567)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:44)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:51)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> 

Failing gradle precommits

[Joel Bernstein]

I'm getting failing gradle precommits in master:

*> Task :solr:contrib:validateSourcePatterns* FAILED

tabs instead spaces:
/Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder.xml

tabs instead spaces:
/Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd.xml

tabs instead spaces:
/Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestErrorHandling.xml

tabs instead spaces:
/Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestScriptTransformer.xml

tabs instead spaces:
/Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestSqlEntityProcessor.xml

tabs instead spaces:
/Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestDocBuilder2.xml

tabs instead spaces:
/Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestZKPropertiesWriter.xml

tabs instead spaces:
/Users/joelbernstein/committer/lucene-solr/solr/contrib/dataimporthandler-extras/build/test-results/test/TEST-org.apache.solr.handler.dataimport.TestTikaEntityProcessor.xml


FAILURE: Build failed with an exception.


* Where:

Script
'/Users/joelbernstein/committer/lucene-solr/gradle/validation/validate-source-patterns.gradle'
line: 324


* What went wrong:

Execution failed for task ':solr:contrib:validateSourcePatterns'.

> Found 8 violations in source files (tabs instead spaces).


Are others seeing this as well? I'm not seeing Jenkins emails about this.


Joel Bernstein
http://joelsolr.blogspot.com/

Re: Requesting a new GH repository for CrossDC modules

[Ishan Chattopadhyaya]

Not necessarily. Most people contribute to Apache Lucene/Solr using
external repositories (forks) and raise pull requests against Apache owned
repositories. There's no SGA needed on such occasions.

I see two paths forward from here.

a) Lets setup a single repository for all packages/plugins, say
lucene-solr-extras or lucene-solr-contribs or lucene-solr-sandbox etc., and
develop it there.
b) All development for this effort happens in an external repository (
https://github.com/apple/solr-dc or https://github.com/anshumg/solr-dc) and
we raise a PR against Apache owned repository (which can be created if
needed once we are all onboard).

What does everyone else think?

On Fri, Jan 8, 2021 at 10:23 AM Mike Drob  wrote:

> An external repository probably ends up requiring a software grant? I know
> there is a material difference between code originating externally and code
> originating within the umbrella of the ASF in terms of IP, copyright, or
> other legal status.
>
>
> On Thu, Jan 7, 2021 at 8:11 PM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
>> If all we need now is a place to commit a PoC for now (and something like
>> sandbox repo or contribs won't suffice), why can't we have a separate
>> repository in GitHub outside Apache and merge into an Apache repository
>> only once the code takes reasonable shape?
>>
>> On Fri, 8 Jan, 2021, 2:31 am Anshum Gupta, 
>> wrote:
>>
>>> Thanks for the feedback, Mike.
>>>
>>> I like the idea of the sandbox, but that might be restricting when we
>>> want to work on more than one repos.
>>>
>>> I'm not sure if that would happen in the near future, but as we can
>>> always discard the repo and it doesn't really come at a cost, I don't see a
>>> problem with having a repo created for this specific reason.
>>>
>>> On Thu, Jan 7, 2021 at 12:45 PM Mike Drob  wrote:
>>>
 I'm not sure where I sit on this, going to start typing things and then
 hopefully I'll reach a conclusion by the end.

 This definitely needs to be outside of the core solr repo so that it
 can be versioned and released independently. And I disagree with Ishan
 about the consequence of abandoning the repository - if we realize that
 it's a bad direction then we can pivot, but we shouldn't let a fear of the
 unknown stop us from doing it in the first place.

 However, if all we need right now is a place to commit code that is
 WIP, then what we really want is a sandbox to play with, and not
 necessarily a strongly directed repo. Lucene has a sandbox in the main
 code. We could similarly start this under Solr contrib and move it out
 before an actual release of 9x happens. Or maybe we start with a
 [lucene-]solr-sandbox repository that we can throw all sorts of stuff into
 and then when components are mature enough they get to graduate into their
 own repo?

 Mike

 On Thu, Jan 7, 2021 at 2:32 PM Anshum Gupta 
 wrote:

> I understand your concern, but this is the placeholder for where the
> code would be, not what the code would look like.
>
> Considering we agreed to do this in a repository outside of the core,
> I believe this is a good place to start. The idea that the release cadence
> for the cross-dc effort should be different from that of core is an
> argument in favor of this approach, but I'm happy to talk more about it.
> I just thought that based on the original email, folks were on-board
> with the idea of this being outside of core Solr artifact/release.
>
> On Thu, Jan 7, 2021 at 11:06 AM Ishan Chattopadhyaya <
> ichattopadhy...@gmail.com> wrote:
>
>> -1 on this. Without finalizing on the shape of how the solution will
>> look like, I don't think we should start a repository: it would be bad if
>> we have to abandon the repository of our approach changes (say we want to
>> keep it tightly integrated inside Solr).
>>
>> On Thu, 7 Jan, 2021, 11:45 pm Anshum Gupta, 
>> wrote:
>>
>>> Hi everyone,
>>>
>>> Inline with my earlier email, I'll be requesting a new repository to
>>> host the cross-dc work. Please let me know if you have any questions or
>>> concerns.
>>>
>>> *Repository name: *solr-crossdc
>>> *Generated name:* lucene-solr-crossdc.git (that's auto-generated,
>>> so can't remove the TLP prefix)
>>> *Commit notification list:* commits-cros...@lucene.apache.org (I
>>> think it makes sense for these commit notifications to go to a new list,
>>> but I'm open to reusing the old one)
>>> *GitHub notification list:* dev@lucene.apache.org
>>>
>>> I'll be submitting a request for the same later in the day today if
>>> there are no concerns.
>>>
>>> --
>>> Anshum Gupta
>>>
>>
>
> --
> Anshum Gupta
>

>>>
>>> --
>>> Anshum Gupta
>>>
>>