date:20201027



[ 
https://issues.apache.org/jira/browse/LUCENE-9586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221856#comment-17221856
 ] 

Aishwarya Dabhade commented on LUCENE-9586:
---

Thanks a lot for your reply, I will keep in mind the users' or dev's list for 
such questions. I was able to run the build task on IntelliJ. The problem was 
that I was using jdk11 that comes bundled with IntelliJ, which for some 
mysterious reason does not have the jdk.javadoc module at all and was thus 
unable to resolve it.

> Intellij not able to resolve jdk.javadoc.doclet
> ---
>
> Key: LUCENE-9586
> URL: https://issues.apache.org/jira/browse/LUCENE-9586
> Project: Lucene - Core
>  Issue Type: Task
>  Components: general/build, general/javadocs, general/tools, luke
>Affects Versions: master (9.0)
> Environment: Windows 10 Home
> jdk11
> IntelliJ IDEA 
> Ubuntu 18.04 on WSL2 ( Windows Subsystem for Linux )
>  
>Reporter: Aishwarya Dabhade
>Priority: Minor
>  Labels: beginner, build, newbie
> Attachments: gradle-build-task-intellij.PNG, gradle-settings.png, 
> intellij-build.PNG
>
>
> I am a complete beginner trying to build Lucene from source on IntelliJ IDEA 
> IDE . Creating issue this as I have already tried it for a couple of days now.
> I have one question mainly:
>  # Does the community use IntelliJ IDE on Windows for building ? If not, then 
> what is the configuration / setup used for development and debugging ? I 
> understand there might not be a single way to do it, but just want to know 
> which one is the easiest. Should I not attempt to build it in Windows and 
> just go for remote debugging?
> I followed the instructions on the website but the assemble task fails in 
> IntelliJ IDE. I am working on Windows 10 Home, 64 bit. 
> In the project root directory ( lucene-solr ) cloned from git:
> on ubuntu 18.04 container on Windows via WSL2, the following works.
> `./gradlew -p lucene assemble`
> The build is successful
> Then, I tried on Windows cmd ( command line ) by executing the following 
> `gradlew.bat -p lucene assemble`
> This time around, though I got an error which said 'command 'perl'' failed. 
> So I installed Strawberry perl on Windows 10. I guess perl is available out 
> of box in most linux distros, so it works by default. After installing perl, 
> the build was successful, so it works via Windows cmdline.
> Then I went on to try it out in IntelliJ IDEA IDE on Windows 10. The only 
> reason I am trying to do it in IDEA is because that's the only method I know 
> that supports CheckStyle and where navigating the code surrounding a 
> breakpoint is easier rather than using say vim and the shell on a Linux 
> machine. If there is any other better or preferred way (maybe like remote 
> debugging with IDE on local Windows installation and source code and 
> ./gradlew on remote Linux machine [haven't tried it yet] ) , please highlight 
> that.
> This is where I got a bunch of red lines in MissingDoclet.java because 
> IntelliJ couldn't resolve it. But javadoc is indeed a part of jdk11, so I'm 
> not sure why IntelliJ cannot resolve it.
> Following is the output of the assemble task
>  * What went wrong:
>  Execution failed for task ':missing-doclet:compileJava'.
>  > Compilation failed; see the compiler error output for details.   
> I have attached a screenshot below of the gradle settings
> !gradle-settings.png!
>  
> !gradle-build-task-intellij.PNG!
> After running the above task:
>  
> !intellij-build.PNG!
>  
> Thanks in advance, appreciate all the efforts by the community, hoping to 
> contribute soon !
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-9586) Intellij not able to resolve jdk.javadoc.doclet

2020-10-27 Thread Erick Erickson (Jira)



 [ 
https://issues.apache.org/jira/browse/LUCENE-9586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson resolved LUCENE-9586.

Resolution: Information Provided

Please use the user's or dev list for questions like this, the JIRA system is 
not a support portal.

But to your question. WARNING: I run on OS X, so it may not apply...

Anyway, I test changes to Solr two ways.

1> build it externally with Gradle, i.e. gradlew assemble and attach remotely. 
If you're developing plugin code, you only have to recompile the plugin and 
(possibly) restart Solr to debug.

2> step through individual unit tests and/or create a new test for new 
functionality.



> Intellij not able to resolve jdk.javadoc.doclet
> ---
>
> Key: LUCENE-9586
> URL: https://issues.apache.org/jira/browse/LUCENE-9586
> Project: Lucene - Core
>  Issue Type: Task
>  Components: general/build, general/javadocs, general/tools, luke
>Affects Versions: master (9.0)
> Environment: Windows 10 Home
> jdk11
> IntelliJ IDEA 
> Ubuntu 18.04 on WSL2 ( Windows Subsystem for Linux )
>  
>Reporter: Aishwarya Dabhade
>Priority: Minor
>  Labels: beginner, build, newbie
> Attachments: gradle-build-task-intellij.PNG, gradle-settings.png, 
> intellij-build.PNG
>
>
> I am a complete beginner trying to build Lucene from source on IntelliJ IDEA 
> IDE . Creating issue this as I have already tried it for a couple of days now.
> I have one question mainly:
>  # Does the community use IntelliJ IDE on Windows for building ? If not, then 
> what is the configuration / setup used for development and debugging ? I 
> understand there might not be a single way to do it, but just want to know 
> which one is the easiest. Should I not attempt to build it in Windows and 
> just go for remote debugging?
> I followed the instructions on the website but the assemble task fails in 
> IntelliJ IDE. I am working on Windows 10 Home, 64 bit. 
> In the project root directory ( lucene-solr ) cloned from git:
> on ubuntu 18.04 container on Windows via WSL2, the following works.
> `./gradlew -p lucene assemble`
> The build is successful
> Then, I tried on Windows cmd ( command line ) by executing the following 
> `gradlew.bat -p lucene assemble`
> This time around, though I got an error which said 'command 'perl'' failed. 
> So I installed Strawberry perl on Windows 10. I guess perl is available out 
> of box in most linux distros, so it works by default. After installing perl, 
> the build was successful, so it works via Windows cmdline.
> Then I went on to try it out in IntelliJ IDEA IDE on Windows 10. The only 
> reason I am trying to do it in IDEA is because that's the only method I know 
> that supports CheckStyle and where navigating the code surrounding a 
> breakpoint is easier rather than using say vim and the shell on a Linux 
> machine. If there is any other better or preferred way (maybe like remote 
> debugging with IDE on local Windows installation and source code and 
> ./gradlew on remote Linux machine [haven't tried it yet] ) , please highlight 
> that.
> This is where I got a bunch of red lines in MissingDoclet.java because 
> IntelliJ couldn't resolve it. But javadoc is indeed a part of jdk11, so I'm 
> not sure why IntelliJ cannot resolve it.
> Following is the output of the assemble task
>  * What went wrong:
>  Execution failed for task ':missing-doclet:compileJava'.
>  > Compilation failed; see the compiler error output for details.   
> I have attached a screenshot below of the gradle settings
> !gradle-settings.png!
>  
> !gradle-build-task-intellij.PNG!
> After running the above task:
>  
> !intellij-build.PNG!
>  
> Thanks in advance, appreciate all the efforts by the community, hoping to 
> contribute soon !
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-14965) Adding metrics to track size of Overseer queues

2020-10-27 Thread Saatchi Bhalla (Jira)



 [ 
https://issues.apache.org/jira/browse/SOLR-14965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Saatchi Bhalla updated SOLR-14965:
--
Status: Patch Available  (was: Open)

> Adding metrics to track size of Overseer queues
> ---
>
> Key: SOLR-14965
> URL: https://issues.apache.org/jira/browse/SOLR-14965
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling, SolrCloud
>Reporter: Saatchi Bhalla
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The Overseer work queues stored in ZK and abstracted by the Overseer can give 
> us a good indication of the health of the cluster - if messages are taking 
> too long to dequeue or the queue is growing too large, we know that the 
> Overseer is overloaded and we are going to overwhelm the cluster.
>  
> This work will add metrics to track the size of the Overseer queues (number 
> of messages enqueued). 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] saatchibhalla opened a new pull request #2040: SOLR-14965 add overseer queue size metrics

saatchibhalla opened a new pull request #2040:
URL: https://github.com/apache/lucene-solr/pull/2040

# Description

Please provide a short description of the changes you're making with this
pull request.

The Overseer work queues stored in ZK and abstracted by the Overseer can
give us a good indication of the health of the cluster - if messages are taking
too long to dequeue or the queue is growing too large, we know that the
Overseer is overloaded and we are going to overwhelm the cluster.

This work adds metrics to track the size of the Overseer queues
(ClusterStateUpdate queue and Collections Work Queue).

# Solution

Please provide a short description of the approach taken to implement your
solution.

Registered these two size metrics in a shared metrics registry for the
Overseer. The Overseer shared metrics registry is only initialized upon
reference so this shouldn't have any impact on metrics when run in non-cloud
mode. Also updated the local solr-exporter-config.xml so that these metrics are
exported and can be viewed locally in Grafana.

# Tests

Please describe the tests you've developed or run to confirm this patch
implements the feature or solves the problem.

Local testing was accomplished by sending CreateCollection load and
monitoring a local grafana dashboard which accessed the metrics through
prometheus-exporter. As the size of these queues change extremely quickly, it
was difficult to see changes in these metrics quickly enough--and if the
requests were sent too quickly we'd see local solr hosts OOMing. However, I was
able to verify that the metrics are updating as the size of the queues change,
and in a real environment with more sustained load we should see these metrics
more accurately represent the state of the Overseer.

Testing confirmed that we don't see the Overseer metrics registry in
non-cloud mode.

**/admin/metrics response containing added metrics**
https://user-images.githubusercontent.com/16807693/97367278-32600980-187f-11eb-98b5-98a321c0f5c9.png;>

**local testing sending CreateCollection request load and seeing metrics
update in real time**

![image](https://user-images.githubusercontent.com/16807693/97367388-5d4a5d80-187f-11eb-8c45-58abb0b0e0cc.png)

# Checklist

Please review the following and check all that apply:

- [x] I have reviewed the guidelines for [How to
Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms
to the standards described there to the best of my ability.
- [x] I have created a Jira issue and added the issue ID to my pull request
title.
- [x] I have given Solr maintainers
[access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork)
to contribute to my PR branch. (optional but recommended)
- [x] I have developed this patch against the `master` branch.
- [x] I have run `./gradlew check`.
- [ ] I have added tests for my changes.
- [x] I have added documentation for the [Ref
Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide)
(for Solr changes only).

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (LUCENE-9587) Tests in gradle do not run with "--illegal-access=deny" like in 8.x



 [ 
https://issues.apache.org/jira/browse/LUCENE-9587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-9587:
--
Fix Version/s: master (9.0)

> Tests in gradle do not run with "--illegal-access=deny" like in 8.x
> ---
>
> Key: LUCENE-9587
> URL: https://issues.apache.org/jira/browse/LUCENE-9587
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/test
>Affects Versions: master (9.0)
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>Priority: Major
>  Labels: Java16
> Fix For: master (9.0)
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> In LUCENE-8035 we added {{--illegal-access=deny}} to the JVM options to 
> enfoce that no code is using any internal JDK APIs.
> https://github.com/apache/lucene-solr/blob/branch_8x/lucene/common-build.xml#L1053-L1055
> While changing to Gradle we lost this option somehow. We need to add it back, 
> as JDK 16 or latest JDK 17 will default to this setting anyways, so we should 
> again make sure that testing works under restricted conditions! Here is the 
> JEP: https://openjdk.java.net/jeps/396



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-9587) Tests in gradle do not run with "--illegal-access=deny" like in 8.x

2020-10-27 Thread ASF subversion and git services (Jira)



 [ 
https://issues.apache.org/jira/browse/LUCENE-9587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler resolved LUCENE-9587.
---
Resolution: Fixed

> Tests in gradle do not run with "--illegal-access=deny" like in 8.x
> ---
>
> Key: LUCENE-9587
> URL: https://issues.apache.org/jira/browse/LUCENE-9587
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/test
>Affects Versions: master (9.0)
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>Priority: Major
>  Labels: Java16
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> In LUCENE-8035 we added {{--illegal-access=deny}} to the JVM options to 
> enfoce that no code is using any internal JDK APIs.
> https://github.com/apache/lucene-solr/blob/branch_8x/lucene/common-build.xml#L1053-L1055
> While changing to Gradle we lost this option somehow. We need to add it back, 
> as JDK 16 or latest JDK 17 will default to this setting anyways, so we should 
> again make sure that testing works under restricted conditions! Here is the 
> JEP: https://openjdk.java.net/jeps/396



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9587) Tests in gradle do not run with "--illegal-access=deny" like in 8.x



[ 
https://issues.apache.org/jira/browse/LUCENE-9587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221805#comment-17221805
 ] 

ASF subversion and git services commented on LUCENE-9587:
-

Commit 9ce4b98af2155ba9d6d41e12ff12017c557a9ea4 in lucene-solr's branch 
refs/heads/master from Uwe Schindler
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=9ce4b98 ]

LUCENE-9587: Add '--illegal-access=deny' to test runner (#2039)



> Tests in gradle do not run with "--illegal-access=deny" like in 8.x
> ---
>
> Key: LUCENE-9587
> URL: https://issues.apache.org/jira/browse/LUCENE-9587
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/test
>Affects Versions: master (9.0)
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>Priority: Major
>  Labels: Java16
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In LUCENE-8035 we added {{--illegal-access=deny}} to the JVM options to 
> enfoce that no code is using any internal JDK APIs.
> https://github.com/apache/lucene-solr/blob/branch_8x/lucene/common-build.xml#L1053-L1055
> While changing to Gradle we lost this option somehow. We need to add it back, 
> as JDK 16 or latest JDK 17 will default to this setting anyways, so we should 
> again make sure that testing works under restricted conditions! Here is the 
> JEP: https://openjdk.java.net/jeps/396



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] uschindler merged pull request #2039: LUCENE-9587: Add '--illegal-access=deny' to test runner

2020-10-27 Thread ASF subversion and git services (Jira)



uschindler merged pull request #2039:
URL: https://github.com/apache/lucene-solr/pull/2039


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14749) Provide a clean API for cluster-level event processing

2020-10-27 Thread Noble Paul (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-14749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221797#comment-17221797
 ] 

Noble Paul commented on SOLR-14749:
---

I mean we have 2 new capabilities
* Registering a cluster singleton
* Registering an event listener

> Provide a clean API for cluster-level event processing
> --
>
> Key: SOLR-14749
> URL: https://issues.apache.org/jira/browse/SOLR-14749
> Project: Solr
>  Issue Type: Improvement
>  Components: AutoScaling
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
>  Labels: clean-api
> Fix For: master (9.0)
>
>  Time Spent: 19.5h
>  Remaining Estimate: 0h
>
> This is a companion issue to SOLR-14613 and it aims at providing a clean, 
> strongly typed API for the functionality formerly known as "triggers" - that 
> is, a component for generating cluster-level events corresponding to changes 
> in the cluster state, and a pluggable API for processing these events.
> The 8x triggers have been removed so this functionality is currently missing 
> in 9.0. However, this functionality is crucial for implementing the automatic 
> collection repair and re-balancing as the cluster state changes (nodes going 
> down / up, becoming overloaded / unused / decommissioned, etc).
> For this reason we need this API and a default implementation of triggers 
> that at least can perform automatic collection repair (maintaining the 
> desired replication factor in presence of live node changes).
> As before, the actual changes to the collections will be executed using 
> existing CollectionAdmin API, which in turn may use the placement plugins 
> from SOLR-14613.
> h3. Division of responsibility
>  * built-in Solr components (non-pluggable):
>  ** cluster state monitoring and event generation,
>  ** simple scheduler to periodically generate scheduled events
>  * plugins:
>  ** automatic collection repair on {{nodeLost}} events (provided by default)
>  ** re-balancing of replicas (periodic or on {{nodeAdded}} events)
>  ** reporting (eg. requesting additional node provisioning)
>  ** scheduled maintenance (eg. removing inactive shards after split)
> h3. Other considerations
> These plugins (unlike the placement plugins) need to execute on one 
> designated node in the cluster. Currently the easiest way to implement this 
> is to run them on the Overseer leader node.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Resolved] (SOLR-14940) ReplicationHandler memory leak through SolrCore.closeHooks

2020-10-27 Thread Mike Drob (Jira)



 [ 
https://issues.apache.org/jira/browse/SOLR-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob resolved SOLR-14940.
--
Fix Version/s: 8.8
   master (9.0)
   Resolution: Fixed

> ReplicationHandler memory leak through SolrCore.closeHooks
> --
>
> Key: SOLR-14940
> URL: https://issues.apache.org/jira/browse/SOLR-14940
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: replication (java)
> Environment: Solr Cloud Cluster on v.8.6.2 configured as 3 TLOG nodes 
> with 2 cores in each JVM.
>  
>Reporter: Anver Sotnikov
>Assignee: Mike Drob
>Priority: Major
> Fix For: master (9.0), 8.8
>
> Attachments: Actual references to hooks that in turn hold references 
> to ReplicationHandlers.png, Memory Analyzer SolrCore.closeHooks .png
>
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> We are experiencing a memory leak in Solr Cloud cluster configured as 3 TLOG 
> nodes.
> Leader does not seem to be affected while Followers are.
>  
> Looking at memory dump we noticed that SolrCore holds lots of references to 
> ReplicationHandler through anonymous inner classes in SolrCore.closeHooks, 
> which in turn holds ReplicationHandlers.
> ReplicationHandler registers hooks as anonymous inner classes in 
> SolrCore.closeHooks through ReplicationHandler.inform() -> 
> ReplicationHandler.registerCloseHook().
>  
> Whenever ZkController.stopReplicationFromLeader is called - it would shutdown 
> ReplicationHandler (ReplicationHandler.shutdown()), BUT reference to 
> ReplicationHandler will stay in SolrCore.closeHooks. Once replication is 
> started again on same SolrCore - new ReplicationHandler will be created and 
> registered in closeHooks.
>  
> It looks like there are few scenarios when replication is stopped and 
> restarted on same core and in our TLOG setup it shows up quite often.
>  
> Potential solutions:
>  # Allow unregistering SolrCore.closeHooks so it can be used from 
> ReplicationHandler.shutdown
>  # Hack but easier - break the link between ReplicationHandler close hooks 
> and full ReplicationHandler object so ReplicationHandler can be GCed even 
> when hooks are still registered in SolrCore.closeHooks



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14940) ReplicationHandler memory leak through SolrCore.closeHooks



[ 
https://issues.apache.org/jira/browse/SOLR-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221781#comment-17221781
 ] 

ASF subversion and git services commented on SOLR-14940:


Commit 6d00843d9705827259fa076ac59b43606720fc81 in lucene-solr's branch 
refs/heads/master from Anver Sotnikov
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6d00843 ]

SOLR-14940: Fix ReplicationHandler memory leak through SolrCore.closeHooks

* Added ability to remove SolrCore.closeHooks
* Keep references to CloseHooks in ReplicationHandler and remove them on 
ReplicationHandler.shutdown()

closes #1997


> ReplicationHandler memory leak through SolrCore.closeHooks
> --
>
> Key: SOLR-14940
> URL: https://issues.apache.org/jira/browse/SOLR-14940
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: replication (java)
> Environment: Solr Cloud Cluster on v.8.6.2 configured as 3 TLOG nodes 
> with 2 cores in each JVM.
>  
>Reporter: Anver Sotnikov
>Assignee: Mike Drob
>Priority: Major
> Attachments: Actual references to hooks that in turn hold references 
> to ReplicationHandlers.png, Memory Analyzer SolrCore.closeHooks .png
>
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> We are experiencing a memory leak in Solr Cloud cluster configured as 3 TLOG 
> nodes.
> Leader does not seem to be affected while Followers are.
>  
> Looking at memory dump we noticed that SolrCore holds lots of references to 
> ReplicationHandler through anonymous inner classes in SolrCore.closeHooks, 
> which in turn holds ReplicationHandlers.
> ReplicationHandler registers hooks as anonymous inner classes in 
> SolrCore.closeHooks through ReplicationHandler.inform() -> 
> ReplicationHandler.registerCloseHook().
>  
> Whenever ZkController.stopReplicationFromLeader is called - it would shutdown 
> ReplicationHandler (ReplicationHandler.shutdown()), BUT reference to 
> ReplicationHandler will stay in SolrCore.closeHooks. Once replication is 
> started again on same SolrCore - new ReplicationHandler will be created and 
> registered in closeHooks.
>  
> It looks like there are few scenarios when replication is stopped and 
> restarted on same core and in our TLOG setup it shows up quite often.
>  
> Potential solutions:
>  # Allow unregistering SolrCore.closeHooks so it can be used from 
> ReplicationHandler.shutdown
>  # Hack but easier - break the link between ReplicationHandler close hooks 
> and full ReplicationHandler object so ReplicationHandler can be GCed even 
> when hooks are still registered in SolrCore.closeHooks



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] madrob closed pull request #1997: SOLR-14940: Add ability to remove SolrCore.closeHooks. Keep reference…

2020-10-27 Thread ASF subversion and git services (Jira)



madrob closed pull request #1997:
URL: https://github.com/apache/lucene-solr/pull/1997


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14940) ReplicationHandler memory leak through SolrCore.closeHooks



[ 
https://issues.apache.org/jira/browse/SOLR-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221780#comment-17221780
 ] 

ASF subversion and git services commented on SOLR-14940:


Commit b8d28555f9f68cd00fc9b65342a175db595c11a3 in lucene-solr's branch 
refs/heads/branch_8x from Anver Sotnikov
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=b8d2855 ]

SOLR-14940: Fix ReplicationHandler memory leak through SolrCore.closeHooks

* Added ability to remove SolrCore.closeHooks
* Keep references to CloseHooks in ReplicationHandler and remove them on 
ReplicationHandler.shutdown()

closes #1997


> ReplicationHandler memory leak through SolrCore.closeHooks
> --
>
> Key: SOLR-14940
> URL: https://issues.apache.org/jira/browse/SOLR-14940
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: replication (java)
> Environment: Solr Cloud Cluster on v.8.6.2 configured as 3 TLOG nodes 
> with 2 cores in each JVM.
>  
>Reporter: Anver Sotnikov
>Assignee: Mike Drob
>Priority: Major
> Attachments: Actual references to hooks that in turn hold references 
> to ReplicationHandlers.png, Memory Analyzer SolrCore.closeHooks .png
>
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> We are experiencing a memory leak in Solr Cloud cluster configured as 3 TLOG 
> nodes.
> Leader does not seem to be affected while Followers are.
>  
> Looking at memory dump we noticed that SolrCore holds lots of references to 
> ReplicationHandler through anonymous inner classes in SolrCore.closeHooks, 
> which in turn holds ReplicationHandlers.
> ReplicationHandler registers hooks as anonymous inner classes in 
> SolrCore.closeHooks through ReplicationHandler.inform() -> 
> ReplicationHandler.registerCloseHook().
>  
> Whenever ZkController.stopReplicationFromLeader is called - it would shutdown 
> ReplicationHandler (ReplicationHandler.shutdown()), BUT reference to 
> ReplicationHandler will stay in SolrCore.closeHooks. Once replication is 
> started again on same SolrCore - new ReplicationHandler will be created and 
> registered in closeHooks.
>  
> It looks like there are few scenarios when replication is stopped and 
> restarted on same core and in our TLOG setup it shows up quite often.
>  
> Potential solutions:
>  # Allow unregistering SolrCore.closeHooks so it can be used from 
> ReplicationHandler.shutdown
>  # Hack but easier - break the link between ReplicationHandler close hooks 
> and full ReplicationHandler object so ReplicationHandler can be GCed even 
> when hooks are still registered in SolrCore.closeHooks



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Created] (SOLR-14967) CloudSolrClient does not pass _stateVer_ if ModifiableSolrParams isn't used

2020-10-27 Thread Varun Thacker (Jira)

Varun Thacker created SOLR-14967:


 Summary: CloudSolrClient does not pass _stateVer_ if 
ModifiableSolrParams isn't used
 Key: SOLR-14967
 URL: https://issues.apache.org/jira/browse/SOLR-14967
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Varun Thacker
 Attachments: zk-cpu-idle.png

If you are using CloudSolrClient and *only* if you are making a request with 
ModifiableSolrParams  , SolrJ passes __stateVer__ to the server on every 
request - 
[code|https://github.com/apache/lucene-solr/blob/3730719ff2ba9051278638e34bac91a4f44e9c3e/solr/solrj/src/java/org/apache/solr/client/solrj/impl/BaseCloudSolrClient.java#L920-L927]

 

This means callers who aren't passing ModifiableSolrParams ( let's say someone 
is using {color:#00}MultiMapSolrParams which is totally allowed and correct 
) never end up updating it's state of the collection{color}

 

{color:#00}The combination of Solr's handling of __stateVer_   
(_https://issues.apache.org/jira/browse/SOLR-14966) and this Jira has a 
terrible effect and causes every request to hit ZK ( this caused ZK cpu to fall 
of a cliff in our case ){color}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Created] (SOLR-14966) Solr should use cache collection state to validate _stateVer_ sent by SolrJ

2020-10-27 Thread Varun Thacker (Jira)

Varun Thacker created SOLR-14966:


 Summary: Solr should use cache collection state to validate 
_stateVer_ sent by SolrJ
 Key: SOLR-14966
 URL: https://issues.apache.org/jira/browse/SOLR-14966
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Varun Thacker


Whenever Solr receives a request with the __stateVer__  parameter it's supposed 
to validate the version the server has vs what the client has. The server was 
meant to cache the state.json for 2 seconds but it's not doing so.

 

If you follow this 
[code|https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/servlet/HttpSolrCall.java#L928]
 you'll see _clusterState.getCollectionOrNull(coll)_ resulting it a ZK fetch ( 
[LazyCollectionRef|https://github.com/apache/lucene-solr/blob/master/solr/solrj/src/java/org/apache/solr/common/cloud/ZkStateReader.java#L656]
 ) . If the code was doing _clusterState.getCollectionOrNull(coll, true)_ then 
we would call ZK only after the 2s TTL has expired



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14749) Provide a clean API for cluster-level event processing

2020-10-27 Thread Andrzej Bialecki (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-14749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221730#comment-17221730
 ] 

Andrzej Bialecki commented on SOLR-14749:
-

[~noble.paul] "this" meaning what part specifically? so far this is an internal 
API, unless you mean the ability to define non-API plugins?

> Provide a clean API for cluster-level event processing
> --
>
> Key: SOLR-14749
> URL: https://issues.apache.org/jira/browse/SOLR-14749
> Project: Solr
>  Issue Type: Improvement
>  Components: AutoScaling
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
>  Labels: clean-api
> Fix For: master (9.0)
>
>  Time Spent: 19.5h
>  Remaining Estimate: 0h
>
> This is a companion issue to SOLR-14613 and it aims at providing a clean, 
> strongly typed API for the functionality formerly known as "triggers" - that 
> is, a component for generating cluster-level events corresponding to changes 
> in the cluster state, and a pluggable API for processing these events.
> The 8x triggers have been removed so this functionality is currently missing 
> in 9.0. However, this functionality is crucial for implementing the automatic 
> collection repair and re-balancing as the cluster state changes (nodes going 
> down / up, becoming overloaded / unused / decommissioned, etc).
> For this reason we need this API and a default implementation of triggers 
> that at least can perform automatic collection repair (maintaining the 
> desired replication factor in presence of live node changes).
> As before, the actual changes to the collections will be executed using 
> existing CollectionAdmin API, which in turn may use the placement plugins 
> from SOLR-14613.
> h3. Division of responsibility
>  * built-in Solr components (non-pluggable):
>  ** cluster state monitoring and event generation,
>  ** simple scheduler to periodically generate scheduled events
>  * plugins:
>  ** automatic collection repair on {{nodeLost}} events (provided by default)
>  ** re-balancing of replicas (periodic or on {{nodeAdded}} events)
>  ** reporting (eg. requesting additional node provisioning)
>  ** scheduled maintenance (eg. removing inactive shards after split)
> h3. Other considerations
> These plugins (unlike the placement plugins) need to execute on one 
> designated node in the cluster. Currently the easiest way to implement this 
> is to run them on the Overseer leader node.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (LUCENE-9587) Tests in gradle do not run with "--illegal-access=deny" like in 8.x



 [ 
https://issues.apache.org/jira/browse/LUCENE-9587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-9587:
--
Description: 
In LUCENE-8035 we added {{--illegal-access=deny}} to the JVM options to enfoce 
that no code is using any internal JDK APIs.

https://github.com/apache/lucene-solr/blob/branch_8x/lucene/common-build.xml#L1053-L1055

While changing to Gradle we lost this option somehow. We need to add it back, 
as JDK 16 or latest JDK 17 will default to this setting anyways, so we should 
again make sure that testing works under restricted conditions! Here is the 
JEP: https://openjdk.java.net/jeps/396

  was:
In LUCENE-8035 we added {{--illegal-access=deny}} to the JVM options to enfoce 
that no code is using any internal JDK APIs.

https://github.com/apache/lucene-solr/blob/branch_8x/lucene/common-build.xml#L1053-L1055

While changing to Gradle we lost this option somehow. We need to add it back, 
as JDK 16 will default to this setting anyways, so we should again make sure 
that testing works under restricted conditions! Here is the JEP: 
https://openjdk.java.net/jeps/396


> Tests in gradle do not run with "--illegal-access=deny" like in 8.x
> ---
>
> Key: LUCENE-9587
> URL: https://issues.apache.org/jira/browse/LUCENE-9587
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/test
>Affects Versions: master (9.0)
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>Priority: Major
>  Labels: Java16
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In LUCENE-8035 we added {{--illegal-access=deny}} to the JVM options to 
> enfoce that no code is using any internal JDK APIs.
> https://github.com/apache/lucene-solr/blob/branch_8x/lucene/common-build.xml#L1053-L1055
> While changing to Gradle we lost this option somehow. We need to add it back, 
> as JDK 16 or latest JDK 17 will default to this setting anyways, so we should 
> again make sure that testing works under restricted conditions! Here is the 
> JEP: https://openjdk.java.net/jeps/396



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (LUCENE-9587) Tests in gradle do not run with "--illegal-access=deny" like in 8.x



 [ 
https://issues.apache.org/jira/browse/LUCENE-9587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-9587:
--
Description: 
In LUCENE-8035 we added {{--illegal-access=deny}} to the JVM options to enfoce 
that no code is using any internal JDK APIs.

https://github.com/apache/lucene-solr/blob/branch_8x/lucene/common-build.xml#L1053-L1055

While changing to Gradle we lost this option somehow. We need to add it back, 
as JDK 16 will default to this setting anyways, so we should again make sure 
that testing works under restricted conditions! Here is the JEP: 
https://openjdk.java.net/jeps/396

  was:
In LUCENE-8035 we added {{--illegal-access=deny}} to the JVM options to enfoce 
that no code is using any internal JDK APIs.

https://github.com/apache/lucene-solr/blob/branch_8x/lucene/common-build.xml#L1053-L1055

While changing to Gradle we lost this option somehow. We need to add it back, 
as JDK 16 will default to this setting anyways, so we should again make sure 
that testing works under restricted conditions!


> Tests in gradle do not run with "--illegal-access=deny" like in 8.x
> ---
>
> Key: LUCENE-9587
> URL: https://issues.apache.org/jira/browse/LUCENE-9587
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/test
>Affects Versions: master (9.0)
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In LUCENE-8035 we added {{--illegal-access=deny}} to the JVM options to 
> enfoce that no code is using any internal JDK APIs.
> https://github.com/apache/lucene-solr/blob/branch_8x/lucene/common-build.xml#L1053-L1055
> While changing to Gradle we lost this option somehow. We need to add it back, 
> as JDK 16 will default to this setting anyways, so we should again make sure 
> that testing works under restricted conditions! Here is the JEP: 
> https://openjdk.java.net/jeps/396



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (LUCENE-9587) Tests in gradle do not run with "--illegal-access=deny" like in 8.x



 [ 
https://issues.apache.org/jira/browse/LUCENE-9587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-9587:
--
Labels: Java16  (was: )

> Tests in gradle do not run with "--illegal-access=deny" like in 8.x
> ---
>
> Key: LUCENE-9587
> URL: https://issues.apache.org/jira/browse/LUCENE-9587
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/test
>Affects Versions: master (9.0)
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>Priority: Major
>  Labels: Java16
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In LUCENE-8035 we added {{--illegal-access=deny}} to the JVM options to 
> enfoce that no code is using any internal JDK APIs.
> https://github.com/apache/lucene-solr/blob/branch_8x/lucene/common-build.xml#L1053-L1055
> While changing to Gradle we lost this option somehow. We need to add it back, 
> as JDK 16 will default to this setting anyways, so we should again make sure 
> that testing works under restricted conditions! Here is the JEP: 
> https://openjdk.java.net/jeps/396



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9587) Tests in gradle do not run with "--illegal-access=deny" like in 8.x



[ 
https://issues.apache.org/jira/browse/LUCENE-9587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221724#comment-17221724
 ] 

Uwe Schindler commented on LUCENE-9587:
---

PR is here: https://github.com/apache/lucene-solr/pull/2039

Will commit this soon, when all tests were run.

On my windows PC command line of JVM looks like this:
{noformat}
"C:\Program Files\Java\jdk-11.0.8\bin\java.exe" "-Dcommon.dir=C:\Users\Uwe 
Schindler\Projects\lucene\trunk-lusolr1\lucene" 
"-Dgradle.lib.dir=C:/Users/Uwe 
Schindler/.gradle/wrapper/dists/gradle-6.6.1-all/ejrtlte9hlw8v6ii20a9584rs/gradle-6.6.1/lib"
 
"-Dgradle.user.home=C:\Users\Uwe Schindler\.gradle" 
"-Dgradle.worker.jar=C:\Users\Uwe 
Schindler\.gradle\caches\6.6.1\workerMain\gradle-worker.jar" 
-Djava.awt.headless=true 
-Djava.security.manager=org.apache.lucene.util.TestSecurityManager 
"-Djava.security.policy=C:\Users\Uwe 
Schindler\Projects\lucene\trunk-lusolr1\gradle\testing\randomization\policies\tests.policy"
 
"-Djava.util.logging.config.file=C:\Users\Uwe 
Schindler\Projects\lucene\trunk-lusolr1\gradle\testing\defaults-tests\logging.properties"
 
-Djdk.map.althashing.threshold=0 -Djetty.insecurerandom=1 -Djetty.testMode=1 
-Djunit4.childvm.count=1 -Djunit4.childvm.id=0 -Dorg.gradle.native=false 
"-DtempDir=C:\Users\Uwe 
Schindler\Projects\lucene\trunk-lusolr1\lucene\highlighter\build\tmp\tests-tmp" 
-Dtests.LUCENE_VERSION=9.0.0 -Dtests.asserts=true -Dtests.badapples=false 
-Dtests.codec=random -Dtests.directory=random -Dtests.docvaluesformat=random 
-Dtests.dups=0 -Dtests.failfast=false -Dtests.file.encoding=US-ASCII 
-Dtests.haltonfailure=true -Dtests.heapsize=512m -Dtests.infostream=false 
-Dtests.jvmargs=-XX:TieredStopAtLevel=1 -Dtests.jvms=4 
-Dtests.leaveTemporary=false -Dtests.linedocsfile=europarl.lines.txt.gz 
-Dtests.locale=random 
-Dtests.minheapsize=256m -Dtests.monster=false -Dtests.multiplier=1 
-Dtests.nightly=false -Dtests.postingsformat=random -Dtests.profile=false 
-Dtests.seed=D3A864B87D8B2694 -Dtests.slow=false -Dtests.timezone=random 
-Dtests.useSecurityManager=true -Dtests.verbose=false 
-Dtests.weekly=false "-Dtests.workDir=C:\Users\Uwe 
Schindler\Projects\lucene\trunk-lusolr1\lucene\highlighter\build\tmp\tests-tmp" 
-XX:TieredStopAtLevel=1 --illegal-access=deny -da:java.util.HashMap 
-esa "@C:\Users\Uwe 
Schindler\Projects\lucene\trunk-lusolr1\.gradle\tmp\gradle-worker-classpath8918994235680622943txt"
 
-Xms256m -Xmx512m -Dfile.encoding=UTF-8 "-Djava.io.tmpdir=C:\Users\Uwe 
Schindler\Projects\lucene\trunk-lusolr1\lucene\highlighter\build\tmp\tests-tmp" 
-Duser.country=DE -Duser.language=de -Duser.variant -ea 
worker.org.gradle.process.internal.worker.GradleWorkerMain "'Gradle Test 
Executor 38'"
{noformat}

> Tests in gradle do not run with "--illegal-access=deny" like in 8.x
> ---
>
> Key: LUCENE-9587
> URL: https://issues.apache.org/jira/browse/LUCENE-9587
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/test
>Affects Versions: master (9.0)
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In LUCENE-8035 we added {{--illegal-access=deny}} to the JVM options to 
> enfoce that no code is using any internal JDK APIs.
> https://github.com/apache/lucene-solr/blob/branch_8x/lucene/common-build.xml#L1053-L1055
> While changing to Gradle we lost this option somehow. We need to add it back, 
> as JDK 16 will default to this setting anyways, so we should again make sure 
> that testing works under restricted conditions!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] uschindler opened a new pull request #2039: LUCENE-9587: Add '--illegal-access=deny' to test runner



uschindler opened a new pull request #2039:
URL: https://github.com/apache/lucene-solr/pull/2039


   This adds back the test runner option, so testing is strict with internal 
APIs.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Created] (LUCENE-9587) Tests in gradle do not run with "--illegal-access=deny" like in 8.x

Uwe Schindler created LUCENE-9587:
-

 Summary: Tests in gradle do not run with "--illegal-access=deny" 
like in 8.x
 Key: LUCENE-9587
 URL: https://issues.apache.org/jira/browse/LUCENE-9587
 Project: Lucene - Core
  Issue Type: Bug
  Components: general/test
Affects Versions: master (9.0)
Reporter: Uwe Schindler
Assignee: Uwe Schindler


In LUCENE-8035 we added {{--illegal-access=deny}} to the JVM options to enfoce 
that no code is using any internal JDK APIs.

https://github.com/apache/lucene-solr/blob/branch_8x/lucene/common-build.xml#L1053-L1055

While changing to Gradle we lost this option somehow. We need to add it back, 
as JDK 16 will default to this setting anyways, so we should again make sure 
that testing works under restricted conditions!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (LUCENE-9586) Intellij not able to resolve jdk.javadoc.doclet



 [ 
https://issues.apache.org/jira/browse/LUCENE-9586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aishwarya Dabhade updated LUCENE-9586:
--
Affects Version/s: (was: 8.6.3)
   master (9.0)

> Intellij not able to resolve jdk.javadoc.doclet
> ---
>
> Key: LUCENE-9586
> URL: https://issues.apache.org/jira/browse/LUCENE-9586
> Project: Lucene - Core
>  Issue Type: Task
>  Components: general/build, general/javadocs, general/tools, luke
>Affects Versions: master (9.0)
> Environment: Windows 10 Home
> jdk11
> IntelliJ IDEA 
> Ubuntu 18.04 on WSL2 ( Windows Subsystem for Linux )
>  
>Reporter: Aishwarya Dabhade
>Priority: Minor
>  Labels: beginner, build, newbie
> Attachments: gradle-build-task-intellij.PNG, gradle-settings.png, 
> intellij-build.PNG
>
>
> I am a complete beginner trying to build Lucene from source on IntelliJ IDEA 
> IDE . Creating issue this as I have already tried it for a couple of days now.
> I have one question mainly:
>  # Does the community use IntelliJ IDE on Windows for building ? If not, then 
> what is the configuration / setup used for development and debugging ? I 
> understand there might not be a single way to do it, but just want to know 
> which one is the easiest. Should I not attempt to build it in Windows and 
> just go for remote debugging?
> I followed the instructions on the website but the assemble task fails in 
> IntelliJ IDE. I am working on Windows 10 Home, 64 bit. 
> In the project root directory ( lucene-solr ) cloned from git:
> on ubuntu 18.04 container on Windows via WSL2, the following works.
> `./gradlew -p lucene assemble`
> The build is successful
> Then, I tried on Windows cmd ( command line ) by executing the following 
> `gradlew.bat -p lucene assemble`
> This time around, though I got an error which said 'command 'perl'' failed. 
> So I installed Strawberry perl on Windows 10. I guess perl is available out 
> of box in most linux distros, so it works by default. After installing perl, 
> the build was successful, so it works via Windows cmdline.
> Then I went on to try it out in IntelliJ IDEA IDE on Windows 10. The only 
> reason I am trying to do it in IDEA is because that's the only method I know 
> that supports CheckStyle and where navigating the code surrounding a 
> breakpoint is easier rather than using say vim and the shell on a Linux 
> machine. If there is any other better or preferred way (maybe like remote 
> debugging with IDE on local Windows installation and source code and 
> ./gradlew on remote Linux machine [haven't tried it yet] ) , please highlight 
> that.
> This is where I got a bunch of red lines in MissingDoclet.java because 
> IntelliJ couldn't resolve it. But javadoc is indeed a part of jdk11, so I'm 
> not sure why IntelliJ cannot resolve it.
> Following is the output of the assemble task
>  * What went wrong:
>  Execution failed for task ':missing-doclet:compileJava'.
>  > Compilation failed; see the compiler error output for details.   
> I have attached a screenshot below of the gradle settings
> !gradle-settings.png!
>  
> !gradle-build-task-intellij.PNG!
> After running the above task:
>  
> !intellij-build.PNG!
>  
> Thanks in advance, appreciate all the efforts by the community, hoping to 
> contribute soon !
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (LUCENE-9586) Intellij not able to resolve jdk.javadoc.doclet

[
https://issues.apache.org/jira/browse/LUCENE-9586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Aishwarya Dabhade updated LUCENE-9586:
--
Description:
I am a complete beginner trying to build Lucene from source on IntelliJ IDEA
IDE . Creating issue this as I have already tried it for a couple of days now.

I have one question mainly:
# Does the community use IntelliJ IDE on Windows for building ? If not, then
what is the configuration / setup used for development and debugging ? I
understand there might not be a single way to do it, but just want to know
which one is the easiest. Should I not attempt to build it in Windows and just
go for remote debugging?

I followed the instructions on the website but the assemble task fails in
IntelliJ IDE. I am working on Windows 10 Home, 64 bit.

In the project root directory ( lucene-solr ) cloned from git:

on ubuntu 18.04 container on Windows via WSL2, the following works.

`./gradlew -p lucene assemble`

The build is successful

Then, I tried on Windows cmd ( command line ) by executing the following

`gradlew.bat -p lucene assemble`

This time around, though I got an error which said 'command 'perl'' failed. So
I installed Strawberry perl on Windows 10. I guess perl is available out of box
in most linux distros, so it works by default. After installing perl, the build
was successful, so it works via Windows cmdline.

Then I went on to try it out in IntelliJ IDEA IDE on Windows 10. The only
reason I am trying to do it in IDEA is because that's the only method I know
that supports CheckStyle and where navigating the code surrounding a breakpoint
is easier rather than using say vim and the shell on a Linux machine. If there
is any other better or preferred way (maybe like remote debugging with IDE on
local Windows installation and source code and ./gradlew on remote Linux
machine [haven't tried it yet] ) , please highlight that.

This is where I got a bunch of red lines in MissingDoclet.java because IntelliJ
couldn't resolve it. But javadoc is indeed a part of jdk11, so I'm not sure why
IntelliJ cannot resolve it.

Following is the output of the assemble task
* What went wrong:
Execution failed for task ':missing-doclet:compileJava'.
> Compilation failed; see the compiler error output for details.

I have attached a screenshot below of the gradle settings

!gradle-settings.png!

!gradle-build-task-intellij.PNG!

After running the above task:

!intellij-build.PNG!

Thanks in advance, appreciate all the efforts by the community, hoping to
contribute soon !

was:
I am a complete beginner trying to build Lucene from source on IntelliJ IDEA
IDE . Creating issue this as I have already tried it for a couple of days now.

I followed the instructions on the website but the assemble task fails in
IntelliJ IDE. I am working on Windows 10 Home, 64 bit.

In the project root directory ( lucene-solr ) cloned from git:

on ubuntu 18.04 container on Windows via WSL2, the following works.

`./gradlew -p lucene assemble`

The build is successful

Then, I tried on Windows cmd ( command line ) by executing the following

`gradlew.bat -p lucene assemble`

This is where I got a bunch of red lines in MissingDoclet.java because IntelliJ
couldn't resolve it. But javadoc is indeed a part of jdk11, so I'm not sure why
IntelliJ cannot resolve it.

Following is the output of the assemble task

* What went wrong:
Execution failed for task ':missing-doclet:compileJava'.
> Compilation failed; see the compiler error output for details.

I have attached a screenshot below of the gradle settings

!gradle-settings.png!

!gradle-build-task-intellij.PNG!

After running the above task:

!intellij-build.PNG!

Thanks in advance, appreciate all the efforts by the community, hoping to
contribute

[jira] [Created] (LUCENE-9586) Intellij not able to resolve jdk.javadoc.doclet

Aishwarya Dabhade created LUCENE-9586:
-

 Summary: Intellij not able to resolve jdk.javadoc.doclet
 Key: LUCENE-9586
 URL: https://issues.apache.org/jira/browse/LUCENE-9586
 Project: Lucene - Core
  Issue Type: Task
  Components: general/build, general/javadocs, general/tools, luke
Affects Versions: 8.6.3
 Environment: Windows 10 Home

jdk11

IntelliJ IDEA 

Ubuntu 18.04 on WSL2 ( Windows Subsystem for Linux )

 
Reporter: Aishwarya Dabhade
 Attachments: gradle-build-task-intellij.PNG, gradle-settings.png, 
intellij-build.PNG

I am a complete beginner trying to build Lucene from source on IntelliJ IDEA 
IDE . Creating issue this as I have already tried it for a couple of days now.

I have one question mainly:
 # Does the community use IntelliJ IDE on Windows for building ? If not, then 
what is the configuration / setup used for development and debugging ? I 
understand there might not be a single way to do it, but just want to know 
which one is the easiest. Should I not attempt to build it in Windows and just 
go for remote debugging?

I followed the instructions on the website but the assemble task fails in 
IntelliJ IDE. I am working on Windows 10 Home, 64 bit. 

In the project root directory ( lucene-solr ) cloned from git:

on ubuntu 18.04 container on Windows via WSL2, the following works.

`./gradlew -p lucene assemble`

The build is successful

Then, I tried on Windows cmd ( command line ) by executing the following 

`gradlew.bat -p lucene assemble`

This time around, though I got an error which said 'command 'perl'' failed. So 
I installed Strawberry perl on Windows 10. I guess perl is available out of box 
in most linux distros, so it works by default.

Then I went on to try it out in IntelliJ IDEA IDE on Windows 10. The only 
reason I am trying to do it in IDEA is because that's the only method I know 
that supports CheckStyle and where navigating the code surrounding a breakpoint 
is easier rather than using say vim and the shell on a Linux machine. If there 
is any other better or preferred way (maybe like remote debugging with IDE on 
local Windows installation and source code and ./gradlew on remote Linux 
machine [haven't tried it yet] ) , please highlight that.

This is where I got a bunch of red lines in MissingDoclet.java because IntelliJ 
couldn't resolve it. But javadoc is indeed a part of jdk11, so I'm not sure why 
IntelliJ cannot resolve it.

Following is the output of the assemble task

* What went wrong:
Execution failed for task ':missing-doclet:compileJava'.
> Compilation failed; see the compiler error output for details.   

I have attached a screenshot below of the gradle settings

!gradle-settings.png!

 

!gradle-build-task-intellij.PNG!

After running the above task:

 

!intellij-build.PNG!

 

Thanks in advance, appreciate all the efforts by the community, hoping to 
contribute soon !

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14940) ReplicationHandler memory leak through SolrCore.closeHooks

2020-10-27 Thread Anver Sotnikov (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221604#comment-17221604
 ] 

Anver Sotnikov commented on SOLR-14940:
---

[~mdrob] It looks good to me. Thank you for adding tests.

> ReplicationHandler memory leak through SolrCore.closeHooks
> --
>
> Key: SOLR-14940
> URL: https://issues.apache.org/jira/browse/SOLR-14940
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: replication (java)
> Environment: Solr Cloud Cluster on v.8.6.2 configured as 3 TLOG nodes 
> with 2 cores in each JVM.
>  
>Reporter: Anver Sotnikov
>Assignee: Mike Drob
>Priority: Major
> Attachments: Actual references to hooks that in turn hold references 
> to ReplicationHandlers.png, Memory Analyzer SolrCore.closeHooks .png
>
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> We are experiencing a memory leak in Solr Cloud cluster configured as 3 TLOG 
> nodes.
> Leader does not seem to be affected while Followers are.
>  
> Looking at memory dump we noticed that SolrCore holds lots of references to 
> ReplicationHandler through anonymous inner classes in SolrCore.closeHooks, 
> which in turn holds ReplicationHandlers.
> ReplicationHandler registers hooks as anonymous inner classes in 
> SolrCore.closeHooks through ReplicationHandler.inform() -> 
> ReplicationHandler.registerCloseHook().
>  
> Whenever ZkController.stopReplicationFromLeader is called - it would shutdown 
> ReplicationHandler (ReplicationHandler.shutdown()), BUT reference to 
> ReplicationHandler will stay in SolrCore.closeHooks. Once replication is 
> started again on same SolrCore - new ReplicationHandler will be created and 
> registered in closeHooks.
>  
> It looks like there are few scenarios when replication is stopped and 
> restarted on same core and in our TLOG setup it shows up quite often.
>  
> Potential solutions:
>  # Allow unregistering SolrCore.closeHooks so it can be used from 
> ReplicationHandler.shutdown
>  # Hack but easier - break the link between ReplicationHandler close hooks 
> and full ReplicationHandler object so ReplicationHandler can be GCed even 
> when hooks are still registered in SolrCore.closeHooks



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14940) ReplicationHandler memory leak through SolrCore.closeHooks

2020-10-27 Thread Mike Drob (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221595#comment-17221595
 ] 

Mike Drob commented on SOLR-14940:
--

[~anverus] - please let me know what you think of the latest patch, and if 
you're happy with it I think we can commit.

> ReplicationHandler memory leak through SolrCore.closeHooks
> --
>
> Key: SOLR-14940
> URL: https://issues.apache.org/jira/browse/SOLR-14940
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: replication (java)
> Environment: Solr Cloud Cluster on v.8.6.2 configured as 3 TLOG nodes 
> with 2 cores in each JVM.
>  
>Reporter: Anver Sotnikov
>Assignee: Mike Drob
>Priority: Major
> Attachments: Actual references to hooks that in turn hold references 
> to ReplicationHandlers.png, Memory Analyzer SolrCore.closeHooks .png
>
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> We are experiencing a memory leak in Solr Cloud cluster configured as 3 TLOG 
> nodes.
> Leader does not seem to be affected while Followers are.
>  
> Looking at memory dump we noticed that SolrCore holds lots of references to 
> ReplicationHandler through anonymous inner classes in SolrCore.closeHooks, 
> which in turn holds ReplicationHandlers.
> ReplicationHandler registers hooks as anonymous inner classes in 
> SolrCore.closeHooks through ReplicationHandler.inform() -> 
> ReplicationHandler.registerCloseHook().
>  
> Whenever ZkController.stopReplicationFromLeader is called - it would shutdown 
> ReplicationHandler (ReplicationHandler.shutdown()), BUT reference to 
> ReplicationHandler will stay in SolrCore.closeHooks. Once replication is 
> started again on same SolrCore - new ReplicationHandler will be created and 
> registered in closeHooks.
>  
> It looks like there are few scenarios when replication is stopped and 
> restarted on same core and in our TLOG setup it shows up quite often.
>  
> Potential solutions:
>  # Allow unregistering SolrCore.closeHooks so it can be used from 
> ReplicationHandler.shutdown
>  # Hack but easier - break the link between ReplicationHandler close hooks 
> and full ReplicationHandler object so ReplicationHandler can be GCed even 
> when hooks are still registered in SolrCore.closeHooks



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14927) Remove Overseer

2020-10-27 Thread Ilan Ginzburg (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-14927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221554#comment-17221554
 ] 

Ilan Ginzburg commented on SOLR-14927:
--

Thanks [~gezapeti]. I use the doc linked in the description to describe what 
I'm currently doing (last section) and questions I have (that I ask myself... 
or others). Would love to collaborate on this!

> Remove Overseer
> ---
>
> Key: SOLR-14927
> URL: https://issues.apache.org/jira/browse/SOLR-14927
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Ilan Ginzburg
>Assignee: Ilan Ginzburg
>Priority: Major
>  Labels: cluster, collection-api, overseer, solrcloud, zookeeper
>
> This Jira is intended to capture sub jiras on the path to remove the Overseer 
> component from SolrCloud and move to all nodes being able to do the work 
> currently done by Overseer.
> See detailed description in [this 
> doc|https://docs.google.com/document/d/1u4QHsIHuIxlglIW6hekYlXGNOP0HjLGVX5N6inkj6Ok/].
> Copying (edited) from the above doc:
> The motivation for removing Overseer include:
>  * Mono threaded state change is slow and doesn’t scale,
>  * Communication between cluster nodes and the Overseer use Zookeeper as a 
> queueing mechanism, this is not a good idea,
>  * Nodes talking to Overseer (then Overseer talking to itself) via Zookeeper 
> is inefficient and adds latency,
>  * Collection API scalability is poor, because not only a single node 
> processes commands for all Collections, but it also depends on the mono 
> threaded state change queue consumption,
>  * The code supporting Overseer in SolrCloud is complex (election, queue 
> management, recovery etc).
> The general idea is that there’s already a central point in the SolrCloud 
> cluster and it’s Zookeeper. It might not be necessary to have a second 
> central point (Overseer) because nodes can interact directly with Zookeeper 
> and synchronize more efficiently by optimistic locking using “conditional 
> updates” (a.k.a compare and swap or CAS).
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] HoustonPutman commented on a change in pull request #2020: SOLR-14949: Ability to customize Solr Docker build



HoustonPutman commented on a change in pull request #2020:
URL: https://github.com/apache/lucene-solr/pull/2020#discussion_r512825250



##
File path: help/docker.txt
##
@@ -0,0 +1,53 @@
+Docker Images for Solr
+==
+
+Solr docker images are built using Palantir's Docker Gradle plugin, 
https://github.com/palantir/gradle-docker.
+
+Common Inputs
+-
+
+The docker image and it's tag can be customized via the following options, all 
accepted via both Environment Variables and Gradle Properties.
+
+Docker Image Repository:
+   Default: "apache/solr"
+   EnvVar: DOCKER_SOLR_IMAGE_REPO
+   Gradle Property: -Pdocker.solr.imageRepo

Review comment:
   Nevermind, I agree with you. Made the changes.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] HoustonPutman commented on pull request #2020: SOLR-14949: Ability to customize Solr Docker build



HoustonPutman commented on pull request #2020:
URL: https://github.com/apache/lucene-solr/pull/2020#issuecomment-717343306


   @dweiss Is the way I am declaring and obtaining variables the correct way of 
doing it?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-9475) Enhance the Gradle build as necessary after removing Ant support

2020-10-27 Thread Erick Erickson (Jira)



 [ 
https://issues.apache.org/jira/browse/LUCENE-9475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson resolved LUCENE-9475.

Fix Version/s: master (9.0)
   Resolution: Fixed

We're using Gradle, Ant has been removed from trunk. We can raise new JIRAs for 
anything that bubbles up.

> Enhance the Gradle build as necessary after removing Ant support
> 
>
> Key: LUCENE-9475
> URL: https://issues.apache.org/jira/browse/LUCENE-9475
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: general/build
>Affects Versions: master (9.0)
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
> Fix For: master (9.0)
>
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> Once the bulk of the Ant build system is removed, stuff will come bubbling up 
> out of the cracks, especially as we try the first 9.0 release which will be 
> Gradle only. Here we list some of the areas we'll have to be aware of. Please 
> add as you see fit. Assigning to myself to track, but I certainly don't want 
> hog all the fun.
>  * Remove Maven support and replace with "The Gradle Way" of doing Maven. See 
> LUCENE-9077 (Dawid)
>  * 
>  ** -Remove all of dev-tools/maven?-
>  ** Other dev-tools files no longer used, check if any Gradle build file 
> references and remove if not.
>  * -Move Jenkins over to use Gradle only-
>  * -Verify reference guide build works under Gradle-
>  * Smoke tester
>  * Remove anything having to to with Clover (obsolete as of Java 11)
>  * -Remove all of {{lucene/tools}} (Ivy, forbiddenapis,...}}- 
>  * -Remove obsolete files in root dirs of lucene and solr (like 
> ivy-versions.props, now integrated into gradle)-
>  * -Remove Maven build files (obsolete with Gradle)-
>  * Hoss's test rollups?
>  * -Enable javadocs after ant stops being used (LUCENE-9441)-
>  * fix some relative links in javadocs which contain ant module names (?)
>  * dev-tools/scripts/* There are a lot of mentions of ant in the *.py* files, 
> and some in the README.md. This one should probably be its own JIRA since 
> it'll require quite a bit of verification...
>  * -Make "the best damn beasting script in the world" work with the Gradle 
> build.- (see LUCENE-9465, LUCENE-9472 for alternatives)
>  * Update the release documentation to reflect Gradle (LUCENE-9488)
>  * -Clean up anything in lucene/tools-
>  * Clean up Confluence, in particular any page that mentions IDEs. The "How 
> to Contribute" page has several links to various bits and pieces of how to 
> use IDEs, and some mention ant.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14927) Remove Overseer

2020-10-27 Thread Jira



[ 
https://issues.apache.org/jira/browse/SOLR-14927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221480#comment-17221480
 ] 

Gézapeti commented on SOLR-14927:
-

I think this is a good direction even if the complete removal of the Overseer 
is far or even unreachable.
I'd be happy to grab pieces of this work and help with it.

> Remove Overseer
> ---
>
> Key: SOLR-14927
> URL: https://issues.apache.org/jira/browse/SOLR-14927
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Ilan Ginzburg
>Assignee: Ilan Ginzburg
>Priority: Major
>  Labels: cluster, collection-api, overseer, solrcloud, zookeeper
>
> This Jira is intended to capture sub jiras on the path to remove the Overseer 
> component from SolrCloud and move to all nodes being able to do the work 
> currently done by Overseer.
> See detailed description in [this 
> doc|https://docs.google.com/document/d/1u4QHsIHuIxlglIW6hekYlXGNOP0HjLGVX5N6inkj6Ok/].
> Copying (edited) from the above doc:
> The motivation for removing Overseer include:
>  * Mono threaded state change is slow and doesn’t scale,
>  * Communication between cluster nodes and the Overseer use Zookeeper as a 
> queueing mechanism, this is not a good idea,
>  * Nodes talking to Overseer (then Overseer talking to itself) via Zookeeper 
> is inefficient and adds latency,
>  * Collection API scalability is poor, because not only a single node 
> processes commands for all Collections, but it also depends on the mono 
> threaded state change queue consumption,
>  * The code supporting Overseer in SolrCloud is complex (election, queue 
> management, recovery etc).
> The general idea is that there’s already a central point in the SolrCloud 
> cluster and it’s Zookeeper. It might not be necessary to have a second 
> central point (Overseer) because nodes can interact directly with Zookeeper 
> and synchronize more efficiently by optimistic locking using “conditional 
> updates” (a.k.a compare and swap or CAS).
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-10059) In SolrCloud, every fq added via is computed twice.

2020-10-27 Thread Christine Poerschke (Jira)



 [ 
https://issues.apache.org/jira/browse/SOLR-10059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christine Poerschke updated SOLR-10059:
---
Attachment: SOLR-10059-RequestHandlerBase-prepareRequestParams.patch

> In SolrCloud, every fq added via  is computed twice.
> 
>
> Key: SOLR-10059
> URL: https://issues.apache.org/jira/browse/SOLR-10059
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Affects Versions: 6.4
>Reporter: Marc Morissette
>Priority: Major
>  Labels: performance
> Attachments: 
> SOLR-10059-RequestHandlerBase-prepareRequestParams.patch, SOLR-10059_7x.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> While researching another issue, I noticed that parameters appended to a 
> query via SearchHandler's  are added to the query twice 
> in SolrCloud: once on the aggregator and again on the shard.
> The FacetComponent corrects this automatically by removing duplicates. Field 
> queries added in this fashion are however computed twice and that hinders 
> performance on filter queries that aren't simple bitsets such as those 
> produced by the CollapsingQueryParser.
> To reproduce the issue, simply test this handler on a large enough 
> collection, then replace "appends" with "defaults". You'll notice significant 
> performance improvements.
> {code}
> 
> 
> {!collapse field=routingKey hint=top_fc}
> 
> 
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-10059) In SolrCloud, every fq added via is computed twice.

2020-10-27 Thread Christine Poerschke (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-10059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221474#comment-17221474
 ] 

Christine Poerschke commented on SOLR-10059:


{quote}... The FacetComponent corrects this automatically by removing 
duplicates. ...
{quote}
Thanks for sharing that detail! 
[https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.6.3/solr/core/src/java/org/apache/solr/handler/component/FacetComponent.java#L86-L101]
 looks to be the area of code where duplicate facet parameters are removed.

I don't know how far or close we are here w.r.t. changing the existing 
behaviour for every handler that inherits from {{RequestHandlerBase}} but 
complementary to any changes, if a protected (say) 
{{RequestHandlerBase.prepareRequestParams}} method was factored out then that 
would slightly shorten the {{handleRequest}} method – 
[https://github.com/apache/lucene-solr/blob/releases/lucene-solr/8.6.3/solr/core/src/java/org/apache/solr/handler/RequestHandlerBase.java#L189-L276]
 – and custom handlers could override this method e.g. to de-duplicate 
parameters.

How would having a custom handler differ from locally patching the 
{{RequestHandlerBase}} code itself?
 * Any {{RequestHandlerBase}} change automatically applies to all handlers that 
inherit from it, no configuration changes would be needed but a locally built 
Solr needs to be deployed.
 * A custom handler means that changes only apply to the handler being 
customised but a configuration change would be needed to use the custom 
handler. Build wise the custom handler could be built separately and deployed 
as a plugin i.e. no need to locally build Solr itself.


Another approach perhaps might be to have custom search component – a bit like 
[~janhoy]'s 
[RequestSanitizerComponent|https://github.com/cominvent/request-sanitizer-component]
 – which dedupes parameters before any other component gets to see the request 
and its parameters?

> In SolrCloud, every fq added via  is computed twice.
> 
>
> Key: SOLR-10059
> URL: https://issues.apache.org/jira/browse/SOLR-10059
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Affects Versions: 6.4
>Reporter: Marc Morissette
>Priority: Major
>  Labels: performance
> Attachments: 
> SOLR-10059-RequestHandlerBase-prepareRequestParams.patch, SOLR-10059_7x.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> While researching another issue, I noticed that parameters appended to a 
> query via SearchHandler's  are added to the query twice 
> in SolrCloud: once on the aggregator and again on the shard.
> The FacetComponent corrects this automatically by removing duplicates. Field 
> queries added in this fashion are however computed twice and that hinders 
> performance on filter queries that aren't simple bitsets such as those 
> produced by the CollapsingQueryParser.
> To reproduce the issue, simply test this handler on a large enough 
> collection, then replace "appends" with "defaults". You'll notice significant 
> performance improvements.
> {code}
> 
> 
> {!collapse field=routingKey hint=top_fc}
> 
> 
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] sigram commented on a change in pull request #1962: SOLR-14749 Provide a clean API for cluster-level event processing



sigram commented on a change in pull request #1962:
URL: https://github.com/apache/lucene-solr/pull/1962#discussion_r512718408



##
File path: 
solr/core/src/java/org/apache/solr/core/ClusterEventProducerFactory.java
##
@@ -0,0 +1,178 @@
+package org.apache.solr.core;
+
+import org.apache.solr.api.CustomContainerPlugins;
+import org.apache.solr.cluster.events.ClusterEvent;
+import org.apache.solr.cluster.events.ClusterEventListener;
+import org.apache.solr.cluster.events.ClusterEventProducer;
+import org.apache.solr.cluster.events.impl.DefaultClusterEventProducer;
+
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * This class helps in handling the initial registration of plugin-based 
listeners,
+ * when both the final {@link ClusterEventProducer} implementation and 
listeners
+ * are configured using plugins.
+ */
+public class ClusterEventProducerFactory implements ClusterEventProducer {
+  private Map> 
initialListeners = new HashMap<>();
+  private CustomContainerPlugins.PluginRegistryListener initialPluginListener;
+  private final CoreContainer cc;
+  private boolean created = false;
+
+  public ClusterEventProducerFactory(CoreContainer cc) {
+this.cc = cc;
+initialPluginListener = new 
CustomContainerPlugins.PluginRegistryListener() {
+  @Override
+  public void added(CustomContainerPlugins.ApiInfo plugin) {
+if (plugin == null || plugin.getInstance() == null) {
+  return;
+}
+Object instance = plugin.getInstance();
+if (instance instanceof ClusterEventListener) {
+  registerListener((ClusterEventListener) instance);
+}
+  }
+
+  @Override
+  public void deleted(CustomContainerPlugins.ApiInfo plugin) {
+if (plugin == null || plugin.getInstance() == null) {
+  return;
+}
+Object instance = plugin.getInstance();
+if (instance instanceof ClusterEventListener) {
+  unregisterListener((ClusterEventListener) instance);
+}
+  }
+
+  @Override
+  public void modified(CustomContainerPlugins.ApiInfo old, 
CustomContainerPlugins.ApiInfo replacement) {
+added(replacement);
+deleted(old);
+  }
+};
+  }
+
+  /**
+   * This method returns an initial plugin registry listener that helps to 
capture the
+   * freshly loaded listener plugins before the final cluster event producer 
is created.
+   * @return initial listener
+   */
+  public CustomContainerPlugins.PluginRegistryListener 
getPluginRegistryListener() {
+return initialPluginListener;
+  }
+
+  /**
+   * Create a {@link ClusterEventProducer} based on the current plugin 
configurations.
+   * NOTE: this method can only be called once because it has side-effects, 
such as
+   * transferring the initially collected listeners to the resulting 
producer's instance, and
+   * installing a {@link 
org.apache.solr.api.CustomContainerPlugins.PluginRegistryListener}.
+   * Calling this method more than once will result in an exception.
+   * @param plugins current plugin configurations
+   * @return configured instance of cluster event producer (with side-effects, 
see above)
+   */
+  public ClusterEventProducer create(CustomContainerPlugins plugins) {
+if (created) {
+  throw new RuntimeException("this factory can be called only once!");
+}
+final ClusterEventProducer clusterEventProducer;
+CustomContainerPlugins.ApiInfo clusterEventProducerInfo = 
plugins.getPlugin(ClusterEventProducer.PLUGIN_NAME);
+if (clusterEventProducerInfo != null) {
+  // the listener in ClusterSingletons already registered it
+  clusterEventProducer = (ClusterEventProducer) 
clusterEventProducerInfo.getInstance();
+} else {
+  // create the default impl
+  clusterEventProducer = new DefaultClusterEventProducer(cc);

Review comment:
   AFAIK there is no mechanism in plugins to require the presence of 
another plugin. If we decide to make this functionality an opt-in then other 
plugins that depend on this may silently fail to work properly because the user 
forgot to specify this opt-in. This problem is not specific to this plugin, but 
rather it's a lack of functionality in our whole plugin ecosystem.
   
   Until we have a working mechanism to specify plugin requirements / 
dependencies I think we have to provide sensible defaults, and a sensible 
default is a working implementation, plus the ability to opt-out or change the 
default impl. I'll add a mechanism to opt-out.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail:

[jira] [Commented] (LUCENE-9583) How should we expose VectorValues.RandomAccess?

2020-10-27 Thread Michael Sokolov (Jira)



[ 
https://issues.apache.org/jira/browse/LUCENE-9583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221445#comment-17221445
 ] 

Michael Sokolov commented on LUCENE-9583:
-

The attached PR follows up on the idea of splitting out RandomAccess into a 
pair of new top-level interfaces: RandomAccessVectorValues and 
RandomAccessVectorValuesProducer. I think it will help with making the public 
API clearer. Is there a blessed way to mark such interfaces as 
internal-use-only, even though they must be public in order to be visible 
across packages internally?

> How should we expose VectorValues.RandomAccess?
> ---
>
> Key: LUCENE-9583
> URL: https://issues.apache.org/jira/browse/LUCENE-9583
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael Sokolov
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In the newly-added {{VectorValues}} API, we have a {{RandomAccess}} 
> sub-interface. [~jtibshirani] pointed out this is not needed by some 
> vector-indexing strategies which can operate solely using a forward-iterator 
> (it is needed by HNSW), and so in the interest of simplifying the public API 
> we should not expose this internal detail (which by the way surfaces internal 
> ordinals that are somewhat uninteresting outside the random access API).
> I looked into how to move this inside the HNSW-specific code and remembered 
> that we do also currently make use of the RA API when merging vector fields 
> over sorted indexes. Without it, we would need to load all vectors into RAM  
> while flushing/merging, as we currently do in 
> {{BinaryDocValuesWriter.BinaryDVs}}. I wonder if it's worth paying this cost 
> for the simpler API.
> Another thing I noticed while reviewing this is that I moved the KNN 
> {{search(float[] target, int topK, int fanout)}} method from {{VectorValues}} 
>  to {{VectorValues.RandomAccess}}. This I think we could move back, and 
> handle the HNSW requirements for search elsewhere. I wonder if that would 
> alleviate the major concern here? 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-14952) Add Jenkins CI support for s390x architecture

2020-10-27 Thread Gajanan kulkarni (Jira)



 [ 
https://issues.apache.org/jira/browse/SOLR-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gajanan kulkarni updated SOLR-14952:

Summary: Add Jenkins CI support for s390x architecture  (was: Add Jenkins 
CI support for IBM zSystems architecture (s390x).)

> Add Jenkins CI support for s390x architecture
> -
>
> Key: SOLR-14952
> URL: https://issues.apache.org/jira/browse/SOLR-14952
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Gajanan kulkarni
>Priority: Minor
>  Labels: None
>
> {color:#172b4d}Add Jenkins CI support for IBM Z systems architecture 
> (s390x).{color}
> {color:#172b4d}Since Solar binary works fine on s390x . We would like to know 
> if support for s390x can be provided in the existing Jenkins CI.{color}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] gerlowskija edited a comment on pull request #936: SOLR-13205 fixed as per diffblue suggestion



gerlowskija edited a comment on pull request #936:
URL: https://github.com/apache/lucene-solr/pull/936#issuecomment-717210773


   Hi @CaderHancock, I'm closing out this pull request - we fixed SOLR-13205 
back in July of this year.  We ended up taking a slightly different approach on 
the fix - resolving things higher up in the callstack (While this PR does fix 
the NPE, we decided we shouldn't allow `field` to be null in this code 
regardless).
   
   Thank you for the contribution - it helped get the ball rolling on the 
ticket and the eventual fix.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] gerlowskija closed pull request #936: SOLR-13205 fixed as per diffblue suggestion



gerlowskija closed pull request #936:
URL: https://github.com/apache/lucene-solr/pull/936


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] gerlowskija commented on pull request #936: SOLR-13205 fixed as per diffblue suggestion



gerlowskija commented on pull request #936:
URL: https://github.com/apache/lucene-solr/pull/936#issuecomment-717210773


   Hi @CaderHancock, I'm closing out this pull request - we fixed SOLR-13205 
back in July of this year.  We ended up taking a slightly different approach on 
the fix - resolving things higher up in the callstack (While this PR does fix 
the NPE, we decided we shouldn't allow `field` to be null in this code 
regardless).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-14749) Provide a clean API for cluster-level event processing

2020-10-27 Thread Noble Paul (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-14749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221293#comment-17221293
 ] 

Noble Paul edited comment on SOLR-14749 at 10/27/20, 10:22 AM:
---

[~ab], this is a user-facing change, Can you please add relevant examples in 
the description?

 


was (Author: noble.paul):
[~ab], this is a user-facing change, Can you please add add relevant examples 
in the description?

 

> Provide a clean API for cluster-level event processing
> --
>
> Key: SOLR-14749
> URL: https://issues.apache.org/jira/browse/SOLR-14749
> Project: Solr
>  Issue Type: Improvement
>  Components: AutoScaling
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
>  Labels: clean-api
> Fix For: master (9.0)
>
>  Time Spent: 19h 20m
>  Remaining Estimate: 0h
>
> This is a companion issue to SOLR-14613 and it aims at providing a clean, 
> strongly typed API for the functionality formerly known as "triggers" - that 
> is, a component for generating cluster-level events corresponding to changes 
> in the cluster state, and a pluggable API for processing these events.
> The 8x triggers have been removed so this functionality is currently missing 
> in 9.0. However, this functionality is crucial for implementing the automatic 
> collection repair and re-balancing as the cluster state changes (nodes going 
> down / up, becoming overloaded / unused / decommissioned, etc).
> For this reason we need this API and a default implementation of triggers 
> that at least can perform automatic collection repair (maintaining the 
> desired replication factor in presence of live node changes).
> As before, the actual changes to the collections will be executed using 
> existing CollectionAdmin API, which in turn may use the placement plugins 
> from SOLR-14613.
> h3. Division of responsibility
>  * built-in Solr components (non-pluggable):
>  ** cluster state monitoring and event generation,
>  ** simple scheduler to periodically generate scheduled events
>  * plugins:
>  ** automatic collection repair on {{nodeLost}} events (provided by default)
>  ** re-balancing of replicas (periodic or on {{nodeAdded}} events)
>  ** reporting (eg. requesting additional node provisioning)
>  ** scheduled maintenance (eg. removing inactive shards after split)
> h3. Other considerations
> These plugins (unlike the placement plugins) need to execute on one 
> designated node in the cluster. Currently the easiest way to implement this 
> is to run them on the Overseer leader node.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (SOLR-14749) Provide a clean API for cluster-level event processing

2020-10-27 Thread Noble Paul (Jira)



[ 
https://issues.apache.org/jira/browse/SOLR-14749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221293#comment-17221293
 ] 

Noble Paul commented on SOLR-14749:
---

[~ab], this is a user-facing change, Can you please add add relevant examples 
in the description?

 

> Provide a clean API for cluster-level event processing
> --
>
> Key: SOLR-14749
> URL: https://issues.apache.org/jira/browse/SOLR-14749
> Project: Solr
>  Issue Type: Improvement
>  Components: AutoScaling
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
>  Labels: clean-api
> Fix For: master (9.0)
>
>  Time Spent: 19h 20m
>  Remaining Estimate: 0h
>
> This is a companion issue to SOLR-14613 and it aims at providing a clean, 
> strongly typed API for the functionality formerly known as "triggers" - that 
> is, a component for generating cluster-level events corresponding to changes 
> in the cluster state, and a pluggable API for processing these events.
> The 8x triggers have been removed so this functionality is currently missing 
> in 9.0. However, this functionality is crucial for implementing the automatic 
> collection repair and re-balancing as the cluster state changes (nodes going 
> down / up, becoming overloaded / unused / decommissioned, etc).
> For this reason we need this API and a default implementation of triggers 
> that at least can perform automatic collection repair (maintaining the 
> desired replication factor in presence of live node changes).
> As before, the actual changes to the collections will be executed using 
> existing CollectionAdmin API, which in turn may use the placement plugins 
> from SOLR-14613.
> h3. Division of responsibility
>  * built-in Solr components (non-pluggable):
>  ** cluster state monitoring and event generation,
>  ** simple scheduler to periodically generate scheduled events
>  * plugins:
>  ** automatic collection repair on {{nodeLost}} events (provided by default)
>  ** re-balancing of replicas (periodic or on {{nodeAdded}} events)
>  ** reporting (eg. requesting additional node provisioning)
>  ** scheduled maintenance (eg. removing inactive shards after split)
> h3. Other considerations
> These plugins (unlike the placement plugins) need to execute on one 
> designated node in the cluster. Currently the easiest way to implement this 
> is to run them on the Overseer leader node.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] noblepaul commented on a change in pull request #1962: SOLR-14749 Provide a clean API for cluster-level event processing



noblepaul commented on a change in pull request #1962:
URL: https://github.com/apache/lucene-solr/pull/1962#discussion_r512566935



##
File path: 
solr/core/src/java/org/apache/solr/core/ClusterEventProducerFactory.java
##
@@ -0,0 +1,178 @@
+package org.apache.solr.core;
+
+import org.apache.solr.api.CustomContainerPlugins;
+import org.apache.solr.cluster.events.ClusterEvent;
+import org.apache.solr.cluster.events.ClusterEventListener;
+import org.apache.solr.cluster.events.ClusterEventProducer;
+import org.apache.solr.cluster.events.impl.DefaultClusterEventProducer;
+
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * This class helps in handling the initial registration of plugin-based 
listeners,
+ * when both the final {@link ClusterEventProducer} implementation and 
listeners
+ * are configured using plugins.
+ */
+public class ClusterEventProducerFactory implements ClusterEventProducer {
+  private Map> 
initialListeners = new HashMap<>();
+  private CustomContainerPlugins.PluginRegistryListener initialPluginListener;
+  private final CoreContainer cc;
+  private boolean created = false;
+
+  public ClusterEventProducerFactory(CoreContainer cc) {
+this.cc = cc;
+initialPluginListener = new 
CustomContainerPlugins.PluginRegistryListener() {
+  @Override
+  public void added(CustomContainerPlugins.ApiInfo plugin) {
+if (plugin == null || plugin.getInstance() == null) {
+  return;
+}
+Object instance = plugin.getInstance();
+if (instance instanceof ClusterEventListener) {
+  registerListener((ClusterEventListener) instance);
+}
+  }
+
+  @Override
+  public void deleted(CustomContainerPlugins.ApiInfo plugin) {
+if (plugin == null || plugin.getInstance() == null) {
+  return;
+}
+Object instance = plugin.getInstance();
+if (instance instanceof ClusterEventListener) {
+  unregisterListener((ClusterEventListener) instance);
+}
+  }
+
+  @Override
+  public void modified(CustomContainerPlugins.ApiInfo old, 
CustomContainerPlugins.ApiInfo replacement) {
+added(replacement);
+deleted(old);
+  }
+};
+  }
+
+  /**
+   * This method returns an initial plugin registry listener that helps to 
capture the
+   * freshly loaded listener plugins before the final cluster event producer 
is created.
+   * @return initial listener
+   */
+  public CustomContainerPlugins.PluginRegistryListener 
getPluginRegistryListener() {
+return initialPluginListener;
+  }
+
+  /**
+   * Create a {@link ClusterEventProducer} based on the current plugin 
configurations.
+   * NOTE: this method can only be called once because it has side-effects, 
such as
+   * transferring the initially collected listeners to the resulting 
producer's instance, and
+   * installing a {@link 
org.apache.solr.api.CustomContainerPlugins.PluginRegistryListener}.
+   * Calling this method more than once will result in an exception.
+   * @param plugins current plugin configurations
+   * @return configured instance of cluster event producer (with side-effects, 
see above)
+   */
+  public ClusterEventProducer create(CustomContainerPlugins plugins) {
+if (created) {
+  throw new RuntimeException("this factory can be called only once!");
+}
+final ClusterEventProducer clusterEventProducer;
+CustomContainerPlugins.ApiInfo clusterEventProducerInfo = 
plugins.getPlugin(ClusterEventProducer.PLUGIN_NAME);
+if (clusterEventProducerInfo != null) {
+  // the listener in ClusterSingletons already registered it
+  clusterEventProducer = (ClusterEventProducer) 
clusterEventProducerInfo.getInstance();
+} else {
+  // create the default impl
+  clusterEventProducer = new DefaultClusterEventProducer(cc);

Review comment:
   This is a core , opt-in functionality
   
   core = part of core
   opt-in =. no traces if user does not explicitly add an event producer





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Updated] (SOLR-14942) Reduce leader election time on node shutdown

2020-10-27 Thread Shalin Shekhar Mangar (Jira)



 [ 
https://issues.apache.org/jira/browse/SOLR-14942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar updated SOLR-14942:
-
Fix Version/s: (was: 8.7)
   8.8

> Reduce leader election time on node shutdown
> 
>
> Key: SOLR-14942
> URL: https://issues.apache.org/jira/browse/SOLR-14942
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.7.3, 8.6.3
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
>Priority: Major
> Fix For: master (9.0), 8.8
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> The credit for this issue and investigation belongs to [~caomanhdat]. I am 
> merely reporting the issue and creating PRs based on his work.
> The shutdown process waits for all replicas/cores to be closed before 
> removing the election node of the leader. This can take some time due to 
> index flush or merge activities on the leader cores and delays new leaders 
> from being elected.
> This process happens at CoreContainer.shutdown():
> # zkController.preClose(): remove current node from live_node and change 
> states of all cores in this node to DOWN state. Assuming that the current 
> node hosting a leader of a shard, the shard becomes leaderless after calling 
> this method, since the state of the leader is DOWN now. The leader election 
> process is not triggered for the shard since the election node is still 
> on-hold by the current node.
> # Waiting for all cores to be loaded (if there are any).
> # SolrCores.close(): close all cores.
> # zkController.close(): this is where all ephemeral nodes are removed from ZK 
> which include election nodes created by this node. Therefore other replicas 
> in the shard can take part in the leader election from now.
> Note that CoreContainer.shutdown() is invoked when Jetty/Solr nodes receive 
> SIGTERM signal. 
> On receiving SIGTERM, Jetty will also stop accepting new connections and new 
> requests. This is a very important factor, since even if the leader replica 
> is ACTIVE and its node in live_nodes, the shard will be considered as 
> leaderless if no-one can index to that shard. Therefore shards become 
> leaderless as soon as the node (which contains shard’s leader) receives 
> SIGTERM.
> Therefore the longer time step 1, 2 and 3 needed to finish, the longer shards 
> remain leaderless. The time needed for step 3 scales with the number of cores 
> so the more cores a node has, the worse. This time is spent in 
> IndexWriter.close() where the system will 
> # Flush all pending updates to disk
> # Waiting for all merge finish (this most likely is the meaty part)
> The shutdown process is proposed to changed to:
> # Wait for all in-flight indexing requests and replication requests to 
> complete
> # Remove election nodes
> # Close all replicas/cores
> This ensures that index flush or merges do not block new leader elections 
> anymore.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Resolved] (SOLR-14942) Reduce leader election time on node shutdown

2020-10-27 Thread Shalin Shekhar Mangar (Jira)



 [ 
https://issues.apache.org/jira/browse/SOLR-14942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar resolved SOLR-14942.
--
Fix Version/s: 8.7
   master (9.0)
   Resolution: Fixed

Thanks Dat, Hoss and Mike!

> Reduce leader election time on node shutdown
> 
>
> Key: SOLR-14942
> URL: https://issues.apache.org/jira/browse/SOLR-14942
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.7.3, 8.6.3
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
>Priority: Major
> Fix For: master (9.0), 8.7
>
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> The credit for this issue and investigation belongs to [~caomanhdat]. I am 
> merely reporting the issue and creating PRs based on his work.
> The shutdown process waits for all replicas/cores to be closed before 
> removing the election node of the leader. This can take some time due to 
> index flush or merge activities on the leader cores and delays new leaders 
> from being elected.
> This process happens at CoreContainer.shutdown():
> # zkController.preClose(): remove current node from live_node and change 
> states of all cores in this node to DOWN state. Assuming that the current 
> node hosting a leader of a shard, the shard becomes leaderless after calling 
> this method, since the state of the leader is DOWN now. The leader election 
> process is not triggered for the shard since the election node is still 
> on-hold by the current node.
> # Waiting for all cores to be loaded (if there are any).
> # SolrCores.close(): close all cores.
> # zkController.close(): this is where all ephemeral nodes are removed from ZK 
> which include election nodes created by this node. Therefore other replicas 
> in the shard can take part in the leader election from now.
> Note that CoreContainer.shutdown() is invoked when Jetty/Solr nodes receive 
> SIGTERM signal. 
> On receiving SIGTERM, Jetty will also stop accepting new connections and new 
> requests. This is a very important factor, since even if the leader replica 
> is ACTIVE and its node in live_nodes, the shard will be considered as 
> leaderless if no-one can index to that shard. Therefore shards become 
> leaderless as soon as the node (which contains shard’s leader) receives 
> SIGTERM.
> Therefore the longer time step 1, 2 and 3 needed to finish, the longer shards 
> remain leaderless. The time needed for step 3 scales with the number of cores 
> so the more cores a node has, the worse. This time is spent in 
> IndexWriter.close() where the system will 
> # Flush all pending updates to disk
> # Waiting for all merge finish (this most likely is the meaty part)
> The shutdown process is proposed to changed to:
> # Wait for all in-flight indexing requests and replication requests to 
> complete
> # Remove election nodes
> # Close all replicas/cores
> This ensures that index flush or merges do not block new leader elections 
> anymore.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] atris closed pull request #2030: Fix CHANGES.txt for 8.7

2020-10-27 Thread ASF subversion and git services (Jira)



atris closed pull request #2030:
URL: https://github.com/apache/lucene-solr/pull/2030


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-9455) ExitableTermsEnum (in ExitableDirectoryReader) should sample next()

2020-10-27 Thread Bruno Roustant (Jira)



 [ 
https://issues.apache.org/jira/browse/LUCENE-9455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bruno Roustant resolved LUCENE-9455.

Fix Version/s: 8.8
   Resolution: Fixed

Thanks [~zacharymorn]

> ExitableTermsEnum (in ExitableDirectoryReader) should sample next()
> ---
>
> Key: LUCENE-9455
> URL: https://issues.apache.org/jira/browse/LUCENE-9455
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/other
>Reporter: David Smiley
>Priority: Major
>  Labels: newdev
> Fix For: 8.8
>
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> ExitableTermsEnum calls "checkAndThrow" on *every* call to next().  This is 
> too expensive; it should sample.  I observed ElasticSearch uses the same 
> approach; I think Lucene would benefit from this:
> https://github.com/elastic/elasticsearch/blob/4af4eb99e18fdaadac879b1223e986227dd2ee71/server/src/main/java/org/elasticsearch/search/internal/ExitableDirectoryReader.java#L151
> CC [~jimczi]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9455) ExitableTermsEnum (in ExitableDirectoryReader) should sample next()



[ 
https://issues.apache.org/jira/browse/LUCENE-9455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221257#comment-17221257
 ] 

ASF subversion and git services commented on LUCENE-9455:
-

Commit f7246581381d304616e109565ab7ef93dd641b43 in lucene-solr's branch 
refs/heads/branch_8x from Zach Chen
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=f724658 ]

LUCENE-9455: ExitableTermsEnum should sample timeout and interruption check 
before calling next()


> ExitableTermsEnum (in ExitableDirectoryReader) should sample next()
> ---
>
> Key: LUCENE-9455
> URL: https://issues.apache.org/jira/browse/LUCENE-9455
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/other
>Reporter: David Smiley
>Priority: Major
>  Labels: newdev
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> ExitableTermsEnum calls "checkAndThrow" on *every* call to next().  This is 
> too expensive; it should sample.  I observed ElasticSearch uses the same 
> approach; I think Lucene would benefit from this:
> https://github.com/elastic/elasticsearch/blob/4af4eb99e18fdaadac879b1223e986227dd2ee71/server/src/main/java/org/elasticsearch/search/internal/ExitableDirectoryReader.java#L151
> CC [~jimczi]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9455) ExitableTermsEnum (in ExitableDirectoryReader) should sample next()

2020-10-27 Thread ASF subversion and git services (Jira)



[ 
https://issues.apache.org/jira/browse/LUCENE-9455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221248#comment-17221248
 ] 

ASF subversion and git services commented on LUCENE-9455:
-

Commit 3730719ff2ba9051278638e34bac91a4f44e9c3e in lucene-solr's branch 
refs/heads/master from Bruno Roustant
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=3730719 ]

Fix CHANGES entry for LUCENE-9455


> ExitableTermsEnum (in ExitableDirectoryReader) should sample next()
> ---
>
> Key: LUCENE-9455
> URL: https://issues.apache.org/jira/browse/LUCENE-9455
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/other
>Reporter: David Smiley
>Priority: Major
>  Labels: newdev
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> ExitableTermsEnum calls "checkAndThrow" on *every* call to next().  This is 
> too expensive; it should sample.  I observed ElasticSearch uses the same 
> approach; I think Lucene would benefit from this:
> https://github.com/elastic/elasticsearch/blob/4af4eb99e18fdaadac879b1223e986227dd2ee71/server/src/main/java/org/elasticsearch/search/internal/ExitableDirectoryReader.java#L151
> CC [~jimczi]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] sigram commented on a change in pull request #1962: SOLR-14749 Provide a clean API for cluster-level event processing



sigram commented on a change in pull request #1962:
URL: https://github.com/apache/lucene-solr/pull/1962#discussion_r512473892



##
File path: 
solr/core/src/java/org/apache/solr/core/ClusterEventProducerFactory.java
##
@@ -0,0 +1,178 @@
+package org.apache.solr.core;
+
+import org.apache.solr.api.CustomContainerPlugins;
+import org.apache.solr.cluster.events.ClusterEvent;
+import org.apache.solr.cluster.events.ClusterEventListener;
+import org.apache.solr.cluster.events.ClusterEventProducer;
+import org.apache.solr.cluster.events.impl.DefaultClusterEventProducer;
+
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * This class helps in handling the initial registration of plugin-based 
listeners,
+ * when both the final {@link ClusterEventProducer} implementation and 
listeners
+ * are configured using plugins.
+ */
+public class ClusterEventProducerFactory implements ClusterEventProducer {
+  private Map> 
initialListeners = new HashMap<>();
+  private CustomContainerPlugins.PluginRegistryListener initialPluginListener;
+  private final CoreContainer cc;
+  private boolean created = false;
+
+  public ClusterEventProducerFactory(CoreContainer cc) {
+this.cc = cc;
+initialPluginListener = new 
CustomContainerPlugins.PluginRegistryListener() {
+  @Override
+  public void added(CustomContainerPlugins.ApiInfo plugin) {
+if (plugin == null || plugin.getInstance() == null) {
+  return;
+}
+Object instance = plugin.getInstance();
+if (instance instanceof ClusterEventListener) {
+  registerListener((ClusterEventListener) instance);
+}
+  }
+
+  @Override
+  public void deleted(CustomContainerPlugins.ApiInfo plugin) {
+if (plugin == null || plugin.getInstance() == null) {
+  return;
+}
+Object instance = plugin.getInstance();
+if (instance instanceof ClusterEventListener) {
+  unregisterListener((ClusterEventListener) instance);
+}
+  }
+
+  @Override
+  public void modified(CustomContainerPlugins.ApiInfo old, 
CustomContainerPlugins.ApiInfo replacement) {
+added(replacement);
+deleted(old);
+  }
+};
+  }
+
+  /**
+   * This method returns an initial plugin registry listener that helps to 
capture the
+   * freshly loaded listener plugins before the final cluster event producer 
is created.
+   * @return initial listener
+   */
+  public CustomContainerPlugins.PluginRegistryListener 
getPluginRegistryListener() {
+return initialPluginListener;
+  }
+
+  /**
+   * Create a {@link ClusterEventProducer} based on the current plugin 
configurations.
+   * NOTE: this method can only be called once because it has side-effects, 
such as
+   * transferring the initially collected listeners to the resulting 
producer's instance, and
+   * installing a {@link 
org.apache.solr.api.CustomContainerPlugins.PluginRegistryListener}.
+   * Calling this method more than once will result in an exception.
+   * @param plugins current plugin configurations
+   * @return configured instance of cluster event producer (with side-effects, 
see above)
+   */
+  public ClusterEventProducer create(CustomContainerPlugins plugins) {
+if (created) {
+  throw new RuntimeException("this factory can be called only once!");
+}
+final ClusterEventProducer clusterEventProducer;
+CustomContainerPlugins.ApiInfo clusterEventProducerInfo = 
plugins.getPlugin(ClusterEventProducer.PLUGIN_NAME);
+if (clusterEventProducerInfo != null) {
+  // the listener in ClusterSingletons already registered it
+  clusterEventProducer = (ClusterEventProducer) 
clusterEventProducerInfo.getInstance();
+} else {
+  // create the default impl
+  clusterEventProducer = new DefaultClusterEventProducer(cc);

Review comment:
   I'm not sure. If we treat this as a part of core functionality (the 
ability to produce and listen to cluster-level events) then we need a 
rudimentary default implementation to be always present.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] sigram commented on a change in pull request #1962: SOLR-14749 Provide a clean API for cluster-level event processing



sigram commented on a change in pull request #1962:
URL: https://github.com/apache/lucene-solr/pull/1962#discussion_r512473129



##
File path: solr/core/src/java/org/apache/solr/core/CoreContainer.java
##
@@ -252,6 +253,11 @@ public CoreLoadFailure(CoreDescriptor cd, Exception 
loadFailure) {
   getZkController().getOverseer() != null &&
   !getZkController().getOverseer().isClosed(),
   (r) -> this.runAsync(r));
+
+  private final ClusterEventProducerFactory clusterEventProducerFactory = new 
ClusterEventProducerFactory(this);

Review comment:
   +1.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] atris merged pull request #2033: Fix changes entry in 8x branch



atris merged pull request #2033:
URL: https://github.com/apache/lucene-solr/pull/2033


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] atris commented on a change in pull request #2033: Fix changes entry in 8x branch



atris commented on a change in pull request #2033:
URL: https://github.com/apache/lucene-solr/pull/2033#discussion_r512467068



##
File path: solr/CHANGES.txt
##
@@ -35,113 +35,13 @@ Consult the lucene/CHANGES.txt file for additional, low 
level, changes in this r
 
 New Features
 -
+
 * SOLR-14151 Make schema components load from packages (noble)
 
 * SOLR-14588: Introduce Circuit Breaker Infrastructure and a JVM heap usage 
memory tracking circuit breaker implementation (Atri Sharma)
 
 * SOLR-14615: CPU Utilization Based Circuit Breaker (Atri Sharma)
 
-Improvements
---
-* LUCENE-8984: MoreLikeThis MLT is biased for uncommon fields (Andy Hind via 
Anshum Gupta)
-

Review comment:
   My bad -- thats already included.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] atris commented on a change in pull request #2033: Fix changes entry in 8x branch



atris commented on a change in pull request #2033:
URL: https://github.com/apache/lucene-solr/pull/2033#discussion_r512463176



##
File path: solr/CHANGES.txt
##
@@ -35,113 +35,13 @@ Consult the lucene/CHANGES.txt file for additional, low 
level, changes in this r
 
 New Features
 -
+
 * SOLR-14151 Make schema components load from packages (noble)
 
 * SOLR-14588: Introduce Circuit Breaker Infrastructure and a JVM heap usage 
memory tracking circuit breaker implementation (Atri Sharma)
 
 * SOLR-14615: CPU Utilization Based Circuit Breaker (Atri Sharma)
 
-Improvements
---
-* LUCENE-8984: MoreLikeThis MLT is biased for uncommon fields (Andy Hind via 
Anshum Gupta)
-

Review comment:
   Should not circuit breaker entries (SOLR-14588 and SOLR-14615) be added 
here?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[jira] [Commented] (LUCENE-9475) Enhance the Gradle build as necessary after removing Ant support

2020-10-27 Thread Dawid Weiss (Jira)



[ 
https://issues.apache.org/jira/browse/LUCENE-9475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17221176#comment-17221176
 ] 

Dawid Weiss commented on LUCENE-9475:
-

Yeah, close it, Erick.

> Enhance the Gradle build as necessary after removing Ant support
> 
>
> Key: LUCENE-9475
> URL: https://issues.apache.org/jira/browse/LUCENE-9475
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: general/build
>Affects Versions: master (9.0)
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> Once the bulk of the Ant build system is removed, stuff will come bubbling up 
> out of the cracks, especially as we try the first 9.0 release which will be 
> Gradle only. Here we list some of the areas we'll have to be aware of. Please 
> add as you see fit. Assigning to myself to track, but I certainly don't want 
> hog all the fun.
>  * Remove Maven support and replace with "The Gradle Way" of doing Maven. See 
> LUCENE-9077 (Dawid)
>  * 
>  ** -Remove all of dev-tools/maven?-
>  ** Other dev-tools files no longer used, check if any Gradle build file 
> references and remove if not.
>  * -Move Jenkins over to use Gradle only-
>  * -Verify reference guide build works under Gradle-
>  * Smoke tester
>  * Remove anything having to to with Clover (obsolete as of Java 11)
>  * -Remove all of {{lucene/tools}} (Ivy, forbiddenapis,...}}- 
>  * -Remove obsolete files in root dirs of lucene and solr (like 
> ivy-versions.props, now integrated into gradle)-
>  * -Remove Maven build files (obsolete with Gradle)-
>  * Hoss's test rollups?
>  * -Enable javadocs after ant stops being used (LUCENE-9441)-
>  * fix some relative links in javadocs which contain ant module names (?)
>  * dev-tools/scripts/* There are a lot of mentions of ant in the *.py* files, 
> and some in the README.md. This one should probably be its own JIRA since 
> it'll require quite a bit of verification...
>  * -Make "the best damn beasting script in the world" work with the Gradle 
> build.- (see LUCENE-9465, LUCENE-9472 for alternatives)
>  * Update the release documentation to reflect Gradle (LUCENE-9488)
>  * -Clean up anything in lucene/tools-
>  * Clean up Confluence, in particular any page that mentions IDEs. The "How 
> to Contribute" page has several links to various bits and pieces of how to 
> use IDEs, and some mention ant.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] MarcusSorealheis commented on a change in pull request #2026: LUCENE-8626: Standardize Lucene Test Files



MarcusSorealheis commented on a change in pull request #2026:
URL: https://github.com/apache/lucene-solr/pull/2026#discussion_r512116365



##
File path: 
lucene/spatial-extras/src/test/org/apache/lucene/spatial/TestQueryEqualsHashCode.java
##
@@ -0,0 +1,119 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.spatial;
+
+import java.util.ArrayList;
+import java.util.Collection;
+
+import org.apache.lucene.spatial.bbox.BBoxStrategy;
+import org.apache.lucene.spatial.composite.CompositeSpatialStrategy;
+import org.apache.lucene.spatial.prefix.RecursivePrefixTreeStrategy;
+import org.apache.lucene.spatial.prefix.TermQueryPrefixTreeStrategy;
+import org.apache.lucene.spatial.prefix.tree.GeohashPrefixTree;
+import org.apache.lucene.spatial.prefix.tree.QuadPrefixTree;
+import org.apache.lucene.spatial.prefix.tree.SpatialPrefixTree;
+import org.apache.lucene.spatial.query.SpatialArgs;
+import org.apache.lucene.spatial.query.SpatialOperation;
+import org.apache.lucene.spatial.serialized.SerializedDVStrategy;
+import org.apache.lucene.spatial.vector.PointVectorStrategy;
+import org.apache.lucene.util.LuceneTestCase;
+import org.junit.Test;
+import org.locationtech.spatial4j.context.SpatialContext;
+import org.locationtech.spatial4j.shape.Shape;
+
+public class TestQueryEqualsHashCode extends LuceneTestCase {

Review comment:
   that's why I am keeping these small





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org

[GitHub] [lucene-solr] MarcusSorealheis commented on a change in pull request #2026: LUCENE-8626: Standardize Lucene Test Files