[jira] [Comment Edited] (SOLR-8673) o.a.s.search.facet classes not public/extendable
[
https://issues.apache.org/jira/browse/SOLR-8673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245647#comment-17245647
]
Mikhail Khludnev edited comment on SOLR-8673 at 12/8/20, 7:31 AM:
--
Sorry to hear that, [~thelabdude]. I'm on it.
was (Author: mkhludnev):
Sorry to hear that, @thelabdude. I'm on it.
> o.a.s.search.facet classes not public/extendable
>
>
> Key: SOLR-8673
> URL: https://issues.apache.org/jira/browse/SOLR-8673
> Project: Solr
> Issue Type: Improvement
> Components: Facet Module
>Affects Versions: 5.4.1
>Reporter: Markus Jelsma
>Priority: Major
> Fix For: 6.2, 7.0
>
> Attachments: SOLR-8673.patch, SOLR-8673.patch
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> It is not easy to create a custom JSON facet function. A simple function
> based on AvgAgg quickly results in the following compilation failures:
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.3:compile (default-compile)
> on project openindex-solr: Compilation failure: Compilation failure:
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[22,36]
> org.apache.solr.search.facet.FacetContext is not public in
> org.apache.solr.search.facet; cannot be accessed from outside package
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[23,36]
> org.apache.solr.search.facet.FacetDoubleMerger is not public in
> org.apache.solr.search.facet; cannot be accessed from outside package
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[40,32]
> cannot find symbol
> [ERROR] symbol: class FacetContext
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[49,39]
> cannot find symbol
> [ERROR] symbol: class FacetDoubleMerger
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[54,43]
> cannot find symbol
> [ERROR] symbol: class Context
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg.Merger
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[41,16]
> cannot find symbol
> [ERROR] symbol: class AvgSlotAcc
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[46,12]
> incompatible types: i.o.s.search.facet.CustomAvgAgg.Merger cannot be
> converted to org.apache.solr.search.facet.FacetMerger
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[53,5]
> method does not override or implement a method from a supertype
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[60,5]
> method does not override or implement a method from a supertype
> {code}
> It seems lots of classes are tucked away in FacetModule, which we can't reach
> from outside.
> Originates from this thread:
> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201602.mbox/%3ccab_8yd9ldbg_0zxm_h1igkfm6bqeypd5ilyy7tty8cztscv...@mail.gmail.com%3E
> ( also available at
> https://lists.apache.org/thread.html/9fddcad3136ec908ce1c57881f8d3069e5d153f08b71f80f3e18d995%401455019826%40%3Csolr-user.lucene.apache.org%3E
> )
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-8673) o.a.s.search.facet classes not public/extendable
[
https://issues.apache.org/jira/browse/SOLR-8673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245647#comment-17245647
]
Mikhail Khludnev edited comment on SOLR-8673 at 12/8/20, 7:30 AM:
--
Sorry to hear that, @thelabdude. I'm on it.
was (Author: mkhludnev):
Sorry to hear that, @thekabdude. I'm on it.
> o.a.s.search.facet classes not public/extendable
>
>
> Key: SOLR-8673
> URL: https://issues.apache.org/jira/browse/SOLR-8673
> Project: Solr
> Issue Type: Improvement
> Components: Facet Module
>Affects Versions: 5.4.1
>Reporter: Markus Jelsma
>Priority: Major
> Fix For: 6.2, 7.0
>
> Attachments: SOLR-8673.patch, SOLR-8673.patch
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> It is not easy to create a custom JSON facet function. A simple function
> based on AvgAgg quickly results in the following compilation failures:
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.3:compile (default-compile)
> on project openindex-solr: Compilation failure: Compilation failure:
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[22,36]
> org.apache.solr.search.facet.FacetContext is not public in
> org.apache.solr.search.facet; cannot be accessed from outside package
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[23,36]
> org.apache.solr.search.facet.FacetDoubleMerger is not public in
> org.apache.solr.search.facet; cannot be accessed from outside package
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[40,32]
> cannot find symbol
> [ERROR] symbol: class FacetContext
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[49,39]
> cannot find symbol
> [ERROR] symbol: class FacetDoubleMerger
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[54,43]
> cannot find symbol
> [ERROR] symbol: class Context
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg.Merger
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[41,16]
> cannot find symbol
> [ERROR] symbol: class AvgSlotAcc
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[46,12]
> incompatible types: i.o.s.search.facet.CustomAvgAgg.Merger cannot be
> converted to org.apache.solr.search.facet.FacetMerger
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[53,5]
> method does not override or implement a method from a supertype
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[60,5]
> method does not override or implement a method from a supertype
> {code}
> It seems lots of classes are tucked away in FacetModule, which we can't reach
> from outside.
> Originates from this thread:
> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201602.mbox/%3ccab_8yd9ldbg_0zxm_h1igkfm6bqeypd5ilyy7tty8cztscv...@mail.gmail.com%3E
> ( also available at
> https://lists.apache.org/thread.html/9fddcad3136ec908ce1c57881f8d3069e5d153f08b71f80f3e18d995%401455019826%40%3Csolr-user.lucene.apache.org%3E
> )
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-8673) o.a.s.search.facet classes not public/extendable
[
https://issues.apache.org/jira/browse/SOLR-8673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245647#comment-17245647
]
Mikhail Khludnev commented on SOLR-8673:
Sorry to hear that, @thekabdude. I'm on it.
> o.a.s.search.facet classes not public/extendable
>
>
> Key: SOLR-8673
> URL: https://issues.apache.org/jira/browse/SOLR-8673
> Project: Solr
> Issue Type: Improvement
> Components: Facet Module
>Affects Versions: 5.4.1
>Reporter: Markus Jelsma
>Priority: Major
> Fix For: 6.2, 7.0
>
> Attachments: SOLR-8673.patch, SOLR-8673.patch
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> It is not easy to create a custom JSON facet function. A simple function
> based on AvgAgg quickly results in the following compilation failures:
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.3:compile (default-compile)
> on project openindex-solr: Compilation failure: Compilation failure:
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[22,36]
> org.apache.solr.search.facet.FacetContext is not public in
> org.apache.solr.search.facet; cannot be accessed from outside package
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[23,36]
> org.apache.solr.search.facet.FacetDoubleMerger is not public in
> org.apache.solr.search.facet; cannot be accessed from outside package
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[40,32]
> cannot find symbol
> [ERROR] symbol: class FacetContext
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[49,39]
> cannot find symbol
> [ERROR] symbol: class FacetDoubleMerger
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[54,43]
> cannot find symbol
> [ERROR] symbol: class Context
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg.Merger
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[41,16]
> cannot find symbol
> [ERROR] symbol: class AvgSlotAcc
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[46,12]
> incompatible types: i.o.s.search.facet.CustomAvgAgg.Merger cannot be
> converted to org.apache.solr.search.facet.FacetMerger
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[53,5]
> method does not override or implement a method from a supertype
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[60,5]
> method does not override or implement a method from a supertype
> {code}
> It seems lots of classes are tucked away in FacetModule, which we can't reach
> from outside.
> Originates from this thread:
> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201602.mbox/%3ccab_8yd9ldbg_0zxm_h1igkfm6bqeypd5ilyy7tty8cztscv...@mail.gmail.com%3E
> ( also available at
> https://lists.apache.org/thread.html/9fddcad3136ec908ce1c57881f8d3069e5d153f08b71f80f3e18d995%401455019826%40%3Csolr-user.lucene.apache.org%3E
> )
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] HoustonPutman opened a new pull request #2130: Adding Apache Reporter step in Release Wizard.
HoustonPutman opened a new pull request #2130: URL: https://github.com/apache/lucene-solr/pull/2130 The release wizard doesn't currently tell the RM to add the new release to the apache release reporter. This is a simple change to add that one step at the end of announcing the new version. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] zacharymorn commented on a change in pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory
zacharymorn commented on a change in pull request #2052:
URL: https://github.com/apache/lucene-solr/pull/2052#discussion_r537975653
##
File path:
lucene/misc/src/java/org/apache/lucene/misc/store/DirectIODirectory.java
##
@@ -51,16 +50,9 @@
* to the provided Directory instance.
*
* See Overview
+ * href="{@docRoot}/overview-summary.html#DirectIODirectory">Overview
* for more details.
*
- * To use this you must compile
- * NativePosixUtil.cpp (exposes Linux-specific APIs through
- * JNI) for your platform, by running ./gradlew build, and then
putting the resulting
- * libLuceneNativeIO.so or libLuceneNativeIO.dylib
- * (from lucene/misc/native/build/lib/release/platform/) onto
your dynamic
- * linker search path.
- *
* WARNING: this code is very new and quite easily
* could contain horrible bugs. For example, here's one
* known issue: if you use seek in IndexOutput, and then
Review comment:
Sounds good that makes sense. Thanks for the detailed explanation Mike!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] zacharymorn commented on a change in pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory
zacharymorn commented on a change in pull request #2052:
URL: https://github.com/apache/lucene-solr/pull/2052#discussion_r537974339
##
File path:
lucene/misc/src/java/org/apache/lucene/misc/store/DirectIODirectory.java
##
@@ -154,14 +146,43 @@ public IndexOutput createOutput(String name, IOContext
context) throws IOExcepti
if (context.context != Context.MERGE ||
context.mergeInfo.estimatedMergeBytes < minBytesDirect) {
return delegate.createOutput(name, context);
} else {
- return new NativeUnixIndexOutput(getDirectory().resolve(name), name,
mergeBufferSize);
+ return new DirectIOIndexOutput(getDirectory().resolve(name), name,
mergeBufferSize);
}
}
- @SuppressForbidden(reason = "java.io.File: native API requires old-style
FileDescriptor")
- private final static class NativeUnixIndexOutput extends IndexOutput {
+ @Override
+ public void deleteFile(String name) throws IOException {
+delegate.deleteFile(name);
+ }
+
+ @Override
+ public Set getPendingDeletions() throws IOException {
Review comment:
Ha I had this same thought for a split second before, but then got
distracted and forgot to come back to it later. I just pushed a commit to make
this change.
##
File path:
lucene/misc/src/java/org/apache/lucene/misc/store/DirectIODirectory.java
##
@@ -154,14 +146,43 @@ public IndexOutput createOutput(String name, IOContext
context) throws IOExcepti
if (context.context != Context.MERGE ||
context.mergeInfo.estimatedMergeBytes < minBytesDirect) {
return delegate.createOutput(name, context);
} else {
- return new NativeUnixIndexOutput(getDirectory().resolve(name), name,
mergeBufferSize);
+ return new DirectIOIndexOutput(getDirectory().resolve(name), name,
mergeBufferSize);
}
}
- @SuppressForbidden(reason = "java.io.File: native API requires old-style
FileDescriptor")
- private final static class NativeUnixIndexOutput extends IndexOutput {
+ @Override
+ public void deleteFile(String name) throws IOException {
+delegate.deleteFile(name);
+ }
+
+ @Override
+ public Set getPendingDeletions() throws IOException {
+return delegate.getPendingDeletions();
+ }
+
+ @Override
+ public String[] listAll() throws IOException {
+return delegate.listAll();
+ }
+
+ @Override
+ public long fileLength(String name) throws IOException {
+return delegate.fileLength(name);
+ }
+
+ @Override
+ public void rename(String source, String dest) throws IOException {
+delegate.rename(source, dest);
+ }
+
+ @Override
+ public void close() throws IOException {
+delegate.close();
Review comment:
I made the update to replace `delegate.close()` with
`IOUtils.close(delegate)`, but I don't think I could achieve `super.close()`
through `IOUtils.close(this)` as well, as that leads to stackoverflow? Or am I
misinterpreting the comment here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] muse-dev[bot] commented on a change in pull request #1963: SOLR-14827: Refactor schema loading to not use XPath
muse-dev[bot] commented on a change in pull request #1963:
URL: https://github.com/apache/lucene-solr/pull/1963#discussion_r537974169
##
File path: solr/core/src/java/org/apache/solr/core/ConfigSetService.java
##
@@ -186,6 +198,20 @@ protected NamedList loadConfigSetFlags(CoreDescriptor cd,
SolrResourceLoader loa
*/
public abstract String configSetName(CoreDescriptor cd);
+ public interface ConfigResource {
+
+ConfigNode get() throws Exception;
+
+ }
+ public static ConfigNode getParsedSchema(InputStream is, SolrResourceLoader
loader, String name) throws IOException, SAXException,
ParserConfigurationException {
+XmlConfigFile schemaConf = null;
+InputSource inputSource = new InputSource(is);
+
inputSource.setSystemId(SystemIdResolver.createSystemIdFromResourceName(name));
+schemaConf = new XmlConfigFile(loader, SCHEMA, inputSource, SLASH + SCHEMA
+ SLASH, null);
+return new DataConfigNode(new
DOMConfigNode(schemaConf.getDocument().getDocumentElement()));
Review comment:
*NULL_DEREFERENCE:* object returned by `getDocument(schemaConf)` could
be null and is dereferenced at line 211.
##
File path:
solr/core/src/java/org/apache/solr/schema/ManagedIndexSchemaFactory.java
##
@@ -210,7 +215,7 @@ private InputStream readSchemaLocally() {
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method
`ManagedIndexSchemaFactory.create(...)` indirectly reads with synchronization
from `this.config`. Potentially races with unsynchronized write in method
`ManagedIndexSchemaFactory.create(...)`.
Reporting because this access may occur on a background thread.
##
File path:
solr/core/src/java/org/apache/solr/schema/ManagedIndexSchemaFactory.java
##
@@ -174,8 +175,12 @@ public ManagedIndexSchema create(String resourceName,
SolrConfig config) {
}
InputSource inputSource = new InputSource(schemaInputStream);
inputSource.setSystemId(SystemIdResolver.createSystemIdFromResourceName(loadedResource));
-schema = new ManagedIndexSchema(config, loadedResource, inputSource,
isMutable,
-managedSchemaResourceName,
schemaZkVersion, getSchemaUpdateLock());
+try {
+ schema = new ManagedIndexSchema(config,
loadedResource,IndexSchemaFactory.getConfigResource(configSetService,
schemaInputStream, loader, managedSchemaResourceName) , isMutable,
Review comment:
*THREAD_SAFETY_VIOLATION:* Read/Write race. Non-private method
`ManagedIndexSchemaFactory.create(...)` indirectly reads without
synchronization from `configSetService.zkController.cloudManager`. Potentially
races with write in method `ManagedIndexSchemaFactory.create(...)`.
Reporting because this access may occur on a background thread.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] zacharymorn commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory
zacharymorn commented on pull request #2052:
URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-740306114
> Hmm, `gradlew precommit` is angry:
>
> ```
> > Task :lucene:misc:compileJava
>
/l/directio/lucene/misc/src/java/org/apache/lucene/misc/store/DirectIODirectory.java:206:
warning: ExtendedOpenOption is internal proprietary API and may be removed in
a future release
>
com.sun.nio.file.ExtendedOpenOption.DIRECT);
> ^
>
/l/directio/lucene/misc/src/java/org/apache/lucene/misc/store/DirectIODirectory.java:325:
warning: ExtendedOpenOption is internal proprietary API and may be removed in
a future release
> channel = FileChannel.open(path, StandardOpenOption.READ,
com.sun.nio.file.ExtendedOpenOption.DIRECT);
>
^
> error: warnings found and -Werror specified
>
> > Task :lucene:sandbox:ecjLintMain
> > Task :lucene:demo:ecjLintMain
> > Task :lucene:codecs:ecjLintMain
>
> > Task :lucene:misc:compileJava FAILED
> 1 error
> 2 warnings
> ```
>
> Maybe we need to add something like `@SuppressWarnings("sunapi")`
annotation?
Hmm this is strange. These warnings should have been suppressed by
`@SuppressForbidden` at the constructors already. Both my local and the github
PR gradle precommit check passed as well
(https://github.com/apache/lucene-solr/pull/2052/checks?check_run_id=1496707362).
Is this error showing up in some CI environment / job execution that I can
take a further look at?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13237) Not all types of index corruption garuntee a leader will "give up its leadership"
[ https://issues.apache.org/jira/browse/SOLR-13237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245556#comment-17245556 ] Mike Drob commented on SOLR-13237: -- Instead of trying to address the underlying corruption issue, I think we can inject a tragic event manually. See https://github.com/apache/lucene-solr/pull/2120 > Not all types of index corruption garuntee a leader will "give up its > leadership" > - > > Key: SOLR-13237 > URL: https://issues.apache.org/jira/browse/SOLR-13237 > Project: Solr > Issue Type: Bug >Reporter: Chris M. Hostetter >Priority: Major > Attachments: SOLR-13237_logging.patch, log-fail-5D803D4699663918.txt, > log-fail-DEADBEEF.txt, log-pass-BEEFBEEF.txt, log-pass-FEEDBEEF.txt > > > While investigating failures from LeaderTragicEventTest, I've found some > reproducible situations where (externally introduced) index corruption can > cause a leader to reject updates, but not automatically give up it's > leadership. > See discussion in LUCENE-8692 – notably simon's comment on why/how some > things are explicitly not treated as tragic today for a disucssion of the > root cause. > *We may need/want to rethink & improve the situations where a leader gives up > leadership, above and beyond IW registering a tragic exception* > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mayya-sharipova opened a new pull request #2129: Fix format indent from 4 to 2 spaces
mayya-sharipova opened a new pull request #2129: URL: https://github.com/apache/lucene-solr/pull/2129 Files in org.apache.lucene.search.comparators package has a wrong indent of 4 spaces instead of 2. This patch fixes only the indent from 4 to correct 2 spaces This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-8673) o.a.s.search.facet classes not public/extendable
[
https://issues.apache.org/jira/browse/SOLR-8673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245498#comment-17245498
]
Tim Owen commented on SOLR-8673:
Good point, yes I found a similar message when I first ported this patch from
8.x to master, one method param changed from int to long, so that'll need
adjusting for the backport.
> o.a.s.search.facet classes not public/extendable
>
>
> Key: SOLR-8673
> URL: https://issues.apache.org/jira/browse/SOLR-8673
> Project: Solr
> Issue Type: Improvement
> Components: Facet Module
>Affects Versions: 5.4.1
>Reporter: Markus Jelsma
>Priority: Major
> Fix For: 6.2, 7.0
>
> Attachments: SOLR-8673.patch, SOLR-8673.patch
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> It is not easy to create a custom JSON facet function. A simple function
> based on AvgAgg quickly results in the following compilation failures:
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.3:compile (default-compile)
> on project openindex-solr: Compilation failure: Compilation failure:
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[22,36]
> org.apache.solr.search.facet.FacetContext is not public in
> org.apache.solr.search.facet; cannot be accessed from outside package
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[23,36]
> org.apache.solr.search.facet.FacetDoubleMerger is not public in
> org.apache.solr.search.facet; cannot be accessed from outside package
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[40,32]
> cannot find symbol
> [ERROR] symbol: class FacetContext
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[49,39]
> cannot find symbol
> [ERROR] symbol: class FacetDoubleMerger
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[54,43]
> cannot find symbol
> [ERROR] symbol: class Context
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg.Merger
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[41,16]
> cannot find symbol
> [ERROR] symbol: class AvgSlotAcc
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[46,12]
> incompatible types: i.o.s.search.facet.CustomAvgAgg.Merger cannot be
> converted to org.apache.solr.search.facet.FacetMerger
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[53,5]
> method does not override or implement a method from a supertype
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[60,5]
> method does not override or implement a method from a supertype
> {code}
> It seems lots of classes are tucked away in FacetModule, which we can't reach
> from outside.
> Originates from this thread:
> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201602.mbox/%3ccab_8yd9ldbg_0zxm_h1igkfm6bqeypd5ilyy7tty8cztscv...@mail.gmail.com%3E
> ( also available at
> https://lists.apache.org/thread.html/9fddcad3136ec908ce1c57881f8d3069e5d153f08b71f80f3e18d995%401455019826%40%3Csolr-user.lucene.apache.org%3E
> )
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14688) First party package implementation design
[ https://issues.apache.org/jira/browse/SOLR-14688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245479#comment-17245479 ] David Smiley commented on SOLR-14688: - I suspect there is something missing where the package manager needs to choose _a supported_ version of a package _for the local node_ that may be different for some other node since some other node may be running some newer/later Solr version. Am I wrong? I think this is really important because Tomas points out that Solr API changes that are binary incompatible (which will happen from time to time) will then be _impossible_ to upgrade Solr. I don't think it's reasonable for us to prevent binary incompatible changes; it's gonna happen. Assuming it's not possible (yet), I'm not sure how much of a blocker this is for package-ifying the contribs. > First party package implementation design > - > > Key: SOLR-14688 > URL: https://issues.apache.org/jira/browse/SOLR-14688 > Project: Solr > Issue Type: Improvement >Reporter: Noble Paul >Priority: Major > Labels: package, packagemanager > > Here's the design document for first party packages: > https://docs.google.com/document/d/1n7gB2JAdZhlJKFrCd4Txcw4HDkdk7hlULyAZBS-wXrE/edit?usp=sharing > Put differently, this is about package-ifying our "contribs". -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-15026) MiniSolrCloudCluster can inconsistently get confused about when it's using SSL
[
https://issues.apache.org/jira/browse/SOLR-15026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245463#comment-17245463
]
Timothy Potter commented on SOLR-15026:
---
Hi [~hossman] I actually think the right fix for this test class is to just add
the {{SuppressSSL}} annotation as you've already done. Alternatively, we could
pass the SSLConfig from the {{SolrTestCaseJ4}} to the JettyConfig builder such
as:
{{JettyConfig.builder().withSSLConfig(sslConfig.buildServerSSLConfig()).build()}}.
But, that's already being done in {{TestMiniSolrCloudClusterSSL}} for SSL
specific tests, seems unnecessary to add SSL randomization to
{{MiniSolrCloudClusterTest}}. If you agree, we can just mark this as won't fix
and leave the {{@SolrTestCaseJ4.SuppressSSL}} annotation in place or I can
update the test to pass the SSLConfig and remove the annotation.
> MiniSolrCloudCluster can inconsistently get confused about when it's using SSL
> --
>
> Key: SOLR-15026
> URL: https://issues.apache.org/jira/browse/SOLR-15026
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Chris M. Hostetter
>Assignee: Timothy Potter
>Priority: Major
>
> A new test added in SOLR-14934 caused the following reproducible failure to
> pop up on jenkins...
> {noformat}
> hossman@slate:~/lucene/dev [j11] [master] $ ./gradlew -p solr/test-framework/
> test --tests MiniSolrCloudClusterTest.testSolrHomeAndResourceLoaders
> -Dtests.seed=806A85748BD81F48 -Dtests.multiplier=2 -Dtests.slow=true
> -Dtests.locale=ln-CG -Dtests.timezone=Asia/Thimbu -Dtests.asserts=true
> -Dtests.file.encoding=UTF-8
> Starting a Gradle Daemon (subsequent builds will be faster)
> > Task :randomizationInfo
> Running tests with randomization seed: tests.seed=806A85748BD81F48
> > Task :solr:test-framework:test
> org.apache.solr.cloud.MiniSolrCloudClusterTest >
> testSolrHomeAndResourceLoaders FAILED
> org.apache.solr.client.solrj.SolrServerException: IOException occurred
> when talking to server at: https://127.0.0.1:38681/solr
> at
> __randomizedtesting.SeedInfo.seed([806A85748BD81F48:37548FA7602CB5FD]:0)
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:712)
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:269)
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
> at
> org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:390)
> at
> org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:360)
> at
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1168)
> at
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:931)
> at
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:865)
> at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:229)
> at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:246)
> at
> org.apache.solr.cloud.MiniSolrCloudClusterTest.testSolrHomeAndResourceLoaders(MiniSolrCloudClusterTest.java:125)
> ...
> Caused by:
> javax.net.ssl.SSLException: Unsupported or unrecognized SSL message
> at
> java.base/sun.security.ssl.SSLSocketInputRecord.handleUnknownRecord(SSLSocketInputRecord.java:439)
> {noformat}
> The problem sems to be that even though the MiniSolrCloudCluster being
> instantiated isn't _intentionally_ using any SSL randomization (it just uses
> {{JettyConfig.builder().build()}} the CloudSolrClient returned by
> {{cluster.getSolrClient()}} is evidnetly picking up the ranodmized SSL and
> trying to use it to talk to the cluster.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on pull request #2123: SOLR-10732: short circuit calls to searcher#numDocs when base is empty
dsmiley commented on pull request #2123: URL: https://github.com/apache/lucene-solr/pull/2123#issuecomment-740122634 Thanks Munendra, I'm glad you moved this from a patch file to GitHub PR -- way more conducive to getting a (good) review. I don't bother with patch files any more because I want reviewers, and patch files are a barrier. I think you already incorporated my suggestion in JIRA; it's good to see Michael's input. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14987) SolrStream ends up creating a new HttpSolrClient for every replica being queried instead of reusing for the same node
[
https://issues.apache.org/jira/browse/SOLR-14987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245421#comment-17245421
]
ASF subversion and git services commented on SOLR-14987:
Commit 8f769c6a82c6801a9c74b12131c1ac43ceb50827 in lucene-solr's branch
refs/heads/branch_8x from Timothy Potter
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=8f769c6 ]
SOLR-14987: Reuse HttpSolrClient per node vs. one per Solr core when using
CloudSolrStream (#2128)
> SolrStream ends up creating a new HttpSolrClient for every replica being
> queried instead of reusing for the same node
> -
>
> Key: SOLR-14987
> URL: https://issues.apache.org/jira/browse/SOLR-14987
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: streaming expressions
>Reporter: Timothy Potter
>Assignee: Timothy Potter
>Priority: Major
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Looking into some streaming expression performance issues when there are many
> collections with many shards being queried and I found that SolrStream's open
> method creates a new \{{HttpSolrClient}} for every replica being queried. For
> instance, in my test case, I have 10 collections with 100 shards each (rf=1)
> and I get 1000 HttpSolrClient instances in my SolrClientCache. If I reuse
> HttpSolrClient's per node hosting a replica (so 10 in my case), the query
> time for my expression drops by half (not too mention the reduced allocation
> load on the JVM).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (SOLR-14987) SolrStream ends up creating a new HttpSolrClient for every replica being queried instead of reusing for the same node
[
https://issues.apache.org/jira/browse/SOLR-14987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothy Potter updated SOLR-14987:
--
Fix Version/s: master (9.0)
8.8
Resolution: Fixed
Status: Resolved (was: Patch Available)
> SolrStream ends up creating a new HttpSolrClient for every replica being
> queried instead of reusing for the same node
> -
>
> Key: SOLR-14987
> URL: https://issues.apache.org/jira/browse/SOLR-14987
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: streaming expressions
>Reporter: Timothy Potter
>Assignee: Timothy Potter
>Priority: Major
> Fix For: 8.8, master (9.0)
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Looking into some streaming expression performance issues when there are many
> collections with many shards being queried and I found that SolrStream's open
> method creates a new \{{HttpSolrClient}} for every replica being queried. For
> instance, in my test case, I have 10 collections with 100 shards each (rf=1)
> and I get 1000 HttpSolrClient instances in my SolrClientCache. If I reuse
> HttpSolrClient's per node hosting a replica (so 10 in my case), the query
> time for my expression drops by half (not too mention the reduced allocation
> load on the JVM).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] thelabdude merged pull request #2128: SOLR-14987: Reuse HttpSolrClient per node vs. one per Solr core when using CloudSolrStream
thelabdude merged pull request #2128: URL: https://github.com/apache/lucene-solr/pull/2128 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] thelabdude opened a new pull request #2128: SOLR-14987: Reuse HttpSolrClient per node vs. one per Solr core when using CloudSolrStream
thelabdude opened a new pull request #2128: URL: https://github.com/apache/lucene-solr/pull/2128 # Description Backport of #2067 to `branch_8x` For collections with many shards (or aliases with many collections and some shards), `CloudSolrStream` will end up creating a new `HttpSolrClient` for every `SolrStream` it opens because the cache key is the full core URL, such as: `http://127.0.0.1:63460/solr/collection4_shard4_replica_n6/` In addition, `CloudSolrStream#getSlices` was calling `clusterState.getCollectionsMap()` which pre-emptively loads all `LazyCollectionRef` from ZK unnecessarily. This could cause an issue with clusters with many collections and slow down the streaming expression execution. # Solution In this PR, I've introduced a new ctor in `SolrStream` to pass the Replica's `baseUrl` and `core` as separate parameters. This leads to reusing the same `HttpSolrClient` for the same node because the cache key is now `http://127.0.0.1:63460/solr/`. I chose this new ctor approach because `CloudSolrStream` is not the only consumer of `SolrStream` and it knows how the list of URLs where constructed from cluster state, so it can safely make the decision about passing the core and reusing clients. When the request is sent to the remote core, we need to add the core name to the path. This happens in `SolrStream#constructParser`. This method was public and takes a SolrClient (even though SolrStream already has an HttpSolrClient created in the `open` method); I've changed the signature to be private and use the client opened in the `open` method. # Tests Added a new test `testCloudStreamClientCache` in `StreamingTest` to verify the SolrStreams created by the CloudStream have the correct baseUrl (without the core). # Checklist Please review the following and check all that apply: - [x] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms to the standards described there to the best of my ability. - [x] I have created a Jira issue and added the issue ID to my pull request title. - [x] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended) - [x] I have developed this patch against the `master` branch. - [x] I have run `./gradlew check`. - [x] I have added tests for my changes. - [ ] I have added documentation for the [Ref Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide) (for Solr changes only). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-8673) o.a.s.search.facet classes not public/extendable
[
https://issues.apache.org/jira/browse/SOLR-8673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245398#comment-17245398
]
Timothy Potter commented on SOLR-8673:
--
[~mkhl] it looks like this commit to 8x is causing precommit to fail, can you
take a look please? Probably just a matter of removing the `@Override`
annotation:
{code}
common.compile-test:
[javac] Compiling 1 source file to 8x/solr/build/solr-core/classes/test
[javac]
8x/solr/core/src/test/org/apache/solr/search/function/AggValueSourceTest.java:47:
error: method does not override or implement a method from a supertype
[javac] @Override
[javac] ^
[javac] 1 error
{code}
Also, looks like the status of this Jira needs to be updated, e.g. fixVersion,
etc.
> o.a.s.search.facet classes not public/extendable
>
>
> Key: SOLR-8673
> URL: https://issues.apache.org/jira/browse/SOLR-8673
> Project: Solr
> Issue Type: Improvement
> Components: Facet Module
>Affects Versions: 5.4.1
>Reporter: Markus Jelsma
>Priority: Major
> Fix For: 6.2, 7.0
>
> Attachments: SOLR-8673.patch, SOLR-8673.patch
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> It is not easy to create a custom JSON facet function. A simple function
> based on AvgAgg quickly results in the following compilation failures:
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.3:compile (default-compile)
> on project openindex-solr: Compilation failure: Compilation failure:
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[22,36]
> org.apache.solr.search.facet.FacetContext is not public in
> org.apache.solr.search.facet; cannot be accessed from outside package
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[23,36]
> org.apache.solr.search.facet.FacetDoubleMerger is not public in
> org.apache.solr.search.facet; cannot be accessed from outside package
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[40,32]
> cannot find symbol
> [ERROR] symbol: class FacetContext
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[49,39]
> cannot find symbol
> [ERROR] symbol: class FacetDoubleMerger
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[54,43]
> cannot find symbol
> [ERROR] symbol: class Context
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg.Merger
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[41,16]
> cannot find symbol
> [ERROR] symbol: class AvgSlotAcc
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[46,12]
> incompatible types: i.o.s.search.facet.CustomAvgAgg.Merger cannot be
> converted to org.apache.solr.search.facet.FacetMerger
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[53,5]
> method does not override or implement a method from a supertype
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[60,5]
> method does not override or implement a method from a supertype
> {code}
> It seems lots of classes are tucked away in FacetModule, which we can't reach
> from outside.
> Originates from this thread:
> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201602.mbox/%3ccab_8yd9ldbg_0zxm_h1igkfm6bqeypd5ilyy7tty8cztscv...@mail.gmail.com%3E
> ( also available at
> https://lists.apache.org/thread.html/9fddcad3136ec908ce1c57881f8d3069e5d153f08b71f80f3e18d995%401455019826%40%3Csolr-user.lucene.apache.org%3E
> )
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-15019) Replica placement API needs a way to fetch existing replica metrics
[
https://issues.apache.org/jira/browse/SOLR-15019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245374#comment-17245374
]
Andrzej Bialecki commented on SOLR-15019:
-
What common strongly-typed metrics we should expose in {{ShardValues}} ?
Previously we used just these:
* {{INDEX.sizeInBytes}} - for easier use I propose to expose only "size in GB"
as that is more practical for any calculations.
* {{QUERY./select.requestTimes:1minRate}} - this was used in the
{{SearchRateTrigger}}. It reflects the search load of the replica
* {{UPDATE./update.requestTimes:1minRate}} - this is new, it reflects the
indexing load of the replica.
Also, IMHO in addition to these strongly-typed metrics the {{ShardValues}}
interface should allow fetching arbitrary weakly-typed metrics.
> Replica placement API needs a way to fetch existing replica metrics
> ---
>
> Key: SOLR-15019
> URL: https://issues.apache.org/jira/browse/SOLR-15019
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Andrzej Bialecki
>Priority: Major
>
> Replica placement API was introduced in SOLR-14613. It offers a few sample
> (and simple) implementations of placement plugins.
> However, this API doesn't offer support for retrieving per-replica metrics,
> which are required for calculating more realistic placements. For example,
> when calculating placements for ADDREPLICA on an already existing collection
> the plugin should know what is the size of replica in order to avoid placing
> large replicas on nodes with insufficient free disk space.
> After discussing this with [~ilan] we propose the following additions to the
> API:
> * use the existing {{AttributeFetcher}} interface as a facade for retrieving
> per-replica values (currently it only retrieves per-node values)
> * add {{ShardValues}} interface to represent strongly-typed API for key
> metrics, such as replica size, number of docs, number of update and search
> requests.
> Plugins could then use this API like this:
> {code}
> AttributeFetcher attributeFetcher = ...
> SolrCollection solrCollection = ...
> Set metricNames = ...
> attributeFetcher.requestCollectionMetrics(solrCollection,
> solrCollection.getShardNames(), metricNames);
> AttributeValues attributeValues = attributeFetcher.fetchAttributes();
> ShardValues shardValues =
> attributeValues.getShardMetrics(solrCollection.getName(), shardName);
> int sizeInGB = shardValues.getSizeInGB(); // retrieves shard leader metrics
> int replicaSizeInGB = shardValues.getSizeInGB(replica);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-15019) Replica placement API needs a way to fetch existing replica metrics
[
https://issues.apache.org/jira/browse/SOLR-15019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245374#comment-17245374
]
Andrzej Bialecki edited comment on SOLR-15019 at 12/7/20, 5:46 PM:
---
What common strongly-typed metrics we should expose in {{ShardValues}} ?
Previously we used just these:
* {{INDEX.sizeInBytes}} - for easier use I propose to expose only "size in GB"
as that is more practical for any calculations.
* {{QUERY./select.requestTimes:1minRate}} - this was used in the
{{SearchRateTrigger}}. It reflects the search load of the replica
* {{UPDATE./update.requestTimes:1minRate}} - this is new, it reflects the
indexing load of the replica.
Also, IMHO in addition to these strongly-typed metrics the {{ShardValues}}
interface should allow fetching arbitrary weakly-typed replica metrics.
was (Author: ab):
What common strongly-typed metrics we should expose in {{ShardValues}} ?
Previously we used just these:
* {{INDEX.sizeInBytes}} - for easier use I propose to expose only "size in GB"
as that is more practical for any calculations.
* {{QUERY./select.requestTimes:1minRate}} - this was used in the
{{SearchRateTrigger}}. It reflects the search load of the replica
* {{UPDATE./update.requestTimes:1minRate}} - this is new, it reflects the
indexing load of the replica.
Also, IMHO in addition to these strongly-typed metrics the {{ShardValues}}
interface should allow fetching arbitrary weakly-typed metrics.
> Replica placement API needs a way to fetch existing replica metrics
> ---
>
> Key: SOLR-15019
> URL: https://issues.apache.org/jira/browse/SOLR-15019
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Andrzej Bialecki
>Priority: Major
>
> Replica placement API was introduced in SOLR-14613. It offers a few sample
> (and simple) implementations of placement plugins.
> However, this API doesn't offer support for retrieving per-replica metrics,
> which are required for calculating more realistic placements. For example,
> when calculating placements for ADDREPLICA on an already existing collection
> the plugin should know what is the size of replica in order to avoid placing
> large replicas on nodes with insufficient free disk space.
> After discussing this with [~ilan] we propose the following additions to the
> API:
> * use the existing {{AttributeFetcher}} interface as a facade for retrieving
> per-replica values (currently it only retrieves per-node values)
> * add {{ShardValues}} interface to represent strongly-typed API for key
> metrics, such as replica size, number of docs, number of update and search
> requests.
> Plugins could then use this API like this:
> {code}
> AttributeFetcher attributeFetcher = ...
> SolrCollection solrCollection = ...
> Set metricNames = ...
> attributeFetcher.requestCollectionMetrics(solrCollection,
> solrCollection.getShardNames(), metricNames);
> AttributeValues attributeValues = attributeFetcher.fetchAttributes();
> ShardValues shardValues =
> attributeValues.getShardMetrics(solrCollection.getName(), shardName);
> int sizeInGB = shardValues.getSizeInGB(); // retrieves shard leader metrics
> int replicaSizeInGB = shardValues.getSizeInGB(replica);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] gf2121 commented on pull request #2113: LUCENE-9629: Use computed masks
gf2121 commented on pull request #2113:
URL: https://github.com/apache/lucene-solr/pull/2113#issuecomment-740050981
There are still several `mask32()` called in `decodeSlow()`, I have fixed
it in the newest commit and checked again to make sure that `mask8()`,
`mask16()` and `mask32()` are only called in the `static{}` code block.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-9628) Make sure to account for ScoreMode.TOP_DOCS in queries
[
https://issues.apache.org/jira/browse/LUCENE-9628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245343#comment-17245343
]
Mayya Sharipova edited comment on LUCENE-9628 at 12/7/20, 5:04 PM:
---
I've create a PR to address the discussed points.
There are other occurrences of checks of scoreMode in boolean scores that we
should probably improve to account for other score modes, but I don't know this
code well enough to make this change.
was (Author: mayyas):
I've create a PR to address the discussed points.
There are other occurrences of checks of scoreMode in boolean scores that we
should probably improve to account for other score modes, but I don't know this
code well enough to make the change.
> Make sure to account for ScoreMode.TOP_DOCS in queries
> --
>
> Key: LUCENE-9628
> URL: https://issues.apache.org/jira/browse/LUCENE-9628
> Project: Lucene - Core
> Issue Type: Test
> Components: core/search
>Reporter: Julie Tibshirani
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I noticed a few places where we are directly check the {{ScoreMode}} type
> that should perhaps be generalized. These could affect whether numeric sort
> optimization is applied:
> * In {{BooleanWeight#bulkScorer}}, we check if score mode is {{TOP_SCORES}}
> and if so, force non-bulk scoring. Should we expand this to include modes
> like {{TOP_DOCS}}?
> * In {{ConstantScoreQuery}}, we create the delegate weight with a hardcoded
> {{COMPLETE_NO_SCORES}}. I'm not sure it actually causes problems, but it
> seems like this doesn't handle {{TOP_DOCS}} correctly.
> Apologies this issue isn’t more precise – I am not up-to-speed on the numeric
> sort optimization but wanted to raise these in case they’re helpful.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9628) Make sure to account for ScoreMode.TOP_DOCS in queries
[
https://issues.apache.org/jira/browse/LUCENE-9628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245343#comment-17245343
]
Mayya Sharipova commented on LUCENE-9628:
-
I've create a PR to address the discussed points.
There are other occurrences of checks of scoreMode in boolean scores that we
should probably improve to account for other score modes, but I don't know this
code well enough to make the change.
> Make sure to account for ScoreMode.TOP_DOCS in queries
> --
>
> Key: LUCENE-9628
> URL: https://issues.apache.org/jira/browse/LUCENE-9628
> Project: Lucene - Core
> Issue Type: Test
> Components: core/search
>Reporter: Julie Tibshirani
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> I noticed a few places where we are directly check the {{ScoreMode}} type
> that should perhaps be generalized. These could affect whether numeric sort
> optimization is applied:
> * In {{BooleanWeight#bulkScorer}}, we check if score mode is {{TOP_SCORES}}
> and if so, force non-bulk scoring. Should we expand this to include modes
> like {{TOP_DOCS}}?
> * In {{ConstantScoreQuery}}, we create the delegate weight with a hardcoded
> {{COMPLETE_NO_SCORES}}. I'm not sure it actually causes problems, but it
> seems like this doesn't handle {{TOP_DOCS}} correctly.
> Apologies this issue isn’t more precise – I am not up-to-speed on the numeric
> sort optimization but wanted to raise these in case they’re helpful.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #2127: LUCENE-9633: Improve match highlighter behavior for degenerate intervals
dweiss commented on a change in pull request #2127:
URL: https://github.com/apache/lucene-solr/pull/2127#discussion_r537669124
##
File path:
lucene/highlighter/src/test/org/apache/lucene/search/matchhighlight/TestMatchRegionRetriever.java
##
@@ -361,6 +374,41 @@ public void testIntervalQueries() throws IOException {
);
}
+ @Test
+ public void testDegenerateIntervalsWithPositions() throws IOException {
+testDegenerateIntervals(FLD_TEXT_POS);
+ }
+
+ @Test @AwaitsFix(bugUrl =
"https://issues.apache.org/jira/browse/LUCENE-9634: " +
Review comment:
@romseygeek Alan - this may be interesting for you. Is this how it's
supposed to work with fields with offsets? If you enable this test and compare
how positions only vs. positions+offsets field behaves you'll see what I mean -
the stored offsets actually cause an incorrect highlight in this case.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss opened a new pull request #2127: LUCENE-9633: Improve match highlighter behavior for degenerate intervals
dweiss opened a new pull request #2127: URL: https://github.com/apache/lucene-solr/pull/2127 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mayya-sharipova opened a new pull request #2126: LUCENE-9628 ScoreMode.TOP_DOCS in bool queries
mayya-sharipova opened a new pull request #2126: URL: https://github.com/apache/lucene-solr/pull/2126 Account for ScoreMode.TOP_DOCS and TOP_DOCS_WITH_SCORES in boolean queries LUCENE-9280 introduced new ScoreModes of TOP_DOCS and TOP_DOCS_WITH_SCORES, but some boolean queries were not considering these score modes. This patch fixes some of these cases and also enhances definition for these score modes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9634) Highlighting of degenerate spans on fields *with offsets* doesn't work properly
[
https://issues.apache.org/jira/browse/LUCENE-9634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dawid Weiss updated LUCENE-9634:
Description: Match highlighter works fine with degenerate interval
positions when {{OffsetsFromPositions}} strategy is used to compute offsets but
will show incorrect offset ranges if offsets are read from directly from the
{{MatchIterator}} ({{OffsetsFromMatchIterator}}).
> Highlighting of degenerate spans on fields *with offsets* doesn't work
> properly
> ---
>
> Key: LUCENE-9634
> URL: https://issues.apache.org/jira/browse/LUCENE-9634
> Project: Lucene - Core
> Issue Type: Sub-task
>Reporter: Dawid Weiss
>Assignee: Dawid Weiss
>Priority: Minor
>
> Match highlighter works fine with degenerate interval positions when
> {{OffsetsFromPositions}} strategy is used to compute offsets but will show
> incorrect offset ranges if offsets are read from directly from the
> {{MatchIterator}} ({{OffsetsFromMatchIterator}}).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9634) Highlighting of degenerate spans on fields *with offsets* doesn't work properly
Dawid Weiss created LUCENE-9634: --- Summary: Highlighting of degenerate spans on fields *with offsets* doesn't work properly Key: LUCENE-9634 URL: https://issues.apache.org/jira/browse/LUCENE-9634 Project: Lucene - Core Issue Type: Sub-task Reporter: Dawid Weiss Assignee: Dawid Weiss -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9633) Improve match highlighter behavior for degenerate intervals (on non-existing positions)
Dawid Weiss created LUCENE-9633:
---
Summary: Improve match highlighter behavior for degenerate
intervals (on non-existing positions)
Key: LUCENE-9633
URL: https://issues.apache.org/jira/browse/LUCENE-9633
Project: Lucene - Core
Issue Type: Improvement
Reporter: Dawid Weiss
Assignee: Dawid Weiss
Interval functions can produce match spans on non-existing or otherwise
degenerate token positions. For example,
{code}
extend(foo 5 5)
{code}
would create an interval to the left and right of each term foo, regardless of
whether such positions actually exist in the token stream.
This issue improves match highlighter to still work in such cases. This is
actually fun to play with as you can highlight and visualize actual interval
spans even for functions that expand or manipulate other sources' context.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14987) SolrStream ends up creating a new HttpSolrClient for every replica being queried instead of reusing for the same node
[
https://issues.apache.org/jira/browse/SOLR-14987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245312#comment-17245312
]
ASF subversion and git services commented on SOLR-14987:
Commit 30e5e38336de49433a7ecc60fb169c2426278565 in lucene-solr's branch
refs/heads/master from Timothy Potter
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=30e5e38 ]
SOLR-14987: Reuse HttpSolrClient per node vs. one per Solr core when using
CloudSolrStream (#2067)
> SolrStream ends up creating a new HttpSolrClient for every replica being
> queried instead of reusing for the same node
> -
>
> Key: SOLR-14987
> URL: https://issues.apache.org/jira/browse/SOLR-14987
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: streaming expressions
>Reporter: Timothy Potter
>Assignee: Timothy Potter
>Priority: Major
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Looking into some streaming expression performance issues when there are many
> collections with many shards being queried and I found that SolrStream's open
> method creates a new \{{HttpSolrClient}} for every replica being queried. For
> instance, in my test case, I have 10 collections with 100 shards each (rf=1)
> and I get 1000 HttpSolrClient instances in my SolrClientCache. If I reuse
> HttpSolrClient's per node hosting a replica (so 10 in my case), the query
> time for my expression drops by half (not too mention the reduced allocation
> load on the JVM).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] thelabdude merged pull request #2067: SOLR-14987: Reuse HttpSolrClient per node vs. one per Solr core when using CloudSolrStream
thelabdude merged pull request #2067: URL: https://github.com/apache/lucene-solr/pull/2067 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mikemccand commented on pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory
mikemccand commented on pull request #2052:
URL: https://github.com/apache/lucene-solr/pull/2052#issuecomment-739974498
Hmm, `gradlew precommit` is angry:
```
> Task :lucene:misc:compileJava
/l/directio/lucene/misc/src/java/org/apache/lucene/misc/store/DirectIODirectory.java:206:
warning: ExtendedOpenOption is internal proprietary API and may be removed in
a future release
com.sun.nio.file.ExtendedOpenOption.DIRECT);
^
/l/directio/lucene/misc/src/java/org/apache/lucene/misc/store/DirectIODirectory.java:325:
warning: ExtendedOpenOption is internal proprietary API and may be removed in
a future release
channel = FileChannel.open(path, StandardOpenOption.READ,
com.sun.nio.file.ExtendedOpenOption.DIRECT);
^
error: warnings found and -Werror specified
> Task :lucene:sandbox:ecjLintMain
> Task :lucene:demo:ecjLintMain
> Task :lucene:codecs:ecjLintMain
> Task :lucene:misc:compileJava FAILED
1 error
2 warnings
```
Maybe we need to add something like `@SuppressWarnings("sunapi")` annotation?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mikemccand commented on a change in pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory
mikemccand commented on a change in pull request #2052:
URL: https://github.com/apache/lucene-solr/pull/2052#discussion_r537569680
##
File path:
lucene/misc/src/java/org/apache/lucene/misc/store/DirectIODirectory.java
##
@@ -154,14 +146,43 @@ public IndexOutput createOutput(String name, IOContext
context) throws IOExcepti
if (context.context != Context.MERGE ||
context.mergeInfo.estimatedMergeBytes < minBytesDirect) {
return delegate.createOutput(name, context);
} else {
- return new NativeUnixIndexOutput(getDirectory().resolve(name), name,
mergeBufferSize);
+ return new DirectIOIndexOutput(getDirectory().resolve(name), name,
mergeBufferSize);
}
}
- @SuppressForbidden(reason = "java.io.File: native API requires old-style
FileDescriptor")
- private final static class NativeUnixIndexOutput extends IndexOutput {
+ @Override
+ public void deleteFile(String name) throws IOException {
+delegate.deleteFile(name);
+ }
+
+ @Override
+ public Set getPendingDeletions() throws IOException {
+return delegate.getPendingDeletions();
+ }
+
+ @Override
+ public String[] listAll() throws IOException {
+return delegate.listAll();
+ }
+
+ @Override
+ public long fileLength(String name) throws IOException {
+return delegate.fileLength(name);
+ }
+
+ @Override
+ public void rename(String source, String dest) throws IOException {
+delegate.rename(source, dest);
+ }
+
+ @Override
+ public void close() throws IOException {
+delegate.close();
Review comment:
Maybe use `IOUtils.close(...)` here instead? It will try to close both
even if one of them throws exception.
##
File path:
lucene/misc/src/java/org/apache/lucene/misc/store/DirectIODirectory.java
##
@@ -154,14 +146,43 @@ public IndexOutput createOutput(String name, IOContext
context) throws IOExcepti
if (context.context != Context.MERGE ||
context.mergeInfo.estimatedMergeBytes < minBytesDirect) {
return delegate.createOutput(name, context);
} else {
- return new NativeUnixIndexOutput(getDirectory().resolve(name), name,
mergeBufferSize);
+ return new DirectIOIndexOutput(getDirectory().resolve(name), name,
mergeBufferSize);
}
}
- @SuppressForbidden(reason = "java.io.File: native API requires old-style
FileDescriptor")
- private final static class NativeUnixIndexOutput extends IndexOutput {
+ @Override
+ public void deleteFile(String name) throws IOException {
+delegate.deleteFile(name);
+ }
+
+ @Override
+ public Set getPendingDeletions() throws IOException {
Review comment:
I wonder if we should simply extend `FilterDirectory` here? Let that
class (designed for such delegation) handle delegating all existing `Directory`
methods? It would lower the risk that a new `Directory` method fails to get
delegated here ...
But, we can do this as followon issue.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mikemccand commented on a change in pull request #2052: LUCENE-8982: Make NativeUnixDirectory pure java with FileChannel direct IO flag, and rename to DirectIODirectory
mikemccand commented on a change in pull request #2052:
URL: https://github.com/apache/lucene-solr/pull/2052#discussion_r537561523
##
File path:
lucene/misc/src/java/org/apache/lucene/misc/store/DirectIODirectory.java
##
@@ -51,16 +50,9 @@
* to the provided Directory instance.
*
* See Overview
+ * href="{@docRoot}/overview-summary.html#DirectIODirectory">Overview
* for more details.
*
- * To use this you must compile
- * NativePosixUtil.cpp (exposes Linux-specific APIs through
- * JNI) for your platform, by running ./gradlew build, and then
putting the resulting
- * libLuceneNativeIO.so or libLuceneNativeIO.dylib
- * (from lucene/misc/native/build/lib/release/platform/) onto
your dynamic
- * linker search path.
- *
* WARNING: this code is very new and quite easily
* could contain horrible bugs. For example, here's one
* known issue: if you use seek in IndexOutput, and then
Review comment:
Let's leave the warning for now? Because this was JNI code, bringing it
risks like SIGSEGV / memory leaks / etc. (even though in this instances those
risks were very small), and requiring C++ compilation to build per-platform,
very few users tried it out.
Now that it will be pure java (thank you @zacharymorn!!), it should see more
usage.
Especially for search applications that do concurrent indexing and searching
on a single box (e.g. Elasticsearch!), this Directory should be a huge win to
bring down interference of a large merge running on a box and impacting queries
as hot pages are swapped out.
So, let's wait and see if we can remove this warning at a later date, if
users really do use this and have good results?
Also, this directory requires some tuning (the buffer size).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-15020) Fix Typo in 8.7 Release Notes
[ https://issues.apache.org/jira/browse/SOLR-15020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cassandra Targett resolved SOLR-15020. -- Fix Version/s: 8.8 Resolution: Fixed Fixed with https://github.com/apache/lucene-solr/commit/e3572d0c4ddff877324750c9d21b0618aed16f1a (did not have Jira issue) > Fix Typo in 8.7 Release Notes > - > > Key: SOLR-15020 > URL: https://issues.apache.org/jira/browse/SOLR-15020 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: documentation >Affects Versions: 8.7 >Reporter: Scott Vanderbilt >Priority: Trivial > Fix For: 8.8 > > > Line 46 should refer to 8.7, not 8.6.x: > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-15021) Fix Typo in 8.7 Release Notes
[ https://issues.apache.org/jira/browse/SOLR-15021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cassandra Targett resolved SOLR-15021. -- Resolution: Duplicate > Fix Typo in 8.7 Release Notes > - > > Key: SOLR-15021 > URL: https://issues.apache.org/jira/browse/SOLR-15021 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: documentation >Affects Versions: 8.7 >Reporter: Scott Vanderbilt >Priority: Trivial > Labels: documentation > > Line 46 should refer to 8.7, not 8.6.x: > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] magibney commented on a change in pull request #2123: SOLR-10732: short circuit calls to searcher#numDocs when base is empty
magibney commented on a change in pull request #2123:
URL: https://github.com/apache/lucene-solr/pull/2123#discussion_r537551055
##
File path: solr/core/src/java/org/apache/solr/request/SimpleFacets.java
##
@@ -325,6 +329,9 @@ public void getFacetQueryCount(ParsedParams parsed,
NamedList res) thro
* @see FacetParams#FACET_QUERY
*/
public int getGroupedFacetQueryCount(Query facetQuery, DocSet docSet) throws
IOException {
+if (docSet.size() == 0) {
+ return 0;
+}
Review comment:
Both callers of this method (`SimpleFacets.getFacetQueryCount(...)` and
`RangeFacetProcessor.rangeCount(...)` check for the base `docSet.size()==0`
case independently. Could perhaps remove this redundant check (or replace with
`assert docSet.size() > 0`)?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] magibney commented on a change in pull request #2123: SOLR-10732: short circuit calls to searcher#numDocs when base is empty
magibney commented on a change in pull request #2123:
URL: https://github.com/apache/lucene-solr/pull/2123#discussion_r537520652
##
File path: solr/core/src/java/org/apache/solr/request/SimpleFacets.java
##
@@ -903,7 +910,7 @@ public void execute(Runnable r) {
private int numDocs(String term, final SchemaField sf, final FieldType ft,
final DocSet baseDocset) {
try {
- return searcher.numDocs(ft.getFieldQuery(null, sf, term), baseDocset);
+ return baseDocset.size() == 0? 0:
searcher.numDocs(ft.getFieldQuery(null, sf, term), baseDocset);
Review comment:
Line 903 (immediately above) is the only caller of this private method.
Would it make sense to do the check in only one of these places? Maybe prefer
the higher-level (line 903) check, where the check could also be used to bypass
`termCountEntries.sorted(...)`? Could optionally leave `assert
baseDocset.size() > 0` here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-15022) RefGuide documentation for /cluster/plugin API
[
https://issues.apache.org/jira/browse/SOLR-15022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245208#comment-17245208
]
ASF subversion and git services commented on SOLR-15022:
Commit 637afadeaaeabd28020e0c47b0ff700fc82571f4 in lucene-solr's branch
refs/heads/master from Andrzej Bialecki
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=637afad ]
SOLR-15022: fix page-children declaration.
> RefGuide documentation for /cluster/plugin API
> --
>
> Key: SOLR-15022
> URL: https://issues.apache.org/jira/browse/SOLR-15022
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Plugin system
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
> Fix For: master (9.0)
>
>
> The {{/cluster/plugin}} API needs user-level documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] munendrasn commented on a change in pull request #2123: SOLR-10732: short circuit calls to searcher#numDocs when base is empty
munendrasn commented on a change in pull request #2123:
URL: https://github.com/apache/lucene-solr/pull/2123#discussion_r537503391
##
File path: solr/core/src/java/org/apache/solr/search/facet/FacetProcessor.java
##
@@ -419,7 +419,7 @@ void fillBucket(SimpleOrderedMap bucket, Query q,
DocSet result, boolean
}
count = result.size(); // don't really need this if we are skipping,
but it's free.
} else {
- if (q == null) {
+ if (q == null || fcontext.base.size() == 0) {
Review comment:
Will remove this before committing
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] magibney commented on a change in pull request #2123: SOLR-10732: short circuit calls to searcher#numDocs when base is empty
magibney commented on a change in pull request #2123:
URL: https://github.com/apache/lucene-solr/pull/2123#discussion_r537501831
##
File path: solr/core/src/java/org/apache/solr/search/facet/FacetProcessor.java
##
@@ -419,7 +419,7 @@ void fillBucket(SimpleOrderedMap bucket, Query q,
DocSet result, boolean
}
count = result.size(); // don't really need this if we are skipping,
but it's free.
} else {
- if (q == null) {
+ if (q == null || fcontext.base.size() == 0) {
Review comment:
The query is already built at this point, so I don't think this
particular change actually helps wrt SOLR-10732? (and the `base.size()==0` case
is already trivially optimized in `SolrIndexSearcher.numDocs(Query, DocSet)`)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-15022) RefGuide documentation for /cluster/plugin API
[
https://issues.apache.org/jira/browse/SOLR-15022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245196#comment-17245196
]
ASF subversion and git services commented on SOLR-15022:
Commit 8dcaa6c6d33c729b92a7415d20f30ccfe7a8c523 in lucene-solr's branch
refs/heads/master from Andrzej Bialecki
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=8dcaa6c ]
SOLR-15022: Add RefGuide documentation for cluster plugins.
> RefGuide documentation for /cluster/plugin API
> --
>
> Key: SOLR-15022
> URL: https://issues.apache.org/jira/browse/SOLR-15022
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Plugin system
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
>
> The {{/cluster/plugin}} API needs user-level documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-15022) RefGuide documentation for /cluster/plugin API
[
https://issues.apache.org/jira/browse/SOLR-15022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrzej Bialecki resolved SOLR-15022.
-
Fix Version/s: master (9.0)
Resolution: Fixed
> RefGuide documentation for /cluster/plugin API
> --
>
> Key: SOLR-15022
> URL: https://issues.apache.org/jira/browse/SOLR-15022
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Plugin system
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
> Fix For: master (9.0)
>
>
> The {{/cluster/plugin}} API needs user-level documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-8673) o.a.s.search.facet classes not public/extendable
[
https://issues.apache.org/jira/browse/SOLR-8673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245130#comment-17245130
]
ASF subversion and git services commented on SOLR-8673:
---
Commit 8d63769795bf25fde29f1762047339d8107b550f in lucene-solr's branch
refs/heads/branch_8x from Mikhail Khludnev
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=8d63769 ]
SOLR-8673: Enable custom aggregate functions by opening up FacetContext (#2125)
Prior to this change, it was not feasible to write a custom aggregate function
in plugin code
because the FacetContext class did not expose its fields for access outside its
package. Also
some of the useful abstract classes for SlotAcc were package-private too
Co-authored-by: Tim Owen
> o.a.s.search.facet classes not public/extendable
>
>
> Key: SOLR-8673
> URL: https://issues.apache.org/jira/browse/SOLR-8673
> Project: Solr
> Issue Type: Improvement
> Components: Facet Module
>Affects Versions: 5.4.1
>Reporter: Markus Jelsma
>Priority: Major
> Fix For: 6.2, 7.0
>
> Attachments: SOLR-8673.patch, SOLR-8673.patch
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> It is not easy to create a custom JSON facet function. A simple function
> based on AvgAgg quickly results in the following compilation failures:
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.3:compile (default-compile)
> on project openindex-solr: Compilation failure: Compilation failure:
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[22,36]
> org.apache.solr.search.facet.FacetContext is not public in
> org.apache.solr.search.facet; cannot be accessed from outside package
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[23,36]
> org.apache.solr.search.facet.FacetDoubleMerger is not public in
> org.apache.solr.search.facet; cannot be accessed from outside package
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[40,32]
> cannot find symbol
> [ERROR] symbol: class FacetContext
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[49,39]
> cannot find symbol
> [ERROR] symbol: class FacetDoubleMerger
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[54,43]
> cannot find symbol
> [ERROR] symbol: class Context
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg.Merger
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[41,16]
> cannot find symbol
> [ERROR] symbol: class AvgSlotAcc
> [ERROR] location: class i.o.s.search.facet.CustomAvgAgg
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[46,12]
> incompatible types: i.o.s.search.facet.CustomAvgAgg.Merger cannot be
> converted to org.apache.solr.search.facet.FacetMerger
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[53,5]
> method does not override or implement a method from a supertype
> [ERROR]
> /home/markus/projects/openindex/solr/trunk/src/main/java/i.o.s.search/facet/CustomAvgAgg.java:[60,5]
> method does not override or implement a method from a supertype
> {code}
> It seems lots of classes are tucked away in FacetModule, which we can't reach
> from outside.
> Originates from this thread:
> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201602.mbox/%3ccab_8yd9ldbg_0zxm_h1igkfm6bqeypd5ilyy7tty8cztscv...@mail.gmail.com%3E
> ( also available at
> https://lists.apache.org/thread.html/9fddcad3136ec908ce1c57881f8d3069e5d153f08b71f80f3e18d995%401455019826%40%3Csolr-user.lucene.apache.org%3E
> )
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mkhludnev merged pull request #2125: SOLR-8673: Enable custom aggregate functions by opening up FacetContext
mkhludnev merged pull request #2125: URL: https://github.com/apache/lucene-solr/pull/2125 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mkhludnev commented on pull request #2125: SOLR-8673: Enable custom aggregate functions by opening up FacetContext
mkhludnev commented on pull request #2125: URL: https://github.com/apache/lucene-solr/pull/2125#issuecomment-739832339 @timatbw, cherry picking https://github.com/apache/lucene-solr/commit/510f96c69c559c53a4f62281b48937a5f887dcc2 to 8x. I have to add `throws IOException` into the test. Should be fine. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] mkhludnev opened a new pull request #2125: SOLR-8673: Enable custom aggregate functions by opening up FacetContext
mkhludnev opened a new pull request #2125: URL: https://github.com/apache/lucene-solr/pull/2125 Prior to this change, it was not feasible to write a custom aggregate function in plugin code because the FacetContext class did not expose its fields for access outside its package. Also some of the useful abstract classes for SlotAcc were package-private too # Description Please provide a short description of the changes you're making with this pull request. # Solution Please provide a short description of the approach taken to implement your solution. # Tests Please describe the tests you've developed or run to confirm this patch implements the feature or solves the problem. # Checklist Please review the following and check all that apply: - [ ] I have reviewed the guidelines for [How to Contribute](https://wiki.apache.org/solr/HowToContribute) and my code conforms to the standards described there to the best of my ability. - [ ] I have created a Jira issue and added the issue ID to my pull request title. - [ ] I have given Solr maintainers [access](https://help.github.com/en/articles/allowing-changes-to-a-pull-request-branch-created-from-a-fork) to contribute to my PR branch. (optional but recommended) - [ ] I have developed this patch against the `master` branch. - [ ] I have run `./gradlew check`. - [ ] I have added tests for my changes. - [ ] I have added documentation for the [Ref Guide](https://github.com/apache/lucene-solr/tree/master/solr/solr-ref-guide) (for Solr changes only). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14182) Move metric reporters config from solr.xml to ZK cluster properties
[ https://issues.apache.org/jira/browse/SOLR-14182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17245123#comment-17245123 ] Andrzej Bialecki commented on SOLR-14182: - Ok, I see your point Tomas - let's work on SOLR-14843 first. > Move metric reporters config from solr.xml to ZK cluster properties > --- > > Key: SOLR-14182 > URL: https://issues.apache.org/jira/browse/SOLR-14182 > Project: Solr > Issue Type: Improvement >Affects Versions: 8.4 >Reporter: Andrzej Bialecki >Assignee: Andrzej Bialecki >Priority: Major > > Metric reporters are currently configured statically in solr.xml, which makes > it difficult to change dynamically or in a containerized environment. > We should move this section to ZK /cluster.properties and add a back-compat > migration shim. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (SOLR-15016) Replica placement plugins should use container plugins API / configs
[
https://issues.apache.org/jira/browse/SOLR-15016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrzej Bialecki resolved SOLR-15016.
-
Fix Version/s: master (9.0)
Resolution: Fixed
> Replica placement plugins should use container plugins API / configs
>
>
> Key: SOLR-15016
> URL: https://issues.apache.org/jira/browse/SOLR-15016
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Plugin system
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
> Fix For: master (9.0)
>
> Time Spent: 6h 20m
> Remaining Estimate: 0h
>
> Replica placement API currently uses its own way of loading plugin
> implementations and their config.
> Instead it should use a more robust mechanism supported by
> {{ContainerPluginsAPI}} and {{ContainerPluginsRegistry}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] muse-dev[bot] commented on a change in pull request #2124: SOLR-8673: Open JSON Facet for extension
muse-dev[bot] commented on a change in pull request #2124:
URL: https://github.com/apache/lucene-solr/pull/2124#discussion_r537321555
##
File path: dev-tools/scripts/reproduceJenkinsFailures.py
##
@@ -106,7 +108,9 @@ def fetchAndParseJenkinsLog(url, numRetries):
tests = {}
print('[repro] Jenkins log URL: %s\n' % url)
try:
-with urllib.request.urlopen(url) as consoleText:
+# HTTPS fails at certificate validation, see LUCENE-9412, PEP-476
+context = ssl._create_unverified_context()
Review comment:
*blacklist:* By default, Python will create a secure, verified ssl
context for use in such classes as HTTPSConnection. However, it still allows
using an insecure context via the _create_unverified_context that reverts to
the previous behavior that does not validate certificates or perform hostname
checks.
##
File path: dev-tools/scripts/githubPRs.py
##
@@ -0,0 +1,153 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""
+Simple script that queries Github for all open PRs, then finds the ones without
+issue number in title, and the ones where the linked JIRA is already closed
+"""
+
+import os
+import sys
+sys.path.append(os.path.dirname(__file__))
+import argparse
+import json
+import re
+from github import Github
+from jira import JIRA
+from datetime import datetime
+from time import strftime
+try:
+ from jinja2 import Environment, BaseLoader
+ can_do_html = True
+except:
+ can_do_html = False
+
+def read_config():
+ parser = argparse.ArgumentParser(description='Find open Pull Requests that
need attention')
+ parser.add_argument('--json', action='store_true', default=False,
help='Output as json')
+ parser.add_argument('--html', action='store_true', default=False,
help='Output as html')
+ parser.add_argument('--token', help='Github access token in case you query
too often anonymously')
+ newconf = parser.parse_args()
+ return newconf
+
+
+def out(text):
+ global conf
+ if not (conf.json or conf.html):
+print(text)
+
+def make_html(dict):
+ if not can_do_html:
+print ("ERROR: Cannot generate HTML. Please install jinja2")
+sys.exit(1)
+ global conf
+ template = Environment(loader=BaseLoader).from_string("""
Review comment:
*jinja2_autoescape_false:* By default, jinja2 sets autoescape to False.
Consider using autoescape=True or use the select_autoescape function to
mitigate XSS vulnerabilities.
##
File path:
lucene/analysis/common/src/java/org/apache/lucene/analysis/classic/ClassicFilter.java
##
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.analysis.classic;
+
+
+import org.apache.lucene.analysis.TokenFilter;
+import org.apache.lucene.analysis.TokenStream;
+import org.apache.lucene.analysis.tokenattributes.CharTermAttribute;
+import org.apache.lucene.analysis.tokenattributes.TypeAttribute;
+
+/** Normalizes tokens extracted with {@link ClassicTokenizer}. */
+
+public class ClassicFilter extends TokenFilter {
+
+ /** Construct filtering in. */
+ public ClassicFilter(TokenStream in) {
+super(in);
+ }
+
+ private static final String APOSTROPHE_TYPE =
ClassicTokenizer.TOKEN_TYPES[ClassicTokenizer.APOSTROPHE];
+ private static final String ACRONYM_TYPE =
ClassicTokenizer.TOKEN_TYPES[ClassicTokenizer.ACRONYM];
+
+ // this filters uses attribute type
+ private final TypeAttribute typeAtt =
