[jira] [Created] (LUCENE-9130) Failed to match when create PhraseQuery with terms analyzed from long query text
Chen Zhixiang created LUCENE-9130: - Summary: Failed to match when create PhraseQuery with terms analyzed from long query text Key: LUCENE-9130 URL: https://issues.apache.org/jira/browse/LUCENE-9130 Project: Lucene - Core Issue Type: Bug Components: core/search Affects Versions: 8.4 Reporter: Chen Zhixiang Attachments: LongTextFieldSearchTest.java When i use a long text (which is euqual to doc's StringField at indexing time) to build a PhraseQuery, i cannot match the document. But BooleanQuery with MUST/AND mode successes. long query text is a address string: "申长路988弄虹桥万科中心地下停车场LG2层2179-2184车位(锡虹路入,LG1层开到底下LG2)" test case is attached. logs: 15:46:11.940 [main] INFO test.LongTextFieldSearchTest - indexed terms: 开, 层, 心, 弄, 万, 停车场, 地下, 科, 虹桥, 底下, 锡, 入, 2184, 中, 路, 到, 1, 2, 申, 2179, 车位, 988, 虹, lg, 长 15:46:11.956 [main] INFO test.LongTextFieldSearchTest - terms: 申, 长, 路, 988, 弄, 虹桥, 万, 科, 中, 心, 地下, 停车场, lg, 2, 层, 2179, 2184, 车位, 锡, 虹, 路, 入, lg, 1, 层, 开, 到, 底下, lg, 2 15:46:11.962 [main] INFO test.LongTextFieldSearchTest - query: +(+address:申 +address:长 +address:路 +address:988 +address:弄 +address:虹桥 +address:万 +address:科 +address:中 +address:心 +address:地下 +address:停车场 +address:lg +address:2 +address:层 +address:2179 +address:2184 +address:车位 +address:锡 +address:虹 +address:路 +address:入 +address:lg +address:1 +address:层 +address:开 +address:到 +address:底下 +address:lg +address:2) 15:46:11.988 [main] INFO test.LongTextFieldSearchTest - results.totalHits.value=1 15:46:12.181 [main] INFO test.LongTextFieldSearchTest - indexed terms: 开, 层, 心, 弄, 万, 停车场, 地下, 科, 虹桥, 底下, 锡, 入, 2184, 中, 路, 到, 1, 2, 申, 2179, 车位, 988, 虹, lg, 长 15:46:12.185 [main] INFO test.LongTextFieldSearchTest - terms: 申, 长, 路, 988, 弄, 虹桥, 万, 科, 中, 心, 地下, 停车场, lg, 2, 层, 2179, 2184, 车位, 锡, 虹, 路, 入, lg, 1, 层, 开, 到, 底下, lg, 2 15:46:12.188 [main] INFO test.LongTextFieldSearchTest - query: +address:"申 长 路 988 弄 虹桥 万 科 中 心 地下 停车场 lg 2 层 2179 2184 车位 锡 虹 路 入 lg 1 层 开 到 底下 lg 2"~2 15:46:12.210 [main] INFO test.LongTextFieldSearchTest - results.totalHits.value=0 15:46:12.214 [main] INFO test.LongTextFieldSearchTest - no matching phrase -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13486) race condition between leader's "replay on startup" and non-leader's "recover from leader" can leave replicas out of sync (TestTlogReplayVsRecovery)
[
https://issues.apache.org/jira/browse/SOLR-13486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014090#comment-17014090
]
ASF subversion and git services commented on SOLR-13486:
Commit 9a2497f6377601d396b1b3b8b83ffcab0fd331a3 in lucene-solr's branch
refs/heads/gradle-master from Chris M. Hostetter
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=9a2497f ]
SOLR-13486: Fix trivial test bug in TestTlogReplayVsRecovery
Add TODOs for future test improvements once underlying race condition is fixed
in core code
> race condition between leader's "replay on startup" and non-leader's "recover
> from leader" can leave replicas out of sync (TestTlogReplayVsRecovery)
>
>
> Key: SOLR-13486
> URL: https://issues.apache.org/jira/browse/SOLR-13486
> Project: Solr
> Issue Type: Bug
>Reporter: Chris M. Hostetter
>Priority: Major
> Attachments: SOLR-13486__test.patch,
> apache_Lucene-Solr-BadApples-NightlyTests-master_61.log.txt.gz,
> apache_Lucene-Solr-BadApples-Tests-8.x_102.log.txt.gz,
> org.apache.solr.cloud.TestCloudConsistency.zip
>
>
> There is a bug in solr cloud that can result in replicas being out of sync
> with the leader if:
> * The leader has uncommitted docs (in the tlog) that didn't make it to the
> replica
> * The leader restarts
> * The replica begins to peer sync from the leader before the leader finishes
> it's own tlog replay on startup
> A "rolling restart" situation is when this is most likeley to affect real
> world users
> This was first discovered via hard to reproduce TestCloudConsistency failures
> in jenkins, but that test has since been modified to work around this bug,
> and a new test "TestTlogReplayVsRecovery" has been added that more
> aggressively demonstrates this error.
> Original jira description below...
>
> I've been investigating some jenkins failures from TestCloudConsistency,
> which at first glance suggest a problem w/replica(s) recovering after a
> network partition from the leader - but in digging into the logs the root
> cause acturally seems to be a thread race conditions when a replica (the
> leader) is first registered...
> * The {{ZkContainer.registerInZk(...)}} method (which is called by
> {{CoreContainer.registerCore(...)}} & {{CoreContainer.load()}}) is typically
> run in a background thread (via the {{ZkContainer.coreZkRegister}}
> ExecutorService)
> * {{ZkContainer.registerInZk(...)}} delegates to
> {{ZKController.register(...)}} which is ultimately responsible for checking
> if there are any "old" tlogs on disk, and if so handling the "Replaying tlog
> for during startup" logic
> * Because this happens in a background thread, other logic/requests can be
> handled by this core/replica in the meantime - before it starts (or while in
> the middle of) replaying the tlogs
> ** Notably: *leader's that have not yet replayed tlogs on startup will
> erroneously respond to RTG / Fingerprint / PeerSync requests from other
> replicas w/incomplete data*
> ...In general, it seems scary / fishy to me that a replica can (aparently)
> become *ACTIVE* before it's finished it's {{registerInZk}} + "Replaying tlog
> ... during startup" logic ... particularly since this can happen even for
> replicas that are/become leaders. It seems like this could potentially cause
> a whole host of problems, only one of which manifests in this particular test
> failure:
> * *BEFORE* replicaX's "coreZkRegister" thread reaches the "Replaying tlog
> ... during startup" check:
> ** replicaX can recognize (via zk terms) that it should be the leader(X)
> ** this leaderX can then instruct some other replicaY to recover from it
> ** replicaY can send RTG / PeerSync / FetchIndex requests to the leaderX
> (either on it's own volition, or because it was instructed to by leaderX) in
> an attempt to recover
> *** the responses to these recovery requests will not include updates in the
> tlog files that existed on leaderX prior to startup that hvae not yet been
> replayed
> * *AFTER* replicaY has finished it's recovery, leaderX's "Replaying tlog ...
> during startup" can finish
> ** replicaY now thinks it is in sync with leaderX, but leaderX has
> (replayed) updates the other replicas know nothing about
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-6613) TextField.analyzeMultiTerm should not throw exception when analyzer returns no term
[
https://issues.apache.org/jira/browse/SOLR-6613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014089#comment-17014089
]
ASF subversion and git services commented on SOLR-6613:
---
Commit 0b072ecedb93202a132612e72cd880fdcc51ea25 in lucene-solr's branch
refs/heads/gradle-master from Bruno Roustant
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=0b072ec ]
SOLR-6613: TextField.analyzeMultiTerm does not throw an exception when Analyzer
returns no terms. (Bruno Roustant)
Closes #1146
> TextField.analyzeMultiTerm should not throw exception when analyzer returns
> no term
> ---
>
> Key: SOLR-6613
> URL: https://issues.apache.org/jira/browse/SOLR-6613
> Project: Solr
> Issue Type: Bug
> Components: Schema and Analysis
>Affects Versions: 4.3.1, 4.10.2, 6.0
>Reporter: Bruno Roustant
>Assignee: Bruno Roustant
>Priority: Major
> Fix For: 8.5
>
> Attachments: TestTextField.java
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> In TextField.analyzeMultiTerm()
> at line
> try {
> if (!source.incrementToken())
> throw new SolrException();
> The method should not throw an exception if there is no token because having
> no token is legitimate because all tokens may be filtered out (e.g. with a
> blocking Filter such as StopFilter).
> In this case it should simply return null (as it already returns null in some
> cases, see first line of method). However, SolrQueryParserBase needs also to
> be fixed to correctly handle null returned by TextField.analyzeMultiTerm().
> See attached TestTextField for the corresponding new test class.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9080) Upgrade ICU4j to 62.2 and make regenerate work
[ https://issues.apache.org/jira/browse/LUCENE-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014091#comment-17014091 ] ASF subversion and git services commented on LUCENE-9080: - Commit 3bae63d215972b8a60c825b5b5716ee0841ecd19 in lucene-solr's branch refs/heads/gradle-master from Erick Erickson [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=3bae63d ] LUCENE-9080: Upgrade ICU4j to 62.2 and make regenerate work > Upgrade ICU4j to 62.2 and make regenerate work > -- > > Key: LUCENE-9080 > URL: https://issues.apache.org/jira/browse/LUCENE-9080 > Project: Lucene - Core > Issue Type: Bug >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Fix For: master (9.0) > > Attachments: after_regen.patch, before_regen.patch, status.res > > Time Spent: 0.5h > Remaining Estimate: 0h > > The root cause is that RamUsageEstimator.NUM_BYTES_INT has been removed and > the python scripts still reference it in the generated scripts. That part's > easy to fix. > Last time I looked, though, the regenerate produces some differences in the > generated files that should be looked at to insure they're benign. > Not really sure whether this should be a Lucene or Solr JIRA. Putting it in > Lucene since one of the failed files is: > lucene/core/src/java/org/apache/lucene/util/packed/Packed8ThreeBlocks.java > I do know that one of the Solr jflex-produced file has an unexplained > difference so it may bleed over. > "ant regenerate" needs about 24G on my machine FWIW. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-14130) Add postlogs command line tool for indexing Solr logs
[ https://issues.apache.org/jira/browse/SOLR-14130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014088#comment-17014088 ] ASF subversion and git services commented on SOLR-14130: Commit d68f3e1a441b39485900e9d94e9686fb12b4ff87 in lucene-solr's branch refs/heads/gradle-master from Joel Bernstein [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=d68f3e1 ] SOLR-14130: Improve robustness of the logs parser > Add postlogs command line tool for indexing Solr logs > - > > Key: SOLR-14130 > URL: https://issues.apache.org/jira/browse/SOLR-14130 > Project: Solr > Issue Type: Task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Joel Bernstein >Assignee: Joel Bernstein >Priority: Major > Attachments: SOLR-14130.patch, SOLR-14130.patch, SOLR-14130.patch, > SOLR-14130.patch, SOLR-14130.patch, SOLR-14130.patch, SOLR-14130.patch, > Screen Shot 2019-12-19 at 2.04.41 PM.png, Screen Shot 2019-12-19 at 2.16.01 > PM.png, Screen Shot 2019-12-19 at 2.35.41 PM.png, Screen Shot 2019-12-21 at > 8.46.51 AM.png > > > This ticket adds a simple command line tool for posting Solr logs to a solr > index. The tool works with the out of the box Solr log format. Still a work > in progress but currently indexes: > * queries > * updates > * commits > * new searchers > * errors - including stack traces > Attached are some sample visualizations using Solr Streaming Expressions and > Math Expressions after the data has been loaded. The visualizations show: > time series, scatter plots, histograms and quantile plots, but really this is > just scratching the surface of the visualizations that can be done with the > Solr logs. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9123) JapaneseTokenizer with search mode doesn't work with SynonymGraphFilter
[
https://issues.apache.org/jira/browse/LUCENE-9123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014065#comment-17014065
]
Tomoko Uchida commented on LUCENE-9123:
---
Hi [~h.kazuaki],
introducing the option {{discardCompoundToken}} looks fine to me, however, I
think we shouldn't change signatures of the existing constructors for backwards
compatibility (they are public interface, so we have to keep them during 8.x
anyways). Instead, we can add a new constructor. Opinions?
> JapaneseTokenizer with search mode doesn't work with SynonymGraphFilter
> ---
>
> Key: LUCENE-9123
> URL: https://issues.apache.org/jira/browse/LUCENE-9123
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/analysis
>Affects Versions: 8.4
>Reporter: Kazuaki Hiraga
>Priority: Major
> Attachments: LUCENE-9123.patch
>
>
> JapaneseTokenizer with `mode=search` or `mode=extended` doesn't work with
> both of SynonymGraphFilter and SynonymFilter when JT generates multiple
> tokens as an output. If we use `mode=normal`, it should be fine. However, we
> would like to use decomposed tokens that can maximize to chance to increase
> recall.
> Snippet of schema:
> {code:xml}
> positionIncrementGap="100" autoGeneratePhraseQueries="false">
>
>
> synonyms="lang/synonyms_ja.txt"
> tokenizerFactory="solr.JapaneseTokenizerFactory"/>
>
>
> tags="lang/stoptags_ja.txt" />
>
>
>
>
>
> minimumLength="4"/>
>
>
>
>
> {code}
> An synonym entry that generates error:
> {noformat}
> 株式会社,コーポレーション
> {noformat}
> The following is an output on console:
> {noformat}
> $ ./bin/solr create_core -c jp_test -d ../config/solrconfs
> ERROR: Error CREATEing SolrCore 'jp_test': Unable to create core [jp_test3]
> Caused by: term: 株式会社 analyzed to a token (株式会社) with position increment != 1
> (got: 0)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-9129) Updating from 7.X to 8.X breaks
[
https://issues.apache.org/jira/browse/LUCENE-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014043#comment-17014043
]
Tomoko Uchida edited comment on LUCENE-9129 at 1/13/20 6:13 AM:
It's not a bug but an intended change related to this optimization.
https://issues.apache.org/jira/browse/LUCENE-4100
The issue isn't mentioned in "API Changes" but "Optimizations" section on the
Change log:
[https://lucene.apache.org/core/8_0_0/changes/Changes.html|https://lucene.apache.org/core/8_0_0/changes/Changes.html].
Although it includes API changes, I think the location is appropriate for the
main purpose of the issue.
In short, you need to implement {{Collector#scoreMode()}} and also discard
{{#needsScores()}} when upgrading to Lucene 8.0+ as the log message says.
See:
[https://lucene.apache.org/core/8_0_0/core/org/apache/lucene/search/Collector.html]
Since there were many changes/optimizations between 7.x and 8.x, it's hard to
make an exhaustive list for every breaking change in the APIs (in addition, the
{{Collector}} interface is marked as "Expert:", that means this is for expert
users who are familiar with Lucene internals). So could you refer the Javadocs
when you encounter errors relating to the library version upgrade. The Git
commit log and diff command will also be help for getting more detailed
information.
was (Author: tomoko uchida):
It's not a bug but an intended change related to this optimization.
https://issues.apache.org/jira/browse/LUCENE-4100
The issue isn't mentioned in "API Changes" but "Optimizations" section on the
Change log:
[https://lucene.apache.org/core/8_0_0/changes/Changes.html|https://lucene.apache.org/core/8_4_0/changes/Changes.html].
Although it includes API changes, I think the location is appropriate for the
main purpose of the issue.
In short, you need to implement {{Collector#scoreMode()}} and also discard
{{#needsScores()}} when upgrading to Lucene 8.0+ as the log message says.
See:
[https://lucene.apache.org/core/8_0_0/core/org/apache/lucene/search/Collector.html]
Since there were many changes/optimizations between 7.x and 8.x, it's hard to
make an exhaustive list for every breaking change in the APIs (in addition, the
{{Collector}} interface is marked as "Expert:", that means this is for expert
users who are familiar with Lucene internals). So could you refer the Javadocs
when you encounter errors relating to the library version upgrade. The Git
commit log and diff command will also be help for getting more detailed
information.
> Updating from 7.X to 8.X breaks
> ---
>
> Key: LUCENE-9129
> URL: https://issues.apache.org/jira/browse/LUCENE-9129
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: xia0c
>Priority: Major
>
> Hi, during my upgrading process from 7.X to 8.X I found another code break.
> {code:java}
> import org.apache.lucene.analysis.Analyzer;
> import org.apache.lucene.index.LeafReaderContext;
> import org.apache.lucene.index.MultiDocValues;
> import org.apache.lucene.index.SortedDocValues;
> import org.apache.lucene.search.IndexSearcher;
> import org.apache.lucene.search.Query;
> import org.apache.lucene.search.SimpleCollector;
> import org.apache.lucene.search.TopDocs;
> import org.apache.lucene.util.LongValues;
> import org.apache.solr.handler.component.FacetComponent.FacetContext;
> import org.apache.solr.search.DocSet;
> import org.apache.solr.search.DocSetUtil;
> import org.apache.lucene.index.IndexReader;
> public class TestLucene {
>
> private FacetContext fcontext;
> private DocSet docs;
> private IndexReader reader;
>
>
> public void demo() throws IOException {
>
> DocSetUtil.collectSortedDocSet(docs, reader, new
> SimpleCollector() {
> @Override
> public boolean needsScores() { return false; }
> @Override
> protected void doSetNextReader(LeafReaderContext ctx) throws
> IOException {
> // TODO
> }
> @Override
> public void collect(int doc) throws IOException {
> // TODO Auto-generated method stub
>
> }
> });
>
> }
> }
> {code}
> The code should pass before, but it throws an error:
> {code:java}
> [ERROR] /TestLucene.java:[32,82] is not abstract and
> does not override abstract method scoreMode() in
> org.apache.lucene.search.Collector
> [ERROR] /TestLucene.java:[36,19] method does not override or implement a
> method from a supertype
> {code}
> I try to find changes in the migration
>
[jira] [Commented] (LUCENE-9129) Updating from 7.X to 8.X breaks
[
https://issues.apache.org/jira/browse/LUCENE-9129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17014043#comment-17014043
]
Tomoko Uchida commented on LUCENE-9129:
---
It's not a bug but an intended change related to this optimization.
https://issues.apache.org/jira/browse/LUCENE-4100
The issue isn't mentioned in "API Changes" but "Optimizations" section on the
Change log:
[https://lucene.apache.org/core/8_0_0/changes/Changes.html|https://lucene.apache.org/core/8_4_0/changes/Changes.html].
Although it includes API changes, I think the location is appropriate for the
main purpose of the issue.
In short, you need to implement {{Collector#scoreMode()}} and also discard
{{#needsScores()}} when upgrading to Lucene 8.0+ as the log message says.
See:
[https://lucene.apache.org/core/8_0_0/core/org/apache/lucene/search/Collector.html]
Since there were many changes/optimizations between 7.x and 8.x, it's hard to
make an exhaustive list for every breaking change in the APIs (in addition, the
{{Collector}} interface is marked as "Expert:", that means this is for expert
users who are familiar with Lucene internals). So could you refer the Javadocs
when you encounter errors relating to the library version upgrade. The Git
commit log and diff command will also be help for getting more detailed
information.
> Updating from 7.X to 8.X breaks
> ---
>
> Key: LUCENE-9129
> URL: https://issues.apache.org/jira/browse/LUCENE-9129
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: xia0c
>Priority: Major
>
> Hi, during my upgrading process from 7.X to 8.X I found another code break.
> {code:java}
> import org.apache.lucene.analysis.Analyzer;
> import org.apache.lucene.index.LeafReaderContext;
> import org.apache.lucene.index.MultiDocValues;
> import org.apache.lucene.index.SortedDocValues;
> import org.apache.lucene.search.IndexSearcher;
> import org.apache.lucene.search.Query;
> import org.apache.lucene.search.SimpleCollector;
> import org.apache.lucene.search.TopDocs;
> import org.apache.lucene.util.LongValues;
> import org.apache.solr.handler.component.FacetComponent.FacetContext;
> import org.apache.solr.search.DocSet;
> import org.apache.solr.search.DocSetUtil;
> import org.apache.lucene.index.IndexReader;
> public class TestLucene {
>
> private FacetContext fcontext;
> private DocSet docs;
> private IndexReader reader;
>
>
> public void demo() throws IOException {
>
> DocSetUtil.collectSortedDocSet(docs, reader, new
> SimpleCollector() {
> @Override
> public boolean needsScores() { return false; }
> @Override
> protected void doSetNextReader(LeafReaderContext ctx) throws
> IOException {
> // TODO
> }
> @Override
> public void collect(int doc) throws IOException {
> // TODO Auto-generated method stub
>
> }
> });
>
> }
> }
> {code}
> The code should pass before, but it throws an error:
> {code:java}
> [ERROR] /TestLucene.java:[32,82] is not abstract and
> does not override abstract method scoreMode() in
> org.apache.lucene.search.Collector
> [ERROR] /TestLucene.java:[36,19] method does not override or implement a
> method from a supertype
> {code}
> I try to find changes in the migration
> guide(https://github.com/apache/lucene-solr/blob/branch_8x/lucene/MIGRATE.txt)
> but I didn't find it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9129) Updating from 7.X to 8.X breaks
xia0c created LUCENE-9129:
-
Summary: Updating from 7.X to 8.X breaks
Key: LUCENE-9129
URL: https://issues.apache.org/jira/browse/LUCENE-9129
Project: Lucene - Core
Issue Type: Bug
Reporter: xia0c
Hi, during my upgrading process from 7.X to 8.X I found another code break.
{code:java}
import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.index.LeafReaderContext;
import org.apache.lucene.index.MultiDocValues;
import org.apache.lucene.index.SortedDocValues;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.Query;
import org.apache.lucene.search.SimpleCollector;
import org.apache.lucene.search.TopDocs;
import org.apache.lucene.util.LongValues;
import org.apache.solr.handler.component.FacetComponent.FacetContext;
import org.apache.solr.search.DocSet;
import org.apache.solr.search.DocSetUtil;
import org.apache.lucene.index.IndexReader;
public class TestLucene {
private FacetContext fcontext;
private DocSet docs;
private IndexReader reader;
public void demo() throws IOException {
DocSetUtil.collectSortedDocSet(docs, reader, new
SimpleCollector() {
@Override
public boolean needsScores() { return false; }
@Override
protected void doSetNextReader(LeafReaderContext ctx) throws
IOException {
// TODO
}
@Override
public void collect(int doc) throws IOException {
// TODO Auto-generated method stub
}
});
}
}
{code}
The code should pass before, but it throws an error:
{code:java}
[ERROR] /TestLucene.java:[32,82] is not abstract and
does not override abstract method scoreMode() in
org.apache.lucene.search.Collector
[ERROR] /TestLucene.java:[36,19] method does not override or implement a method
from a supertype
{code}
I try to find changes in the migration
guide(https://github.com/apache/lucene-solr/blob/branch_8x/lucene/MIGRATE.txt)
but I didn't find it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9077) Gradle build
[
https://issues.apache.org/jira/browse/LUCENE-9077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013948#comment-17013948
]
Erick Erickson commented on LUCENE-9077:
Now that I've finally gotten LUCENE-9080 done I'm ready to start dealing with
making the equivalent of the old "ant regenerate" target work in the Gradle
build, any pointers gratefully received ;)
> Gradle build
>
>
> Key: LUCENE-9077
> URL: https://issues.apache.org/jira/browse/LUCENE-9077
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Dawid Weiss
>Assignee: Dawid Weiss
>Priority: Major
> Fix For: master (9.0)
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> This task focuses on providing gradle-based build equivalent for Lucene and
> Solr (on master branch). See notes below on why this respin is needed.
> The code lives on *gradle-master* branch. It is kept with sync with *master*.
> Try running the following to see an overview of helper guides concerning
> typical workflow, testing and ant-migration helpers:
> gradlew :help
> A list of items that needs to be added or requires work. If you'd like to
> work on any of these, please add your name to the list. Once you have a
> patch/ pull request let me (dweiss) know - I'll try to coordinate the merges.
> * (/) Apply forbiddenAPIs
> * (/) Generate hardware-aware gradle defaults for parallelism (count of
> workers and test JVMs).
> * (/) Fail the build if --tests filter is applied and no tests execute
> during the entire build (this allows for an empty set of filtered tests at
> single project level).
> * (/) Port other settings and randomizations from common-build.xml
> * (/) Configure security policy/ sandboxing for tests.
> * (/) test's console output on -Ptests.verbose=true
> * (/) add a :helpDeps explanation to how the dependency system works
> (palantir plugin, lockfile) and how to retrieve structured information about
> current dependencies of a given module (in a tree-like output).
> * (/) jar checksums, jar checksum computation and validation. This should be
> done without intermediate folders (directly on dependency sets).
> * (/) verify min. JVM version and exact gradle version on build startup to
> minimize odd build side-effects
> * (/) Repro-line for failed tests/ runs.
> * (/) add a top-level README note about building with gradle (and the
> required JVM).
> * (/) add an equivalent of 'validate-source-patterns'
> (check-source-patterns.groovy) to precommit.
> * add an equivalent of 'rat-sources' to precommit.
> * (/) add an equivalent of 'check-example-lucene-match-version' (solr only)
> to precommit.
> * add an equivalent of 'documentation-lint" to precommit.
> Hard-to-implement stuff already investigated:
> * (/) (done) -*Printing console output of failed tests.* There doesn't seem
> to be any way to do this in a reasonably efficient way. There are onOutput
> listeners but they're slow to operate and solr tests emit *tons* of output so
> it's an overkill.-
> * (!) (LUCENE-9120) *Tests working with security-debug logs or other
> JVM-early log output*. Gradle's test runner works by redirecting Java's
> stdout/ syserr so this just won't work. Perhaps we can spin the ant-based
> test runner for such corner-cases.
> Of lesser importance:
> * add rendering of javadocs (gradlew javadoc) and attaching them to maven
> publications.
> * Add test 'beasting' (rerunning the same suite multiple times). I'm afraid
> it'll be difficult to run it sensibly because gradle doesn't offer cwd
> separation for the forked test runners.
> * if you diff solr packaged distribution against ant-created distribution
> there are minor differences in library versions and some JARs are excluded/
> moved around. I didn't try to force these as everything seems to work (tests,
> etc.) – perhaps these differences should be fixed in the ant build instead.
> * [EOE] identify and port various "regenerate" tasks from ant builds
> (javacc, precompiled automata, etc.)
> * fill in POM details in gradle/defaults-maven.gradle so that they reflect
> the previous content better (dependencies aside).
> * Add any IDE integration layers that should be added (I use IntelliJ and it
> imports the project out of the box, without the need for any special tuning).
> * *Clean up dependencies, especially for Solr*: any \{ transitive = false }
> should just explicitly exclude whatever they don't need (and their
> dependencies currently declared explicitly should be folded). Figure out
> which scope to import a dependency to.
> * Add Solr packaging for docs/* (see TODO in packaging/build.gradle;
> currently XSLT...)
> * I didn't bother adding Solr dist/test-framework to packaging (who'd use it
> from a binary distribution?
>
> *{color:#ff}Note:{color}* this
[jira] [Commented] (LUCENE-9128) Updating from 7.X to 8.X breaks due to change of TotalHits
[
https://issues.apache.org/jira/browse/LUCENE-9128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013924#comment-17013924
]
Michael McCandless commented on LUCENE-9128:
You're welcome [~lingchao]!
> Updating from 7.X to 8.X breaks due to change of TotalHits
> --
>
> Key: LUCENE-9128
> URL: https://issues.apache.org/jira/browse/LUCENE-9128
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: xia0c
>Priority: Major
>
> When I try to upgrade lucene-core from 7.X to version 8.X. The following code
> breaks.
> {code:java}
> public class TestLucene {
>
> private IndexSearcher searcher;
> public Analyzer analyzer;
>
>
> public void demo(String keyword, int pageNum, int pageSize) throws
> ParseException, IOException {
>
> String[] fields = {"title"};
> MultiFieldQueryParser multiFieldQueryParser = new
> MultiFieldQueryParser(fields, analyzer);
> Query query = multiFieldQueryParser.parse(keyword);
> TopDocs topDocs = searcher.search(query, 100);
> int start = (pageNum - 1) * pageSize;
> int end = (int) Math.min(start + pageSize,topDocs.totalHits);
>
> }
> }
> {code}
> The code should pass, but it throws an error:
> {code:java}
> [ERROR] /TestLucene.java:[26,29] no suitable method found for
> min(int,org.apache.lucene.search.TotalHits)
> [ERROR] method java.lang.Math.min(int,int) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to int)
> [ERROR] method java.lang.Math.min(long,long) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to long)
> [ERROR] method java.lang.Math.min(float,float) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to float)
> [ERROR] method java.lang.Math.min(double,double) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to double)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9128) Updating from 7.X to 8.X breaks due to change of TotalHits
[
https://issues.apache.org/jira/browse/LUCENE-9128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013923#comment-17013923
]
xia0c commented on LUCENE-9128:
---
Alright. Thank you!
> Updating from 7.X to 8.X breaks due to change of TotalHits
> --
>
> Key: LUCENE-9128
> URL: https://issues.apache.org/jira/browse/LUCENE-9128
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: xia0c
>Priority: Major
>
> When I try to upgrade lucene-core from 7.X to version 8.X. The following code
> breaks.
> {code:java}
> public class TestLucene {
>
> private IndexSearcher searcher;
> public Analyzer analyzer;
>
>
> public void demo(String keyword, int pageNum, int pageSize) throws
> ParseException, IOException {
>
> String[] fields = {"title"};
> MultiFieldQueryParser multiFieldQueryParser = new
> MultiFieldQueryParser(fields, analyzer);
> Query query = multiFieldQueryParser.parse(keyword);
> TopDocs topDocs = searcher.search(query, 100);
> int start = (pageNum - 1) * pageSize;
> int end = (int) Math.min(start + pageSize,topDocs.totalHits);
>
> }
> }
> {code}
> The code should pass, but it throws an error:
> {code:java}
> [ERROR] /TestLucene.java:[26,29] no suitable method found for
> min(int,org.apache.lucene.search.TotalHits)
> [ERROR] method java.lang.Math.min(int,int) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to int)
> [ERROR] method java.lang.Math.min(long,long) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to long)
> [ERROR] method java.lang.Math.min(float,float) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to float)
> [ERROR] method java.lang.Math.min(double,double) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to double)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9128) Updating from 7.X to 8.X breaks due to change of TotalHits
[
https://issues.apache.org/jira/browse/LUCENE-9128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013922#comment-17013922
]
Michael McCandless commented on LUCENE-9128:
Hi [~lingchao], each Lucene release includes MIGRATE.txt to explain big API
changes like this ... here's the 8.x entry about the totalHits change:
[https://github.com/apache/lucene-solr/blob/branch_8x/lucene/MIGRATE.txt#L100-L108]
> Updating from 7.X to 8.X breaks due to change of TotalHits
> --
>
> Key: LUCENE-9128
> URL: https://issues.apache.org/jira/browse/LUCENE-9128
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: xia0c
>Priority: Major
>
> When I try to upgrade lucene-core from 7.X to version 8.X. The following code
> breaks.
> {code:java}
> public class TestLucene {
>
> private IndexSearcher searcher;
> public Analyzer analyzer;
>
>
> public void demo(String keyword, int pageNum, int pageSize) throws
> ParseException, IOException {
>
> String[] fields = {"title"};
> MultiFieldQueryParser multiFieldQueryParser = new
> MultiFieldQueryParser(fields, analyzer);
> Query query = multiFieldQueryParser.parse(keyword);
> TopDocs topDocs = searcher.search(query, 100);
> int start = (pageNum - 1) * pageSize;
> int end = (int) Math.min(start + pageSize,topDocs.totalHits);
>
> }
> }
> {code}
> The code should pass, but it throws an error:
> {code:java}
> [ERROR] /TestLucene.java:[26,29] no suitable method found for
> min(int,org.apache.lucene.search.TotalHits)
> [ERROR] method java.lang.Math.min(int,int) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to int)
> [ERROR] method java.lang.Math.min(long,long) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to long)
> [ERROR] method java.lang.Math.min(float,float) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to float)
> [ERROR] method java.lang.Math.min(double,double) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to double)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9128) Updating from 7.X to 8.X breaks due to change of TotalHits
[
https://issues.apache.org/jira/browse/LUCENE-9128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013921#comment-17013921
]
Uwe Schindler commented on LUCENE-9128:
---
Check Migration guide. It's also mentioned on the documentation page:
http://lucene.apache.org/core/8_4_0/MIGRATE.html
> Updating from 7.X to 8.X breaks due to change of TotalHits
> --
>
> Key: LUCENE-9128
> URL: https://issues.apache.org/jira/browse/LUCENE-9128
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: xia0c
>Priority: Major
>
> When I try to upgrade lucene-core from 7.X to version 8.X. The following code
> breaks.
> {code:java}
> public class TestLucene {
>
> private IndexSearcher searcher;
> public Analyzer analyzer;
>
>
> public void demo(String keyword, int pageNum, int pageSize) throws
> ParseException, IOException {
>
> String[] fields = {"title"};
> MultiFieldQueryParser multiFieldQueryParser = new
> MultiFieldQueryParser(fields, analyzer);
> Query query = multiFieldQueryParser.parse(keyword);
> TopDocs topDocs = searcher.search(query, 100);
> int start = (pageNum - 1) * pageSize;
> int end = (int) Math.min(start + pageSize,topDocs.totalHits);
>
> }
> }
> {code}
> The code should pass, but it throws an error:
> {code:java}
> [ERROR] /TestLucene.java:[26,29] no suitable method found for
> min(int,org.apache.lucene.search.TotalHits)
> [ERROR] method java.lang.Math.min(int,int) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to int)
> [ERROR] method java.lang.Math.min(long,long) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to long)
> [ERROR] method java.lang.Math.min(float,float) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to float)
> [ERROR] method java.lang.Math.min(double,double) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to double)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9128) Updating from 7.X to 8.X breaks due to change of TotalHits
[
https://issues.apache.org/jira/browse/LUCENE-9128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013920#comment-17013920
]
xia0c commented on LUCENE-9128:
---
Hi [~mikemccand], thanks for your quick reply and help.
Alright. Would it be better to add a breaking announcement in the release
note? Then all other developers who use Lucene-core can know it. Actually, I
didn't find it at
https://lucene.apache.org/core/8_0_0/changes/Changes.html#v8.0.0.api_changes.
> Updating from 7.X to 8.X breaks due to change of TotalHits
> --
>
> Key: LUCENE-9128
> URL: https://issues.apache.org/jira/browse/LUCENE-9128
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: xia0c
>Priority: Major
>
> When I try to upgrade lucene-core from 7.X to version 8.X. The following code
> breaks.
> {code:java}
> public class TestLucene {
>
> private IndexSearcher searcher;
> public Analyzer analyzer;
>
>
> public void demo(String keyword, int pageNum, int pageSize) throws
> ParseException, IOException {
>
> String[] fields = {"title"};
> MultiFieldQueryParser multiFieldQueryParser = new
> MultiFieldQueryParser(fields, analyzer);
> Query query = multiFieldQueryParser.parse(keyword);
> TopDocs topDocs = searcher.search(query, 100);
> int start = (pageNum - 1) * pageSize;
> int end = (int) Math.min(start + pageSize,topDocs.totalHits);
>
> }
> }
> {code}
> The code should pass, but it throws an error:
> {code:java}
> [ERROR] /TestLucene.java:[26,29] no suitable method found for
> min(int,org.apache.lucene.search.TotalHits)
> [ERROR] method java.lang.Math.min(int,int) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to int)
> [ERROR] method java.lang.Math.min(long,long) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to long)
> [ERROR] method java.lang.Math.min(float,float) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to float)
> [ERROR] method java.lang.Math.min(double,double) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to double)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9128) Updating from 7.X to 8.X breaks due to change of TotalHits
[
https://issues.apache.org/jira/browse/LUCENE-9128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013919#comment-17013919
]
Michael McCandless commented on LUCENE-9128:
Hi [~lingchao], this is expected – this was an API change, which is allowed in
a new major Lucene release.
This was done because Lucene can now return "approximate" total hits, enable
powerful optimizations like Block-MAX Weak And
([https://www.elastic.co/blog/faster-retrieval-of-top-hits-in-elasticsearch-with-block-max-wand]).
> Updating from 7.X to 8.X breaks due to change of TotalHits
> --
>
> Key: LUCENE-9128
> URL: https://issues.apache.org/jira/browse/LUCENE-9128
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: xia0c
>Priority: Major
>
> When I try to upgrade lucene-core from 7.X to version 8.X. The following code
> breaks.
> {code:java}
> public class TestLucene {
>
> private IndexSearcher searcher;
> public Analyzer analyzer;
>
>
> public void demo(String keyword, int pageNum, int pageSize) throws
> ParseException, IOException {
>
> String[] fields = {"title"};
> MultiFieldQueryParser multiFieldQueryParser = new
> MultiFieldQueryParser(fields, analyzer);
> Query query = multiFieldQueryParser.parse(keyword);
> TopDocs topDocs = searcher.search(query, 100);
> int start = (pageNum - 1) * pageSize;
> int end = (int) Math.min(start + pageSize,topDocs.totalHits);
>
> }
> }
> {code}
> The code should pass, but it throws an error:
> {code:java}
> [ERROR] /TestLucene.java:[26,29] no suitable method found for
> min(int,org.apache.lucene.search.TotalHits)
> [ERROR] method java.lang.Math.min(int,int) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to int)
> [ERROR] method java.lang.Math.min(long,long) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to long)
> [ERROR] method java.lang.Math.min(float,float) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to float)
> [ERROR] method java.lang.Math.min(double,double) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to double)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-9128) Updating from 7.X to 8.X breaks due to change of TotalHits
[
https://issues.apache.org/jira/browse/LUCENE-9128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael McCandless resolved LUCENE-9128.
Resolution: Not A Problem
> Updating from 7.X to 8.X breaks due to change of TotalHits
> --
>
> Key: LUCENE-9128
> URL: https://issues.apache.org/jira/browse/LUCENE-9128
> Project: Lucene - Core
> Issue Type: Bug
>Reporter: xia0c
>Priority: Major
>
> When I try to upgrade lucene-core from 7.X to version 8.X. The following code
> breaks.
> {code:java}
> public class TestLucene {
>
> private IndexSearcher searcher;
> public Analyzer analyzer;
>
>
> public void demo(String keyword, int pageNum, int pageSize) throws
> ParseException, IOException {
>
> String[] fields = {"title"};
> MultiFieldQueryParser multiFieldQueryParser = new
> MultiFieldQueryParser(fields, analyzer);
> Query query = multiFieldQueryParser.parse(keyword);
> TopDocs topDocs = searcher.search(query, 100);
> int start = (pageNum - 1) * pageSize;
> int end = (int) Math.min(start + pageSize,topDocs.totalHits);
>
> }
> }
> {code}
> The code should pass, but it throws an error:
> {code:java}
> [ERROR] /TestLucene.java:[26,29] no suitable method found for
> min(int,org.apache.lucene.search.TotalHits)
> [ERROR] method java.lang.Math.min(int,int) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to int)
> [ERROR] method java.lang.Math.min(long,long) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to long)
> [ERROR] method java.lang.Math.min(float,float) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to float)
> [ERROR] method java.lang.Math.min(double,double) is not applicable
> [ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
> converted to double)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Created] (LUCENE-9128) Updating from 7.X to 8.X breaks due to change of TotalHits
xia0c created LUCENE-9128:
-
Summary: Updating from 7.X to 8.X breaks due to change of TotalHits
Key: LUCENE-9128
URL: https://issues.apache.org/jira/browse/LUCENE-9128
Project: Lucene - Core
Issue Type: Bug
Reporter: xia0c
When I try to upgrade lucene-core from 7.X to version 8.X. The following code
breaks.
{code:java}
public class TestLucene {
private IndexSearcher searcher;
public Analyzer analyzer;
public void demo(String keyword, int pageNum, int pageSize) throws
ParseException, IOException {
String[] fields = {"title"};
MultiFieldQueryParser multiFieldQueryParser = new
MultiFieldQueryParser(fields, analyzer);
Query query = multiFieldQueryParser.parse(keyword);
TopDocs topDocs = searcher.search(query, 100);
int start = (pageNum - 1) * pageSize;
int end = (int) Math.min(start + pageSize,topDocs.totalHits);
}
}
{code}
The code should pass, but it throws an error:
{code:java}
[ERROR] /TestLucene.java:[26,29] no suitable method found for
min(int,org.apache.lucene.search.TotalHits)
[ERROR] method java.lang.Math.min(int,int) is not applicable
[ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
converted to int)
[ERROR] method java.lang.Math.min(long,long) is not applicable
[ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
converted to long)
[ERROR] method java.lang.Math.min(float,float) is not applicable
[ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
converted to float)
[ERROR] method java.lang.Math.min(double,double) is not applicable
[ERROR] (argument mismatch; org.apache.lucene.search.TotalHits cannot be
converted to double)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9080) Upgrade ICU4j to 62.2 and make regenerate work
[ https://issues.apache.org/jira/browse/LUCENE-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson updated LUCENE-9080: --- Fix Version/s: master (9.0) Resolution: Fixed Status: Resolved (was: Patch Available) I decided not to open a separate Jira for 8x, if anyone is moved to backport this Jira will show up easily enough in a search. > Upgrade ICU4j to 62.2 and make regenerate work > -- > > Key: LUCENE-9080 > URL: https://issues.apache.org/jira/browse/LUCENE-9080 > Project: Lucene - Core > Issue Type: Bug >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Fix For: master (9.0) > > Attachments: after_regen.patch, before_regen.patch, status.res > > Time Spent: 0.5h > Remaining Estimate: 0h > > The root cause is that RamUsageEstimator.NUM_BYTES_INT has been removed and > the python scripts still reference it in the generated scripts. That part's > easy to fix. > Last time I looked, though, the regenerate produces some differences in the > generated files that should be looked at to insure they're benign. > Not really sure whether this should be a Lucene or Solr JIRA. Putting it in > Lucene since one of the failed files is: > lucene/core/src/java/org/apache/lucene/util/packed/Packed8ThreeBlocks.java > I do know that one of the Solr jflex-produced file has an unexplained > difference so it may bleed over. > "ant regenerate" needs about 24G on my machine FWIW. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (LUCENE-9080) Upgrade ICU4j to 62.2 and make regenerate work
[ https://issues.apache.org/jira/browse/LUCENE-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013908#comment-17013908 ] ASF subversion and git services commented on LUCENE-9080: - Commit 3bae63d215972b8a60c825b5b5716ee0841ecd19 in lucene-solr's branch refs/heads/master from Erick Erickson [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=3bae63d ] LUCENE-9080: Upgrade ICU4j to 62.2 and make regenerate work > Upgrade ICU4j to 62.2 and make regenerate work > -- > > Key: LUCENE-9080 > URL: https://issues.apache.org/jira/browse/LUCENE-9080 > Project: Lucene - Core > Issue Type: Bug >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: after_regen.patch, before_regen.patch, status.res > > Time Spent: 10m > Remaining Estimate: 0h > > The root cause is that RamUsageEstimator.NUM_BYTES_INT has been removed and > the python scripts still reference it in the generated scripts. That part's > easy to fix. > Last time I looked, though, the regenerate produces some differences in the > generated files that should be looked at to insure they're benign. > Not really sure whether this should be a Lucene or Solr JIRA. Putting it in > Lucene since one of the failed files is: > lucene/core/src/java/org/apache/lucene/util/packed/Packed8ThreeBlocks.java > I do know that one of the Solr jflex-produced file has an unexplained > difference so it may bleed over. > "ant regenerate" needs about 24G on my machine FWIW. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] ErickErickson closed pull request #1148: LUCENE-9080: Upgrade ICU4j to 62.2 and regenerate
ErickErickson closed pull request #1148: LUCENE-9080: Upgrade ICU4j to 62.2 and regenerate URL: https://github.com/apache/lucene-solr/pull/1148 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] ErickErickson commented on issue #1148: LUCENE-9080: Upgrade ICU4j to 62.2 and regenerate
ErickErickson commented on issue #1148: LUCENE-9080: Upgrade ICU4j to 62.2 and regenerate URL: https://github.com/apache/lucene-solr/pull/1148#issuecomment-573463126 Merged in to master This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Updated] (LUCENE-9080) Upgrade ICU4j to 62.2 and make regenerate work
[ https://issues.apache.org/jira/browse/LUCENE-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson updated LUCENE-9080: --- Summary: Upgrade ICU4j to 62.2 and make regenerate work (was: Upgrade ICU4j to 62.2 and regenerate) > Upgrade ICU4j to 62.2 and make regenerate work > -- > > Key: LUCENE-9080 > URL: https://issues.apache.org/jira/browse/LUCENE-9080 > Project: Lucene - Core > Issue Type: Bug >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: after_regen.patch, before_regen.patch, status.res > > Time Spent: 10m > Remaining Estimate: 0h > > The root cause is that RamUsageEstimator.NUM_BYTES_INT has been removed and > the python scripts still reference it in the generated scripts. That part's > easy to fix. > Last time I looked, though, the regenerate produces some differences in the > generated files that should be looked at to insure they're benign. > Not really sure whether this should be a Lucene or Solr JIRA. Putting it in > Lucene since one of the failed files is: > lucene/core/src/java/org/apache/lucene/util/packed/Packed8ThreeBlocks.java > I do know that one of the Solr jflex-produced file has an unexplained > difference so it may bleed over. > "ant regenerate" needs about 24G on my machine FWIW. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
[jira] [Commented] (SOLR-13759) Optimize Queries when query filtering by TRA router.field
[
https://issues.apache.org/jira/browse/SOLR-13759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013794#comment-17013794
]
Gus Heck commented on SOLR-13759:
-
I've looked at your patch, here are my comments:
# Generally the init() methd of HttpSolrCall is way to long to begin with.
Everything inside the if block should be swapped out to a method called
optimizeTraFilter or something of the sort.
# QueriesCollectorQueryVisitor is not very generic, and I don’t think it
deserves it’s own class. I’d rather see this as a private inner class or
anonymous class. Folks should not be seeing it in their ide suggestions etc…
Also the name is long and hard to read anyway, so moving it down should allow
us to use a shorter name or no name at all.
# Similarly ResponseHelper doesn't look like it deserves a top level class.
Making a new class at the top level should be reserved for cases where A) the
code gets reused in at least 2 if not 3 other classes, B) some data and logic
should be encapsulated together, or C) there is a large coherent set of logic
which is all closely related and including it in the main class would
significantly pollute or distract from the main class in which it is used. In
this case these can probably just be private instance methods in your test.
# if (path.contains("select") && …. My first reaction is I don’t like any sort
of contains(“foo”) on the path, because this would get triggered by a
collection name such as “selected” etc. There’s logic similar to that for
“update” later in the method but it requires / chars too which would be safer…
# Secondarily, though I haven’t thought too deeply about it yet, I don’t see
good reason to limit this only to the case of the /select search handler. There
would be no reason for any handler that accepts fq to pester Sub-collections in
a TRA that are out of range. I’m not terribly concerned about cases where
people add fq to parameters for any handlers that don’t normally accept fq.
That falls in the “well… um... don’t do that?” category :). I think we are
probably safe cuing off the presence of fq.
# Finally, I know it came from my example, but {{QueryParser queryParser = new
QueryParser("_text_", new StandardAnalyzer());}} makes me nervous. When I wrote
it in the example I was thinking that we might have a way to use the parser
already being created for the request, but given where this code lives that may
be difficult. In it's current form this makes me nervous. If we can't find a
better way we will need to ask ourselves if this cost in excess object creation
is greater than the benefits of this filtering (testing/benchmarks would be
good) It's possible that this will be a tradeoff in GC vs response time and not
right for everyone. That may imply the need for a mechanism to turn this
feature on/off
# the isTRA method: I'd like to suggest a more future proof name: isTimeRouted
- "TRA" is a specific notion and a special case of the DRA's which may have
time dimensions. We might or might not ever extend this to the general case
(depending on how beneficial this turns out to be) but not using the TRA
acronym will make it read better if we do.
> Optimize Queries when query filtering by TRA router.field
> -
>
> Key: SOLR-13759
> URL: https://issues.apache.org/jira/browse/SOLR-13759
> Project: Solr
> Issue Type: Sub-task
>Reporter: mosh
>Assignee: Gus Heck
>Priority: Minor
> Attachments: QueryVisitorExample.java, SOLR-13759.patch,
> SOLR-13759.patch, SOLR-13759.patch, SOLR-13759.patch, SOLR-13759.patch,
> image-2019-12-09-22-45-51-721.png
>
>
> We are currently testing TRA using Solr 7.7, having >300 shards in the alias,
> with much growth in the coming months.
> The "hot" data(in our case, more recent) will be stored on stronger
> nodes(SSD, more RAM, etc).
> A proposal of optimizing queries will be by filtering query by date range, by
> that we will be able to querying the specific TRA collections taking
> advantage of the TRA mechanism of partitioning data based on date.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1157: Add RAT check using Gradle
dweiss commented on a change in pull request #1157: Add RAT check using Gradle
URL: https://github.com/apache/lucene-solr/pull/1157#discussion_r365565204
##
File path: gradle/validation/rat-sources.gradle
##
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import groovy.xml.NamespaceBuilder
+
+allprojects {
+configurations {
+rat
+}
+
+dependencies {
+rat 'org.apache.rat:apache-rat-tasks:0.13'
+}
+}
+
+subprojects {
+plugins.withId("java", {
+task("rat", type: RatTask) {
+group = 'Verification'
+description = 'Runs Apache Rat checks.'
+}
+})
+}
+
+// Somewhat inspired by existing task from Apache Kafka
+class RatTask extends DefaultTask {
+@Input
+List includes = []
+
+@Input
+List excludes = []
+
+@Input
+List additionalExcludes = []
+
+def reportDir = new File(project.buildDir, 'rat')
+def xmlReport = new File(reportDir, 'rat-report.xml')
+
+def generateXmlReport(File reportDir) {
+def uri = 'antlib:org.apache.rat.anttasks'
+def ratClasspath = project.configurations.rat.asPath
+ant.taskdef(resource: 'org/apache/rat/anttasks/antlib.xml', uri: uri,
classpath: ratClasspath)
+
+def projectPath = project.projectDir.absolutePath
+
+def rat = NamespaceBuilder.newInstance(ant, uri)
+rat.report(format: 'xml', reportFile: xmlReport,
addDefaultLicenseMatchers: true) {
+ant.fileset(dir: projectPath) {
+ant.include(name: '*.xml')
+includes.each {
+ant.include(name: it)
+}
+additionalExcludes.each {
+ant.exclude(name: it)
+}
+}
+
+// There may be a more efficient way to write these next two
blocks since they use the same logic
Review comment:
something like [project.sourceSets.main, project.sourceSets.test].each {
location -> ... }
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1157: Add RAT check using Gradle
dweiss commented on a change in pull request #1157: Add RAT check using Gradle
URL: https://github.com/apache/lucene-solr/pull/1157#discussion_r365565474
##
File path: gradle/validation/rat-sources.gradle
##
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import groovy.xml.NamespaceBuilder
+
+allprojects {
+configurations {
+rat
+}
+
+dependencies {
+rat 'org.apache.rat:apache-rat-tasks:0.13'
+}
+}
+
+subprojects {
+plugins.withId("java", {
+task("rat", type: RatTask) {
+group = 'Verification'
+description = 'Runs Apache Rat checks.'
+}
+})
+}
+
+// Somewhat inspired by existing task from Apache Kafka
+class RatTask extends DefaultTask {
+@Input
+List includes = []
+
+@Input
+List excludes = []
+
+@Input
+List additionalExcludes = []
+
+def reportDir = new File(project.buildDir, 'rat')
+def xmlReport = new File(reportDir, 'rat-report.xml')
+
+def generateXmlReport(File reportDir) {
+def uri = 'antlib:org.apache.rat.anttasks'
+def ratClasspath = project.configurations.rat.asPath
+ant.taskdef(resource: 'org/apache/rat/anttasks/antlib.xml', uri: uri,
classpath: ratClasspath)
+
+def projectPath = project.projectDir.absolutePath
+
+def rat = NamespaceBuilder.newInstance(ant, uri)
+rat.report(format: 'xml', reportFile: xmlReport,
addDefaultLicenseMatchers: true) {
+ant.fileset(dir: projectPath) {
+ant.include(name: '*.xml')
+includes.each {
+ant.include(name: it)
+}
+additionalExcludes.each {
+ant.exclude(name: it)
+}
+}
+
+// There may be a more efficient way to write these next two
blocks since they use the same logic
+project.sourceSets.main.java.srcDirs.each {
+ant.fileset(dir: it, erroronmissingdir: false) {
+excludes.each { x -> ant.exclude(name: x) }
+}
+}
+project.sourceSets.test.java.srcDirs.each {
+ant.fileset(dir: it, erroronmissingdir: false) {
+excludes.each { x -> ant.exclude(name: x) }
+}
+}
+
+project.sourceSets.main.resources.srcDirs.each {
+ant.fileset(dir: it, erroronmissingdir: false) {
+ant.include(name: 'META-INF/**')
+}
+}
+
+// The license rules below were manually copied from
lucene/common-build.xml, there is currently no mechanism to sync them
+
+// BSD 4-clause stuff (is disallowed below)
+substringMatcher(licenseFamilyCategory: "BSD4 ",
licenseFamilyName: "Original BSD License (with advertising clause)") {
+pattern(substring: "All advertising materials")
+}
+
+// BSD-like stuff
+substringMatcher(licenseFamilyCategory: "BSD ",
licenseFamilyName: "Modified BSD License") {
+// brics automaton
+pattern(substring: "Copyright (c) 2001-2009 Anders Moeller")
+// snowball
+pattern(substring: "Copyright (c) 2001, Dr Martin Porter")
+// UMASS kstem
+pattern(substring: "THIS SOFTWARE IS PROVIDED BY UNIVERSITY OF
MASSACHUSETTS AND OTHER CONTRIBUTORS")
+// Egothor
+pattern(substring: "Egothor Software License version 1.00")
+// JaSpell
+pattern(substring: "Copyright (c) 2005 Bruno Martins")
+// d3.js
+pattern(substring: "THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT
HOLDERS AND CONTRIBUTORS")
+// highlight.js
+pattern(substring: "THIS SOFTWARE IS PROVIDED BY THE REGENTS
AND CONTRIBUTORS")
+}
+
+// MIT-like
+substringMatcher(licenseFamilyCategory: "MIT ",
licenseFamilyName:"Modified BSD License") {
+// ICU license
+pattern(substring: "Permission is hereby granted, free of
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1157: Add RAT check using Gradle
dweiss commented on a change in pull request #1157: Add RAT check using Gradle
URL: https://github.com/apache/lucene-solr/pull/1157#discussion_r365565027
##
File path: gradle/validation/rat-sources.gradle
##
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import groovy.xml.NamespaceBuilder
+
+allprojects {
+configurations {
+rat
+}
+
+dependencies {
+rat 'org.apache.rat:apache-rat-tasks:0.13'
+}
+}
+
+subprojects {
+plugins.withId("java", {
Review comment:
These rat checks seem to apply not just to java projects but to other files
(non-java files) as well. If so then the task should be applicable to any
project, not just those with the java plugin?
I think we'd need to cross-check with what the ant script does (whether it
runs those checks on non-java folders).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1157: Add RAT check using Gradle
dweiss commented on a change in pull request #1157: Add RAT check using Gradle
URL: https://github.com/apache/lucene-solr/pull/1157#discussion_r365565232
##
File path: gradle/validation/rat-sources.gradle
##
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import groovy.xml.NamespaceBuilder
+
+allprojects {
+configurations {
+rat
+}
+
+dependencies {
+rat 'org.apache.rat:apache-rat-tasks:0.13'
+}
+}
+
+subprojects {
+plugins.withId("java", {
+task("rat", type: RatTask) {
+group = 'Verification'
+description = 'Runs Apache Rat checks.'
+}
+})
+}
+
+// Somewhat inspired by existing task from Apache Kafka
+class RatTask extends DefaultTask {
+@Input
+List includes = []
+
+@Input
+List excludes = []
+
+@Input
+List additionalExcludes = []
+
+def reportDir = new File(project.buildDir, 'rat')
+def xmlReport = new File(reportDir, 'rat-report.xml')
+
+def generateXmlReport(File reportDir) {
+def uri = 'antlib:org.apache.rat.anttasks'
+def ratClasspath = project.configurations.rat.asPath
+ant.taskdef(resource: 'org/apache/rat/anttasks/antlib.xml', uri: uri,
classpath: ratClasspath)
+
+def projectPath = project.projectDir.absolutePath
+
+def rat = NamespaceBuilder.newInstance(ant, uri)
+rat.report(format: 'xml', reportFile: xmlReport,
addDefaultLicenseMatchers: true) {
+ant.fileset(dir: projectPath) {
+ant.include(name: '*.xml')
+includes.each {
+ant.include(name: it)
+}
+additionalExcludes.each {
+ant.exclude(name: it)
+}
+}
+
+// There may be a more efficient way to write these next two
blocks since they use the same logic
+project.sourceSets.main.java.srcDirs.each {
+ant.fileset(dir: it, erroronmissingdir: false) {
Review comment:
I'd be explicit and use variable for closures rather than "it", especially
in larger code blocks.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
[GitHub] [lucene-solr] dweiss commented on a change in pull request #1157: Add RAT check using Gradle
dweiss commented on a change in pull request #1157: Add RAT check using Gradle
URL: https://github.com/apache/lucene-solr/pull/1157#discussion_r365565330
##
File path: gradle/validation/rat-sources.gradle
##
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import groovy.xml.NamespaceBuilder
+
+allprojects {
+configurations {
+rat
+}
+
+dependencies {
+rat 'org.apache.rat:apache-rat-tasks:0.13'
+}
+}
+
+subprojects {
+plugins.withId("java", {
+task("rat", type: RatTask) {
+group = 'Verification'
+description = 'Runs Apache Rat checks.'
+}
+})
+}
+
+// Somewhat inspired by existing task from Apache Kafka
+class RatTask extends DefaultTask {
+@Input
+List includes = []
+
+@Input
+List excludes = []
+
+@Input
+List additionalExcludes = []
+
+def reportDir = new File(project.buildDir, 'rat')
+def xmlReport = new File(reportDir, 'rat-report.xml')
+
+def generateXmlReport(File reportDir) {
+def uri = 'antlib:org.apache.rat.anttasks'
+def ratClasspath = project.configurations.rat.asPath
+ant.taskdef(resource: 'org/apache/rat/anttasks/antlib.xml', uri: uri,
classpath: ratClasspath)
+
+def projectPath = project.projectDir.absolutePath
+
+def rat = NamespaceBuilder.newInstance(ant, uri)
+rat.report(format: 'xml', reportFile: xmlReport,
addDefaultLicenseMatchers: true) {
+ant.fileset(dir: projectPath) {
+ant.include(name: '*.xml')
+includes.each {
+ant.include(name: it)
+}
+additionalExcludes.each {
+ant.exclude(name: it)
+}
+}
+
+// There may be a more efficient way to write these next two
blocks since they use the same logic
+project.sourceSets.main.java.srcDirs.each {
+ant.fileset(dir: it, erroronmissingdir: false) {
+excludes.each { x -> ant.exclude(name: x) }
+}
+}
+project.sourceSets.test.java.srcDirs.each {
+ant.fileset(dir: it, erroronmissingdir: false) {
+excludes.each { x -> ant.exclude(name: x) }
+}
+}
+
+project.sourceSets.main.resources.srcDirs.each {
+ant.fileset(dir: it, erroronmissingdir: false) {
+ant.include(name: 'META-INF/**')
+}
+}
+
+// The license rules below were manually copied from
lucene/common-build.xml, there is currently no mechanism to sync them
+
+// BSD 4-clause stuff (is disallowed below)
+substringMatcher(licenseFamilyCategory: "BSD4 ",
licenseFamilyName: "Original BSD License (with advertising clause)") {
+pattern(substring: "All advertising materials")
+}
+
+// BSD-like stuff
+substringMatcher(licenseFamilyCategory: "BSD ",
licenseFamilyName: "Modified BSD License") {
+// brics automaton
+pattern(substring: "Copyright (c) 2001-2009 Anders Moeller")
+// snowball
+pattern(substring: "Copyright (c) 2001, Dr Martin Porter")
+// UMASS kstem
+pattern(substring: "THIS SOFTWARE IS PROVIDED BY UNIVERSITY OF
MASSACHUSETTS AND OTHER CONTRIBUTORS")
+// Egothor
+pattern(substring: "Egothor Software License version 1.00")
+// JaSpell
+pattern(substring: "Copyright (c) 2005 Bruno Martins")
+// d3.js
+pattern(substring: "THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT
HOLDERS AND CONTRIBUTORS")
+// highlight.js
+pattern(substring: "THIS SOFTWARE IS PROVIDED BY THE REGENTS
AND CONTRIBUTORS")
+}
+
+// MIT-like
+substringMatcher(licenseFamilyCategory: "MIT ",
licenseFamilyName:"Modified BSD License") {
+// ICU license
+pattern(substring: "Permission is hereby granted, free of
