[jira] [Commented] (LUCENE-7717) UnifiedHighlighter doesn't highlight PrefixQuery with multi-byte chars

[Dmitry Malinin (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891753#comment-15891753
 ] 

Dmitry Malinin commented on LUCENE-7717:


Hello Christine
I saw your test. I think that 
- "I" is not suitable data for test, because StandardAnalyzer converts data to 
lower case (but PrefixQuery doesn't)
- why "random"? Сould be better make text array String[] texts = {"i", "я", } 
and run for each?
- and I mean that result should be, for example, "q" for data "q" and 
query "q*" and so on for multi-byte data (assertEquals(""+text+"", 
snippetString);)
Best regards


> UnifiedHighlighter doesn't highlight PrefixQuery with multi-byte chars
> --
>
> Key: LUCENE-7717
> URL: https://issues.apache.org/jira/browse/LUCENE-7717
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/highlighter
>Affects Versions: 5.1, 6.3, 6.4.1
>Reporter: Dmitry Malinin
>Assignee: David Smiley
> Fix For: 6.4.2
>
> Attachments: LUCENE-7717.patch, LUCENE-7717.patch
>
>
> UnifiedHighlighter highlighter = new UnifiedHighlighter(null, new 
> StandardAnalyzer());
> Query query = new PrefixQuery(new Term("title", "я"));
> String testData = "я";
> Object snippet = highlighter.highlightWithoutSearcher(fieldName, query, 
> testData, 1);
> System.out.printf("testData=[%s] Query=%s snippet=[%s]\n", testData, query, 
> snippet==null?null:snippet.toString());



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_121) - Build # 19085 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/19085/
Java: 64bit/jdk1.8.0_121 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI

Error Message:
expected:<3> but was:<2>

Stack Trace:
java.lang.AssertionError: expected:<3> but was:<2>
at 
__randomizedtesting.SeedInfo.seed([517E18C9699946AA:190B6C7D6FAA693F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:523)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11916 lines...]
   [junit4] Suite: 

[JENKINS] Lucene-Solr-Tests-6.4 - Build # 25 - Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.4/25/

4 tests failed.
FAILED:  org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv_stored

Error Message:
There are still nodes recoverying - waited for 330 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 330 
seconds
at 
__randomizedtesting.SeedInfo.seed([BDF6EB09ECC5047B:2FCA8A8D508239A3]:0)
at org.junit.Assert.fail(Assert.java:93)
at 
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:187)
at 
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.waitForRecoveriesToFinish(TestStressCloudBlindAtomicUpdates.java:459)
at 
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:304)
at 
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv_stored(TestStressCloudBlindAtomicUpdates.java:203)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[JENKINS] Lucene-Solr-6.x-Linux (32bit/jdk1.8.0_121) - Build # 2976 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/2976/
Java: 32bit/jdk1.8.0_121 -server -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.TestDistributedSearch.test

Error Message:
Timeout occured while waiting response from server at: 
http://127.0.0.1:37002/aeoo/collection1

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting 
response from server at: http://127.0.0.1:37002/aeoo/collection1
at 
__randomizedtesting.SeedInfo.seed([540414553544FB1C:DC502B8F9BB896E4]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:621)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:957)
at 
org.apache.solr.TestDistributedSearch.queryRandomUpServer(TestDistributedSearch.java:1153)
at 
org.apache.solr.TestDistributedSearch.queryPartialResults(TestDistributedSearch.java:1115)
at 
org.apache.solr.TestDistributedSearch.test(TestDistributedSearch.java:959)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsRepeatStatement.callStatement(BaseDistributedSearchTestCase.java:1018)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Commented] (SOLR-10205) Evaluate and reduce BlockCache store failures

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891600#comment-15891600
 ] 

Yonik Seeley commented on SOLR-10205:
-

I also plan on trying out using a higher number of reserved blocks (like 2 or 
3) instead of the current 1.  This helps because if 2 threads both try to cache 
blocks at the same time, one will grab the reserved block first, then the other 
will have to wait until an older entry is evicted from the map (caused by the 
fact that the first thread will insert a new entry).

> Evaluate and reduce BlockCache store failures
> -
>
> Key: SOLR-10205
> URL: https://issues.apache.org/jira/browse/SOLR-10205
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Yonik Seeley
>Assignee: Yonik Seeley
> Attachments: SOLR-10205.patch
>
>
> The BlockCache is written such that requests to cache a block 
> (BlockCache.store call) can fail, making caching less effective.  We should 
> evaluate the impact of this storage failure and potentially reduce the number 
> of storage failures.
> The implementation reserves a single block of memory.  In store, a block of 
> memory is allocated, and then a pointer is inserted into the underling map.  
> A block is only freed when the underlying map evicts the map entry.
> This means that when two store() operations are called concurrently (even 
> under low load), one can fail.  This is made worse by the fact that 
> concurrent maps typically tend to amortize the cost of eviction over many 
> keys (i.e. the actual size of the map can grow beyond the configured maximum 
> number of entries... both the older ConcurrentLinkedHashMap and newer 
> Caffeine do this).  When this is the case, store() won't be able to find a 
> free block of memory, even if there aren't any other concurrently operating 
> stores.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7728) TestGrouping.testRandom() failures

[Steve Rowe (JIRA)]

Steve Rowe created LUCENE-7728:
--

 Summary: TestGrouping.testRandom() failures
 Key: LUCENE-7728
 URL: https://issues.apache.org/jira/browse/LUCENE-7728
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Steve Rowe


My Jenkins found a reproducing branch_6x seed:

{noformat}
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGrouping 
-Dtests.method=testRandom -Dtests.seed=6F0B519BBDC786B3 -Dtests.slow=true 
-Dtests.locale=en-CA -Dtests.timezone=Europe/Zagreb -Dtests.asserts=true 
-Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 2.50s J0 | TestGrouping.testRandom <<<
   [junit4]> Throwable #1: java.lang.AssertionError: expected:<10> but 
was:<29>
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([6F0B519BBDC786B3:1D4774940CA730C0]:0)
   [junit4]>at 
org.apache.lucene.search.grouping.TestGrouping.assertEquals(TestGrouping.java:1288)
   [junit4]>at 
org.apache.lucene.search.grouping.TestGrouping.testRandom(TestGrouping.java:1130)
   [junit4]>at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): 
{groupend=Lucene50(blocksize=128), sort1=PostingsFormat(name=Memory doPackFST= 
true), sort2=Lucene50(blocksize=128), content=PostingsFormat(name=Memory 
doPackFST= false), group=PostingsFormat(name=Memory doPackFST= true)}, 
docValues:{groupend=DocValuesFormat(name=Direct), 
author=DocValuesFormat(name=Memory), sort1=DocValuesFormat(name=Memory), 
id=DocValuesFormat(name=Memory), sort2=DocValuesFormat(name=Direct), 
content=DocValuesFormat(name=Lucene54), group=DocValuesFormat(name=Memory)}, 
maxPointsInLeafNode=454, maxMBSortInHeap=6.048552291438714, 
sim=RandomSimilarity(queryNorm=false,coord=no): {content=DFI(Saturated)}, 
locale=en-CA, timezone=Europe/Zagreb
   [junit4]   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 
1.8.0_77 (64-bit)/cpus=16,threads=1,free=485264792,total=514850816
{noformat}

{{git bisect}} says the first bad commit is this one: 
[http://git-wip-us.apache.org/repos/asf/lucene-solr/commit/6d952076], but that 
just removed {{BooleanSimilarity}} from the {{RandomSimilarity}}'s candidates 
list, which in this case likely has the side effect of picking a different 
similarity with this seed.  So the problem is almost certainly older than this.

Policeman Jenkins reported a master seed four weeks ago 
[https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18890/] for a failure 
that's still reproducing for me on Linux w/ Java8:

{noformat}
  [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestGrouping 
-Dtests.method=testRandom -Dtests.seed=381A6A2BB3B21736 -Dtests.multiplier=3 
-Dtests.slow=true -Dtests.locale=nyn -Dtests.timezone=Antarctica/McMurdo 
-Dtests.asserts=true -Dtests.file.encoding=US-ASCII
  [junit4] FAILURE 8.72s J0 | TestGrouping.testRandom <<<
  [junit4]> Throwable #1: java.lang.AssertionError: expected:<2526> but 
was:<1818>
  [junit4]> at 
__randomizedtesting.SeedInfo.seed([381A6A2BB3B21736:4A564F2402D2A145]:0)
  [junit4]> at 
org.apache.lucene.search.grouping.TestGrouping.assertEquals(TestGrouping.java:1299)
  [junit4]> at 
org.apache.lucene.search.grouping.TestGrouping.testRandom(TestGrouping.java:1141)
  [junit4]> at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  [junit4]> at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  [junit4]> at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  [junit4]> at 
java.base/java.lang.reflect.Method.invoke(Method.java:543)
  [junit4]> at java.base/java.lang.Thread.run(Thread.java:844)
  [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): 
{groupend=PostingsFormat(name=Direct), 
sort1=PostingsFormat(name=LuceneVarGapFixedInterval), 
sort2=PostingsFormat(name=Direct), 
content=PostingsFormat(name=LuceneVarGapDocFreqInterval), 
group=PostingsFormat(name=LuceneVarGapFixedInterval)}, 
docValues:{groupend=DocValuesFormat(name=Lucene70), 
author=DocValuesFormat(name=Memory), sort1=DocValuesFormat(name=Memory), 
id=DocValuesFormat(name=Memory), sort2=DocValuesFormat(name=Lucene70), 
content=DocValuesFormat(name=Direct), group=DocValuesFormat(name=Memory)}, 
maxPointsInLeafNode=1292, maxMBSortInHeap=6.668104559353747, 
sim=RandomSimilarity(queryNorm=true): {content=DFI(Standardized)}, locale=nyn, 
timezone=Antarctica/McMurdo
  [junit4]   2> NOTE: Linux 4.4.0-53-generic i386/Oracle Corporation 9-ea 
(32-bit)/cpus=12,threads=1,free=23290544,total=79691776
{noformat}

{{git bisect}} points to the exact same first bad commit for this seed too (see 
above).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (SOLR-10220) The failure message will not disappeared after the error was fixed on solr web UI.

[shangpingping (JIRA)]

shangpingping created SOLR-10220:


 Summary: The failure message will not disappeared after the error 
was fixed on solr web UI.
 Key: SOLR-10220
 URL: https://issues.apache.org/jira/browse/SOLR-10220
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 5.3.1
Reporter: shangpingping


The failure message will not disappeared after the error was fixed on solr web 
UI.
The solr web UI always shows  "SolrCore Initialization Failures" and the detail 
message even if the error already be fixed.
How to remove that or how to update that?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7722) Remove BoostedQuery

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891495#comment-15891495
 ] 

David Smiley commented on LUCENE-7722:
--

It's wonderful to see some cleanup in the works of these confusingly similar 
queries :-)

> Remove BoostedQuery
> ---
>
> Key: LUCENE-7722
> URL: https://issues.apache.org/jira/browse/LUCENE-7722
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Adrien Grand
>Priority: Minor
>
> We already  have FunctionScoreQuery, which is more flexible than BoostedQuery 
> as it can combine scores in arbitrary ways and only requests scores on the 
> underlying scorer if they are needed. So let's remove BoostedQuery?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7682) UnifiedHighlighter not highlighting all terms relevant in SpanNearQuery

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891492#comment-15891492
 ] 

David Smiley commented on LUCENE-7682:
--

Michael, don't worry about how you filed it; this happens often -- report the 
bug and then dig deeper and see it's indirectly caused by something else.  
Here, it's not clear is LUCENE-7398 or LUCENE-7580 will fix it; I've been 
following these issues; we'll see.   [~paul.elsc...@xs4all.nl] I'm glad you 
noticed this issue; this is in your area of interest :-)

> UnifiedHighlighter not highlighting all terms relevant in SpanNearQuery
> ---
>
> Key: LUCENE-7682
> URL: https://issues.apache.org/jira/browse/LUCENE-7682
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/highlighter
>Reporter: Michael Braun
>
> Original text: "Something for protecting wildlife feed in a feed thing."
> Query is:
>SpanNearQuery with Slop 9 - in order - 
>   1. SpanTermQuery(wildlife)
>   2. SpanTermQuery(feed)
> This should highlight both instances of "feed" since they are both within 
> slop of 9 of "wildlife". However, only the first instance is highlighted. 
> This occurs with unordered SpanNearQuery as well.  Test below replicates. 
> Affects both the current 6.x line and master.
> Test that fits within TestUnifiedHighlighterMTQ:
> {code}
>   public void testOrderedSpanNearQueryWithDupeTerms() throws Exception {
> RandomIndexWriter iw = new RandomIndexWriter(random(), dir, 
> indexAnalyzer);
> Document doc = new Document();
> doc.add(new Field("body", "Something for protecting wildlife feed in a 
> feed thing.", fieldType));
> doc.add(newTextField("id", "id", Field.Store.YES));
> iw.addDocument(doc);
> IndexReader ir = iw.getReader();
> iw.close();
> IndexSearcher searcher = newSearcher(ir);
> UnifiedHighlighter highlighter = new UnifiedHighlighter(searcher, 
> indexAnalyzer);
> int docID = searcher.search(new TermQuery(new Term("id", "id")), 
> 1).scoreDocs[0].doc;
> SpanTermQuery termOne = new SpanTermQuery(new Term("body", "wildlife"));
> SpanTermQuery termTwo = new SpanTermQuery(new Term("body", "feed"));
> SpanNearQuery topQuery = new SpanNearQuery.Builder("body", true)
> .setSlop(9)
> .addClause(termOne)
> .addClause(termTwo)
> .build();
> int[] docIds = new int[] {docID};
> String snippets[] = highlighter.highlightFields(new String[] {"body"}, 
> topQuery, docIds, new int[] {2}).get("body");
> assertEquals(1, snippets.length);
> assertEquals("Something for protecting wildlife feed in a 
> feed thing.", snippets[0]);
> ir.close();
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-10152) PostingsSolrHighlighter support for CustomSeparatorBreakIterator (LUCENE-6485)

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley resolved SOLR-10152.
-
   Resolution: Fixed
 Assignee: David Smiley
Fix Version/s: 6.5

> PostingsSolrHighlighter support for CustomSeparatorBreakIterator (LUCENE-6485)
> --
>
> Key: SOLR-10152
> URL: https://issues.apache.org/jira/browse/SOLR-10152
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Amrit Sarkar
>Assignee: David Smiley
> Fix For: 6.5
>
> Attachments: SOLR-10152.patch
>
>
> Lucene 5.3 added a CustomSeparatorBreakIterator (see LUCENE-6485)
> SOLR-10152.patch uploaded which incorporates CustomSeparatorBreakIterator in 
> PostingsSolrHighlighter.
> - added a new request param option to specify which separator char to use. 
> *customSeparatorChar*.
> - changed PostingsSolrHighlighter.getBreakIterator to check 
> HighlightParams.BS_TYPE first.
> - if type=='CUSTOM', look for the new separator param, in getBreakIterator, 
> validate it's a single char, & skip locale parsing.
> - 'WHOLE' option moved from parseBreakIterator to getBreakIterator, as it 
> doesn't depend on locale.
> Changes made in:
> * HighlightParams.java
> * PostingsSolrHighlighter.java
> * test cases added in TestPostingsSolrHighlighter



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-10153) UnifiedSolrHighlighter support for CustomSeparatorBreakIterator (LUCENE-6485)

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley resolved SOLR-10153.
-
   Resolution: Fixed
 Assignee: David Smiley
Fix Version/s: 6.5

> UnifiedSolrHighlighter support for CustomSeparatorBreakIterator (LUCENE-6485)
> -
>
> Key: SOLR-10153
> URL: https://issues.apache.org/jira/browse/SOLR-10153
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Amrit Sarkar
>Assignee: David Smiley
> Fix For: 6.5
>
> Attachments: SOLR-10153.patch, SOLR-10153.patch, 
> SOLR_10153_UH_and_PH_hl_customSeparatorChar.patch, 
> SOLR_10153_UH_and_PH_hl_customSeparatorChar.patch
>
>
> Lucene 5.3 added a CustomSeparatorBreakIterator (see LUCENE-6485)
> UnifiedSolrHighlighter should support *CustomSeparatorBreakIterator* along 
> with existing ones, WholeBreakIterator etc.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10152) PostingsSolrHighlighter support for CustomSeparatorBreakIterator (LUCENE-6485)

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891481#comment-15891481
 ] 

ASF subversion and git services commented on SOLR-10152:


Commit a607a2c6cfdeb191b3da4474e87d4242b1270fd1 in lucene-solr's branch 
refs/heads/branch_6x from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a607a2c ]

SOLR-10153: (and SOLR-10152): UH & PH: Add hl.bs.type=SEPARATOR with new param 
hl.bs.separator

(cherry picked from commit d1d73bf)


> PostingsSolrHighlighter support for CustomSeparatorBreakIterator (LUCENE-6485)
> --
>
> Key: SOLR-10152
> URL: https://issues.apache.org/jira/browse/SOLR-10152
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Amrit Sarkar
> Attachments: SOLR-10152.patch
>
>
> Lucene 5.3 added a CustomSeparatorBreakIterator (see LUCENE-6485)
> SOLR-10152.patch uploaded which incorporates CustomSeparatorBreakIterator in 
> PostingsSolrHighlighter.
> - added a new request param option to specify which separator char to use. 
> *customSeparatorChar*.
> - changed PostingsSolrHighlighter.getBreakIterator to check 
> HighlightParams.BS_TYPE first.
> - if type=='CUSTOM', look for the new separator param, in getBreakIterator, 
> validate it's a single char, & skip locale parsing.
> - 'WHOLE' option moved from parseBreakIterator to getBreakIterator, as it 
> doesn't depend on locale.
> Changes made in:
> * HighlightParams.java
> * PostingsSolrHighlighter.java
> * test cases added in TestPostingsSolrHighlighter



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10153) UnifiedSolrHighlighter support for CustomSeparatorBreakIterator (LUCENE-6485)

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891480#comment-15891480
 ] 

ASF subversion and git services commented on SOLR-10153:


Commit a607a2c6cfdeb191b3da4474e87d4242b1270fd1 in lucene-solr's branch 
refs/heads/branch_6x from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a607a2c ]

SOLR-10153: (and SOLR-10152): UH & PH: Add hl.bs.type=SEPARATOR with new param 
hl.bs.separator

(cherry picked from commit d1d73bf)


> UnifiedSolrHighlighter support for CustomSeparatorBreakIterator (LUCENE-6485)
> -
>
> Key: SOLR-10153
> URL: https://issues.apache.org/jira/browse/SOLR-10153
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Amrit Sarkar
> Attachments: SOLR-10153.patch, SOLR-10153.patch, 
> SOLR_10153_UH_and_PH_hl_customSeparatorChar.patch, 
> SOLR_10153_UH_and_PH_hl_customSeparatorChar.patch
>
>
> Lucene 5.3 added a CustomSeparatorBreakIterator (see LUCENE-6485)
> UnifiedSolrHighlighter should support *CustomSeparatorBreakIterator* along 
> with existing ones, WholeBreakIterator etc.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10152) PostingsSolrHighlighter support for CustomSeparatorBreakIterator (LUCENE-6485)

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891478#comment-15891478
 ] 

ASF subversion and git services commented on SOLR-10152:


Commit d1d73bfbea3db4adead960fae3597bec7647fba6 in lucene-solr's branch 
refs/heads/master from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d1d73bf ]

SOLR-10153: (and SOLR-10152): UH & PH: Add hl.bs.type=SEPARATOR with new param 
hl.bs.separator


> PostingsSolrHighlighter support for CustomSeparatorBreakIterator (LUCENE-6485)
> --
>
> Key: SOLR-10152
> URL: https://issues.apache.org/jira/browse/SOLR-10152
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Amrit Sarkar
> Attachments: SOLR-10152.patch
>
>
> Lucene 5.3 added a CustomSeparatorBreakIterator (see LUCENE-6485)
> SOLR-10152.patch uploaded which incorporates CustomSeparatorBreakIterator in 
> PostingsSolrHighlighter.
> - added a new request param option to specify which separator char to use. 
> *customSeparatorChar*.
> - changed PostingsSolrHighlighter.getBreakIterator to check 
> HighlightParams.BS_TYPE first.
> - if type=='CUSTOM', look for the new separator param, in getBreakIterator, 
> validate it's a single char, & skip locale parsing.
> - 'WHOLE' option moved from parseBreakIterator to getBreakIterator, as it 
> doesn't depend on locale.
> Changes made in:
> * HighlightParams.java
> * PostingsSolrHighlighter.java
> * test cases added in TestPostingsSolrHighlighter



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10153) UnifiedSolrHighlighter support for CustomSeparatorBreakIterator (LUCENE-6485)

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891477#comment-15891477
 ] 

ASF subversion and git services commented on SOLR-10153:


Commit d1d73bfbea3db4adead960fae3597bec7647fba6 in lucene-solr's branch 
refs/heads/master from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d1d73bf ]

SOLR-10153: (and SOLR-10152): UH & PH: Add hl.bs.type=SEPARATOR with new param 
hl.bs.separator


> UnifiedSolrHighlighter support for CustomSeparatorBreakIterator (LUCENE-6485)
> -
>
> Key: SOLR-10153
> URL: https://issues.apache.org/jira/browse/SOLR-10153
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Amrit Sarkar
> Attachments: SOLR-10153.patch, SOLR-10153.patch, 
> SOLR_10153_UH_and_PH_hl_customSeparatorChar.patch, 
> SOLR_10153_UH_and_PH_hl_customSeparatorChar.patch
>
>
> Lucene 5.3 added a CustomSeparatorBreakIterator (see LUCENE-6485)
> UnifiedSolrHighlighter should support *CustomSeparatorBreakIterator* along 
> with existing ones, WholeBreakIterator etc.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10205) Evaluate and reduce BlockCache store failures

[Yonik Seeley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yonik Seeley updated SOLR-10205:

Attachment: SOLR-10205.patch

Here's a snapshot of the modifications and tests I'm using to reduce store 
failures and evaluate the potential impact.

It's messy... yet another hacked up version of the HDFS performance test I used 
to uncover the blockcache corruption issues.   I don't plan on committing most 
of this.  It's just for evaluating what should be committed to reduce store 
failures.

> Evaluate and reduce BlockCache store failures
> -
>
> Key: SOLR-10205
> URL: https://issues.apache.org/jira/browse/SOLR-10205
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Yonik Seeley
>Assignee: Yonik Seeley
> Attachments: SOLR-10205.patch
>
>
> The BlockCache is written such that requests to cache a block 
> (BlockCache.store call) can fail, making caching less effective.  We should 
> evaluate the impact of this storage failure and potentially reduce the number 
> of storage failures.
> The implementation reserves a single block of memory.  In store, a block of 
> memory is allocated, and then a pointer is inserted into the underling map.  
> A block is only freed when the underlying map evicts the map entry.
> This means that when two store() operations are called concurrently (even 
> under low load), one can fail.  This is made worse by the fact that 
> concurrent maps typically tend to amortize the cost of eviction over many 
> keys (i.e. the actual size of the map can grow beyond the configured maximum 
> number of entries... both the older ConcurrentLinkedHashMap and newer 
> Caffeine do this).  When this is the case, store() won't be able to find a 
> free block of memory, even if there aren't any other concurrently operating 
> stores.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Issue Comment Deleted] (SOLR-10153) UnifiedSolrHighlighter support for CustomSeparatorBreakIterator (LUCENE-6485)

[Amrit Sarkar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amrit Sarkar updated SOLR-10153:

Comment: was deleted

(was: It seems it was bigger pain for you refactoring everything than me 
cooking up the patch, really appreciate it.)

> UnifiedSolrHighlighter support for CustomSeparatorBreakIterator (LUCENE-6485)
> -
>
> Key: SOLR-10153
> URL: https://issues.apache.org/jira/browse/SOLR-10153
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Amrit Sarkar
> Attachments: SOLR-10153.patch, SOLR-10153.patch, 
> SOLR_10153_UH_and_PH_hl_customSeparatorChar.patch, 
> SOLR_10153_UH_and_PH_hl_customSeparatorChar.patch
>
>
> Lucene 5.3 added a CustomSeparatorBreakIterator (see LUCENE-6485)
> UnifiedSolrHighlighter should support *CustomSeparatorBreakIterator* along 
> with existing ones, WholeBreakIterator etc.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7727) Replace EOL'ed pegdown by flexmark-java for Java 9 compatibility

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-7727:
--
Attachment: LUCENE-7727.patch

Rewrite to use flexmark.

I changed the task names to use "markdown" and let the implementation be hidden 
(which is flexmark after this patch).

There is some room for improvements by not using the PegDown Adapter (this 
removes many dependencies). I will look into this tomorrow. For now build works 
with Java 9 and the HTML output matches.

> Replace EOL'ed pegdown by flexmark-java for Java 9 compatibility
> 
>
> Key: LUCENE-7727
> URL: https://issues.apache.org/jira/browse/LUCENE-7727
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: general/build
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>  Labels: Java9
> Attachments: LUCENE-7727.patch
>
>
> The documentation tasks use a library called "pegdown" to convert Markdown to 
> HTML. Unfortunately, the developer of pegdown EOLed it and points the users 
> to a faster replacement: flexmark-java 
> (https://github.com/vsch/flexmark-java).
> This would not be important for us, if pegdown would work with Java 9, but it 
> is also affected by the usual "setAccessible into private Java APIs" issue 
> (see my talk at FOSDEM: 
> https://fosdem.org/2017/schedule/event/jigsaw_challenges/).
> The migration should not be too hard, its just a bit of Groovy Code rewriting 
> and dependency changes.
> This is the pegdown problem:
> {noformat}
> Caused by: java.lang.RuntimeException: Could not determine whether class 
> 'org.pegdown.Parser$$parboiled' has already been loaded
> at org.parboiled.transform.AsmUtils.findLoadedClass(AsmUtils.java:213)
> at 
> org.parboiled.transform.ParserTransformer.transformParser(ParserTransformer.java:35)
> at org.parboiled.Parboiled.createParser(Parboiled.java:54)
> ... 50 more
> Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make 
> protected final java.lang.Class 
> java.lang.ClassLoader.findLoadedClass(java.lang.String) accessible: module 
> java.base does not "opens java.lang" to unnamed module @551b6736
> at 
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:335)
> at 
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:278)
> at 
> java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:196)
> at java.base/java.lang.reflect.Method.setAccessible(Method.java:190)
> at org.parboiled.transform.AsmUtils.findLoadedClass(AsmUtils.java:206)
> ... 52 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7726) Fix Javadocs HTML entity bugs

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891278#comment-15891278
 ] 

Uwe Schindler commented on LUCENE-7726:
---

FYI, here is the HTML5 definition of "ambiguous ampersand": 
[https://www.w3.org/TR/html5/syntax.html#syntax-ambiguous-ampersand]
In fact, Java 9's Javadocs and Javac parser do not use that definition, which 
may be seen as a bug, but in fact this definition is just horrible and was 
added to make sloppy web developers happy...

> Fix Javadocs HTML entity bugs
> -
>
> Key: LUCENE-7726
> URL: https://issues.apache.org/jira/browse/LUCENE-7726
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/javadocs
>Reporter: Hoss Man
>Assignee: Uwe Schindler
>  Labels: Java9
> Fix For: master (7.0), 6.5
>
> Attachments: LUCENE-7726.patch
>
>
> As of jdk9-ea-b158, {{ant documentation}} seems to build the core javadocs 
> just fine, but fails on the  {{lucene/memory/}} javadocs...
> {noformat}
> javadocs:
> [mkdir] Created dir: /home/hossman/lucene/dev/lucene/build/docs/memory
> download-java8-javadoc-packagelist:
>   [javadoc] Generating Javadoc
>   [javadoc] Javadoc execution
>   [javadoc] Loading source files for package org.apache.lucene.index.memory...
>   [javadoc] Constructing Javadoc information...
>   [javadoc] Standard Doclet version 9-ea
>   [javadoc] Building tree for all the packages and classes...
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] Building index for all the packages and classes...
>   [javadoc] Building index for all classes...
>   [javadoc] Generating 
> /home/hossman/lucene/dev/lucene/build/docs/memory/help-doc.html...
>   [javadoc] Note: Custom tags that were not seen:  @lucene.internal
>   [javadoc] 3 warnings
> BUILD FAILED
> /home/hossman/lucene/dev/build.xml:93: The following error occurred while 
> executing this line:
> /home/hossman/lucene/dev/lucene/build.xml:251: The following error occurred 
> while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2179: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:549: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:65: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:78: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2155: Javadocs warnings were 
> found!
> Total time: 1 minute 0 seconds
> {noformat}
> looking at the generated html files turns up this...
> {noformat}
> hossman@tray:~/lucene/dev [master] $ find lucene/build/docs/memory -name 
> \*.html | xargs grep -C5 ""
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- but 
> rather thrown away immediately after tokenization.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- For 
> some interesting background information on search technology, see Bob Wyman's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://bobwyman.pubsub.com/main/2005/05/mary_hodder_poi.html;>Prospective
>  Search, 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- Jim 
> Gray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html:  target="_blank" 
> href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- A 
> Call to Arms - Custom subscriptions, and Tim Bray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://www.tbray.org/ongoing/When/200x/2003/07/30/OnSearchTOC;>On 
> Search, the Series.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> Example Usage 
> {noformat}
> The source java file has this...
> {noformat}
>  * Jim Gray's
>  *  href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
>  * A Call to Arms - Custom subscriptions, and Tim Bray's
> {noformat}
> ...which does in fact seem to be invalid HTML ... aren't {{&}} always suppose 
> to be encoded as {{}} ... even in URLs?
> I'm suprised the java8 javadocs/linter don't warn about this.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (SOLR-10153) UnifiedSolrHighlighter support for CustomSeparatorBreakIterator (LUCENE-6485)

[Amrit Sarkar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891270#comment-15891270
 ] 

Amrit Sarkar commented on SOLR-10153:
-

It seems it was bigger pain for you refactoring everything than me cooking up 
the patch, really appreciate it.

> UnifiedSolrHighlighter support for CustomSeparatorBreakIterator (LUCENE-6485)
> -
>
> Key: SOLR-10153
> URL: https://issues.apache.org/jira/browse/SOLR-10153
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Amrit Sarkar
> Attachments: SOLR-10153.patch, SOLR-10153.patch, 
> SOLR_10153_UH_and_PH_hl_customSeparatorChar.patch, 
> SOLR_10153_UH_and_PH_hl_customSeparatorChar.patch
>
>
> Lucene 5.3 added a CustomSeparatorBreakIterator (see LUCENE-6485)
> UnifiedSolrHighlighter should support *CustomSeparatorBreakIterator* along 
> with existing ones, WholeBreakIterator etc.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7727) Replace EOL'ed pegdown by flexmark-java for Java 9 compatibility

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-7727:
--
Labels: Java9  (was: )

> Replace EOL'ed pegdown by flexmark-java for Java 9 compatibility
> 
>
> Key: LUCENE-7727
> URL: https://issues.apache.org/jira/browse/LUCENE-7727
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: general/build
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>  Labels: Java9
>
> The documentation tasks use a library called "pegdown" to convert Markdown to 
> HTML. Unfortunately, the developer of pegdown EOLed it and points the users 
> to a faster replacement: flexmark-java 
> (https://github.com/vsch/flexmark-java).
> This would not be important for us, if pegdown would work with Java 9, but it 
> is also affected by the usual "setAccessible into private Java APIs" issue 
> (see my talk at FOSDEM: 
> https://fosdem.org/2017/schedule/event/jigsaw_challenges/).
> The migration should not be too hard, its just a bit of Groovy Code rewriting 
> and dependency changes.
> This is the pegdown problem:
> {noformat}
> Caused by: java.lang.RuntimeException: Could not determine whether class 
> 'org.pegdown.Parser$$parboiled' has already been loaded
> at org.parboiled.transform.AsmUtils.findLoadedClass(AsmUtils.java:213)
> at 
> org.parboiled.transform.ParserTransformer.transformParser(ParserTransformer.java:35)
> at org.parboiled.Parboiled.createParser(Parboiled.java:54)
> ... 50 more
> Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make 
> protected final java.lang.Class 
> java.lang.ClassLoader.findLoadedClass(java.lang.String) accessible: module 
> java.base does not "opens java.lang" to unnamed module @551b6736
> at 
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:335)
> at 
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:278)
> at 
> java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:196)
> at java.base/java.lang.reflect.Method.setAccessible(Method.java:190)
> at org.parboiled.transform.AsmUtils.findLoadedClass(AsmUtils.java:206)
> ... 52 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (LUCENE-7727) Replace EOL'ed pegdown by flexmark-java for Java 9 compatibility

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler reassigned LUCENE-7727:
-

Assignee: Uwe Schindler

> Replace EOL'ed pegdown by flexmark-java for Java 9 compatibility
> 
>
> Key: LUCENE-7727
> URL: https://issues.apache.org/jira/browse/LUCENE-7727
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: general/build
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>  Labels: Java9
>
> The documentation tasks use a library called "pegdown" to convert Markdown to 
> HTML. Unfortunately, the developer of pegdown EOLed it and points the users 
> to a faster replacement: flexmark-java 
> (https://github.com/vsch/flexmark-java).
> This would not be important for us, if pegdown would work with Java 9, but it 
> is also affected by the usual "setAccessible into private Java APIs" issue 
> (see my talk at FOSDEM: 
> https://fosdem.org/2017/schedule/event/jigsaw_challenges/).
> The migration should not be too hard, its just a bit of Groovy Code rewriting 
> and dependency changes.
> This is the pegdown problem:
> {noformat}
> Caused by: java.lang.RuntimeException: Could not determine whether class 
> 'org.pegdown.Parser$$parboiled' has already been loaded
> at org.parboiled.transform.AsmUtils.findLoadedClass(AsmUtils.java:213)
> at 
> org.parboiled.transform.ParserTransformer.transformParser(ParserTransformer.java:35)
> at org.parboiled.Parboiled.createParser(Parboiled.java:54)
> ... 50 more
> Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make 
> protected final java.lang.Class 
> java.lang.ClassLoader.findLoadedClass(java.lang.String) accessible: module 
> java.base does not "opens java.lang" to unnamed module @551b6736
> at 
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:335)
> at 
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:278)
> at 
> java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:196)
> at java.base/java.lang.reflect.Method.setAccessible(Method.java:190)
> at org.parboiled.transform.AsmUtils.findLoadedClass(AsmUtils.java:206)
> ... 52 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7727) Replace EOL'ed pegdown by flexmark-java for Java 9 compatibility

[Uwe Schindler (JIRA)]

Uwe Schindler created LUCENE-7727:
-

 Summary: Replace EOL'ed pegdown by flexmark-java for Java 9 
compatibility
 Key: LUCENE-7727
 URL: https://issues.apache.org/jira/browse/LUCENE-7727
 Project: Lucene - Core
  Issue Type: Improvement
  Components: general/build
Reporter: Uwe Schindler


The documentation tasks use a library called "pegdown" to convert Markdown to 
HTML. Unfortunately, the developer of pegdown EOLed it and points the users to 
a faster replacement: flexmark-java (https://github.com/vsch/flexmark-java).

This would not be important for us, if pegdown would work with Java 9, but it 
is also affected by the usual "setAccessible into private Java APIs" issue (see 
my talk at FOSDEM: https://fosdem.org/2017/schedule/event/jigsaw_challenges/).

The migration should not be too hard, its just a bit of Groovy Code rewriting 
and dependency changes.

This is the pegdown problem:

{noformat}
Caused by: java.lang.RuntimeException: Could not determine whether class 
'org.pegdown.Parser$$parboiled' has already been loaded
at org.parboiled.transform.AsmUtils.findLoadedClass(AsmUtils.java:213)
at 
org.parboiled.transform.ParserTransformer.transformParser(ParserTransformer.java:35)
at org.parboiled.Parboiled.createParser(Parboiled.java:54)
... 50 more
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make 
protected final java.lang.Class 
java.lang.ClassLoader.findLoadedClass(java.lang.String) accessible: module 
java.base does not "opens java.lang" to unnamed module @551b6736
at 
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:335)
at 
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:278)
at 
java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:196)
at java.base/java.lang.reflect.Method.setAccessible(Method.java:190)
at org.parboiled.transform.AsmUtils.findLoadedClass(AsmUtils.java:206)
... 52 more
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-7726) Fix Javadocs HTML entity bugs

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler resolved LUCENE-7726.
---
Resolution: Fixed

Thanks [~hossman] for reminding me!

> Fix Javadocs HTML entity bugs
> -
>
> Key: LUCENE-7726
> URL: https://issues.apache.org/jira/browse/LUCENE-7726
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/javadocs
>Reporter: Hoss Man
>Assignee: Uwe Schindler
>  Labels: Java9
> Fix For: master (7.0), 6.5
>
> Attachments: LUCENE-7726.patch
>
>
> As of jdk9-ea-b158, {{ant documentation}} seems to build the core javadocs 
> just fine, but fails on the  {{lucene/memory/}} javadocs...
> {noformat}
> javadocs:
> [mkdir] Created dir: /home/hossman/lucene/dev/lucene/build/docs/memory
> download-java8-javadoc-packagelist:
>   [javadoc] Generating Javadoc
>   [javadoc] Javadoc execution
>   [javadoc] Loading source files for package org.apache.lucene.index.memory...
>   [javadoc] Constructing Javadoc information...
>   [javadoc] Standard Doclet version 9-ea
>   [javadoc] Building tree for all the packages and classes...
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] Building index for all the packages and classes...
>   [javadoc] Building index for all classes...
>   [javadoc] Generating 
> /home/hossman/lucene/dev/lucene/build/docs/memory/help-doc.html...
>   [javadoc] Note: Custom tags that were not seen:  @lucene.internal
>   [javadoc] 3 warnings
> BUILD FAILED
> /home/hossman/lucene/dev/build.xml:93: The following error occurred while 
> executing this line:
> /home/hossman/lucene/dev/lucene/build.xml:251: The following error occurred 
> while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2179: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:549: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:65: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:78: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2155: Javadocs warnings were 
> found!
> Total time: 1 minute 0 seconds
> {noformat}
> looking at the generated html files turns up this...
> {noformat}
> hossman@tray:~/lucene/dev [master] $ find lucene/build/docs/memory -name 
> \*.html | xargs grep -C5 ""
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- but 
> rather thrown away immediately after tokenization.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- For 
> some interesting background information on search technology, see Bob Wyman's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://bobwyman.pubsub.com/main/2005/05/mary_hodder_poi.html;>Prospective
>  Search, 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- Jim 
> Gray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html:  target="_blank" 
> href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- A 
> Call to Arms - Custom subscriptions, and Tim Bray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://www.tbray.org/ongoing/When/200x/2003/07/30/OnSearchTOC;>On 
> Search, the Series.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> Example Usage 
> {noformat}
> The source java file has this...
> {noformat}
>  * Jim Gray's
>  *  href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
>  * A Call to Arms - Custom subscriptions, and Tim Bray's
> {noformat}
> ...which does in fact seem to be invalid HTML ... aren't {{&}} always suppose 
> to be encoded as {{}} ... even in URLs?
> I'm suprised the java8 javadocs/linter don't warn about this.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7726) Fix Javadocs HTML entity bugs

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891250#comment-15891250
 ] 

ASF subversion and git services commented on LUCENE-7726:
-

Commit e51e50e557fe7d2652080179994d08bea3b74239 in lucene-solr's branch 
refs/heads/branch_6x from [~thetaphi]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e51e50e ]

LUCENE-7726: Fix HTML entity bugs in Javadocs to be able to build with Java 9


> Fix Javadocs HTML entity bugs
> -
>
> Key: LUCENE-7726
> URL: https://issues.apache.org/jira/browse/LUCENE-7726
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/javadocs
>Reporter: Hoss Man
>Assignee: Uwe Schindler
>  Labels: Java9
> Fix For: master (7.0), 6.5
>
> Attachments: LUCENE-7726.patch
>
>
> As of jdk9-ea-b158, {{ant documentation}} seems to build the core javadocs 
> just fine, but fails on the  {{lucene/memory/}} javadocs...
> {noformat}
> javadocs:
> [mkdir] Created dir: /home/hossman/lucene/dev/lucene/build/docs/memory
> download-java8-javadoc-packagelist:
>   [javadoc] Generating Javadoc
>   [javadoc] Javadoc execution
>   [javadoc] Loading source files for package org.apache.lucene.index.memory...
>   [javadoc] Constructing Javadoc information...
>   [javadoc] Standard Doclet version 9-ea
>   [javadoc] Building tree for all the packages and classes...
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] Building index for all the packages and classes...
>   [javadoc] Building index for all classes...
>   [javadoc] Generating 
> /home/hossman/lucene/dev/lucene/build/docs/memory/help-doc.html...
>   [javadoc] Note: Custom tags that were not seen:  @lucene.internal
>   [javadoc] 3 warnings
> BUILD FAILED
> /home/hossman/lucene/dev/build.xml:93: The following error occurred while 
> executing this line:
> /home/hossman/lucene/dev/lucene/build.xml:251: The following error occurred 
> while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2179: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:549: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:65: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:78: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2155: Javadocs warnings were 
> found!
> Total time: 1 minute 0 seconds
> {noformat}
> looking at the generated html files turns up this...
> {noformat}
> hossman@tray:~/lucene/dev [master] $ find lucene/build/docs/memory -name 
> \*.html | xargs grep -C5 ""
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- but 
> rather thrown away immediately after tokenization.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- For 
> some interesting background information on search technology, see Bob Wyman's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://bobwyman.pubsub.com/main/2005/05/mary_hodder_poi.html;>Prospective
>  Search, 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- Jim 
> Gray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html:  target="_blank" 
> href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- A 
> Call to Arms - Custom subscriptions, and Tim Bray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://www.tbray.org/ongoing/When/200x/2003/07/30/OnSearchTOC;>On 
> Search, the Series.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> Example Usage 
> {noformat}
> The source java file has this...
> {noformat}
>  * Jim Gray's
>  *  href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
>  * A Call to Arms - Custom subscriptions, and Tim Bray's
> {noformat}
> ...which does in fact seem to be invalid HTML ... aren't {{&}} always suppose 
> to be encoded as {{}} ... even in URLs?
> I'm suprised the java8 javadocs/linter don't warn about this.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, 

[jira] [Commented] (LUCENE-7726) Fix Javadocs HTML entity bugs

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891249#comment-15891249
 ] 

ASF subversion and git services commented on LUCENE-7726:
-

Commit 8684fe794437e483039a027dcd21a14d56773a12 in lucene-solr's branch 
refs/heads/master from [~thetaphi]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=8684fe7 ]

LUCENE-7726: Fix HTML entity bugs in Javadocs to be able to build with Java 9


> Fix Javadocs HTML entity bugs
> -
>
> Key: LUCENE-7726
> URL: https://issues.apache.org/jira/browse/LUCENE-7726
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/javadocs
>Reporter: Hoss Man
>Assignee: Uwe Schindler
>  Labels: Java9
> Fix For: master (7.0), 6.5
>
> Attachments: LUCENE-7726.patch
>
>
> As of jdk9-ea-b158, {{ant documentation}} seems to build the core javadocs 
> just fine, but fails on the  {{lucene/memory/}} javadocs...
> {noformat}
> javadocs:
> [mkdir] Created dir: /home/hossman/lucene/dev/lucene/build/docs/memory
> download-java8-javadoc-packagelist:
>   [javadoc] Generating Javadoc
>   [javadoc] Javadoc execution
>   [javadoc] Loading source files for package org.apache.lucene.index.memory...
>   [javadoc] Constructing Javadoc information...
>   [javadoc] Standard Doclet version 9-ea
>   [javadoc] Building tree for all the packages and classes...
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] Building index for all the packages and classes...
>   [javadoc] Building index for all classes...
>   [javadoc] Generating 
> /home/hossman/lucene/dev/lucene/build/docs/memory/help-doc.html...
>   [javadoc] Note: Custom tags that were not seen:  @lucene.internal
>   [javadoc] 3 warnings
> BUILD FAILED
> /home/hossman/lucene/dev/build.xml:93: The following error occurred while 
> executing this line:
> /home/hossman/lucene/dev/lucene/build.xml:251: The following error occurred 
> while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2179: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:549: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:65: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:78: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2155: Javadocs warnings were 
> found!
> Total time: 1 minute 0 seconds
> {noformat}
> looking at the generated html files turns up this...
> {noformat}
> hossman@tray:~/lucene/dev [master] $ find lucene/build/docs/memory -name 
> \*.html | xargs grep -C5 ""
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- but 
> rather thrown away immediately after tokenization.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- For 
> some interesting background information on search technology, see Bob Wyman's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://bobwyman.pubsub.com/main/2005/05/mary_hodder_poi.html;>Prospective
>  Search, 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- Jim 
> Gray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html:  target="_blank" 
> href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- A 
> Call to Arms - Custom subscriptions, and Tim Bray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://www.tbray.org/ongoing/When/200x/2003/07/30/OnSearchTOC;>On 
> Search, the Series.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> Example Usage 
> {noformat}
> The source java file has this...
> {noformat}
>  * Jim Gray's
>  *  href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
>  * A Call to Arms - Custom subscriptions, and Tim Bray's
> {noformat}
> ...which does in fact seem to be invalid HTML ... aren't {{&}} always suppose 
> to be encoded as {{}} ... even in URLs?
> I'm suprised the java8 javadocs/linter don't warn about this.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, 

[jira] [Updated] (LUCENE-7726) Fix Javadocs HTML entity bugs

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-7726:
--
Attachment: LUCENE-7726.patch

Patch fixing this issue. You can still not build full documentation because of 
failing "pegdown" processor, but that's a separate issue.

> Fix Javadocs HTML entity bugs
> -
>
> Key: LUCENE-7726
> URL: https://issues.apache.org/jira/browse/LUCENE-7726
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/javadocs
>Reporter: Hoss Man
>Assignee: Uwe Schindler
>  Labels: Java9
> Fix For: master (7.0), 6.5
>
> Attachments: LUCENE-7726.patch
>
>
> As of jdk9-ea-b158, {{ant documentation}} seems to build the core javadocs 
> just fine, but fails on the  {{lucene/memory/}} javadocs...
> {noformat}
> javadocs:
> [mkdir] Created dir: /home/hossman/lucene/dev/lucene/build/docs/memory
> download-java8-javadoc-packagelist:
>   [javadoc] Generating Javadoc
>   [javadoc] Javadoc execution
>   [javadoc] Loading source files for package org.apache.lucene.index.memory...
>   [javadoc] Constructing Javadoc information...
>   [javadoc] Standard Doclet version 9-ea
>   [javadoc] Building tree for all the packages and classes...
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] Building index for all the packages and classes...
>   [javadoc] Building index for all classes...
>   [javadoc] Generating 
> /home/hossman/lucene/dev/lucene/build/docs/memory/help-doc.html...
>   [javadoc] Note: Custom tags that were not seen:  @lucene.internal
>   [javadoc] 3 warnings
> BUILD FAILED
> /home/hossman/lucene/dev/build.xml:93: The following error occurred while 
> executing this line:
> /home/hossman/lucene/dev/lucene/build.xml:251: The following error occurred 
> while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2179: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:549: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:65: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:78: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2155: Javadocs warnings were 
> found!
> Total time: 1 minute 0 seconds
> {noformat}
> looking at the generated html files turns up this...
> {noformat}
> hossman@tray:~/lucene/dev [master] $ find lucene/build/docs/memory -name 
> \*.html | xargs grep -C5 ""
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- but 
> rather thrown away immediately after tokenization.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- For 
> some interesting background information on search technology, see Bob Wyman's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://bobwyman.pubsub.com/main/2005/05/mary_hodder_poi.html;>Prospective
>  Search, 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- Jim 
> Gray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html:  target="_blank" 
> href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- A 
> Call to Arms - Custom subscriptions, and Tim Bray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://www.tbray.org/ongoing/When/200x/2003/07/30/OnSearchTOC;>On 
> Search, the Series.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> Example Usage 
> {noformat}
> The source java file has this...
> {noformat}
>  * Jim Gray's
>  *  href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
>  * A Call to Arms - Custom subscriptions, and Tim Bray's
> {noformat}
> ...which does in fact seem to be invalid HTML ... aren't {{&}} always suppose 
> to be encoded as {{}} ... even in URLs?
> I'm suprised the java8 javadocs/linter don't warn about this.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7726) Fix Javadocs HTML entity bugs

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-7726:
--
Fix Version/s: 6.5
   master (7.0)

> Fix Javadocs HTML entity bugs
> -
>
> Key: LUCENE-7726
> URL: https://issues.apache.org/jira/browse/LUCENE-7726
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/javadocs
>Reporter: Hoss Man
>Assignee: Uwe Schindler
>  Labels: Java9
> Fix For: master (7.0), 6.5
>
>
> As of jdk9-ea-b158, {{ant documentation}} seems to build the core javadocs 
> just fine, but fails on the  {{lucene/memory/}} javadocs...
> {noformat}
> javadocs:
> [mkdir] Created dir: /home/hossman/lucene/dev/lucene/build/docs/memory
> download-java8-javadoc-packagelist:
>   [javadoc] Generating Javadoc
>   [javadoc] Javadoc execution
>   [javadoc] Loading source files for package org.apache.lucene.index.memory...
>   [javadoc] Constructing Javadoc information...
>   [javadoc] Standard Doclet version 9-ea
>   [javadoc] Building tree for all the packages and classes...
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] Building index for all the packages and classes...
>   [javadoc] Building index for all classes...
>   [javadoc] Generating 
> /home/hossman/lucene/dev/lucene/build/docs/memory/help-doc.html...
>   [javadoc] Note: Custom tags that were not seen:  @lucene.internal
>   [javadoc] 3 warnings
> BUILD FAILED
> /home/hossman/lucene/dev/build.xml:93: The following error occurred while 
> executing this line:
> /home/hossman/lucene/dev/lucene/build.xml:251: The following error occurred 
> while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2179: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:549: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:65: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:78: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2155: Javadocs warnings were 
> found!
> Total time: 1 minute 0 seconds
> {noformat}
> looking at the generated html files turns up this...
> {noformat}
> hossman@tray:~/lucene/dev [master] $ find lucene/build/docs/memory -name 
> \*.html | xargs grep -C5 ""
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- but 
> rather thrown away immediately after tokenization.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- For 
> some interesting background information on search technology, see Bob Wyman's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://bobwyman.pubsub.com/main/2005/05/mary_hodder_poi.html;>Prospective
>  Search, 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- Jim 
> Gray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html:  target="_blank" 
> href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- A 
> Call to Arms - Custom subscriptions, and Tim Bray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://www.tbray.org/ongoing/When/200x/2003/07/30/OnSearchTOC;>On 
> Search, the Series.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> Example Usage 
> {noformat}
> The source java file has this...
> {noformat}
>  * Jim Gray's
>  *  href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
>  * A Call to Arms - Custom subscriptions, and Tim Bray's
> {noformat}
> ...which does in fact seem to be invalid HTML ... aren't {{&}} always suppose 
> to be encoded as {{}} ... even in URLs?
> I'm suprised the java8 javadocs/linter don't warn about this.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7726) Fix Javadocs HTML entity bugs

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-7726:
--
Labels: Java9  (was: )

> Fix Javadocs HTML entity bugs
> -
>
> Key: LUCENE-7726
> URL: https://issues.apache.org/jira/browse/LUCENE-7726
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/javadocs
>Reporter: Hoss Man
>Assignee: Uwe Schindler
>  Labels: Java9
> Fix For: master (7.0), 6.5
>
>
> As of jdk9-ea-b158, {{ant documentation}} seems to build the core javadocs 
> just fine, but fails on the  {{lucene/memory/}} javadocs...
> {noformat}
> javadocs:
> [mkdir] Created dir: /home/hossman/lucene/dev/lucene/build/docs/memory
> download-java8-javadoc-packagelist:
>   [javadoc] Generating Javadoc
>   [javadoc] Javadoc execution
>   [javadoc] Loading source files for package org.apache.lucene.index.memory...
>   [javadoc] Constructing Javadoc information...
>   [javadoc] Standard Doclet version 9-ea
>   [javadoc] Building tree for all the packages and classes...
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] Building index for all the packages and classes...
>   [javadoc] Building index for all classes...
>   [javadoc] Generating 
> /home/hossman/lucene/dev/lucene/build/docs/memory/help-doc.html...
>   [javadoc] Note: Custom tags that were not seen:  @lucene.internal
>   [javadoc] 3 warnings
> BUILD FAILED
> /home/hossman/lucene/dev/build.xml:93: The following error occurred while 
> executing this line:
> /home/hossman/lucene/dev/lucene/build.xml:251: The following error occurred 
> while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2179: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:549: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:65: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:78: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2155: Javadocs warnings were 
> found!
> Total time: 1 minute 0 seconds
> {noformat}
> looking at the generated html files turns up this...
> {noformat}
> hossman@tray:~/lucene/dev [master] $ find lucene/build/docs/memory -name 
> \*.html | xargs grep -C5 ""
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- but 
> rather thrown away immediately after tokenization.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- For 
> some interesting background information on search technology, see Bob Wyman's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://bobwyman.pubsub.com/main/2005/05/mary_hodder_poi.html;>Prospective
>  Search, 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- Jim 
> Gray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html:  target="_blank" 
> href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- A 
> Call to Arms - Custom subscriptions, and Tim Bray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://www.tbray.org/ongoing/When/200x/2003/07/30/OnSearchTOC;>On 
> Search, the Series.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> Example Usage 
> {noformat}
> The source java file has this...
> {noformat}
>  * Jim Gray's
>  *  href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
>  * A Call to Arms - Custom subscriptions, and Tim Bray's
> {noformat}
> ...which does in fact seem to be invalid HTML ... aren't {{&}} always suppose 
> to be encoded as {{}} ... even in URLs?
> I'm suprised the java8 javadocs/linter don't warn about this.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (LUCENE-7726) Fix Javadocs HTML entity bugs

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler reassigned LUCENE-7726:
-

Assignee: Uwe Schindler

> Fix Javadocs HTML entity bugs
> -
>
> Key: LUCENE-7726
> URL: https://issues.apache.org/jira/browse/LUCENE-7726
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/javadocs
>Reporter: Hoss Man
>Assignee: Uwe Schindler
>
> As of jdk9-ea-b158, {{ant documentation}} seems to build the core javadocs 
> just fine, but fails on the  {{lucene/memory/}} javadocs...
> {noformat}
> javadocs:
> [mkdir] Created dir: /home/hossman/lucene/dev/lucene/build/docs/memory
> download-java8-javadoc-packagelist:
>   [javadoc] Generating Javadoc
>   [javadoc] Javadoc execution
>   [javadoc] Loading source files for package org.apache.lucene.index.memory...
>   [javadoc] Constructing Javadoc information...
>   [javadoc] Standard Doclet version 9-ea
>   [javadoc] Building tree for all the packages and classes...
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] Building index for all the packages and classes...
>   [javadoc] Building index for all classes...
>   [javadoc] Generating 
> /home/hossman/lucene/dev/lucene/build/docs/memory/help-doc.html...
>   [javadoc] Note: Custom tags that were not seen:  @lucene.internal
>   [javadoc] 3 warnings
> BUILD FAILED
> /home/hossman/lucene/dev/build.xml:93: The following error occurred while 
> executing this line:
> /home/hossman/lucene/dev/lucene/build.xml:251: The following error occurred 
> while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2179: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:549: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:65: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:78: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2155: Javadocs warnings were 
> found!
> Total time: 1 minute 0 seconds
> {noformat}
> looking at the generated html files turns up this...
> {noformat}
> hossman@tray:~/lucene/dev [master] $ find lucene/build/docs/memory -name 
> \*.html | xargs grep -C5 ""
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- but 
> rather thrown away immediately after tokenization.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- For 
> some interesting background information on search technology, see Bob Wyman's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://bobwyman.pubsub.com/main/2005/05/mary_hodder_poi.html;>Prospective
>  Search, 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- Jim 
> Gray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html:  target="_blank" 
> href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- A 
> Call to Arms - Custom subscriptions, and Tim Bray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://www.tbray.org/ongoing/When/200x/2003/07/30/OnSearchTOC;>On 
> Search, the Series.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> Example Usage 
> {noformat}
> The source java file has this...
> {noformat}
>  * Jim Gray's
>  *  href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
>  * A Call to Arms - Custom subscriptions, and Tim Bray's
> {noformat}
> ...which does in fact seem to be invalid HTML ... aren't {{&}} always suppose 
> to be encoded as {{}} ... even in URLs?
> I'm suprised the java8 javadocs/linter don't warn about this.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7726) Fix Javadocs HTML entity bugs

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891231#comment-15891231
 ] 

Uwe Schindler commented on LUCENE-7726:
---

There is also another TODO (separate issue, I'll open one): pegdown (the 
markdown processor) is also incompatible to Java 9; but it reached its end of 
life. The developer points to a new replacement lib - I just have to rewrite 
the Java code a bit. This is on my "mental TODO list" since build 154 of 
Java 9.

> Fix Javadocs HTML entity bugs
> -
>
> Key: LUCENE-7726
> URL: https://issues.apache.org/jira/browse/LUCENE-7726
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/javadocs
>Reporter: Hoss Man
>
> As of jdk9-ea-b158, {{ant documentation}} seems to build the core javadocs 
> just fine, but fails on the  {{lucene/memory/}} javadocs...
> {noformat}
> javadocs:
> [mkdir] Created dir: /home/hossman/lucene/dev/lucene/build/docs/memory
> download-java8-javadoc-packagelist:
>   [javadoc] Generating Javadoc
>   [javadoc] Javadoc execution
>   [javadoc] Loading source files for package org.apache.lucene.index.memory...
>   [javadoc] Constructing Javadoc information...
>   [javadoc] Standard Doclet version 9-ea
>   [javadoc] Building tree for all the packages and classes...
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] Building index for all the packages and classes...
>   [javadoc] Building index for all classes...
>   [javadoc] Generating 
> /home/hossman/lucene/dev/lucene/build/docs/memory/help-doc.html...
>   [javadoc] Note: Custom tags that were not seen:  @lucene.internal
>   [javadoc] 3 warnings
> BUILD FAILED
> /home/hossman/lucene/dev/build.xml:93: The following error occurred while 
> executing this line:
> /home/hossman/lucene/dev/lucene/build.xml:251: The following error occurred 
> while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2179: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:549: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:65: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:78: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2155: Javadocs warnings were 
> found!
> Total time: 1 minute 0 seconds
> {noformat}
> looking at the generated html files turns up this...
> {noformat}
> hossman@tray:~/lucene/dev [master] $ find lucene/build/docs/memory -name 
> \*.html | xargs grep -C5 ""
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- but 
> rather thrown away immediately after tokenization.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- For 
> some interesting background information on search technology, see Bob Wyman's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://bobwyman.pubsub.com/main/2005/05/mary_hodder_poi.html;>Prospective
>  Search, 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- Jim 
> Gray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html:  target="_blank" 
> href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- A 
> Call to Arms - Custom subscriptions, and Tim Bray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://www.tbray.org/ongoing/When/200x/2003/07/30/OnSearchTOC;>On 
> Search, the Series.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> Example Usage 
> {noformat}
> The source java file has this...
> {noformat}
>  * Jim Gray's
>  *  href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
>  * A Call to Arms - Custom subscriptions, and Tim Bray's
> {noformat}
> ...which does in fact seem to be invalid HTML ... aren't {{&}} always suppose 
> to be encoded as {{}} ... even in URLs?
> I'm suprised the java8 javadocs/linter don't warn about this.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7725) it should be possible to run "ant precommit" with java9

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891221#comment-15891221
 ] 

Uwe Schindler commented on LUCENE-7725:
---

BTW: If you try to run ECJ with current Java 9, it fails horribly with 24667 
errors on Lucene-Core :-) Last one is most funny:

{noformat}
 [ecj-lint] 24667. ERROR in C:\Users\Uwe 
Schindler\Projects\lucene\trunk-lusolr1\lucene\core\src\java\org\apache\lucene\util\packed\PagedMutable.java
 (at line 66)
 [ecj-lint] @Override
 [ecj-lint]  
 [ecj-lint] Override cannot be resolved to a type
{noformat}

This is because ECJ cannot find the JAR file with the runtime classes (rt.jar). 
We have to wait for a new version (see above).

> it should be possible to run "ant precommit" with java9
> ---
>
> Key: LUCENE-7725
> URL: https://issues.apache.org/jira/browse/LUCENE-7725
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Hoss Man
>  Labels: Java9
>
> Eventually we'll want to be sure that {{ant precommit}} works even if you are 
> using java9.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7725) it should be possible to run "ant precommit" with java9

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891215#comment-15891215
 ] 

Uwe Schindler commented on LUCENE-7725:
---

We (Robert and I) added the hard fail, because we wanted to prevent anybody 
from running precommit and missing the extra checks, just because they ran Java 
9. In any case, if you run precommit, you should really, really only do that on 
Java 8. Because that's the version we compile against.

Nevertheless, we can remove the hard fail and just print the warning. But I'd 
not be happy.

So the fix is: {{ant precommit -Dis.jenkins.build=true}}

> it should be possible to run "ant precommit" with java9
> ---
>
> Key: LUCENE-7725
> URL: https://issues.apache.org/jira/browse/LUCENE-7725
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Hoss Man
>  Labels: Java9
>
> Eventually we'll want to be sure that {{ant precommit}} works even if you are 
> using java9.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7725) it should be possible to run "ant precommit" with java9

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891212#comment-15891212
 ] 

Uwe Schindler commented on LUCENE-7725:
---

Hi,

this target explains it:

{code:xml}
  

  

  


  
{code}

> it should be possible to run "ant precommit" with java9
> ---
>
> Key: LUCENE-7725
> URL: https://issues.apache.org/jira/browse/LUCENE-7725
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Hoss Man
>  Labels: Java9
>
> Eventually we'll want to be sure that {{ant precommit}} works even if you are 
> using java9.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10153) UnifiedSolrHighlighter support for CustomSeparatorBreakIterator (LUCENE-6485)

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley updated SOLR-10153:

Attachment: SOLR_10153_UH_and_PH_hl_customSeparatorChar.patch

Here's an updated patch (same patch filename even though the file name 
references the old param).  hl.bs.type=SEPARATOR and hl.bs.separator=# (or 
whatever) as discussed.  Tests are running now.  I moved it's location in 
HighlightParams to the right section too; it was wrong.  And added CHANGES.txt. 
 If all goes well I'll commit later tonight.

> UnifiedSolrHighlighter support for CustomSeparatorBreakIterator (LUCENE-6485)
> -
>
> Key: SOLR-10153
> URL: https://issues.apache.org/jira/browse/SOLR-10153
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Amrit Sarkar
> Attachments: SOLR-10153.patch, SOLR-10153.patch, 
> SOLR_10153_UH_and_PH_hl_customSeparatorChar.patch, 
> SOLR_10153_UH_and_PH_hl_customSeparatorChar.patch
>
>
> Lucene 5.3 added a CustomSeparatorBreakIterator (see LUCENE-6485)
> UnifiedSolrHighlighter should support *CustomSeparatorBreakIterator* along 
> with existing ones, WholeBreakIterator etc.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-7726) Fix Javadocs HTML entity bugs

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891201#comment-15891201
 ] 

Uwe Schindler edited comment on LUCENE-7726 at 3/1/17 10:30 PM:


bq. I'm suprised the java8 javadocs/linter don't warn about this.

We don't have a full HTML validator involved. In addition, for HTML5, the 
entity escaping _can_ be left out, if it is unambiguous. This mimics the 
behaviour most browsers out there always had (because most web devs out there 
did this wrong). So the produced HTML is valid (HTML5) and also leads to no 
problems in HTML4 browsers. But we should still fix it.

The requirement to escape also attributes is a requirement of just Java 9's 
Javac (which is a bug from the HTML5 perspective, but a good thing, too).


was (Author: thetaphi):
bq. I'm suprised the java8 javadocs/linter don't warn about this.

We don't have a full HTML validator involved. In addition, for HTML5, the 
entity escaping _can_ be left out, if it is unambiguous. This mimics the 
behavious most earlier browsers had (because most web devs out there did this 
wrong). So the produced HTML is valid (HTML5) and also leads to no problems in 
HTML4 browsers. But we should still fix it.

The requirement to escape also attributes is a requirement of just Java 9's 
Javac (which is a bug from the HTML5 perspective, but a good thing, too).

> Fix Javadocs HTML entity bugs
> -
>
> Key: LUCENE-7726
> URL: https://issues.apache.org/jira/browse/LUCENE-7726
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/javadocs
>Reporter: Hoss Man
>
> As of jdk9-ea-b158, {{ant documentation}} seems to build the core javadocs 
> just fine, but fails on the  {{lucene/memory/}} javadocs...
> {noformat}
> javadocs:
> [mkdir] Created dir: /home/hossman/lucene/dev/lucene/build/docs/memory
> download-java8-javadoc-packagelist:
>   [javadoc] Generating Javadoc
>   [javadoc] Javadoc execution
>   [javadoc] Loading source files for package org.apache.lucene.index.memory...
>   [javadoc] Constructing Javadoc information...
>   [javadoc] Standard Doclet version 9-ea
>   [javadoc] Building tree for all the packages and classes...
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] Building index for all the packages and classes...
>   [javadoc] Building index for all classes...
>   [javadoc] Generating 
> /home/hossman/lucene/dev/lucene/build/docs/memory/help-doc.html...
>   [javadoc] Note: Custom tags that were not seen:  @lucene.internal
>   [javadoc] 3 warnings
> BUILD FAILED
> /home/hossman/lucene/dev/build.xml:93: The following error occurred while 
> executing this line:
> /home/hossman/lucene/dev/lucene/build.xml:251: The following error occurred 
> while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2179: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:549: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:65: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:78: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2155: Javadocs warnings were 
> found!
> Total time: 1 minute 0 seconds
> {noformat}
> looking at the generated html files turns up this...
> {noformat}
> hossman@tray:~/lucene/dev [master] $ find lucene/build/docs/memory -name 
> \*.html | xargs grep -C5 ""
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- but 
> rather thrown away immediately after tokenization.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- For 
> some interesting background information on search technology, see Bob Wyman's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://bobwyman.pubsub.com/main/2005/05/mary_hodder_poi.html;>Prospective
>  Search, 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- Jim 
> Gray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html:  target="_blank" 
> href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- A 
> Call to Arms - Custom subscriptions, and Tim Bray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://www.tbray.org/ongoing/When/200x/2003/07/30/OnSearchTOC;>On 
> Search, the Series.
> 

[jira] [Commented] (LUCENE-7726) Fix Javadocs HTML entity bugs

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891201#comment-15891201
 ] 

Uwe Schindler commented on LUCENE-7726:
---

bq. I'm suprised the java8 javadocs/linter don't warn about this.

We don't have a full HTML validator involved. In addition, for HTML5, the 
entity escaping _can_ be left out, if it is unambiguous. This mimics the 
behavious most earlier browsers had (because most web devs out there did this 
wrong). So the produced HTML is valid (HTML5) and also leads to no problems in 
HTML4 browsers. But we should still fix it.

The requirement to escape also attributes is a requirement of just Java 9's 
Javac (which is a bug from the HTML5 perspective, but a good thing, too).

> Fix Javadocs HTML entity bugs
> -
>
> Key: LUCENE-7726
> URL: https://issues.apache.org/jira/browse/LUCENE-7726
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/javadocs
>Reporter: Hoss Man
>
> As of jdk9-ea-b158, {{ant documentation}} seems to build the core javadocs 
> just fine, but fails on the  {{lucene/memory/}} javadocs...
> {noformat}
> javadocs:
> [mkdir] Created dir: /home/hossman/lucene/dev/lucene/build/docs/memory
> download-java8-javadoc-packagelist:
>   [javadoc] Generating Javadoc
>   [javadoc] Javadoc execution
>   [javadoc] Loading source files for package org.apache.lucene.index.memory...
>   [javadoc] Constructing Javadoc information...
>   [javadoc] Standard Doclet version 9-ea
>   [javadoc] Building tree for all the packages and classes...
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] Building index for all the packages and classes...
>   [javadoc] Building index for all classes...
>   [javadoc] Generating 
> /home/hossman/lucene/dev/lucene/build/docs/memory/help-doc.html...
>   [javadoc] Note: Custom tags that were not seen:  @lucene.internal
>   [javadoc] 3 warnings
> BUILD FAILED
> /home/hossman/lucene/dev/build.xml:93: The following error occurred while 
> executing this line:
> /home/hossman/lucene/dev/lucene/build.xml:251: The following error occurred 
> while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2179: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:549: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:65: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:78: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2155: Javadocs warnings were 
> found!
> Total time: 1 minute 0 seconds
> {noformat}
> looking at the generated html files turns up this...
> {noformat}
> hossman@tray:~/lucene/dev [master] $ find lucene/build/docs/memory -name 
> \*.html | xargs grep -C5 ""
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- but 
> rather thrown away immediately after tokenization.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- For 
> some interesting background information on search technology, see Bob Wyman's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://bobwyman.pubsub.com/main/2005/05/mary_hodder_poi.html;>Prospective
>  Search, 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- Jim 
> Gray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html:  target="_blank" 
> href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- A 
> Call to Arms - Custom subscriptions, and Tim Bray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://www.tbray.org/ongoing/When/200x/2003/07/30/OnSearchTOC;>On 
> Search, the Series.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> Example Usage 
> {noformat}
> The source java file has this...
> {noformat}
>  * Jim Gray's
>  *  href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
>  * A Call to Arms - Custom subscriptions, and Tim Bray's
> {noformat}
> ...which does in fact seem to be invalid HTML ... aren't {{&}} always suppose 
> to be encoded as {{}} ... even in URLs?
> I'm suprised the java8 javadocs/linter don't warn about this.



--
This message was sent 

[jira] [Issue Comment Deleted] (LUCENE-7294) Figure out why building Javadocs fails with "unknown error" in Java 9 build 118

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-7294:
--
Comment: was deleted

(was: This should be fixed in source code, but is unrelated to the bug this 
issue is about! I will resolve it now, as the underlying problem was fixed in 
recent JDK 9 b154 (I just forgot to resolve this bug, sorry).

bq. ...which does in fact seem to be invalid HTML ... aren't & always suppose 
to be encoded as & ... even in URLs?

That's the problem. Easy to fix. Just commit a fix or open issue about it. I 
noticed it already about a month ago, but had no time to fix it.)

> Figure out why building Javadocs fails with "unknown error" in Java 9 build 
> 118
> ---
>
> Key: LUCENE-7294
> URL: https://issues.apache.org/jira/browse/LUCENE-7294
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.x, master (7.0)
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>  Labels: Java9
>
> When building Javadocs with java 9 it fails like:
> {noformat}
> [javadoc] Loading source files for package org.apache.lucene.util.mutable...
> [javadoc] Loading source files for package org.apache.lucene.util.packed...
> [javadoc] Constructing Javadoc information...
> [javadoc] Standard Doclet version 9-ea
> [javadoc] Building tree for all the packages and classes...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\LucenePackage.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\analysis\Analyzer.html...
> [...]
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\util\bkd\package-use.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\util\fst\package-use.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\util\mutable\package-use.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\util\packed\package-use.html...
> [javadoc] Building index for all the packages and classes...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\overview-tree.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\deprecated-list.html...
> [javadoc] Building index for all classes...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\allclasses-frame.html...
> [javadoc] javadoc: error - an unknown error has occurred
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\allclasses-noframe.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\index.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\overview-summary.html...
> [javadoc] 1 error
> {noformat}
> This looks like a bug in "javadoc". I started a question on OpenJDK mailing 
> list: 
> [http://mail.openjdk.java.net/pipermail/javadoc-dev/2016-May/000244.html]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-7726) Fix Javadocs HTML entity bugs

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891185#comment-15891185
 ] 

Uwe Schindler edited comment on LUCENE-7726 at 3/1/17 10:22 PM:


bq. ...which does in fact seem to be invalid HTML ... aren't & always suppose 
to be encoded as & ... even in URLs?

That's the problem. Easy to fix. Just commit a fix or open issue about it. I 
noticed it already about a month ago, but had no time to fix it.


was (Author: thetaphi):
This should be fixed in source code, but is unrelated to the bug this issue is 
about! I will resolve it now, as the underlying problem was fixed in recent JDK 
9 b154 (I just forgot to resolve this bug, sorry).

bq. ...which does in fact seem to be invalid HTML ... aren't & always suppose 
to be encoded as & ... even in URLs?

That's the problem. Easy to fix. Just commit a fix or open issue about it. I 
noticed it already about a month ago, but had no time to fix it.

> Fix Javadocs HTML entity bugs
> -
>
> Key: LUCENE-7726
> URL: https://issues.apache.org/jira/browse/LUCENE-7726
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/javadocs
>Reporter: Hoss Man
>
> As of jdk9-ea-b158, {{ant documentation}} seems to build the core javadocs 
> just fine, but fails on the  {{lucene/memory/}} javadocs...
> {noformat}
> javadocs:
> [mkdir] Created dir: /home/hossman/lucene/dev/lucene/build/docs/memory
> download-java8-javadoc-packagelist:
>   [javadoc] Generating Javadoc
>   [javadoc] Javadoc execution
>   [javadoc] Loading source files for package org.apache.lucene.index.memory...
>   [javadoc] Constructing Javadoc information...
>   [javadoc] Standard Doclet version 9-ea
>   [javadoc] Building tree for all the packages and classes...
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] Building index for all the packages and classes...
>   [javadoc] Building index for all classes...
>   [javadoc] Generating 
> /home/hossman/lucene/dev/lucene/build/docs/memory/help-doc.html...
>   [javadoc] Note: Custom tags that were not seen:  @lucene.internal
>   [javadoc] 3 warnings
> BUILD FAILED
> /home/hossman/lucene/dev/build.xml:93: The following error occurred while 
> executing this line:
> /home/hossman/lucene/dev/lucene/build.xml:251: The following error occurred 
> while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2179: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:549: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:65: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:78: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2155: Javadocs warnings were 
> found!
> Total time: 1 minute 0 seconds
> {noformat}
> looking at the generated html files turns up this...
> {noformat}
> hossman@tray:~/lucene/dev [master] $ find lucene/build/docs/memory -name 
> \*.html | xargs grep -C5 ""
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- but 
> rather thrown away immediately after tokenization.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- For 
> some interesting background information on search technology, see Bob Wyman's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://bobwyman.pubsub.com/main/2005/05/mary_hodder_poi.html;>Prospective
>  Search, 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- Jim 
> Gray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html:  target="_blank" 
> href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- A 
> Call to Arms - Custom subscriptions, and Tim Bray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://www.tbray.org/ongoing/When/200x/2003/07/30/OnSearchTOC;>On 
> Search, the Series.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> Example Usage 
> {noformat}
> The source java file has this...
> {noformat}
>  * Jim Gray's
>  *  href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
>  * A Call to Arms - Custom subscriptions, and 

[jira] [Comment Edited] (LUCENE-7294) Figure out why building Javadocs fails with "unknown error" in Java 9 build 118

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891184#comment-15891184
 ] 

Uwe Schindler edited comment on LUCENE-7294 at 3/1/17 10:21 PM:


I moved the HTML entity bugs in memory module to a new issue: LUCENE-7726


was (Author: thetaphi):
I moved the entity bugs to a new issue: LUCENE-7726

> Figure out why building Javadocs fails with "unknown error" in Java 9 build 
> 118
> ---
>
> Key: LUCENE-7294
> URL: https://issues.apache.org/jira/browse/LUCENE-7294
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.x, master (7.0)
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>  Labels: Java9
>
> When building Javadocs with java 9 it fails like:
> {noformat}
> [javadoc] Loading source files for package org.apache.lucene.util.mutable...
> [javadoc] Loading source files for package org.apache.lucene.util.packed...
> [javadoc] Constructing Javadoc information...
> [javadoc] Standard Doclet version 9-ea
> [javadoc] Building tree for all the packages and classes...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\LucenePackage.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\analysis\Analyzer.html...
> [...]
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\util\bkd\package-use.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\util\fst\package-use.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\util\mutable\package-use.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\util\packed\package-use.html...
> [javadoc] Building index for all the packages and classes...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\overview-tree.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\deprecated-list.html...
> [javadoc] Building index for all classes...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\allclasses-frame.html...
> [javadoc] javadoc: error - an unknown error has occurred
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\allclasses-noframe.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\index.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\overview-summary.html...
> [javadoc] 1 error
> {noformat}
> This looks like a bug in "javadoc". I started a question on OpenJDK mailing 
> list: 
> [http://mail.openjdk.java.net/pipermail/javadoc-dev/2016-May/000244.html]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7726) Fix Javadocs HTML entity bugs

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891185#comment-15891185
 ] 

Uwe Schindler commented on LUCENE-7726:
---

This should be fixed in source code, but is unrelated to the bug this issue is 
about! I will resolve it now, as the underlying problem was fixed in recent JDK 
9 b154 (I just forgot to resolve this bug, sorry).

bq. ...which does in fact seem to be invalid HTML ... aren't & always suppose 
to be encoded as & ... even in URLs?

That's the problem. Easy to fix. Just commit a fix or open issue about it. I 
noticed it already about a month ago, but had no time to fix it.

> Fix Javadocs HTML entity bugs
> -
>
> Key: LUCENE-7726
> URL: https://issues.apache.org/jira/browse/LUCENE-7726
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/javadocs
>Reporter: Hoss Man
>
> As of jdk9-ea-b158, {{ant documentation}} seems to build the core javadocs 
> just fine, but fails on the  {{lucene/memory/}} javadocs...
> {noformat}
> javadocs:
> [mkdir] Created dir: /home/hossman/lucene/dev/lucene/build/docs/memory
> download-java8-javadoc-packagelist:
>   [javadoc] Generating Javadoc
>   [javadoc] Javadoc execution
>   [javadoc] Loading source files for package org.apache.lucene.index.memory...
>   [javadoc] Constructing Javadoc information...
>   [javadoc] Standard Doclet version 9-ea
>   [javadoc] Building tree for all the packages and classes...
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] javadoc: warning - invalid usage of tag 
>   [javadoc] Building index for all the packages and classes...
>   [javadoc] Building index for all classes...
>   [javadoc] Generating 
> /home/hossman/lucene/dev/lucene/build/docs/memory/help-doc.html...
>   [javadoc] Note: Custom tags that were not seen:  @lucene.internal
>   [javadoc] 3 warnings
> BUILD FAILED
> /home/hossman/lucene/dev/build.xml:93: The following error occurred while 
> executing this line:
> /home/hossman/lucene/dev/lucene/build.xml:251: The following error occurred 
> while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2179: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:549: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:65: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/module-build.xml:78: The following error 
> occurred while executing this line:
> /home/hossman/lucene/dev/lucene/common-build.xml:2155: Javadocs warnings were 
> found!
> Total time: 1 minute 0 seconds
> {noformat}
> looking at the generated html files turns up this...
> {noformat}
> hossman@tray:~/lucene/dev [master] $ find lucene/build/docs/memory -name 
> \*.html | xargs grep -C5 ""
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- but 
> rather thrown away immediately after tokenization.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- For 
> some interesting background information on search technology, see Bob Wyman's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://bobwyman.pubsub.com/main/2005/05/mary_hodder_poi.html;>Prospective
>  Search, 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- Jim 
> Gray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html:  target="_blank" 
> href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- A 
> Call to Arms - Custom subscriptions, and Tim Bray's
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html-  target="_blank" 
> href="http://www.tbray.org/ongoing/When/200x/2003/07/30/OnSearchTOC;>On 
> Search, the Series.
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
> Example Usage 
> {noformat}
> The source java file has this...
> {noformat}
>  * Jim Gray's
>  *  href="http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
>  * A Call to Arms - Custom subscriptions, and Tim Bray's
> {noformat}
> ...which does in fact seem to be invalid HTML ... aren't {{&}} always suppose 
> to be encoded as {{}} ... even in URLs?
> I'm suprised the java8 javadocs/linter don't warn about this.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (LUCENE-7294) Figure out why building Javadocs fails with "unknown error" in Java 9 build 118

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891184#comment-15891184
 ] 

Uwe Schindler commented on LUCENE-7294:
---

I moved the entity bugs to a new issue: LUCENE-7726

> Figure out why building Javadocs fails with "unknown error" in Java 9 build 
> 118
> ---
>
> Key: LUCENE-7294
> URL: https://issues.apache.org/jira/browse/LUCENE-7294
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.x, master (7.0)
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>  Labels: Java9
>
> When building Javadocs with java 9 it fails like:
> {noformat}
> [javadoc] Loading source files for package org.apache.lucene.util.mutable...
> [javadoc] Loading source files for package org.apache.lucene.util.packed...
> [javadoc] Constructing Javadoc information...
> [javadoc] Standard Doclet version 9-ea
> [javadoc] Building tree for all the packages and classes...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\LucenePackage.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\analysis\Analyzer.html...
> [...]
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\util\bkd\package-use.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\util\fst\package-use.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\util\mutable\package-use.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\util\packed\package-use.html...
> [javadoc] Building index for all the packages and classes...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\overview-tree.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\deprecated-list.html...
> [javadoc] Building index for all classes...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\allclasses-frame.html...
> [javadoc] javadoc: error - an unknown error has occurred
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\allclasses-noframe.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\index.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\overview-summary.html...
> [javadoc] 1 error
> {noformat}
> This looks like a bug in "javadoc". I started a question on OpenJDK mailing 
> list: 
> [http://mail.openjdk.java.net/pipermail/javadoc-dev/2016-May/000244.html]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Issue Comment Deleted] (LUCENE-7294) Figure out why building Javadocs fails with "unknown error" in Java 9 build 118

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-7294:
--
Comment: was deleted

(was: As of jdk9-ea-b158, {{ant documentation}} seems to build the core 
javadocs just fine, but fails on the  {{lucene/memory/}} javadocs...

{noformat}
javadocs:
[mkdir] Created dir: /home/hossman/lucene/dev/lucene/build/docs/memory

download-java8-javadoc-packagelist:
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] Loading source files for package org.apache.lucene.index.memory...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 9-ea
  [javadoc] Building tree for all the packages and classes...
  [javadoc] javadoc: warning - invalid usage of tag 
  [javadoc] javadoc: warning - invalid usage of tag 
  [javadoc] javadoc: warning - invalid usage of tag 
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating 
/home/hossman/lucene/dev/lucene/build/docs/memory/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal
  [javadoc] 3 warnings

BUILD FAILED
/home/hossman/lucene/dev/build.xml:93: The following error occurred while 
executing this line:
/home/hossman/lucene/dev/lucene/build.xml:251: The following error occurred 
while executing this line:
/home/hossman/lucene/dev/lucene/common-build.xml:2179: The following error 
occurred while executing this line:
/home/hossman/lucene/dev/lucene/module-build.xml:549: The following error 
occurred while executing this line:
/home/hossman/lucene/dev/lucene/module-build.xml:65: The following error 
occurred while executing this line:
/home/hossman/lucene/dev/lucene/module-build.xml:78: The following error 
occurred while executing this line:
/home/hossman/lucene/dev/lucene/common-build.xml:2155: Javadocs warnings were 
found!

Total time: 1 minute 0 seconds
{noformat}

looking at the generated html files turns up this...

{noformat}
hossman@tray:~/lucene/dev [master] $ find lucene/build/docs/memory -name 
\*.html | xargs grep -C5 ""
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- but 
rather thrown away immediately after tokenization.
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- For 
some interesting background information on search technology, see Bob Wyman's
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- http://bobwyman.pubsub.com/main/2005/05/mary_hodder_poi.html;>Prospective 
Search, 
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- Jim 
Gray's
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html: http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- A 
Call to Arms - Custom subscriptions, and Tim Bray's
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- http://www.tbray.org/ongoing/When/200x/2003/07/30/OnSearchTOC;>On Search, 
the Series.
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
Example Usage 
{noformat}

The source java file has this...

{noformat}
 * Jim Gray's
 * http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
 * A Call to Arms - Custom subscriptions, and Tim Bray's
{noformat}

...which does in fact seem to be invalid HTML ... aren't {{&}} always suppose 
to be encoded as {{}} ... even in URLs?

I'm suprised the java8 javadocs/linter don't warn about this.

)

> Figure out why building Javadocs fails with "unknown error" in Java 9 build 
> 118
> ---
>
> Key: LUCENE-7294
> URL: https://issues.apache.org/jira/browse/LUCENE-7294
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.x, master (7.0)
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>  Labels: Java9
>
> When building Javadocs with java 9 it fails like:
> {noformat}
> [javadoc] Loading source files for package org.apache.lucene.util.mutable...
> [javadoc] Loading source files for package org.apache.lucene.util.packed...
> [javadoc] Constructing Javadoc information...
> [javadoc] Standard Doclet version 9-ea
> [javadoc] Building tree for all the packages and classes...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\LucenePackage.html...
> [javadoc] Generating C:\Users\Uwe 
> 

[jira] [Created] (LUCENE-7726) Fix Javadocs HTML entity bugs

[Uwe Schindler (JIRA)]

Uwe Schindler created LUCENE-7726:
-

 Summary: Fix Javadocs HTML entity bugs
 Key: LUCENE-7726
 URL: https://issues.apache.org/jira/browse/LUCENE-7726
 Project: Lucene - Core
  Issue Type: Bug
  Components: general/javadocs
Reporter: Hoss Man


As of jdk9-ea-b158, {{ant documentation}} seems to build the core javadocs just 
fine, but fails on the  {{lucene/memory/}} javadocs...

{noformat}
javadocs:
[mkdir] Created dir: /home/hossman/lucene/dev/lucene/build/docs/memory

download-java8-javadoc-packagelist:
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] Loading source files for package org.apache.lucene.index.memory...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 9-ea
  [javadoc] Building tree for all the packages and classes...
  [javadoc] javadoc: warning - invalid usage of tag 
  [javadoc] javadoc: warning - invalid usage of tag 
  [javadoc] javadoc: warning - invalid usage of tag 
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating 
/home/hossman/lucene/dev/lucene/build/docs/memory/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal
  [javadoc] 3 warnings

BUILD FAILED
/home/hossman/lucene/dev/build.xml:93: The following error occurred while 
executing this line:
/home/hossman/lucene/dev/lucene/build.xml:251: The following error occurred 
while executing this line:
/home/hossman/lucene/dev/lucene/common-build.xml:2179: The following error 
occurred while executing this line:
/home/hossman/lucene/dev/lucene/module-build.xml:549: The following error 
occurred while executing this line:
/home/hossman/lucene/dev/lucene/module-build.xml:65: The following error 
occurred while executing this line:
/home/hossman/lucene/dev/lucene/module-build.xml:78: The following error 
occurred while executing this line:
/home/hossman/lucene/dev/lucene/common-build.xml:2155: Javadocs warnings were 
found!

Total time: 1 minute 0 seconds
{noformat}

looking at the generated html files turns up this...

{noformat}
hossman@tray:~/lucene/dev [master] $ find lucene/build/docs/memory -name 
\*.html | xargs grep -C5 ""
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- but 
rather thrown away immediately after tokenization.
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- For 
some interesting background information on search technology, see Bob Wyman's
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- http://bobwyman.pubsub.com/main/2005/05/mary_hodder_poi.html;>Prospective 
Search, 
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- Jim 
Gray's
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html: http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- A 
Call to Arms - Custom subscriptions, and Tim Bray's
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- http://www.tbray.org/ongoing/When/200x/2003/07/30/OnSearchTOC;>On Search, 
the Series.
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
Example Usage 
{noformat}

The source java file has this...

{noformat}
 * Jim Gray's
 * http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
 * A Call to Arms - Custom subscriptions, and Tim Bray's
{noformat}

...which does in fact seem to be invalid HTML ... aren't {{&}} always suppose 
to be encoded as {{}} ... even in URLs?

I'm suprised the java8 javadocs/linter don't warn about this.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-7294) Figure out why building Javadocs fails with "unknown error" in Java 9 build 118

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler resolved LUCENE-7294.
---
Resolution: Fixed

Javadocs Issue was fixed in Java 9 build 154.

> Figure out why building Javadocs fails with "unknown error" in Java 9 build 
> 118
> ---
>
> Key: LUCENE-7294
> URL: https://issues.apache.org/jira/browse/LUCENE-7294
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.x, master (7.0)
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>  Labels: Java9
>
> When building Javadocs with java 9 it fails like:
> {noformat}
> [javadoc] Loading source files for package org.apache.lucene.util.mutable...
> [javadoc] Loading source files for package org.apache.lucene.util.packed...
> [javadoc] Constructing Javadoc information...
> [javadoc] Standard Doclet version 9-ea
> [javadoc] Building tree for all the packages and classes...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\LucenePackage.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\analysis\Analyzer.html...
> [...]
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\util\bkd\package-use.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\util\fst\package-use.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\util\mutable\package-use.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\util\packed\package-use.html...
> [javadoc] Building index for all the packages and classes...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\overview-tree.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\deprecated-list.html...
> [javadoc] Building index for all classes...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\allclasses-frame.html...
> [javadoc] javadoc: error - an unknown error has occurred
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\allclasses-noframe.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\index.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\overview-summary.html...
> [javadoc] 1 error
> {noformat}
> This looks like a bug in "javadoc". I started a question on OpenJDK mailing 
> list: 
> [http://mail.openjdk.java.net/pipermail/javadoc-dev/2016-May/000244.html]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7294) Figure out why building Javadocs fails with "unknown error" in Java 9 build 118

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891177#comment-15891177
 ] 

Uwe Schindler commented on LUCENE-7294:
---

This should be fixed in source code, but is unrelated to the bug this issue is 
about! I will resolve it now, as the underlying problem was fixed in recent JDK 
9 b154 (I just forgot to resolve this bug, sorry).

bq. ...which does in fact seem to be invalid HTML ... aren't & always suppose 
to be encoded as & ... even in URLs?

That's the problem. Easy to fix. Just commit a fix or open issue about it. I 
noticed it already about a month ago, but had no time to fix it.

> Figure out why building Javadocs fails with "unknown error" in Java 9 build 
> 118
> ---
>
> Key: LUCENE-7294
> URL: https://issues.apache.org/jira/browse/LUCENE-7294
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.x, master (7.0)
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>  Labels: Java9
>
> When building Javadocs with java 9 it fails like:
> {noformat}
> [javadoc] Loading source files for package org.apache.lucene.util.mutable...
> [javadoc] Loading source files for package org.apache.lucene.util.packed...
> [javadoc] Constructing Javadoc information...
> [javadoc] Standard Doclet version 9-ea
> [javadoc] Building tree for all the packages and classes...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\LucenePackage.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\analysis\Analyzer.html...
> [...]
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\util\bkd\package-use.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\util\fst\package-use.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\util\mutable\package-use.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\util\packed\package-use.html...
> [javadoc] Building index for all the packages and classes...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\overview-tree.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\deprecated-list.html...
> [javadoc] Building index for all classes...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\allclasses-frame.html...
> [javadoc] javadoc: error - an unknown error has occurred
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\allclasses-noframe.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\index.html...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\overview-summary.html...
> [javadoc] 1 error
> {noformat}
> This looks like a bug in "javadoc". I started a question on OpenJDK mailing 
> list: 
> [http://mail.openjdk.java.net/pipermail/javadoc-dev/2016-May/000244.html]



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7725) it should be possible to run "ant precommit" with java9

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891173#comment-15891173
 ] 

Hoss Man commented on LUCENE-7725:
--

bq. Btw, you can run precommit also with Java 9 if you set an extra build 
property to activate "Jenkins mode".

really? what exactly does that look like?

we should document that here as a workaround.

> it should be possible to run "ant precommit" with java9
> ---
>
> Key: LUCENE-7725
> URL: https://issues.apache.org/jira/browse/LUCENE-7725
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Hoss Man
>  Labels: Java9
>
> Eventually we'll want to be sure that {{ant precommit}} works even if you are 
> using java9.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-7294) Figure out why building Javadocs fails with "unknown error" in Java 9 build 118

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891167#comment-15891167
 ] 

Hoss Man edited comment on LUCENE-7294 at 3/1/17 10:13 PM:
---

As of jdk9-ea-b158, {{ant documentation}} seems to build the core javadocs just 
fine, but fails on the  {{lucene/memory/}} javadocs...

{noformat}
javadocs:
[mkdir] Created dir: /home/hossman/lucene/dev/lucene/build/docs/memory

download-java8-javadoc-packagelist:
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] Loading source files for package org.apache.lucene.index.memory...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 9-ea
  [javadoc] Building tree for all the packages and classes...
  [javadoc] javadoc: warning - invalid usage of tag 
  [javadoc] javadoc: warning - invalid usage of tag 
  [javadoc] javadoc: warning - invalid usage of tag 
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating 
/home/hossman/lucene/dev/lucene/build/docs/memory/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal
  [javadoc] 3 warnings

BUILD FAILED
/home/hossman/lucene/dev/build.xml:93: The following error occurred while 
executing this line:
/home/hossman/lucene/dev/lucene/build.xml:251: The following error occurred 
while executing this line:
/home/hossman/lucene/dev/lucene/common-build.xml:2179: The following error 
occurred while executing this line:
/home/hossman/lucene/dev/lucene/module-build.xml:549: The following error 
occurred while executing this line:
/home/hossman/lucene/dev/lucene/module-build.xml:65: The following error 
occurred while executing this line:
/home/hossman/lucene/dev/lucene/module-build.xml:78: The following error 
occurred while executing this line:
/home/hossman/lucene/dev/lucene/common-build.xml:2155: Javadocs warnings were 
found!

Total time: 1 minute 0 seconds
{noformat}

looking at the generated html files turns up this...

{noformat}
hossman@tray:~/lucene/dev [master] $ find lucene/build/docs/memory -name 
\*.html | xargs grep -C5 ""
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- but 
rather thrown away immediately after tokenization.
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- For 
some interesting background information on search technology, see Bob Wyman's
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- http://bobwyman.pubsub.com/main/2005/05/mary_hodder_poi.html;>Prospective 
Search, 
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- Jim 
Gray's
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html: http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- A 
Call to Arms - Custom subscriptions, and Tim Bray's
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- http://www.tbray.org/ongoing/When/200x/2003/07/30/OnSearchTOC;>On Search, 
the Series.
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
Example Usage 
{noformat}

The source java file has this...

{noformat}
 * Jim Gray's
 * http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
 * A Call to Arms - Custom subscriptions, and Tim Bray's
{noformat}

...which does in fact seem to be invalid HTML ... aren't {{&}} always suppose 
to be encoded as {{}} ... even in URLs?

I'm suprised the java8 javadocs/linter don't warn about this.




was (Author: hossman):
As of jdk9-ea-b148, {{ant documentation}} seems to build the core javadocs just 
fine, but fails on the  {{lucene/memory/}} javadocs...

{noformat}
javadocs:
[mkdir] Created dir: /home/hossman/lucene/dev/lucene/build/docs/memory

download-java8-javadoc-packagelist:
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] Loading source files for package org.apache.lucene.index.memory...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 9-ea
  [javadoc] Building tree for all the packages and classes...
  [javadoc] javadoc: warning - invalid usage of tag 
  [javadoc] javadoc: warning - invalid usage of tag 
  [javadoc] javadoc: warning - invalid usage of tag 
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating 
/home/hossman/lucene/dev/lucene/build/docs/memory/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal
  [javadoc] 3 warnings

BUILD FAILED

[jira] [Commented] (LUCENE-7294) Figure out why building Javadocs fails with "unknown error" in Java 9 build 118

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891167#comment-15891167
 ] 

Hoss Man commented on LUCENE-7294:
--

As of jdk9-ea-b148, {{ant documentation}} seems to build the core javadocs just 
fine, but fails on the  {{lucene/memory/}} javadocs...

{noformat}
javadocs:
[mkdir] Created dir: /home/hossman/lucene/dev/lucene/build/docs/memory

download-java8-javadoc-packagelist:
  [javadoc] Generating Javadoc
  [javadoc] Javadoc execution
  [javadoc] Loading source files for package org.apache.lucene.index.memory...
  [javadoc] Constructing Javadoc information...
  [javadoc] Standard Doclet version 9-ea
  [javadoc] Building tree for all the packages and classes...
  [javadoc] javadoc: warning - invalid usage of tag 
  [javadoc] javadoc: warning - invalid usage of tag 
  [javadoc] javadoc: warning - invalid usage of tag 
  [javadoc] Building index for all the packages and classes...
  [javadoc] Building index for all classes...
  [javadoc] Generating 
/home/hossman/lucene/dev/lucene/build/docs/memory/help-doc.html...
  [javadoc] Note: Custom tags that were not seen:  @lucene.internal
  [javadoc] 3 warnings

BUILD FAILED
/home/hossman/lucene/dev/build.xml:93: The following error occurred while 
executing this line:
/home/hossman/lucene/dev/lucene/build.xml:251: The following error occurred 
while executing this line:
/home/hossman/lucene/dev/lucene/common-build.xml:2179: The following error 
occurred while executing this line:
/home/hossman/lucene/dev/lucene/module-build.xml:549: The following error 
occurred while executing this line:
/home/hossman/lucene/dev/lucene/module-build.xml:65: The following error 
occurred while executing this line:
/home/hossman/lucene/dev/lucene/module-build.xml:78: The following error 
occurred while executing this line:
/home/hossman/lucene/dev/lucene/common-build.xml:2155: Javadocs warnings were 
found!

Total time: 1 minute 0 seconds
{noformat}

looking at the generated html files turns up this...

{noformat}
hossman@tray:~/lucene/dev [master] $ find lucene/build/docs/memory -name 
\*.html | xargs grep -C5 ""
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- but 
rather thrown away immediately after tokenization.
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- For 
some interesting background information on search technology, see Bob Wyman's
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- http://bobwyman.pubsub.com/main/2005/05/mary_hodder_poi.html;>Prospective 
Search, 
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- Jim 
Gray's
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html: http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- A 
Call to Arms - Custom subscriptions, and Tim Bray's
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- http://www.tbray.org/ongoing/When/200x/2003/07/30/OnSearchTOC;>On Search, 
the Series.
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
lucene/build/docs/memory/org/apache/lucene/index/memory/MemoryIndex.html- 
Example Usage 
{noformat}

The source java file has this...

{noformat}
 * Jim Gray's
 * http://www.acmqueue.org/modules.php?name=Content=showpage=293=4;>
 * A Call to Arms - Custom subscriptions, and Tim Bray's
{noformat}

...which does in fact seem to be invalid HTML ... aren't {{&}} always suppose 
to be encoded as {{}} ... even in URLs?

I'm suprised the java8 javadocs/linter don't warn about this.



> Figure out why building Javadocs fails with "unknown error" in Java 9 build 
> 118
> ---
>
> Key: LUCENE-7294
> URL: https://issues.apache.org/jira/browse/LUCENE-7294
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.x, master (7.0)
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>  Labels: Java9
>
> When building Javadocs with java 9 it fails like:
> {noformat}
> [javadoc] Loading source files for package org.apache.lucene.util.mutable...
> [javadoc] Loading source files for package org.apache.lucene.util.packed...
> [javadoc] Constructing Javadoc information...
> [javadoc] Standard Doclet version 9-ea
> [javadoc] Building tree for all the packages and classes...
> [javadoc] Generating C:\Users\Uwe 
> Schindler\Projects\lucene\trunk-lusolr1\lucene\build\docs\core\org\apache\lucene\LucenePackage.html...
> [javadoc] Generating C:\Users\Uwe 
> 

[jira] [Commented] (SOLR-10219) diagnose HDFS test problems with Java9 and/or re-enable these tests

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891152#comment-15891152
 ] 

Uwe Schindler commented on SOLR-10219:
--

Thanks Hoss!

> diagnose HDFS test problems with Java9 and/or re-enable these tests
> ---
>
> Key: SOLR-10219
> URL: https://issues.apache.org/jira/browse/SOLR-10219
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hoss Man
>Assignee: Hoss Man
>  Labels: Java9
>
> As part of SOLR-8874, Uwe disabled all HDFS based tests under java9 at the 
> build.xml/pom.xml level by adding a conditional to the existing 
> {{tests.disableHdfs}} system property (Note: this property exists so that 
> HDFS tests can be disabled by default on windows, but still run on cygwin if 
> users wish to set that property)
> We need to get to the bottom of what exactly the issue(s) are with HDFS and 
> file specific bugs to track the problems



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7725) it should be possible to run "ant precommit" with java9

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891148#comment-15891148
 ] 

Uwe Schindler commented on LUCENE-7725:
---

I can take this issue, but we have to wait for Java 9 to come out until ECJ 
works.

Btw, you can run precommit also with Java 9 if you set an extra build property 
to activate "Jenkins mode". 

> it should be possible to run "ant precommit" with java9
> ---
>
> Key: LUCENE-7725
> URL: https://issues.apache.org/jira/browse/LUCENE-7725
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Hoss Man
>  Labels: Java9
>
> Eventually we'll want to be sure that {{ant precommit}} works even if you are 
> using java9.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7725) it should be possible to run "ant precommit" with java9

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891138#comment-15891138
 ] 

Uwe Schindler commented on LUCENE-7725:
---

The problem was Javadocs and it's linter. In addition, current ECJ cannot 
handle the missing rt.jar. There is work going on to release a new version 
that's able to handle Java 9.

> it should be possible to run "ant precommit" with java9
> ---
>
> Key: LUCENE-7725
> URL: https://issues.apache.org/jira/browse/LUCENE-7725
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Hoss Man
>  Labels: Java9
>
> Eventually we'll want to be sure that {{ant precommit}} works even if you are 
> using java9.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10215) Cannot use the namenode for HDFS HA as of Solr 6.4

[Cassandra Targett (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891104#comment-15891104
 ] 

Cassandra Targett commented on SOLR-10215:
--

I just checked this again with the 6.4.2 release candidate, and it's looking 
fixed to me.

> Cannot use the namenode for HDFS HA as of Solr 6.4
> --
>
> Key: SOLR-10215
> URL: https://issues.apache.org/jira/browse/SOLR-10215
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Hadoop Integration
>Affects Versions: 6.4.1, 6.4.0
>Reporter: Cassandra Targett
>Priority: Blocker
> Fix For: 6.4.2
>
>
> As of Solr 6.4, it seems it's no longer possible to use a namenode instead of 
> a server address with the {{solr.hdfs.home}} parameter when configuring Solr 
> with HDFS high availability (HA).
> Startup is fine, but when trying to create a collection, this error is in the 
> logs:
> {code}
> 2017-02-27 22:22:57.359 ERROR (qtp401424608-21) [c:testing s:shard1  
> x:testing_shard1_replica1] o.a.s.c.CoreContainer Error creating core 
> [testing_shard1_replica1]: Error Instantiating Update Handler, 
> solr.DirectUpdateHandler2 failed to instantiate 
> org.apache.solr.update.UpdateHandler
> org.apache.solr.common.SolrException: Error Instantiating Update Handler, 
> solr.DirectUpdateHandler2 failed to instantiate 
> org.apache.solr.update.UpdateHandler
> {code}
> And after the full stack trace (which I will put in a comment), there is this:
> {code}
> Caused by: java.lang.IllegalArgumentException: java.net.UnknownHostException: 
> mycluster
> {code}
> I started Solr with the params configured as system params instead of in 
> {{solrconfig.xml}}, so my {{solr.in.sh}} has this:
> {code}
> SOLR_OPTS="$SOLR_OPTS $SOLR_ZK_CREDS_AND_ACLS 
> -Dsolr.directoryFactory=HdfsDirectoryFactory -Dsolr.lock.type=hdfs 
> -Dsolr.hdfs.home=hdfs://mycluster:8020/solr-index 
> -Dsolr.hdfs.confdir=/etc/hadoop/conf/"
> {code}
> Solr in this case is running on the same nodes as Hadoop (Hortonworks HDP 
> 2.5).
> I tried with a couple variations of defining the Solr home parameter:
> * {{hdfs://mycluster:8020/solr-index}}
> * {{hdfs://mycluster/solr-index}}
> * {{solr-index}}
> None of these variations worked with Solr 6.4.1 (the first 2 got the same 
> error as above, the last was just wrong so it got a different error).
> I believe this problem is isolated to Solr 6.4.x. I tested the same setup (as 
> in the {{solr.in.sh}} above) with 6.3.0 and it worked fine. Using the server 
> address also works fine, but that negates the High Availability feature 
> (which is like failover, for those who don't know).
> _edit: the problem isn't just 6.4.1, I believe it's probably in 6.4.0 also_



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7682) UnifiedHighlighter not highlighting all terms relevant in SpanNearQuery

[Michael Braun (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891101#comment-15891101
 ] 

Michael Braun commented on LUCENE-7682:
---

Should this be marked as a bug for a module other than the highlighter then 
since it also affects scoring? 

> UnifiedHighlighter not highlighting all terms relevant in SpanNearQuery
> ---
>
> Key: LUCENE-7682
> URL: https://issues.apache.org/jira/browse/LUCENE-7682
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/highlighter
>Reporter: Michael Braun
>
> Original text: "Something for protecting wildlife feed in a feed thing."
> Query is:
>SpanNearQuery with Slop 9 - in order - 
>   1. SpanTermQuery(wildlife)
>   2. SpanTermQuery(feed)
> This should highlight both instances of "feed" since they are both within 
> slop of 9 of "wildlife". However, only the first instance is highlighted. 
> This occurs with unordered SpanNearQuery as well.  Test below replicates. 
> Affects both the current 6.x line and master.
> Test that fits within TestUnifiedHighlighterMTQ:
> {code}
>   public void testOrderedSpanNearQueryWithDupeTerms() throws Exception {
> RandomIndexWriter iw = new RandomIndexWriter(random(), dir, 
> indexAnalyzer);
> Document doc = new Document();
> doc.add(new Field("body", "Something for protecting wildlife feed in a 
> feed thing.", fieldType));
> doc.add(newTextField("id", "id", Field.Store.YES));
> iw.addDocument(doc);
> IndexReader ir = iw.getReader();
> iw.close();
> IndexSearcher searcher = newSearcher(ir);
> UnifiedHighlighter highlighter = new UnifiedHighlighter(searcher, 
> indexAnalyzer);
> int docID = searcher.search(new TermQuery(new Term("id", "id")), 
> 1).scoreDocs[0].doc;
> SpanTermQuery termOne = new SpanTermQuery(new Term("body", "wildlife"));
> SpanTermQuery termTwo = new SpanTermQuery(new Term("body", "feed"));
> SpanNearQuery topQuery = new SpanNearQuery.Builder("body", true)
> .setSlop(9)
> .addClause(termOne)
> .addClause(termTwo)
> .build();
> int[] docIds = new int[] {docID};
> String snippets[] = highlighter.highlightFields(new String[] {"body"}, 
> topQuery, docIds, new int[] {2}).get("body");
> assertEquals(1, snippets.length);
> assertEquals("Something for protecting wildlife feed in a 
> feed thing.", snippets[0]);
> ir.close();
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7725) it should be possible to run "ant precommit" with java9

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891063#comment-15891063
 ] 

Hoss Man commented on LUCENE-7725:
--

Currently precommit on master with java9 fails because...

{noformat}
-ecj-javadoc-lint-unsupported:

BUILD FAILED
/home/hossman/lucene/dev/lucene/common-build.xml:1993: Linting documentation 
with ECJ is not supported on this Java version (9).

{noformat}

...i haven't dug into this enough to figure out if there is a genuine problem 
running ECJ on java9, or if it's just an overly aggressive version restriction 
in the build file.


> it should be possible to run "ant precommit" with java9
> ---
>
> Key: LUCENE-7725
> URL: https://issues.apache.org/jira/browse/LUCENE-7725
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Hoss Man
>  Labels: Java9
>
> Eventually we'll want to be sure that {{ant precommit}} works even if you are 
> using java9.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7725) it should be possible to run "ant precommit" with java9

[Hoss Man (JIRA)]

Hoss Man created LUCENE-7725:


 Summary: it should be possible to run "ant precommit" with java9
 Key: LUCENE-7725
 URL: https://issues.apache.org/jira/browse/LUCENE-7725
 Project: Lucene - Core
  Issue Type: Task
Reporter: Hoss Man


Eventually we'll want to be sure that {{ant precommit}} works even if you are 
using java9.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 298 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/298/

2 tests failed.
FAILED:  
org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload

Error Message:
expected:<[{indexVersion=1488399241239,generation=2,filelist=[_4x.cfe, _4x.cfs, 
_4x.si, _9u.cfe, _9u.cfs, _9u.si, _bo.cfe, _bo.cfs, _bo.si, _bp.cfe, _bp.cfs, 
_bp.si, _bq.cfe, _bq.cfs, _bq.si, _br.cfe, _br.cfs, _br.si, _bs.cfe, _bs.cfs, 
_bs.si, _bt.cfe, _bt.cfs, _bt.si, _bu.cfe, _bu.cfs, _bu.si, _bv.cfe, _bv.cfs, 
_bv.si, _bw.cfe, _bw.cfs, _bw.si, _bx.cfe, _bx.cfs, _bx.si, _by.cfe, _by.cfs, 
_by.si, segments_2]}]> but 
was:<[{indexVersion=1488399241239,generation=2,filelist=[_4x.cfe, _4x.cfs, 
_4x.si, _9u.cfe, _9u.cfs, _9u.si, _bo.cfe, _bo.cfs, _bo.si, _bp.cfe, _bp.cfs, 
_bp.si, _bq.cfe, _bq.cfs, _bq.si, _br.cfe, _br.cfs, _br.si, _bs.cfe, _bs.cfs, 
_bs.si, _bt.cfe, _bt.cfs, _bt.si, _bu.cfe, _bu.cfs, _bu.si, _bv.cfe, _bv.cfs, 
_bv.si, _bw.cfe, _bw.cfs, _bw.si, _bx.cfe, _bx.cfs, _bx.si, _by.cfe, _by.cfs, 
_by.si, segments_2]}, 
{indexVersion=1488399241239,generation=3,filelist=[_4x.cfe, _4x.cfs, _4x.si, 
_9u.cfe, _9u.cfs, _9u.si, _bz.cfe, _bz.cfs, _bz.si, segments_3]}]>

Stack Trace:
java.lang.AssertionError: 
expected:<[{indexVersion=1488399241239,generation=2,filelist=[_4x.cfe, _4x.cfs, 
_4x.si, _9u.cfe, _9u.cfs, _9u.si, _bo.cfe, _bo.cfs, _bo.si, _bp.cfe, _bp.cfs, 
_bp.si, _bq.cfe, _bq.cfs, _bq.si, _br.cfe, _br.cfs, _br.si, _bs.cfe, _bs.cfs, 
_bs.si, _bt.cfe, _bt.cfs, _bt.si, _bu.cfe, _bu.cfs, _bu.si, _bv.cfe, _bv.cfs, 
_bv.si, _bw.cfe, _bw.cfs, _bw.si, _bx.cfe, _bx.cfs, _bx.si, _by.cfe, _by.cfs, 
_by.si, segments_2]}]> but 
was:<[{indexVersion=1488399241239,generation=2,filelist=[_4x.cfe, _4x.cfs, 
_4x.si, _9u.cfe, _9u.cfs, _9u.si, _bo.cfe, _bo.cfs, _bo.si, _bp.cfe, _bp.cfs, 
_bp.si, _bq.cfe, _bq.cfs, _bq.si, _br.cfe, _br.cfs, _br.si, _bs.cfe, _bs.cfs, 
_bs.si, _bt.cfe, _bt.cfs, _bt.si, _bu.cfe, _bu.cfs, _bu.si, _bv.cfe, _bv.cfs, 
_bv.si, _bw.cfe, _bw.cfs, _bw.si, _bx.cfe, _bx.cfs, _bx.si, _by.cfe, _by.cfs, 
_by.si, segments_2]}, 
{indexVersion=1488399241239,generation=3,filelist=[_4x.cfe, _4x.cfs, _4x.si, 
_9u.cfe, _9u.cfs, _9u.si, _bz.cfe, _bz.cfs, _bz.si, segments_3]}]>
at 
__randomizedtesting.SeedInfo.seed([2813066EAC1DCBD3:DC41D5EDC55C5D0]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:147)
at 
org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1281)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

Re: [JENKINS] Lucene-Solr-SmokeRelease-6.4 - Build # 15 - Still Failing

[Ishan Chattopadhyaya]

Thanks, Adrien.

On Wed, Mar 1, 2017 at 1:56 PM, Adrien Grand  wrote:

> I just added the missing backward index.
>
> Le mer. 1 mars 2017 à 08:39, Apache Jenkins Server <
> jenk...@builds.apache.org> a écrit :
>
> Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-6.4/15/
>
> No tests ran.
>
> Build Log:
> [...truncated 41918 lines...]
> prepare-release-no-sign:
> [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-
> SmokeRelease-6.4/lucene/build/smokeTestRelease/dist
>  [copy] Copying 476 files to /x1/jenkins/jenkins-slave/
> workspace/Lucene-Solr-SmokeRelease-6.4/lucene/build/
> smokeTestRelease/dist/lucene
>  [copy] Copying 260 files to /x1/jenkins/jenkins-slave/
> workspace/Lucene-Solr-SmokeRelease-6.4/lucene/build/
> smokeTestRelease/dist/solr
>[smoker] Java 1.8 JAVA_HOME=/home/jenkins/tools/java/latest1.8
>[smoker] NOTE: output encoding is UTF-8
>[smoker]
>[smoker] Load release URL "file:/x1/jenkins/jenkins-
> slave/workspace/Lucene-Solr-SmokeRelease-6.4/lucene/build/
> smokeTestRelease/dist/"...
>[smoker]
>[smoker] Test Lucene...
>[smoker]   test basics...
>[smoker]   get KEYS
>[smoker] 0.2 MB in 0.01 sec (29.9 MB/sec)
>[smoker]   check changes HTML...
>[smoker]   download lucene-6.4.2-src.tgz...
>[smoker] 30.6 MB in 0.03 sec (1172.7 MB/sec)
>[smoker] verify md5/sha1 digests
>[smoker]   download lucene-6.4.2.tgz...
>[smoker] 65.0 MB in 0.06 sec (1158.7 MB/sec)
>[smoker] verify md5/sha1 digests
>[smoker]   download lucene-6.4.2.zip...
>[smoker] 75.3 MB in 0.06 sec (1160.0 MB/sec)
>[smoker] verify md5/sha1 digests
>[smoker]   unpack lucene-6.4.2.tgz...
>[smoker] verify JAR metadata/identity/no javax.* or java.*
> classes...
>[smoker] test demo with 1.8...
>[smoker]   got 6206 hits for query "lucene"
>[smoker] checkindex with 1.8...
>[smoker] check Lucene's javadoc JAR
>[smoker]   unpack lucene-6.4.2.zip...
>[smoker] verify JAR metadata/identity/no javax.* or java.*
> classes...
>[smoker] test demo with 1.8...
>[smoker]   got 6206 hits for query "lucene"
>[smoker] checkindex with 1.8...
>[smoker] check Lucene's javadoc JAR
>[smoker]   unpack lucene-6.4.2-src.tgz...
>[smoker] make sure no JARs/WARs in src dist...
>[smoker] run "ant validate"
>[smoker] run tests w/ Java 8 and testArgs='-Dtests.slow=false'...
>[smoker] test demo with 1.8...
>[smoker]   got 229 hits for query "lucene"
>[smoker] checkindex with 1.8...
>[smoker] generate javadocs w/ Java 8...
>[smoker]
>[smoker] Crawl/parse...
>[smoker]
>[smoker] Verify...
>[smoker]   confirm all releases have coverage in
> TestBackwardsCompatibility
>[smoker] find all past Lucene releases...
>[smoker] run TestBackwardsCompatibility..
>[smoker] Traceback (most recent call last):
>[smoker]   File "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-
> SmokeRelease-6.4/dev-tools/scripts/smokeTestRelease.py", line 1472, in
> 
>[smoker] main()
>[smoker]   File "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-
> SmokeRelease-6.4/dev-tools/scripts/smokeTestRelease.py", line 1416, in
> main
>[smoker] smokeTest(c.java, c.url, c.revision, c.version, c.tmp_dir,
> c.is_signed, ' '.join(c.test_args))
>[smoker]   File "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-
> SmokeRelease-6.4/dev-tools/scripts/smokeTestRelease.py", line 1454, in
> smokeTest
>[smoker] unpackAndVerify(java, 'lucene', tmpDir,
> 'lucene-%s-src.tgz' % version, gitRevision, version, testArgs, baseURL)
>[smoker]   File "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-
> SmokeRelease-6.4/dev-tools/scripts/smokeTestRelease.py", line 622, in
> unpackAndVerify
>[smoker] verifyUnpacked(java, project, artifact, unpackPath,
> gitRevision, version, testArgs, tmpDir, baseURL)
>[smoker]   File "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-
> SmokeRelease-6.4/dev-tools/scripts/smokeTestRelease.py", line 768, in
> verifyUnpacked
>[smoker] confirmAllReleasesAreTestedForBackCompat(version,
> unpackPath)
>[smoker]   File "/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-
> SmokeRelease-6.4/dev-tools/scripts/smokeTestRelease.py", line 1392, in
> confirmAllReleasesAreTestedForBackCompat
>[smoker] raise RuntimeError('some releases are not tested by
> TestBackwardsCompatibility?')
>[smoker] RuntimeError: some releases are not tested by
> TestBackwardsCompatibility?
>[smoker] Releases that don't seem to be tested:
>[smoker]   6.4.1
>
> BUILD FAILED
> /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-
> SmokeRelease-6.4/build.xml:571: exec returned: 1
>
> Total time: 39 minutes 22 seconds
> Build step 'Invoke Ant' marked build as failure
> Email was triggered for: Failure - Any
> Sending email for 

[jira] [Commented] (SOLR-10219) diagnose HDFS test problems with Java9 and/or re-enable these tests

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891016#comment-15891016
 ] 

Hoss Man commented on SOLR-10219:
-

[~thetaphi]: i think we're fine ... even with b158 ... i've pushed a "revert" 
of {{tests.disableHdfs=true if java9}} to master and i'll let it soak for a few 
days before backporting.

If you notice any suspcious hadoop/hdfs related failures under java9, can you 
please file new jiras for them w/stack traces -- and (when you feel it's 
warranted) add {{assumeFalse(Constants.JRE_IS_MINIMUM_JAVA9)}} pointed at those 
distinct jiras.

(I'd prefer we leave {{tests.disableHdfs}} alone for now so we can more 
precisely assess individual bugs)

> diagnose HDFS test problems with Java9 and/or re-enable these tests
> ---
>
> Key: SOLR-10219
> URL: https://issues.apache.org/jira/browse/SOLR-10219
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hoss Man
>Assignee: Hoss Man
>  Labels: Java9
>
> As part of SOLR-8874, Uwe disabled all HDFS based tests under java9 at the 
> build.xml/pom.xml level by adding a conditional to the existing 
> {{tests.disableHdfs}} system property (Note: this property exists so that 
> HDFS tests can be disabled by default on windows, but still run on cygwin if 
> users wish to set that property)
> We need to get to the bottom of what exactly the issue(s) are with HDFS and 
> file specific bugs to track the problems



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10219) diagnose HDFS test problems with Java9 and/or re-enable these tests

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891001#comment-15891001
 ] 

ASF subversion and git services commented on SOLR-10219:


Commit 4851f399d4b25f76eeb494d7c63844bf6b858fd5 in lucene-solr's branch 
refs/heads/master from Chris Hostetter
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4851f39 ]

SOLR-10219: stop defaulting to tests.disableHdfs=true under java9


> diagnose HDFS test problems with Java9 and/or re-enable these tests
> ---
>
> Key: SOLR-10219
> URL: https://issues.apache.org/jira/browse/SOLR-10219
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hoss Man
>Assignee: Hoss Man
>  Labels: Java9
>
> As part of SOLR-8874, Uwe disabled all HDFS based tests under java9 at the 
> build.xml/pom.xml level by adding a conditional to the existing 
> {{tests.disableHdfs}} system property (Note: this property exists so that 
> HDFS tests can be disabled by default on windows, but still run on cygwin if 
> users wish to set that property)
> We need to get to the bottom of what exactly the issue(s) are with HDFS and 
> file specific bugs to track the problems



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[VOTE] Release Lucene/Solr 6.4.2 RC1

[Ishan Chattopadhyaya]

Please vote for release candidate 1 for Lucene/Solr 6.4.2The artifacts
can be downloaded
from:https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-6.4.2-RC1-rev34a975ca3d4bd7fa121340e5bcbf165929e0542fYou
can run the smoke tester directly with this command:python3 -u
dev-tools/scripts/smokeTestRelease.py
\https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-6.4.2-RC1-rev34a975ca3d4bd7fa121340e5bcbf165929e0542fHere's
my +1
SUCCESS! [0:52:41.429385]

[jira] [Commented] (SOLR-10219) diagnose HDFS test problems with Java9 and/or re-enable these tests

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890932#comment-15890932
 ] 

Hoss Man commented on SOLR-10219:
-

Uwe: I'm still testing ... i'll take care of it.


> diagnose HDFS test problems with Java9 and/or re-enable these tests
> ---
>
> Key: SOLR-10219
> URL: https://issues.apache.org/jira/browse/SOLR-10219
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hoss Man
>Assignee: Hoss Man
>  Labels: Java9
>
> As part of SOLR-8874, Uwe disabled all HDFS based tests under java9 at the 
> build.xml/pom.xml level by adding a conditional to the existing 
> {{tests.disableHdfs}} system property (Note: this property exists so that 
> HDFS tests can be disabled by default on windows, but still run on cygwin if 
> users wish to set that property)
> We need to get to the bottom of what exactly the issue(s) are with HDFS and 
> file specific bugs to track the problems



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10219) diagnose HDFS test problems with Java9 and/or re-enable these tests

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890847#comment-15890847
 ] 

Uwe Schindler commented on SOLR-10219:
--

It could also be that the Java 9 people made some adjustments, so it is more 
compatible to Hadoop. So I have no idea, but in my Linux VM at that time I have 
seen tons of failures primarily on Hadoop tests, so I disabled them.

Be sure to check again with build 158, as there were some changes, including 
SecurityManager.

Otherwise +1 to revert the above 2 commits done by me. [~hossman]: Should I do 
it, or will you do it? I'd take the issue in that case.

> diagnose HDFS test problems with Java9 and/or re-enable these tests
> ---
>
> Key: SOLR-10219
> URL: https://issues.apache.org/jira/browse/SOLR-10219
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hoss Man
>Assignee: Hoss Man
>  Labels: Java9
>
> As part of SOLR-8874, Uwe disabled all HDFS based tests under java9 at the 
> build.xml/pom.xml level by adding a conditional to the existing 
> {{tests.disableHdfs}} system property (Note: this property exists so that 
> HDFS tests can be disabled by default on windows, but still run on cygwin if 
> users wish to set that property)
> We need to get to the bottom of what exactly the issue(s) are with HDFS and 
> file specific bugs to track the problems



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-8593) Integrate Apache Calcite into the SQLHandler

[Julian Hyde (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890784#comment-15890784
 ] 

Julian Hyde edited comment on SOLR-8593 at 3/1/17 6:49 PM:
---

[~risdenk], Regarding the Turkish locale issue. We have to explicitly pass 
user.timezone from maven into surefire (see 
[pom.xml|https://github.com/apache/calcite/blob/0372d23b847d4d145917dd786d1c9e3570cb8041/pom.xml#L733]),
 so I suspect we'd have to do the same with the locale. Can you log a Calcite 
case please? Even if we can't reproduce, I'd rather that we tracked it.


was (Author: julianhyde):
[~risdenk], Regarding the Turkish locale issue. We have to explicitly pass 
user.timezone from maven into surefire, so I suspect we'd have to do the same 
with the locale. Can you log a Calcite case please? Even if we can't reproduce, 
I'd rather that we tracked it.

> Integrate Apache Calcite into the SQLHandler
> 
>
> Key: SOLR-8593
> URL: https://issues.apache.org/jira/browse/SOLR-8593
> Project: Solr
>  Issue Type: Improvement
>  Components: Parallel SQL
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: 6.5, master (7.0)
>
> Attachments: SOLR-8593.patch, SOLR-8593.patch, SOLR-8593.patch
>
>
>The Presto SQL Parser was perfect for phase one of the SQLHandler. It was 
> nicely split off from the larger Presto project and it did everything that 
> was needed for the initial implementation.
> Phase two of the SQL work though will require an optimizer. Here is where 
> Apache Calcite comes into play. It has a battle tested cost based optimizer 
> and has been integrated into Apache Drill and Hive.
> This work can begin in trunk following the 6.0 release. The final query plans 
> will continue to be translated to Streaming API objects (TupleStreams), so 
> continued work on the JDBC driver should plug in nicely with the Calcite work.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8593) Integrate Apache Calcite into the SQLHandler

[Julian Hyde (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890784#comment-15890784
 ] 

Julian Hyde commented on SOLR-8593:
---

[~risdenk], Regarding the Turkish locale issue. We have to explicitly pass 
user.timezone from maven into surefire, so I suspect we'd have to do the same 
with the locale. Can you log a Calcite case please? Even if we can't reproduce, 
I'd rather that we tracked it.

> Integrate Apache Calcite into the SQLHandler
> 
>
> Key: SOLR-8593
> URL: https://issues.apache.org/jira/browse/SOLR-8593
> Project: Solr
>  Issue Type: Improvement
>  Components: Parallel SQL
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: 6.5, master (7.0)
>
> Attachments: SOLR-8593.patch, SOLR-8593.patch, SOLR-8593.patch
>
>
>The Presto SQL Parser was perfect for phase one of the SQLHandler. It was 
> nicely split off from the larger Presto project and it did everything that 
> was needed for the initial implementation.
> Phase two of the SQL work though will require an optimizer. Here is where 
> Apache Calcite comes into play. It has a battle tested cost based optimizer 
> and has been integrated into Apache Drill and Hive.
> This work can begin in trunk following the 6.0 release. The final query plans 
> will continue to be translated to Streaming API objects (TupleStreams), so 
> continued work on the JDBC driver should plug in nicely with the Calcite work.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10059) In SolrCloud, every fq added via is computed twice.

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890736#comment-15890736
 ] 

Hoss Man commented on SOLR-10059:
-

SOme historical context here is that when "distributed search" was first added, 
before there was any native "cloud support" the want to trigger a distributed 
search was to specify a list of shard URLs (as a request param) for the 
coordinator node to query & aggregate the responses from.  A common 
configuration pattern was for people to put the shards (URLS) in their handler 
defaults in solrconfig.xml -- but also have a "shards.qt" param that pointed at 
a different handler name. (to some other handler registration w/o the shards 
list) ... alternatively, some people deployed one solrconfig.xml file to the 
nodes that had data one them (and included things like defaults/appends fqs), 
and had completely diff solrconfig.xml for their coordinator nodes that only 
know about the shards param and the list of nodes to aggregate from.

you're definitely correct -- as things evolved into solr cloud, the fact that 
things like appends fqs are being computed multiple times because they come 
from both the coordinator node's init params and the individual shard's 
(identical) init params.

I think the the general approach #2 you suggested makes the most sense ... the 
bit of code (in RequestHandlerBase i believe?) where the 
defaults/invariants/appends are wrapped around/under the request params should 
be skipped in (some) solr cloud shard requests -- but i think checking IS_SHARD 
is really only 1 piece of the puzzle? for completeness we should probably also 
check that the SolrCore says we are in solrcloud mode (to ensure the user isn't 
rolling their own distributed search via pre-solrcloud shard requests like i 
described above)

the only other thing to worry about i guess is what should happen when 
multi-collection requests are issued? -- such as when a collection alias points 
to multiple collections.  Shouldn't the "appends" FQ params from collection1 be 
applied anytime a query includes collection1, and the appends FQ params from 
collection1 be applied any time a query includes collection2; even if those are 
both a single query that originated via a request to "both_collections" (which 
is an alias for "collection1,collection2") ?

I suppose the coordinating node could include the "source collection (alias)" 
of the request as a param that the individual shards could compare with 
themselves to decide when they need to wrap the params?

(just thinking outloud -- probably a better solution)






> In SolrCloud, every fq added via  is computed twice.
> 
>
> Key: SOLR-10059
> URL: https://issues.apache.org/jira/browse/SOLR-10059
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 6.4.0
>Reporter: Marc Morissette
>  Labels: performance
>
> While researching another issue, I noticed that parameters appended to a 
> query via SearchHandler's  are added to the query twice 
> in SolrCloud: once on the aggregator and again on the shard.
> The FacetComponent corrects this automatically by removing duplicates. Field 
> queries added in this fashion are however computed twice and that hinders 
> performance on filter queries that aren't simple bitsets such as those 
> produced by the CollapsingQueryParser.
> To reproduce the issue, simply test this handler on a large enough 
> collection, then replace "appends" with "defaults". You'll notice significant 
> performance improvements.
> {code}
> 
> 
> {!collapse field=routingKey hint=top_fc}
> 
> 
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7410) Make cache keys and closed listeners less trappy

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890723#comment-15890723
 ] 

Adrien Grand commented on LUCENE-7410:
--

It was a test bug introduced by this change, which happens when the created 
IndexSearcher wraps a threadpool. I just pushed a fix.

> Make cache keys and closed listeners less trappy
> 
>
> Key: LUCENE-7410
> URL: https://issues.apache.org/jira/browse/LUCENE-7410
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Adrien Grand
> Fix For: master (7.0)
>
> Attachments: LUCENE-7410.patch, LUCENE-7410.patch, LUCENE-7410.patch
>
>
> IndexReader currently exposes getCoreCacheKey(), 
> getCombinedCoreAndDeletesKey(), addCoreClosedListener() and 
> addReaderClosedListener(). They are typically used to manage resources whose 
> lifetime needs to mimic the lifetime of segments/indexes, typically caches.
> I think this is trappy for various reasons:
> h3. Memory leaks
> When maintaining a cache, entries are added to the cache based on the cache 
> key and then evicted using the cache key that is given back by the close 
> listener, so it is very important that both keys are the same.
> But if a filter reader happens to delegate get*Key() and not 
> add*ClosedListener() or vice-versa then there is potential for a memory leak 
> since the closed listener will be called on a different key and entries will 
> never be evicted from the cache.
> h3. Lifetime expectations
> The expectation of using the core cache key is that it will not change in 
> case of deletions, but this is only true on SegmentReader and LeafReader 
> impls that delegate to it. Other implementations such as composite readers or 
> parallel leaf readers use the same key for "core" and "combined core and 
> deletes".
> h3. Throw-away wrappers cause cache trashing
> An application might want to either expose more (with a ParrallelReader or 
> MultiReader) or less information (by filtering fields/docs that can be seen) 
> depending on the user who is logged in. In that case the application would 
> typically maintain a DirectoryReader and then wrap it per request depending 
> on the logged user and throw away the wrapper once the request is completed.
> The problem is that these wrappers have their own cache keys and the 
> application may build something costly and put it in a cache to throw it away 
> a couple milliseconds later. I would rather like for such readers to have a 
> way to opt out from caching on order to avoid this performance trap.
> h3. Type safety
> The keys that are exposed are plain java.lang.Object instances, which 
> requires caches to look like a {{Map}} which makes it very easy to 
> either try to get, put or remove on the wrong object since any object would 
> be accepted.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7410) Make cache keys and closed listeners less trappy

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890721#comment-15890721
 ] 

ASF subversion and git services commented on LUCENE-7410:
-

Commit 540a23723104e250a4fce94042fb90c86fcf3720 in lucene-solr's branch 
refs/heads/master from [~jpountz]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=540a237 ]

LUCENE-7410: Make TestReaderClosed pass if the IndexSearcher wraps a threadpool.


> Make cache keys and closed listeners less trappy
> 
>
> Key: LUCENE-7410
> URL: https://issues.apache.org/jira/browse/LUCENE-7410
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Adrien Grand
> Fix For: master (7.0)
>
> Attachments: LUCENE-7410.patch, LUCENE-7410.patch, LUCENE-7410.patch
>
>
> IndexReader currently exposes getCoreCacheKey(), 
> getCombinedCoreAndDeletesKey(), addCoreClosedListener() and 
> addReaderClosedListener(). They are typically used to manage resources whose 
> lifetime needs to mimic the lifetime of segments/indexes, typically caches.
> I think this is trappy for various reasons:
> h3. Memory leaks
> When maintaining a cache, entries are added to the cache based on the cache 
> key and then evicted using the cache key that is given back by the close 
> listener, so it is very important that both keys are the same.
> But if a filter reader happens to delegate get*Key() and not 
> add*ClosedListener() or vice-versa then there is potential for a memory leak 
> since the closed listener will be called on a different key and entries will 
> never be evicted from the cache.
> h3. Lifetime expectations
> The expectation of using the core cache key is that it will not change in 
> case of deletions, but this is only true on SegmentReader and LeafReader 
> impls that delegate to it. Other implementations such as composite readers or 
> parallel leaf readers use the same key for "core" and "combined core and 
> deletes".
> h3. Throw-away wrappers cause cache trashing
> An application might want to either expose more (with a ParrallelReader or 
> MultiReader) or less information (by filtering fields/docs that can be seen) 
> depending on the user who is logged in. In that case the application would 
> typically maintain a DirectoryReader and then wrap it per request depending 
> on the logged user and throw away the wrapper once the request is completed.
> The problem is that these wrappers have their own cache keys and the 
> application may build something costly and put it in a cache to throw it away 
> a couple milliseconds later. I would rather like for such readers to have a 
> way to opt out from caching on order to avoid this performance trap.
> h3. Type safety
> The keys that are exposed are plain java.lang.Object instances, which 
> requires caches to look like a {{Map}} which makes it very easy to 
> either try to get, put or remove on the wrong object since any object would 
> be accepted.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7724) TestIndexWriterExceptions.testNoLostDeletesOrUpdates() failure

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890657#comment-15890657
 ] 

Steve Rowe commented on LUCENE-7724:


Beasting with the repro line (after s/test/test-nocompile/ \[1\]) on the tip of 
branch_6x failed 3/150 iterations on my box -> 2% failure rate.

\[1\] {{ant compile-test ; (for a in \{1..500\} ; do ant test-nocompile 
 ; done) 2>&1 | tee ~/output.txt}}

> TestIndexWriterExceptions.testNoLostDeletesOrUpdates() failure
> --
>
> Key: LUCENE-7724
> URL: https://issues.apache.org/jira/browse/LUCENE-7724
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Steve Rowe
>
> Non-reproducing branch_6x seed from my Jenkins:
> {noformat}
> Checking out Revision bce1417fceeed2054f16565e96dc49268c1b2ea1 
> (refs/remotes/origin/branch_6x)
> [...]
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestIndexWriterExceptions 
> -Dtests.method=testNoLostDeletesOrUpdates -Dtests.seed=611686EB379EF662 
> -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true 
> -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt 
> -Dtests.locale=en-US -Dtests.timezone=America/Cordoba -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
>[junit4] ERROR   9.90s J6  | 
> TestIndexWriterExceptions.testNoLostDeletesOrUpdates <<<
>[junit4]> Throwable #1: 
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an 
> uncaught exception in thread: Thread[id=343, name=Lucene Merge Thread #21, 
> state=RUNNABLE, group=TGRP-TestIndexWriterExceptions]
>[junit4]> Caused by: 
> org.apache.lucene.index.MergePolicy$MergeException: 
> org.apache.lucene.store.AlreadyClosedException: refusing to delete any files: 
> this IndexWriter hit an unrecoverable exception
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([611686EB379EF662]:0)
>[junit4]>  at 
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
>[junit4]>  at 
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
>[junit4]> Caused by: org.apache.lucene.store.AlreadyClosedException: 
> refusing to delete any files: this IndexWriter hit an unrecoverable exception
>[junit4]>  at 
> org.apache.lucene.index.IndexFileDeleter.ensureOpen(IndexFileDeleter.java:345)
>[junit4]>  at 
> org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:696)
>[junit4]>  at 
> org.apache.lucene.index.IndexFileDeleter.deleteNewFiles(IndexFileDeleter.java:691)
>[junit4]>  at 
> org.apache.lucene.index.IndexWriter.deleteNewFiles(IndexWriter.java:4964)
>[junit4]>  at 
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4455)
>[junit4]>  at 
> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3940)
>[junit4]>  at 
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
>[junit4]>  at 
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
>[junit4]> Caused by: 
> org.apache.lucene.store.MockDirectoryWrapper$FakeIOException
>[junit4]>  at 
> org.apache.lucene.index.TestIndexWriterExceptions$11.eval(TestIndexWriterExceptions.java:1875)
>[junit4]>  at 
> org.apache.lucene.store.MockDirectoryWrapper.maybeThrowDeterministicException(MockDirectoryWrapper.java:1022)
>[junit4]>  at 
> org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:144)
>[junit4]>  at 
> org.apache.lucene.store.MockIndexOutputWrapper.writeByte(MockIndexOutputWrapper.java:126)
>[junit4]>  at 
> org.apache.lucene.store.DataOutput.writeInt(DataOutput.java:70)
>[junit4]>  at 
> org.apache.lucene.codecs.CodecUtil.writeHeader(CodecUtil.java:90)
>[junit4]>  at 
> org.apache.lucene.codecs.CodecUtil.writeIndexHeader(CodecUtil.java:133)
>[junit4]>  at 
> org.apache.lucene.codecs.lucene54.Lucene54DocValuesConsumer.(Lucene54DocValuesConsumer.java:74)
>[junit4]>  at 
> org.apache.lucene.codecs.lucene54.Lucene54DocValuesFormat.fieldsConsumer(Lucene54DocValuesFormat.java:108)
>[junit4]>  at 
> org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.getInstance(PerFieldDocValuesFormat.java:213)
>[junit4]>  at 
> org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addNumericField(PerFieldDocValuesFormat.java:111)
>[junit4]>  at 
> org.apache.lucene.index.ReadersAndUpdates.handleNumericDVUpdates(ReadersAndUpdates.java:328)
>[junit4]>  at 
> 

[jira] [Created] (LUCENE-7724) TestIndexWriterExceptions.testNoLostDeletesOrUpdates() failure

[Steve Rowe (JIRA)]

Steve Rowe created LUCENE-7724:
--

 Summary: TestIndexWriterExceptions.testNoLostDeletesOrUpdates() 
failure
 Key: LUCENE-7724
 URL: https://issues.apache.org/jira/browse/LUCENE-7724
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Steve Rowe


Non-reproducing branch_6x seed from my Jenkins:

{noformat}
Checking out Revision bce1417fceeed2054f16565e96dc49268c1b2ea1 
(refs/remotes/origin/branch_6x)
[...]
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestIndexWriterExceptions -Dtests.method=testNoLostDeletesOrUpdates 
-Dtests.seed=611686EB379EF662 -Dtests.multiplier=2 -Dtests.nightly=true 
-Dtests.slow=true 
-Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt 
-Dtests.locale=en-US -Dtests.timezone=America/Cordoba -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
   [junit4] ERROR   9.90s J6  | 
TestIndexWriterExceptions.testNoLostDeletesOrUpdates <<<
   [junit4]> Throwable #1: 
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=343, name=Lucene Merge Thread #21, 
state=RUNNABLE, group=TGRP-TestIndexWriterExceptions]
   [junit4]> Caused by: org.apache.lucene.index.MergePolicy$MergeException: 
org.apache.lucene.store.AlreadyClosedException: refusing to delete any files: 
this IndexWriter hit an unrecoverable exception
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([611686EB379EF662]:0)
   [junit4]>at 
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
   [junit4]>at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
   [junit4]> Caused by: org.apache.lucene.store.AlreadyClosedException: 
refusing to delete any files: this IndexWriter hit an unrecoverable exception
   [junit4]>at 
org.apache.lucene.index.IndexFileDeleter.ensureOpen(IndexFileDeleter.java:345)
   [junit4]>at 
org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:696)
   [junit4]>at 
org.apache.lucene.index.IndexFileDeleter.deleteNewFiles(IndexFileDeleter.java:691)
   [junit4]>at 
org.apache.lucene.index.IndexWriter.deleteNewFiles(IndexWriter.java:4964)
   [junit4]>at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4455)
   [junit4]>at 
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3940)
   [junit4]>at 
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
   [junit4]>at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
   [junit4]> Caused by: 
org.apache.lucene.store.MockDirectoryWrapper$FakeIOException
   [junit4]>at 
org.apache.lucene.index.TestIndexWriterExceptions$11.eval(TestIndexWriterExceptions.java:1875)
   [junit4]>at 
org.apache.lucene.store.MockDirectoryWrapper.maybeThrowDeterministicException(MockDirectoryWrapper.java:1022)
   [junit4]>at 
org.apache.lucene.store.MockIndexOutputWrapper.writeBytes(MockIndexOutputWrapper.java:144)
   [junit4]>at 
org.apache.lucene.store.MockIndexOutputWrapper.writeByte(MockIndexOutputWrapper.java:126)
   [junit4]>at 
org.apache.lucene.store.DataOutput.writeInt(DataOutput.java:70)
   [junit4]>at 
org.apache.lucene.codecs.CodecUtil.writeHeader(CodecUtil.java:90)
   [junit4]>at 
org.apache.lucene.codecs.CodecUtil.writeIndexHeader(CodecUtil.java:133)
   [junit4]>at 
org.apache.lucene.codecs.lucene54.Lucene54DocValuesConsumer.(Lucene54DocValuesConsumer.java:74)
   [junit4]>at 
org.apache.lucene.codecs.lucene54.Lucene54DocValuesFormat.fieldsConsumer(Lucene54DocValuesFormat.java:108)
   [junit4]>at 
org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.getInstance(PerFieldDocValuesFormat.java:213)
   [junit4]>at 
org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addNumericField(PerFieldDocValuesFormat.java:111)
   [junit4]>at 
org.apache.lucene.index.ReadersAndUpdates.handleNumericDVUpdates(ReadersAndUpdates.java:328)
   [junit4]>at 
org.apache.lucene.index.ReadersAndUpdates.writeFieldUpdates(ReadersAndUpdates.java:521)
   [junit4]>at 
org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:272)
   [junit4]>at 
org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:4119)
   [junit4]>at 
org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:4077)
   [junit4]>at 
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3931)
   [junit4]>... 2 moreThrowable #2: 

[jira] [Commented] (SOLR-10153) UnifiedSolrHighlighter support for CustomSeparatorBreakIterator (LUCENE-6485)

[Amrit Sarkar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890544#comment-15890544
 ] 

Amrit Sarkar commented on SOLR-10153:
-

Alright :) Yes, +1 for the suggestion. I hate long worded arguments and just 
_seperator_ is meaningful and apt. CustomSeparatorBreakIterator itself is 
naming the char '_seperator_' in its implementation. 

Regarding using string as separator in near future, looking at the 
CustomSeparatorBreakIterator's code, it seems it can be done with not much 
effort. I will give it a shot soon. 

We should go with: *hl.bs.type=SEPARATOR with e.g. hl.bs.separator=|*

> UnifiedSolrHighlighter support for CustomSeparatorBreakIterator (LUCENE-6485)
> -
>
> Key: SOLR-10153
> URL: https://issues.apache.org/jira/browse/SOLR-10153
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Amrit Sarkar
> Attachments: SOLR-10153.patch, SOLR-10153.patch, 
> SOLR_10153_UH_and_PH_hl_customSeparatorChar.patch
>
>
> Lucene 5.3 added a CustomSeparatorBreakIterator (see LUCENE-6485)
> UnifiedSolrHighlighter should support *CustomSeparatorBreakIterator* along 
> with existing ones, WholeBreakIterator etc.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7410) Make cache keys and closed listeners less trappy

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890523#comment-15890523
 ] 

Adrien Grand commented on LUCENE-7410:
--

Thanks, I'll dig!

> Make cache keys and closed listeners less trappy
> 
>
> Key: LUCENE-7410
> URL: https://issues.apache.org/jira/browse/LUCENE-7410
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Adrien Grand
> Fix For: master (7.0)
>
> Attachments: LUCENE-7410.patch, LUCENE-7410.patch, LUCENE-7410.patch
>
>
> IndexReader currently exposes getCoreCacheKey(), 
> getCombinedCoreAndDeletesKey(), addCoreClosedListener() and 
> addReaderClosedListener(). They are typically used to manage resources whose 
> lifetime needs to mimic the lifetime of segments/indexes, typically caches.
> I think this is trappy for various reasons:
> h3. Memory leaks
> When maintaining a cache, entries are added to the cache based on the cache 
> key and then evicted using the cache key that is given back by the close 
> listener, so it is very important that both keys are the same.
> But if a filter reader happens to delegate get*Key() and not 
> add*ClosedListener() or vice-versa then there is potential for a memory leak 
> since the closed listener will be called on a different key and entries will 
> never be evicted from the cache.
> h3. Lifetime expectations
> The expectation of using the core cache key is that it will not change in 
> case of deletions, but this is only true on SegmentReader and LeafReader 
> impls that delegate to it. Other implementations such as composite readers or 
> parallel leaf readers use the same key for "core" and "combined core and 
> deletes".
> h3. Throw-away wrappers cause cache trashing
> An application might want to either expose more (with a ParrallelReader or 
> MultiReader) or less information (by filtering fields/docs that can be seen) 
> depending on the user who is logged in. In that case the application would 
> typically maintain a DirectoryReader and then wrap it per request depending 
> on the logged user and throw away the wrapper once the request is completed.
> The problem is that these wrappers have their own cache keys and the 
> application may build something costly and put it in a cache to throw it away 
> a couple milliseconds later. I would rather like for such readers to have a 
> way to opt out from caching on order to avoid this performance trap.
> h3. Type safety
> The keys that are exposed are plain java.lang.Object instances, which 
> requires caches to look like a {{Map}} which makes it very easy to 
> either try to get, put or remove on the wrong object since any object would 
> be accepted.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_121) - Build # 759 - Still unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/759/
Java: 32bit/jdk1.8.0_121 -server -XX:+UseG1GC

2 tests failed.
FAILED:  org.apache.solr.cloud.OverseerRolesTest.testOverseerRole

Error Message:
Timed out waiting for overseer state change

Stack Trace:
java.lang.AssertionError: Timed out waiting for overseer state change
at 
__randomizedtesting.SeedInfo.seed([1EAC1C4DCAAFB171:FF67E1D9F11C87A0]:0)
at org.junit.Assert.fail(Assert.java:93)
at 
org.apache.solr.cloud.OverseerRolesTest.waitForNewOverseer(OverseerRolesTest.java:62)
at 
org.apache.solr.cloud.OverseerRolesTest.testOverseerRole(OverseerRolesTest.java:140)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestLazyCores

Error Message:
Could not remove the following files (in the order of attempts):

[jira] [Commented] (LUCENE-7410) Make cache keys and closed listeners less trappy

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890516#comment-15890516
 ] 

Steve Rowe commented on LUCENE-7410:


My Jenkins found a reproducing seed for a 
{{TestReaderClosed.testReaderChaining()}} failure, and {{git bisect}} running 
the repro line says:

{noformat}
df6f83072303b4891a296b700a50c743284d3c30 is the first bad commit
commit df6f83072303b4891a296b700a50c743284d3c30
Author: Adrien Grand 
Date:   Tue Feb 28 14:21:30 2017 +0100

LUCENE-7410: Make cache keys and close listeners less trappy.
{noformat}

{noformat}
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestReaderClosed 
-Dtests.method=testReaderChaining -Dtests.seed=C4374342D2D99B8F 
-Dtests.slow=true -Dtests.locale=hi -Dtests.timezone=America/Dominica 
-Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 0.04s J1 | TestReaderClosed.testReaderChaining <<<
   [junit4]> Throwable #1: java.lang.AssertionError: Query failed, but not 
due to an AlreadyClosedException
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([C4374342D2D99B8F:530116CD6543D942]:0)
   [junit4]>at 
org.apache.lucene.index.TestReaderClosed.testReaderChaining(TestReaderClosed.java:96)
   [junit4]>at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): 
{field=PostingsFormat(name=LuceneVarGapFixedInterval)}, docValues:{}, 
maxPointsInLeafNode=1885, maxMBSortInHeap=6.663525927605304, 
sim=RandomSimilarity(queryNorm=true): {}, locale=hi, timezone=America/Dominica
   [junit4]   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 
1.8.0_77 (64-bit)/cpus=16,threads=1,free=82013152,total=289931264
{noformat}

> Make cache keys and closed listeners less trappy
> 
>
> Key: LUCENE-7410
> URL: https://issues.apache.org/jira/browse/LUCENE-7410
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Adrien Grand
> Fix For: master (7.0)
>
> Attachments: LUCENE-7410.patch, LUCENE-7410.patch, LUCENE-7410.patch
>
>
> IndexReader currently exposes getCoreCacheKey(), 
> getCombinedCoreAndDeletesKey(), addCoreClosedListener() and 
> addReaderClosedListener(). They are typically used to manage resources whose 
> lifetime needs to mimic the lifetime of segments/indexes, typically caches.
> I think this is trappy for various reasons:
> h3. Memory leaks
> When maintaining a cache, entries are added to the cache based on the cache 
> key and then evicted using the cache key that is given back by the close 
> listener, so it is very important that both keys are the same.
> But if a filter reader happens to delegate get*Key() and not 
> add*ClosedListener() or vice-versa then there is potential for a memory leak 
> since the closed listener will be called on a different key and entries will 
> never be evicted from the cache.
> h3. Lifetime expectations
> The expectation of using the core cache key is that it will not change in 
> case of deletions, but this is only true on SegmentReader and LeafReader 
> impls that delegate to it. Other implementations such as composite readers or 
> parallel leaf readers use the same key for "core" and "combined core and 
> deletes".
> h3. Throw-away wrappers cause cache trashing
> An application might want to either expose more (with a ParrallelReader or 
> MultiReader) or less information (by filtering fields/docs that can be seen) 
> depending on the user who is logged in. In that case the application would 
> typically maintain a DirectoryReader and then wrap it per request depending 
> on the logged user and throw away the wrapper once the request is completed.
> The problem is that these wrappers have their own cache keys and the 
> application may build something costly and put it in a cache to throw it away 
> a couple milliseconds later. I would rather like for such readers to have a 
> way to opt out from caching on order to avoid this performance trap.
> h3. Type safety
> The keys that are exposed are plain java.lang.Object instances, which 
> requires caches to look like a {{Map}} which makes it very easy to 
> either try to get, put or remove on the wrong object since any object would 
> be accepted.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9858) Collect aggregated metrics from nodes and shard leaders in overseer

[Andrzej Bialecki (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890512#comment-15890512
 ] 

Andrzej Bialecki  commented on SOLR-9858:
-

Comments to your comments:
1. Fixing.
2. We can make this handler smarter. I thought it would be less complicated to 
allow any updates to be processed, especially since we're not sure yet what we 
want to aggregate where, and also to allow for future extensibility of this 
mechanism. We can lock it down later. The issue of sending reports to the node 
that recently changed its role (eg. stopped being Overseer) - I don't think 
it's a big deal, in the worst case we lose just one report, because the next 
report will go to the right node.
3.  (and 8) I've modified {{SolrClientCache}} to use the provided instance of 
{{HttpClient}} and both {{SolrOverseerReporter}} and {{SolrReplicaReporter}} 
use the one obtained from {{UpdateShardHandler}}.
4. After refactoring the way to construct these names stopped being so simple - 
I'm moving these tests now to {{SolrCloudReportersTest}}.
5. Good point - fixing.
6. Fixing.
7. I'll make a separate issue for this.
9. This is on purpose. Imagine a collection with replication 1 - the only 
replica is also a shard leader. If we prevented the leader from reporting its 
state we wouldn't get any data in the "leader" registry.
10. Indeed, there seems to be a bug somewhere - investigating...

Updated patch will follow. 

> Collect aggregated metrics from nodes and shard leaders in overseer
> ---
>
> Key: SOLR-9858
> URL: https://issues.apache.org/jira/browse/SOLR-9858
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Affects Versions: master (7.0)
>Reporter: Andrzej Bialecki 
>Assignee: Andrzej Bialecki 
>Priority: Minor
> Attachments: SOLR-9858.patch
>
>
> Overseer can collect metrics from Solr nodes and shard leaders in order to 
> have a notion of the indexing / query / replication / system load on each 
> node, shard and its replicas. This information then can be used for cluster 
> (auto)scaling.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6819) Deprecate index-time boosts?

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890496#comment-15890496
 ] 

Uwe Schindler commented on LUCENE-6819:
---

Atomic part is fine. When proposing not to use atomics I did not notice that 
the getAndSet does not happen for the common case (without boost), so there is 
no performance issue.

> Deprecate index-time boosts?
> 
>
> Key: LUCENE-6819
> URL: https://issues.apache.org/jira/browse/LUCENE-6819
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-6819-deprecation.patch, 
> LUCENE-6819-deprecation.patch, LUCENE-6819.patch, LUCENE-6819.patch, 
> LUCENE-6819-wip.patch
>
>
> Follow-up of this comment: 
> https://issues.apache.org/jira/browse/LUCENE-6818?focusedCommentId=14934801=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14934801
> Index-time boosts are a very expert feature whose behaviour is tight to the 
> Similarity impl. Additionally users have often be confused by the poor 
> precision due to the fact that we encode values on a single byte. But now we 
> have doc values that allow you to encode any values the way you want with as 
> much precision as you need so maybe we should deprecate index-time boosts and 
> recommend to encode index-time scoring factors into doc values fields instead.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7723) LongValuesSource/DoubleValuesSource impls should implement equals/hashCode

[Adrien Grand (JIRA)]

Adrien Grand created LUCENE-7723:


 Summary: LongValuesSource/DoubleValuesSource impls should 
implement equals/hashCode
 Key: LUCENE-7723
 URL: https://issues.apache.org/jira/browse/LUCENE-7723
 Project: Lucene - Core
  Issue Type: Task
Reporter: Adrien Grand


Given that we embed values sources in queries and sorts, from which we expect 
that they have working equals/hashCode, we should also implement 
equals/hashCode on LongValuesSource/DoubleValuesSource impls.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6819) Deprecate index-time boosts?

[JIRA]

[ 
https://issues.apache.org/jira/browse/LUCENE-6819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890472#comment-15890472
 ] 

Jan Høydahl commented on LUCENE-6819:
-

I like the Atomic part and alternating between WARN on first-time and DEBUG 
thereafter!

Should the deprecation patch have a solr/CHANGES.txt entry even if it is a 
LUCENE issue? Well, it's a hybrid issue, and we have explicitly referenced 
LUCENE issues in solr CHANGES earlier.

> Deprecate index-time boosts?
> 
>
> Key: LUCENE-6819
> URL: https://issues.apache.org/jira/browse/LUCENE-6819
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-6819-deprecation.patch, 
> LUCENE-6819-deprecation.patch, LUCENE-6819.patch, LUCENE-6819.patch, 
> LUCENE-6819-wip.patch
>
>
> Follow-up of this comment: 
> https://issues.apache.org/jira/browse/LUCENE-6818?focusedCommentId=14934801=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14934801
> Index-time boosts are a very expert feature whose behaviour is tight to the 
> Similarity impl. Additionally users have often be confused by the poor 
> precision due to the fact that we encode values on a single byte. But now we 
> have doc values that allow you to encode any values the way you want with as 
> much precision as you need so maybe we should deprecate index-time boosts and 
> recommend to encode index-time scoring factors into doc values fields instead.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7722) Remove BoostedQuery

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890466#comment-15890466
 ] 

Adrien Grand commented on LUCENE-7722:
--

bq. There's a bit of an interface mismatch here, though, as DoubleValuesSource 
is always IndexReader-independant, so you need to make sure that the passed in 
Weight was generated against the same IndexReader as the FunctionScoreQuery is 
being run against.

I don't think that would be much of an issue if it is documented properly? It's 
appealing that it would allow us to remove all these custom-score queries.

> Remove BoostedQuery
> ---
>
> Key: LUCENE-7722
> URL: https://issues.apache.org/jira/browse/LUCENE-7722
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Adrien Grand
>Priority: Minor
>
> We already  have FunctionScoreQuery, which is more flexible than BoostedQuery 
> as it can combine scores in arbitrary ways and only requests scores on the 
> underlying scorer if they are needed. So let's remove BoostedQuery?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7449) Add CROSSES query support to RangeField

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890464#comment-15890464
 ] 

Steve Rowe commented on LUCENE-7449:


ASF Jenkins found a reproducing seed for the 
{{Test\*RangeFieldQueries.testRandomMedium()}} failures 
[https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1251/]:

{noformat}
Checking out Revision ec13032a948a29f69d50d41e4859fd38ed5ca377 
(refs/remotes/origin/master)
[...]
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestLongRangeFieldQueries -Dtests.method=testRandomMedium 
-Dtests.seed=372308A94C0C5F3B -Dtests.multiplier=2 -Dtests.nightly=true 
-Dtests.slow=true 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt
 -Dtests.locale=be-BY -Dtests.timezone=America/Indiana/Winamac 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 7.53s J2 | TestLongRangeFieldQueries.testRandomMedium <<<
   [junit4]> Throwable #1: java.lang.AssertionError: wrong hit (first of 
possibly more):
   [junit4]> FAIL (iter 75): id=52600 should not match but did
   [junit4]>  queryRange=Box(-9223372036854775808 TO 9223372036854775807)
   [junit4]>  box=Box(2938302894405237373 TO 3114577670663594333)
   [junit4]>  queryType=CROSSES
   [junit4]>  deleted?=false
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([372308A94C0C5F3B:8AFD3F010D693C5D]:0)
   [junit4]>at 
org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:287)
   [junit4]>at 
org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:158)
   [junit4]>at 
org.apache.lucene.search.BaseRangeFieldQueryTestCase.testRandomMedium(BaseRangeFieldQueryTestCase.java:68)
[...]
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): 
{id=PostingsFormat(name=LuceneFixedGap)}, 
docValues:{id=DocValuesFormat(name=Direct), 
longRangeField=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=41, 
maxMBSortInHeap=5.573918837864136, sim=RandomSimilarity(queryNorm=true): {}, 
locale=be-BY, timezone=America/Indiana/Winamac
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 
1.8.0_121 (64-bit)/cpus=4,threads=1,free=136358544,total=484966400
{noformat}
{noformat}
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestIntRangeFieldQueries -Dtests.method=testRandomMedium 
-Dtests.seed=372308A94C0C5F3B -Dtests.multiplier=2 -Dtests.nightly=true 
-Dtests.slow=true 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt
 -Dtests.locale=en-ZA -Dtests.timezone=Europe/Helsinki -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
   [junit4] FAILURE 5.30s J2 | TestIntRangeFieldQueries.testRandomMedium <<<
   [junit4]> Throwable #1: java.lang.AssertionError: wrong hit (first of 
possibly more):
   [junit4]> FAIL (iter 75): id=52600 should not match but did
   [junit4]>  queryRange=Box(-2147483648 TO 2147483647)
   [junit4]>  box=Box(-2059116195 TO -410600835)
   [junit4]>  queryType=CROSSES
   [junit4]>  deleted?=false
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([372308A94C0C5F3B:8AFD3F010D693C5D]:0)
   [junit4]>at 
org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:287)
   [junit4]>at 
org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:158)
   [junit4]>at 
org.apache.lucene.search.BaseRangeFieldQueryTestCase.testRandomMedium(BaseRangeFieldQueryTestCase.java:68)
[...]
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): 
{id=PostingsFormat(name=LuceneVarGapFixedInterval)}, 
docValues:{id=DocValuesFormat(name=Memory), 
intRangeField=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=1034, 
maxMBSortInHeap=7.960284461233806, sim=RandomSimilarity(queryNorm=true): {}, 
locale=en-ZA, timezone=Europe/Helsinki
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 
1.8.0_121 (64-bit)/cpus=4,threads=1,free=254900272,total=515375104
{noformat}
{noformat}
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestDoubleRangeFieldQueries -Dtests.method=testRandomMedium 
-Dtests.seed=372308A94C0C5F3B -Dtests.multiplier=2 -Dtests.nightly=true 
-Dtests.slow=true 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt
 -Dtests.locale=mk-MK -Dtests.timezone=Japan -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
   [junit4] FAILURE 6.37s J0 | TestDoubleRangeFieldQueries.testRandomMedium <<<
   [junit4]> Throwable #1: java.lang.AssertionError: wrong hit (first of 
possibly more):
   [junit4]> FAIL (iter 75): id=52600 should not match but did
 

[jira] [Updated] (LUCENE-5143) rm or formalize dealing with "general" KEYS files in our dist dir

[JIRA]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl updated LUCENE-5143:

Attachment: LUCENE-5143.patch

This patch fixes buildAndPushRelease.py so that it will not push KEYS inside 
versioned release folder anymore.

It also adds an automatic check that the committer's key used for signing is 
actually present on the KEYS file. It does this by fetching the KEYS file and 
then finding the key ID using a regex.

Question is if we should still publish {{java/KEYS}} and {{solr/KEYS}}, in 
which case we could discontinue also the root one) or if both these README's 
should refer to the common file in the dist root? 

> rm or formalize dealing with "general" KEYS files in our dist dir
> -
>
> Key: LUCENE-5143
> URL: https://issues.apache.org/jira/browse/LUCENE-5143
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Hoss Man
> Attachments: LUCENE-5143.patch, LUCENE-5143_READMEs.patch
>
>
> At some point in the past, we started creating a snapshots of KEYS (taken 
> from the auto-generated data from id.apache.org) in the release dir of each 
> release...
> http://www.apache.org/dist/lucene/solr/4.4.0/KEYS
> http://www.apache.org/dist/lucene/java/4.4.0/KEYS
> http://archive.apache.org/dist/lucene/java/4.3.0/KEYS
> http://archive.apache.org/dist/lucene/solr/4.3.0/KEYS
> etc...
> But we also still have some "general" KEYS files...
> https://www.apache.org/dist/lucene/KEYS
> https://www.apache.org/dist/lucene/java/KEYS
> https://www.apache.org/dist/lucene/solr/KEYS
> ...which (as i discovered when i went to add my key to them today) are stale 
> and don't seem to be getting updated.
> I vaguely remember someone (rmuir?) explaining to me at one point the reason 
> we started creating a fresh copy of KEYS in each release dir, but i no longer 
> remember what they said, and i can't find any mention of a reason in any of 
> the release docs, or in any sort of comment in buildAndPushRelease.py
> we probably do one of the following:
>  * remove these "general" KEYS files
>  * add a disclaimer to the top of these files that they are legacy files for 
> verifying old releases and are no longer used for new releases
>  * ensure these files are up to date stop generating per-release KEYS file 
> copies
>  * update our release process to ensure that the general files get updated on 
> each release as well



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8593) Integrate Apache Calcite into the SQLHandler

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890442#comment-15890442
 ] 

Kevin Risden commented on SOLR-8593:


You can generate a patch from the jira/solr-8593 branch (if you have it 
locally) and then just apply the patch to branch_6x. That would avoid the merge 
and all the other commits.

> Integrate Apache Calcite into the SQLHandler
> 
>
> Key: SOLR-8593
> URL: https://issues.apache.org/jira/browse/SOLR-8593
> Project: Solr
>  Issue Type: Improvement
>  Components: Parallel SQL
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: 6.5, master (7.0)
>
> Attachments: SOLR-8593.patch, SOLR-8593.patch, SOLR-8593.patch
>
>
>The Presto SQL Parser was perfect for phase one of the SQLHandler. It was 
> nicely split off from the larger Presto project and it did everything that 
> was needed for the initial implementation.
> Phase two of the SQL work though will require an optimizer. Here is where 
> Apache Calcite comes into play. It has a battle tested cost based optimizer 
> and has been integrated into Apache Drill and Hive.
> This work can begin in trunk following the 6.0 release. The final query plans 
> will continue to be translated to Streaming API objects (TupleStreams), so 
> continued work on the JDBC driver should plug in nicely with the Calcite work.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-8593) Integrate Apache Calcite into the SQLHandler

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890431#comment-15890431
 ] 

Joel Bernstein edited comment on SOLR-8593 at 3/1/17 3:56 PM:
--

The backport is turning out be trickier then I thought.

The reason is that jira/solr-8593 has lots of master commits in it that are not 
in branch_6x. So merging is not going to work.

Also getting an isolated list of the 80+ commits done in jira/solr-8593 for the 
Calcite integration is tricky because the original pull request is closed. 
Creating a new pull request against branch_6x includes all the master commits.

So I'm looking at ways to isolate all the commits to cherry-pick.


was (Author: joel.bernstein):
The backport is turning out be trickier then I thought.

The reason is that jira/solr-8593 has lost of master commits in it that are not 
in branch_6x. So merging is not going to work.

Also getting an isolated list of the 80+ commits done in jira/solr-8593 for the 
Calcite integration is tricky because the original pull request is closed. 
Creating a new pull request against branch_6x includes all the master commits.

So I'm looking at ways to isolate all the commits to cherry-pick.

> Integrate Apache Calcite into the SQLHandler
> 
>
> Key: SOLR-8593
> URL: https://issues.apache.org/jira/browse/SOLR-8593
> Project: Solr
>  Issue Type: Improvement
>  Components: Parallel SQL
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: 6.5, master (7.0)
>
> Attachments: SOLR-8593.patch, SOLR-8593.patch, SOLR-8593.patch
>
>
>The Presto SQL Parser was perfect for phase one of the SQLHandler. It was 
> nicely split off from the larger Presto project and it did everything that 
> was needed for the initial implementation.
> Phase two of the SQL work though will require an optimizer. Here is where 
> Apache Calcite comes into play. It has a battle tested cost based optimizer 
> and has been integrated into Apache Drill and Hive.
> This work can begin in trunk following the 6.0 release. The final query plans 
> will continue to be translated to Streaming API objects (TupleStreams), so 
> continued work on the JDBC driver should plug in nicely with the Calcite work.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8593) Integrate Apache Calcite into the SQLHandler

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890431#comment-15890431
 ] 

Joel Bernstein commented on SOLR-8593:
--

The backport is turning out be trickier then I thought.

The reason is that jira/solr-8593 has lost of master commits in it that are not 
in branch_6x. So merging is not going to work.

Also getting an isolated list of the 80+ commits done in jira/solr-8593 for the 
Calcite integration is tricky because the original pull request is closed. 
Creating a new pull request against branch_6x includes all the master commits.

So I'm looking at ways to isolate all the commits to cherry-pick.

> Integrate Apache Calcite into the SQLHandler
> 
>
> Key: SOLR-8593
> URL: https://issues.apache.org/jira/browse/SOLR-8593
> Project: Solr
>  Issue Type: Improvement
>  Components: Parallel SQL
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: 6.5, master (7.0)
>
> Attachments: SOLR-8593.patch, SOLR-8593.patch, SOLR-8593.patch
>
>
>The Presto SQL Parser was perfect for phase one of the SQLHandler. It was 
> nicely split off from the larger Presto project and it did everything that 
> was needed for the initial implementation.
> Phase two of the SQL work though will require an optimizer. Here is where 
> Apache Calcite comes into play. It has a battle tested cost based optimizer 
> and has been integrated into Apache Drill and Hive.
> This work can begin in trunk following the 6.0 release. The final query plans 
> will continue to be translated to Streaming API objects (TupleStreams), so 
> continued work on the JDBC driver should plug in nicely with the Calcite work.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-5143) rm or formalize dealing with "general" KEYS files in our dist dir

[JIRA]

[ 
https://issues.apache.org/jira/browse/LUCENE-5143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890418#comment-15890418
 ] 

Jan Høydahl edited comment on LUCENE-5143 at 3/1/17 3:51 PM:
-

Attaching a patch against DIST site which updates the README.html files

* Clone Lucene's README for Solr, remove old HEADER.html, so it will display 
below the download links, not above
* For all three sub folders java, pylucene and solr, add a paragraph "Use a 
mirror" to encourage downloading from mirrors
* Added logos to the README.html files (where to find a smaller official Solr 
logo online?)
* Remove warning about tar for macOS, as this is not a problem anymore
* Revise the "Signatures" paragraph to a "Signatures and hashes" paragraph also 
describing what .md5 and .sha1 files are for
* Rewrite the Signatures paragraph
** It was still talking about {{binaries/}} and {{source/}} folders
** Make it clear that the {{.asc}} file must be downloaded from apache site
** Use x.y.z instead of random version number in example commands
** Changed link to the official apache dist to be HTTPS and write a note about 
always downloading KEYS file over HTTPS

There will be a separate git patch for {{buildAndPushRelease.py}} to stop 
publishing KEYS inside release folders.


was (Author: janhoy):
Attaching a patch against DIST site which updates the README.html files

* Clone Lucene's README for Solr, remove old HEADER.html, so it will display 
below the download links, not above
* For all three sub folders java, pylucene and solr, add a paragraph "Use a 
mirror" to encourage downloading from mirrors
* Remove warning about tar for macOS, as this is not a problem anymore
* Revise the "Signatures" paragraph to a "Signatures and hashes" paragraph also 
describing what .md5 and .sha1 files are for
* Rewrite the Signatures paragraph
** It was still talking about {{binaries/}} and {{source/}} folders
** Make it clear that the {{.asc}} file must be downloaded from apache site
** Use x.y.z instead of random version number in example commands
** Changed link to the official apache dist to be HTTPS and write a note about 
always downloading KEYS file over HTTPS

There will be a separate git patch for {{buildAndPushRelease.py}} to stop 
publishing KEYS inside release folders.

> rm or formalize dealing with "general" KEYS files in our dist dir
> -
>
> Key: LUCENE-5143
> URL: https://issues.apache.org/jira/browse/LUCENE-5143
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Hoss Man
> Attachments: LUCENE-5143_READMEs.patch
>
>
> At some point in the past, we started creating a snapshots of KEYS (taken 
> from the auto-generated data from id.apache.org) in the release dir of each 
> release...
> http://www.apache.org/dist/lucene/solr/4.4.0/KEYS
> http://www.apache.org/dist/lucene/java/4.4.0/KEYS
> http://archive.apache.org/dist/lucene/java/4.3.0/KEYS
> http://archive.apache.org/dist/lucene/solr/4.3.0/KEYS
> etc...
> But we also still have some "general" KEYS files...
> https://www.apache.org/dist/lucene/KEYS
> https://www.apache.org/dist/lucene/java/KEYS
> https://www.apache.org/dist/lucene/solr/KEYS
> ...which (as i discovered when i went to add my key to them today) are stale 
> and don't seem to be getting updated.
> I vaguely remember someone (rmuir?) explaining to me at one point the reason 
> we started creating a fresh copy of KEYS in each release dir, but i no longer 
> remember what they said, and i can't find any mention of a reason in any of 
> the release docs, or in any sort of comment in buildAndPushRelease.py
> we probably do one of the following:
>  * remove these "general" KEYS files
>  * add a disclaimer to the top of these files that they are legacy files for 
> verifying old releases and are no longer used for new releases
>  * ensure these files are up to date stop generating per-release KEYS file 
> copies
>  * update our release process to ensure that the general files get updated on 
> each release as well



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr pull request #163: Jira/solr 8593

[joel-bernstein]

GitHub user joel-bernstein opened a pull request:

https://github.com/apache/lucene-solr/pull/163

Jira/solr 8593



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apache/lucene-solr jira/solr-8593

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/lucene-solr/pull/163.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #163


commit 283b329bbbd8b6a8f9c75ec4e79ca4034c910e88
Author: Karl Wright 
Date:   2016-12-28T00:41:55Z

LUCENE-7511: Introduce Vector.MINIMUM_ANGULAR_RESOLUTION.

commit c2292faaf1f4993bf1cec666f4286ac71f786506
Author: Shalin Shekhar Mangar 
Date:   2016-12-28T13:30:13Z

SOLR-9877: Remove assertion because many tests use UpdateShardHandler 
without metrics

commit e4ef4239f1b23afb116868e8528f1cd947287bd9
Author: Christine Poerschke 
Date:   2016-12-28T10:41:17Z

SOLR-9787, SOLR-9442: Replace json.nl=arrnvp with json.nl=arrntv (array of 
Name Type Value) style in JSONResponseWriter

commit dc6dcdda8078eb9f100fd6c66b5d488d057b019b
Author: Adrien Grand 
Date:   2016-12-28T19:12:02Z

LUCENE-7605: Use codec-specific impl of live docs when sorting.

commit 96ed221fb6924dd167591004a5eaf70d53f92e4f
Author: markrmiller 
Date:   2016-12-28T22:40:03Z

SOLR-9859: replication.properties cannot be updated after being written and 
neither eplication.properties or ndex.properties are durable in the face of a 
crash.

commit 262049fc8f60a166f0eed0aef5d7ddd1e7c90bc7
Author: markrmiller 
Date:   2016-12-28T22:42:41Z

SOLR-9899: StandardDirectoryFactory should use optimizations for all 
FilterDirectorys not just NRTCachingDirectory.

commit f29d2b5668296dfcdb8d650305449674faa29847
Author: Uwe Schindler 
Date:   2016-12-29T00:56:23Z

LUCENE-7595: Improve RAMUsageTester in test-framework to estimate memory 
usage of runtime classes and work with Java 9 EA (b148+). Disable static field 
heap usage checker in LuceneTestCase

commit 20362deb7e6814c1922163595e7edeb652d3ce37
Author: David Smiley 
Date:   2016-12-29T03:57:44Z

SOLR-9897: Add hl.requireFieldMatch=false support when using the 
UnifiedHighlighter

commit 662be93ed11abebaff1d13711f3bffca478ba61e
Author: Shalin Shekhar Mangar 
Date:   2016-12-29T04:27:03Z

SOLR-9877: Null check for metric registry before attempting to use it

commit 2781145eb3760489922530fd92d5f1d4c35215a9
Author: markrmiller 
Date:   2016-12-29T10:29:51Z

SOLR-9902: StandardDirectoryFactory should use Files API for it's move 
implementation.

commit a5e5c4a04385eb030aac1ec6126ff9b82407158f
Author: markrmiller 
Date:   2016-12-29T10:40:45Z

tests: bump up fudge

commit 197590a928cfefa51b1a8307046e5a11e5400e34
Author: markrmiller 
Date:   2016-12-28T21:16:14Z

SOLR-9901: Implement move in HdfsDirectoryFactory.

commit 5f55ae0b73ec546132f7188490065798bba0ffad
Author: markrmiller 
Date:   2016-12-29T10:53:51Z

tests: raise commit time to avoid false fails

commit b4de6288fb739b53ad138a16bc862130dc9318a8
Author: markrmiller 
Date:   2016-12-29T10:59:25Z

tests: bump timeout

commit c58eaa1a49bf518f3b0f70701ffd31f0cca79c17
Author: markrmiller 
Date:   2016-12-28T13:36:08Z

tests: speed up very slow test

commit fa959ad25d2460ebb41fae6bcf496a5ce785e989
Author: markrmiller 
Date:   2016-12-29T11:42:14Z

tests: speed up non nightly run

commit 12aff1cfcc48d7c89008447d482bf610242e0431
Author: Alan Woodward 
Date:   2016-10-27T15:50:28Z

SOLR-9132: Cut over some more tests

commit 87b6c2c8fcdc3a5f4adc3516f249af89b479d77a
Author: Alan Woodward 
Date:   2016-12-28T19:48:16Z

LUCENE-7607: FieldLeafComparator.setScorer() should throw IOException

commit 3f24fd81c836982be96b9b60082b53177fffe504
Author: Alan Woodward 
Date:   2016-12-28T20:10:47Z

LUCENE-5325: Add LongValuesSource and DoubleValuesSource in core

commit db9190db9372ae88a7392a7186397441ce070a96
Author: Uwe Schindler 
Date:   2016-12-29T19:31:47Z

LUCENE-7595: Fix bug with RamUsageTester incorrectly handling Iterables 
outside Java Runtime

commit 7dcb557ab73da7fb7af0e8f698895e28dde4bbca
Author: Joel Bernstein 
Date:   2016-12-29T18:46:04Z

SOLR-9905: Add NullStream to isolate the performance of the ExportWriter

commit 00723827ff5ad5c129d3d8487d2c64469ea03239
Author: Joel Bernstein 
Date:   2016-12-29T19:42:31Z

SOLR-9905: Update CHANGES.txt

commit 

[jira] [Updated] (LUCENE-5143) rm or formalize dealing with "general" KEYS files in our dist dir

[JIRA]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl updated LUCENE-5143:

Attachment: LUCENE-5143_READMEs.patch

Attaching a patch against DIST site which updates the README.html files

* Clone Lucene's README for Solr, remove old HEADER.html, so it will display 
below the download links, not above
* For all three sub folders java, pylucene and solr, add a paragraph "Use a 
mirror" to encourage downloading from mirrors
* Remove warning about tar for macOS, as this is not a problem anymore
* Revise the "Signatures" paragraph to a "Signatures and hashes" paragraph also 
describing what .md5 and .sha1 files are for
* Rewrite the Signatures paragraph
** It was still talking about {{binaries/}} and {{source/}} folders
** Make it clear that the {{.asc}} file must be downloaded from apache site
** Use x.y.z instead of random version number in example commands
** Changed link to the official apache dist to be HTTPS and write a note about 
always downloading KEYS file over HTTPS

There will be a separate git patch for {{buildAndPushRelease.py}} to stop 
publishing KEYS inside release folders.

> rm or formalize dealing with "general" KEYS files in our dist dir
> -
>
> Key: LUCENE-5143
> URL: https://issues.apache.org/jira/browse/LUCENE-5143
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Hoss Man
> Attachments: LUCENE-5143_READMEs.patch
>
>
> At some point in the past, we started creating a snapshots of KEYS (taken 
> from the auto-generated data from id.apache.org) in the release dir of each 
> release...
> http://www.apache.org/dist/lucene/solr/4.4.0/KEYS
> http://www.apache.org/dist/lucene/java/4.4.0/KEYS
> http://archive.apache.org/dist/lucene/java/4.3.0/KEYS
> http://archive.apache.org/dist/lucene/solr/4.3.0/KEYS
> etc...
> But we also still have some "general" KEYS files...
> https://www.apache.org/dist/lucene/KEYS
> https://www.apache.org/dist/lucene/java/KEYS
> https://www.apache.org/dist/lucene/solr/KEYS
> ...which (as i discovered when i went to add my key to them today) are stale 
> and don't seem to be getting updated.
> I vaguely remember someone (rmuir?) explaining to me at one point the reason 
> we started creating a fresh copy of KEYS in each release dir, but i no longer 
> remember what they said, and i can't find any mention of a reason in any of 
> the release docs, or in any sort of comment in buildAndPushRelease.py
> we probably do one of the following:
>  * remove these "general" KEYS files
>  * add a disclaimer to the top of these files that they are legacy files for 
> verifying old releases and are no longer used for new releases
>  * ensure these files are up to date stop generating per-release KEYS file 
> copies
>  * update our release process to ensure that the general files get updated on 
> each release as well



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_121) - Build # 19080 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/19080/
Java: 64bit/jdk1.8.0_121 -XX:+UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
Expected 2 of 3 replicas to be active but only found 1; 
[core_node3:{"core":"c8n_1x3_lf_shard1_replica2","base_url":"http://127.0.0.1:43964/sj_a","node_name":"127.0.0.1:43964_sj_a","state":"active","leader":"true"}];
 clusterState: DocCollection(c8n_1x3_lf//clusterstate.json/33)={   
"replicationFactor":"3",   "shards":{"shard1":{   
"range":"8000-7fff",   "state":"active",   "replicas":{ 
"core_node1":{   "core":"c8n_1x3_lf_shard1_replica1",   
"base_url":"http://127.0.0.1:33369/sj_a;,   
"node_name":"127.0.0.1:33369_sj_a",   "state":"down"}, 
"core_node2":{   "state":"down",   
"base_url":"http://127.0.0.1:36651/sj_a;,   
"core":"c8n_1x3_lf_shard1_replica3",   
"node_name":"127.0.0.1:36651_sj_a"}, "core_node3":{   
"core":"c8n_1x3_lf_shard1_replica2",   
"base_url":"http://127.0.0.1:43964/sj_a;,   
"node_name":"127.0.0.1:43964_sj_a",   "state":"active",   
"leader":"true",   "router":{"name":"compositeId"},   
"maxShardsPerNode":"1",   "autoAddReplicas":"false"}

Stack Trace:
java.lang.AssertionError: Expected 2 of 3 replicas to be active but only found 
1; 
[core_node3:{"core":"c8n_1x3_lf_shard1_replica2","base_url":"http://127.0.0.1:43964/sj_a","node_name":"127.0.0.1:43964_sj_a","state":"active","leader":"true"}];
 clusterState: DocCollection(c8n_1x3_lf//clusterstate.json/33)={
  "replicationFactor":"3",
  "shards":{"shard1":{
  "range":"8000-7fff",
  "state":"active",
  "replicas":{
"core_node1":{
  "core":"c8n_1x3_lf_shard1_replica1",
  "base_url":"http://127.0.0.1:33369/sj_a;,
  "node_name":"127.0.0.1:33369_sj_a",
  "state":"down"},
"core_node2":{
  "state":"down",
  "base_url":"http://127.0.0.1:36651/sj_a;,
  "core":"c8n_1x3_lf_shard1_replica3",
  "node_name":"127.0.0.1:36651_sj_a"},
"core_node3":{
  "core":"c8n_1x3_lf_shard1_replica2",
  "base_url":"http://127.0.0.1:43964/sj_a;,
  "node_name":"127.0.0.1:43964_sj_a",
  "state":"active",
  "leader":"true",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false"}
at 
__randomizedtesting.SeedInfo.seed([6F8A169181351238:E7DE294B2FC97FC0]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:170)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:57)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 

[jira] [Commented] (SOLR-10153) UnifiedSolrHighlighter support for CustomSeparatorBreakIterator (LUCENE-6485)

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890402#comment-15890402
 ] 

David Smiley commented on SOLR-10153:
-

It's okay; I think _all_ software developers struggle with naming :-).  Goes 
with the job.  Thanks again for the contribution.

Now that I think about naming again... hmmm, maybe a more useful/memorable Solr 
parameter for this might be {{hl.bs.separatorChar}} ? I understand why you 
chose the name you did since it's based on a class name but users don't 
know/char about that. The "hl.bs" namespaces this param as related to the other 
break iterator related params, which this is.  "Custom" is superfluous; we 
provide it directly via a parameter.  Hmmm; do we even need 
{{hl.bs.type==CUSTOM}} if the user sets {{hl.bs.separatorChar}}?  I guess it 
should be set so that there is consistency in mapping the type to an algorithm. 
 Though maybe use the value {{SEPARATOR}}? And maybe just name this 
{{hl.bs.separator}} to open the door for possibly using an arbitrary length 
string in the future?

Thus I propose {{hl.bs.type=SEPARATOR}} with e.g. {{hl.bs.separator=|}}

> UnifiedSolrHighlighter support for CustomSeparatorBreakIterator (LUCENE-6485)
> -
>
> Key: SOLR-10153
> URL: https://issues.apache.org/jira/browse/SOLR-10153
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: Amrit Sarkar
> Attachments: SOLR-10153.patch, SOLR-10153.patch, 
> SOLR_10153_UH_and_PH_hl_customSeparatorChar.patch
>
>
> Lucene 5.3 added a CustomSeparatorBreakIterator (see LUCENE-6485)
> UnifiedSolrHighlighter should support *CustomSeparatorBreakIterator* along 
> with existing ones, WholeBreakIterator etc.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6819) Deprecate index-time boosts?

[Adrien Grand (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand updated LUCENE-6819:
-
Attachment: LUCENE-6819-deprecation.patch

And the 6.x patch now also logs messages to warn users that this feature will 
go away.

> Deprecate index-time boosts?
> 
>
> Key: LUCENE-6819
> URL: https://issues.apache.org/jira/browse/LUCENE-6819
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-6819-deprecation.patch, 
> LUCENE-6819-deprecation.patch, LUCENE-6819.patch, LUCENE-6819.patch, 
> LUCENE-6819-wip.patch
>
>
> Follow-up of this comment: 
> https://issues.apache.org/jira/browse/LUCENE-6818?focusedCommentId=14934801=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14934801
> Index-time boosts are a very expert feature whose behaviour is tight to the 
> Similarity impl. Additionally users have often be confused by the poor 
> precision due to the fact that we encode values on a single byte. But now we 
> have doc values that allow you to encode any values the way you want with as 
> much precision as you need so maybe we should deprecate index-time boosts and 
> recommend to encode index-time scoring factors into doc values fields instead.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6819) Deprecate index-time boosts?

[Adrien Grand (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand updated LUCENE-6819:
-
Attachment: LUCENE-6819.patch

I updated the master patch to use the warning level for the first message, and 
them debug.

> Deprecate index-time boosts?
> 
>
> Key: LUCENE-6819
> URL: https://issues.apache.org/jira/browse/LUCENE-6819
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-6819-deprecation.patch, LUCENE-6819.patch, 
> LUCENE-6819.patch, LUCENE-6819-wip.patch
>
>
> Follow-up of this comment: 
> https://issues.apache.org/jira/browse/LUCENE-6818?focusedCommentId=14934801=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14934801
> Index-time boosts are a very expert feature whose behaviour is tight to the 
> Similarity impl. Additionally users have often be confused by the poor 
> precision due to the fact that we encode values on a single byte. But now we 
> have doc values that allow you to encode any values the way you want with as 
> much precision as you need so maybe we should deprecate index-time boosts and 
> recommend to encode index-time scoring factors into doc values fields instead.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7722) Remove BoostedQuery

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890358#comment-15890358
 ] 

Alan Woodward commented on LUCENE-7722:
---

bq. BoostedQuery vs. BoostQuery is confusing anyways!

Don't forget BoostingQuery!  Which I think we may be able to remove as well, by 
creating a DoubleValuesSource that wraps a Weight and returns scores as values. 
 There's a bit of an interface mismatch here, though, as DoubleValuesSource is 
always IndexReader-independant, so you need to make sure that the passed in 
Weight was generated against the same IndexReader as the FunctionScoreQuery is 
being run against.

> Remove BoostedQuery
> ---
>
> Key: LUCENE-7722
> URL: https://issues.apache.org/jira/browse/LUCENE-7722
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Adrien Grand
>Priority: Minor
>
> We already  have FunctionScoreQuery, which is more flexible than BoostedQuery 
> as it can combine scores in arbitrary ways and only requests scores on the 
> underlying scorer if they are needed. So let's remove BoostedQuery?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6819) Deprecate index-time boosts?

[Adrien Grand (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand updated LUCENE-6819:
-
Attachment: LUCENE-6819-deprecation.patch

Here is a proposed patch for 6.x.

> Deprecate index-time boosts?
> 
>
> Key: LUCENE-6819
> URL: https://issues.apache.org/jira/browse/LUCENE-6819
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-6819-deprecation.patch, LUCENE-6819.patch, 
> LUCENE-6819-wip.patch
>
>
> Follow-up of this comment: 
> https://issues.apache.org/jira/browse/LUCENE-6818?focusedCommentId=14934801=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14934801
> Index-time boosts are a very expert feature whose behaviour is tight to the 
> Similarity impl. Additionally users have often be confused by the poor 
> precision due to the fact that we encode values on a single byte. But now we 
> have doc values that allow you to encode any values the way you want with as 
> much precision as you need so maybe we should deprecate index-time boosts and 
> recommend to encode index-time scoring factors into doc values fields instead.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Stop using SHA-1 and MD5 hashes?

[Jan Høydahl]

Hi devs,

Working on LUCENE-5143 I’m revising the README.html files we place in the dist 
folders.
Then I started documenting how to validate checksum of the downloads in 
addition to GPG signature,
Looks like MD5 can still be used for integrity checks 
(https://en.wikipedia.org/wiki/MD5),
while the Ant guys claim otherwise in 
https://ant.apache.org/manual/Tasks/checksum.html
Will our .md5 and .sha1 files still provide security for the downloader after 
Google releases their 
recent findings or are they only useful to check that the download was complete 
and not partial?

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7722) Remove BoostedQuery

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890344#comment-15890344
 ] 

Uwe Schindler commented on LUCENE-7722:
---

BoostedQuery vs. BoostQuery is confusing anyways!

> Remove BoostedQuery
> ---
>
> Key: LUCENE-7722
> URL: https://issues.apache.org/jira/browse/LUCENE-7722
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Adrien Grand
>Priority: Minor
>
> We already  have FunctionScoreQuery, which is more flexible than BoostedQuery 
> as it can combine scores in arbitrary ways and only requests scores on the 
> underlying scorer if they are needed. So let's remove BoostedQuery?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7722) Remove BoostedQuery

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890339#comment-15890339
 ] 

Alan Woodward commented on LUCENE-7722:
---

+1

I think this also applies to FunctionQuery and CustomScoreQuery?

> Remove BoostedQuery
> ---
>
> Key: LUCENE-7722
> URL: https://issues.apache.org/jira/browse/LUCENE-7722
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Adrien Grand
>Priority: Minor
>
> We already  have FunctionScoreQuery, which is more flexible than BoostedQuery 
> as it can combine scores in arbitrary ways and only requests scores on the 
> underlying scorer if they are needed. So let's remove BoostedQuery?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6819) Deprecate index-time boosts?

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890337#comment-15890337
 ] 

Uwe Schindler commented on LUCENE-6819:
---

"correct" way would be to use an AtomicBoolean, but a conventional, static 
boolean next to the LOGGER declaration should be fine, too. On concurrent use, 
you may get mutliple warnings for a short time until cache flush, but who cares?

> Deprecate index-time boosts?
> 
>
> Key: LUCENE-6819
> URL: https://issues.apache.org/jira/browse/LUCENE-6819
> Project: Lucene - Core
>  Issue Type: Task
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-6819.patch, LUCENE-6819-wip.patch
>
>
> Follow-up of this comment: 
> https://issues.apache.org/jira/browse/LUCENE-6818?focusedCommentId=14934801=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14934801
> Index-time boosts are a very expert feature whose behaviour is tight to the 
> Similarity impl. Additionally users have often be confused by the poor 
> precision due to the fact that we encode values on a single byte. But now we 
> have doc values that allow you to encode any values the way you want with as 
> much precision as you need so maybe we should deprecate index-time boosts and 
> recommend to encode index-time scoring factors into doc values fields instead.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9401) TestPKIAuthenticationPlugin NPE

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890319#comment-15890319
 ] 

Steve Rowe commented on SOLR-9401:
--

+1, LGTM, 300 beasting iterations passed on my box using the latest patch.

> TestPKIAuthenticationPlugin NPE
> ---
>
> Key: SOLR-9401
> URL: https://issues.apache.org/jira/browse/SOLR-9401
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>Assignee: Noble Paul
> Attachments: SOLR-9401.patch, SOLR-9401.patch, SOLR-9401.patch
>
>
> Failure from my Jenkins, doesn't reproduce for me (this is 
> {{tests-failures.txt}}):
> {noformat}
>   2> Creating dataDir: 
> /var/lib/jenkins/jobs/Lucene-Solr-tests-master/workspace/solr/build/solr-core/test/J7/temp/solr.security.TestPKIAuthenticationPlugi
> n_7AC33B2240CB767D-001/init-core-data-001
>   2> 14521 INFO  
> (SUITE-TestPKIAuthenticationPlugin-seed#[7AC33B2240CB767D]-worker) [] 
> o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (fal
> se) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, 
> clientAuth=NaN)
>   2> 14540 INFO  
> (TEST-TestPKIAuthenticationPlugin.test-seed#[7AC33B2240CB767D]) [] 
> o.a.s.SolrTestCaseJ4 ###Starting test
>   2> 15553 ERROR 
> (TEST-TestPKIAuthenticationPlugin.test-seed#[7AC33B2240CB767D]) [] 
> o.a.s.s.PKIAuthenticationPlugin No SolrAuth header present
>   2> 15843 ERROR 
> (TEST-TestPKIAuthenticationPlugin.test-seed#[7AC33B2240CB767D]) [] 
> o.a.s.s.PKIAuthenticationPlugin Invalid key request timestamp: 9 ,
>  received timestamp: 1470760833176 , TTL: 5000
>   2> 15843 INFO  
> (TEST-TestPKIAuthenticationPlugin.test-seed#[7AC33B2240CB767D]) [] 
> o.a.s.SolrTestCaseJ4 ###Ending test
>   2> NOTE: download the large Jenkins line-docs file by running 'ant 
> get-jenkins-line-docs' in the lucene directory.
>   2> NOTE: reproduce with: ant test  -Dtestcase=TestPKIAuthenticationPlugin 
> -Dtests.method=test -Dtests.seed=7AC33B2240CB767D -Dtests.slow=true -Dtests.li
> nedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt 
> -Dtests.locale=cs -Dtests.timezone=Europe/Tirane -Dtests.asserts=true 
> -Dtests.file.encoding=U
> TF-8
> [12:40:32.094] ERROR   1.35s J7  | TestPKIAuthenticationPlugin.test <<<
>> Throwable #1: java.lang.NullPointerException
>>at 
> __randomizedtesting.SeedInfo.seed([7AC33B2240CB767D:F29704F8EE371B85]:0)
>>at 
> org.apache.solr.security.TestPKIAuthenticationPlugin.test(TestPKIAuthenticationPlugin.java:144)
> [...]
>   2> 15867 INFO  
> (SUITE-TestPKIAuthenticationPlugin-seed#[7AC33B2240CB767D]-worker) [] 
> o.a.s.SolrTestCaseJ4 ###deleteCore
>   2> NOTE: leaving temporary files on disk at: 
> /var/lib/jenkins/jobs/Lucene-Solr-tests-master/workspace/solr/build/solr-core/test/J7/temp/solr.security.TestPKIAuthenticationPlugin_7AC33B2240CB767D-001
>   2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, 
> maxPointsInLeafNode=752, maxMBSortInHeap=5.390190554185364, 
> sim=RandomSimilarity(queryNorm=true): {}, locale=cs, timezone=Europe/Tirane
>   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 1.8.0_77 
> (64-bit)/cpus=16,threads=1,free=255922760,total=336592896
>   2> NOTE: All tests run in this JVM: [TestIndexingPerformance, 
> TestPKIAuthenticationPlugin]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8593) Integrate Apache Calcite into the SQLHandler

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890314#comment-15890314
 ] 

Kevin Risden commented on SOLR-8593:


[~joel.bernstein] & [~julianhyde] - I tried to set user.locale=tr for the 
Calcite and Avatica proper tests and couldn't get it to fail. I haven't been 
able to isolate the issue. The few things I tried from the Calcite repo:
{code}
MAVEN_OPTS="$MAVEN_OPTS -Duser.locale=tr" mvn clean test
Add -Duser.locale=tr to the surefire argLinein pom.xml
{code}

Neither of the above made tests in either Calcite or Avatica fail. Not sure 
where to go from here.

> Integrate Apache Calcite into the SQLHandler
> 
>
> Key: SOLR-8593
> URL: https://issues.apache.org/jira/browse/SOLR-8593
> Project: Solr
>  Issue Type: Improvement
>  Components: Parallel SQL
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: 6.5, master (7.0)
>
> Attachments: SOLR-8593.patch, SOLR-8593.patch, SOLR-8593.patch
>
>
>The Presto SQL Parser was perfect for phase one of the SQLHandler. It was 
> nicely split off from the larger Presto project and it did everything that 
> was needed for the initial implementation.
> Phase two of the SQL work though will require an optimizer. Here is where 
> Apache Calcite comes into play. It has a battle tested cost based optimizer 
> and has been integrated into Apache Drill and Hive.
> This work can begin in trunk following the 6.0 release. The final query plans 
> will continue to be translated to Streaming API objects (TupleStreams), so 
> continued work on the JDBC driver should plug in nicely with the Calcite work.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7722) Remove BoostedQuery

[Adrien Grand (JIRA)]

Adrien Grand created LUCENE-7722:


 Summary: Remove BoostedQuery
 Key: LUCENE-7722
 URL: https://issues.apache.org/jira/browse/LUCENE-7722
 Project: Lucene - Core
  Issue Type: Task
Reporter: Adrien Grand
Priority: Minor


We already  have FunctionScoreQuery, which is more flexible than BoostedQuery 
as it can combine scores in arbitrary ways and only requests scores on the 
underlying scorer if they are needed. So let's remove BoostedQuery?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org