[jira] [Commented] (SOLR-8868) SolrCloud: if zookeeper loses and then regains a quorum, Solr nodes and SolrJ Client do not recover and need to be restarted

[zarnigar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15766366#comment-15766366
 ] 

zarnigar commented on SOLR-8868:


any solution found on this issue?


> SolrCloud: if zookeeper loses and then regains a quorum, Solr nodes and SolrJ 
> Client do not recover and need to be restarted
> 
>
> Key: SOLR-8868
> URL: https://issues.apache.org/jira/browse/SOLR-8868
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud, SolrJ
>Affects Versions: 5.3.1
>Reporter: Frank J Kelly
>
> Tried mailing list on 3/15 and 3/16 to no avail. Hopefully I gave enough 
> details.
> 
> Just wondering if my observation of SolrCloud behavior after ZooKeeper loses 
> a quorum is normal or to-be-expected
> Version of Solr: 5.3.1
> Version of ZooKeeper: 3.4.7
> Using SolrCloud with external ZooKeeper
> Deployed on AWS
> Our Solr cluster has 3 nodes (m3.large)
> Our Zookeeper ensemble consists of three nodes (t2.small) with the same 
> config using DNS names e.g.
> {noformat}
> $ more ../conf/zoo.cfg
> tickTime=2000
> dataDir=/var/zookeeper
> dataLogDir=/var/log/zookeeper
> clientPort=2181
> initLimit=10
> syncLimit=5
> standaloneEnabled=false
> server.1=zookeeper1.qa.eu-west-1.mysearch.com:2888:3888
> server.2=zookeeper2.qa.eu-west-1.mysearch.com:2888:3888
> server.3=zookeeper3.qa.eu-west-1.mysearch.com:2888:3888
> {noformat}
> If we terminate one of the zookeeper nodes we get a ZK election (and I think) 
> a quorum is maintained.
> Operation continues OK and we detect the terminated instance and relaunch a 
> new ZK node which comes up fine
> If we terminate two of the ZK nodes we lose a quorum and then we observe the 
> following
> 1.1) Admin UI shows an error that it is unable to contact ZooKeeper “Could 
> not connect to ZooKeeper"
> 1.2) SolrJ returns the following
> {noformat}
> org.apache.solr.common.SolrException: Could not load collection from 
> ZK:qa_eu-west-1_public_index
> at 
> org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:850)
> at org.apache.solr.common.cloud.ZkStateReader$7.get(ZkStateReader.java:515)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1205)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:837)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:805)
> at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)
> at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:107)
> at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:72)
> at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:86)
> at 
> com.here.scbe.search.solr.SolrFacadeImpl.addToSearchIndex(SolrFacadeImpl.java:112)
> Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: 
> KeeperErrorCode = ConnectionLoss for 
> /collections/qa_eu-west-1_public_index/state.json
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:345)
> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:342)
> at 
> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:61)
> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:342)
> at 
> org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:841)
> ... 24 more
> {noformat}
> This makes sense based on our understanding.
> When our AutoScale groups launch two new ZooKeeper nodes, initialize them, 
> fix the DNS etc. we regain a quorum but at this point
> 2.1) Admin UI shows the shards as “GONE” (all greyed out)
> 2.2) SolrJ returns the same error even though the ZooKeeper DNS names are now 
> bound to new IP addresses
> So at this point I restart the Solr nodes. At this point then
> 3.1) Admin UI shows the collections as OK (all shards are green) – yeah the 
> nodes are back!
> 3.2) SolrJ Client still shows the same error – namely
> {noformat}
> org.apache.solr.common.SolrException: Could not load collection from 
> ZK:qa_eu-west-1_here_account
> at 
> org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:850)
> at org.apache.solr.common.cloud.ZkStateReader$7.get(ZkStateReader.java:515)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1205)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:837)
> at 
> 

[jira] [Commented] (LUCENE-7590) Add DocValues statistics helpers

[Shai Erera (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15766325#comment-15766325
 ] 

Shai Erera commented on LUCENE-7590:


[~shia] where do you see that? I checked master and there's no {{description}} 
in the file at all. Here's the code:

{code}
public LongDocValuesStats(String field) {
  super(field, Long.MAX_VALUE, Long.MIN_VALUE);
}
{code}

> Add DocValues statistics helpers
> 
>
> Key: LUCENE-7590
> URL: https://issues.apache.org/jira/browse/LUCENE-7590
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: modules/misc
>Reporter: Shai Erera
>Assignee: Shai Erera
> Fix For: master (7.0), 6.4
>
> Attachments: LUCENE-7590-2.patch, LUCENE-7590-sorted-numeric.patch, 
> LUCENE-7590-sorted-set.patch, LUCENE-7590.patch, LUCENE-7590.patch, 
> LUCENE-7590.patch, LUCENE-7590.patch, LUCENE-7590.patch, LUCENE-7590.patch, 
> LUCENE-7590.patch
>
>
> I think it can be useful to have DocValues statistics helpers, that can allow 
> users to query for the min/max/avg etc. stats of a DV field. In this issue 
> I'd like to cover numeric DV, but there's no reason not to add it to other DV 
> types too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-6.x-Linux (32bit/jdk-9-ea+147) - Build # 2473 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/2473/
Java: 32bit/jdk-9-ea+147 -client -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.cloud.TestLeaderElectionWithEmptyReplica.test

Error Message:


Stack Trace:
java.util.concurrent.TimeoutException
at 
__randomizedtesting.SeedInfo.seed([8C5BBA6CC906A148:40F85B667FACCB0]:0)
at 
org.apache.solr.common.cloud.ZkStateReader.waitForState(ZkStateReader.java:1251)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.waitForState(CloudSolrClient.java:697)
at 
org.apache.solr.cloud.TestLeaderElectionWithEmptyReplica.test(TestLeaderElectionWithEmptyReplica.java:97)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:538)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 12304 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestLeaderElectionWithEmptyReplica
   [junit4]   2> Creating dataDir: 

[jira] [Commented] (SOLR-9878) Code smell in if statement

[Mikhail Khludnev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15766287#comment-15766287
 ] 

Mikhail Khludnev commented on SOLR-9878:


when a fieldType doesn't have RWFF it's cached as null value and isn't checked 
anymore. I agree it's not obvious and probably over-engineered, but I decided 
to fix as is. You can check the test which demonstrate this invariant. 
Thanks for reporting.

> Code smell in if statement
> --
>
> Key: SOLR-9878
> URL: https://issues.apache.org/jira/browse/SOLR-9878
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Jaechang Nam
>Priority: Trivial
> Attachments: SOLR-9878.patch
>
>
> In recent code snapshot (Github mirror's commit id: 
> c8542b2bd0470af9f8d64bb8133f31828b342604 as today), there is an illogical 
> condition that can be a code smell or a potential bug:
> {code}
> ReversedWildcardFilterFactory fac = leadingWildcards.get(fieldType);
> if (fac != null || leadingWildcards.containsKey(fac)) {
>   return fac;
> }
> {code}
> In SOLR-3492, it said there was a fix in SOLR-4093. However, the fix still 
> has an issue as above: containsKey will always have null in this if 
> statement. The second condition could be unnecessary. Does leadingWildcards 
> allow a null object as a key? If so, it will return null that might cause NPE 
> in some other locations.
> Patch could be just like in SOLR-3492?:
> {code}
> if (fac != null)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7590) Add DocValues statistics helpers

[Zhang Yuan (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15766041#comment-15766041
 ] 

Zhang Yuan commented on LUCENE-7590:


[~shaie] I'm a little confused about the description defined here.

{code}
 public LongDocValuesStats(String description) {
  super(description, Long.MAX_VALUE, Long.MIN_VALUE);
 }
{code}

it was in turn passed to NumericDocValueStats as the name of DV field.
Why not use the {{field}} as the name of parameter in LongDocValuesStats?


> Add DocValues statistics helpers
> 
>
> Key: LUCENE-7590
> URL: https://issues.apache.org/jira/browse/LUCENE-7590
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: modules/misc
>Reporter: Shai Erera
>Assignee: Shai Erera
> Fix For: master (7.0), 6.4
>
> Attachments: LUCENE-7590-2.patch, LUCENE-7590-sorted-numeric.patch, 
> LUCENE-7590-sorted-set.patch, LUCENE-7590.patch, LUCENE-7590.patch, 
> LUCENE-7590.patch, LUCENE-7590.patch, LUCENE-7590.patch, LUCENE-7590.patch, 
> LUCENE-7590.patch
>
>
> I think it can be useful to have DocValues statistics helpers, that can allow 
> users to query for the min/max/avg etc. stats of a DV field. In this issue 
> I'd like to cover numeric DV, but there's no reason not to add it to other DV 
> types too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+147) - Build # 18578 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18578/
Java: 32bit/jdk-9-ea+147 -client -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.handler.admin.MetricsHandlerTest.test

Error Message:
Unable to make member of class com.sun.management.internal.OperatingSystemImpl 
accessible:  module jdk.management does not export com.sun.management.internal 
to unnamed module @189e714

Stack Trace:
java.lang.reflect.InaccessibleObjectException: Unable to make member of class 
com.sun.management.internal.OperatingSystemImpl accessible:  module 
jdk.management does not export com.sun.management.internal to unnamed module 
@189e714
at 
__randomizedtesting.SeedInfo.seed([685BDEACF8AA2CAF:E00FE17656564157]:0)
at 
java.base/jdk.internal.reflect.Reflection.throwInaccessibleObjectException(Reflection.java:424)
at 
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:174)
at 
java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:192)
at java.base/java.lang.reflect.Method.setAccessible(Method.java:186)
at 
com.codahale.metrics.jvm.FileDescriptorRatioGauge.invoke(FileDescriptorRatioGauge.java:48)
at 
com.codahale.metrics.jvm.FileDescriptorRatioGauge.getRatio(FileDescriptorRatioGauge.java:35)
at com.codahale.metrics.RatioGauge.getValue(RatioGauge.java:64)
at com.codahale.metrics.RatioGauge.getValue(RatioGauge.java:11)
at 
org.apache.solr.util.stats.MetricUtils.gaugeToNamedList(MetricUtils.java:135)
at 
org.apache.solr.util.stats.MetricUtils.lambda$toNamedList$2(MetricUtils.java:87)
at 
java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at 
java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
at 
java.base/java.util.TreeMap$KeySpliterator.forEachRemaining(TreeMap.java:2739)
at 
java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at 
java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at 
java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at 
java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at 
java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at 
java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:430)
at 
org.apache.solr.util.stats.MetricUtils.toNamedList(MetricUtils.java:80)
at 
org.apache.solr.handler.admin.MetricsHandler.handleRequestBody(MetricsHandler.java:93)
at 
org.apache.solr.handler.admin.MetricsHandlerTest.test(MetricsHandlerTest.java:41)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:538)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
   

[jira] [Comment Edited] (SOLR-7027) ExtractingRequestHandler indiscriminantly dumps all source HTML attributes into the catch-all field when captureAttr=false, but it should be more selective, somethi

[JIRA]

[ 
https://issues.apache.org/jira/browse/SOLR-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765748#comment-15765748
 ] 

Ole Jørgen Brønner edited comment on SOLR-7027 at 12/21/16 1:08 AM:


A semi-related issue that caught me off guard is that it doesn't seem to be 
possible to capture both attribute values ({{captureAttr}}) and content 
({{capture=h1}}) and be able to distinguish between the content and attributes?

Without {{captureAttr}} the content captured in the {{h1}} field will be very 
low quality since h1 tags commonly contain eg. {{class}} attributes, but with 
{{captureAttr}} the attribute values will be stored in the same field. (it 
doesn't seem possible to map the attributes and the content to different 
fields). They will be stored as different values in the multivalued field, but 
I don't think that helps much.

The documentation also says that when capturing elements ({{capture=h1}}) the 
content should also be present in the catch-all content field, but that doesn't 
align with my observations.


was (Author: olejorgenb):
A semi-related issue that caught me off guard is that it doesn't seem to be 
possible to capture both attribute values ({{captureAttr}}) and content 
({{capture=h1}}) and be able to distinguish between the content and attributes?

Without {{captureAttr}} the content captured in the {{h1}} field will be very 
low quality since h1 tags commonly contain eg. {{class}} attributes, but with 
{{captureAttr}} the attribute values will be stored in the same field. (it 
doesn't seem possible to map the attributes and the content to different 
fields). They will be stored as different values in the multivalued field, but 
I don't think that helps much.

The documentation says that when capturing elements ({{capture=h1}}) the 
content should also be present in the catch-all content field, but that doesn't 
align with my observations.

> ExtractingRequestHandler indiscriminantly dumps all source HTML attributes 
> into the catch-all field when captureAttr=false, but it should be more 
> selective, something like only href, title, alt, etc. attributes
> --
>
> Key: SOLR-7027
> URL: https://issues.apache.org/jira/browse/SOLR-7027
> Project: Solr
>  Issue Type: Improvement
>  Components: contrib - Solr Cell (Tika extraction)
>Affects Versions: 5.0
>Reporter: Steve Rowe
>Priority: Minor
> Fix For: 5.2, 6.0
>
>
> On line 283 in {{SolrContentHandler}}, the catch-all field gets *all* source 
> HTML attribute values dumped into it:
> {code:java}
> 270:  @Override
> 271:  public void startElement(String uri, String localName, String qName, 
> Attributes attributes) throws SAXException {
> 272:StringBuilder theBldr = fieldBuilders.get(localName);
> 273:if (theBldr != null) {
> 274:  //we need to switch the currentBuilder
> 275:  bldrStack.add(theBldr);
> 276:}
> 277:if (captureAttribs == true) {
> 278:  for (int i = 0; i < attributes.getLength(); i++) {
> 279:addField(localName, attributes.getValue(i), null);
> 280:  }
> 281:} else {
> 282:  for (int i = 0; i < attributes.getLength(); i++) {
> 283:bldrStack.getLast().append(' ').append(attributes.getValue(i));
> 284:  }
> 285:}
> 286:bldrStack.getLast().append(' ');
> 287:  }
> {code}
> But this will contains lots of unwanted cruft: {{class}} and {{style}} tags, 
> etc.
> It would be much better if only attribute values containing addresses or 
> tooltip text, etc. were dumped into the catch-all field.  Here are a couple 
> of places where this kind of attribute are described:
> http://jericho.htmlparser.net/docs/javadoc/net/htmlparser/jericho/TextExtractor.html#includeAttribute(net.htmlparser.jericho.StartTag,%20net.htmlparser.jericho.Attribute)
> From Tika's {{HtmlHandler}} class:
> {code:java}
> // List of attributes that need to be resolved.
> private static final Set URI_ATTRIBUTES =
> new HashSet(Arrays.asList("src", "href", "longdesc", "cite"));
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7027) ExtractingRequestHandler indiscriminantly dumps all source HTML attributes into the catch-all field when captureAttr=false, but it should be more selective, something li

[JIRA]

[ 
https://issues.apache.org/jira/browse/SOLR-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765748#comment-15765748
 ] 

Ole Jørgen Brønner commented on SOLR-7027:
--

A semi-related issue that caught me off guard is that it doesn't seem to be 
possible to capture both attribute values ({{captureAttr}}) and content 
({{capture=h1}}) and be able to distinguish between the content and attributes?

Without {{captureAttr}} the content captured in the {{h1}} field will be very 
low quality since h1 tags commonly contain eg. {{class}} attributes, but with 
{{captureAttr}} the attribute values will be stored in the same field. (it 
doesn't seem possible to map the attributes and the content to different 
fields). They will be stored as different values in the multivalued field, but 
I don't think that helps much.

The documentation says that when capturing elements ({{capture=h1}}) the 
content should also be present in the catch-all content field, but that doesn't 
align with my observations.

> ExtractingRequestHandler indiscriminantly dumps all source HTML attributes 
> into the catch-all field when captureAttr=false, but it should be more 
> selective, something like only href, title, alt, etc. attributes
> --
>
> Key: SOLR-7027
> URL: https://issues.apache.org/jira/browse/SOLR-7027
> Project: Solr
>  Issue Type: Improvement
>  Components: contrib - Solr Cell (Tika extraction)
>Affects Versions: 5.0
>Reporter: Steve Rowe
>Priority: Minor
> Fix For: 5.2, 6.0
>
>
> On line 283 in {{SolrContentHandler}}, the catch-all field gets *all* source 
> HTML attribute values dumped into it:
> {code:java}
> 270:  @Override
> 271:  public void startElement(String uri, String localName, String qName, 
> Attributes attributes) throws SAXException {
> 272:StringBuilder theBldr = fieldBuilders.get(localName);
> 273:if (theBldr != null) {
> 274:  //we need to switch the currentBuilder
> 275:  bldrStack.add(theBldr);
> 276:}
> 277:if (captureAttribs == true) {
> 278:  for (int i = 0; i < attributes.getLength(); i++) {
> 279:addField(localName, attributes.getValue(i), null);
> 280:  }
> 281:} else {
> 282:  for (int i = 0; i < attributes.getLength(); i++) {
> 283:bldrStack.getLast().append(' ').append(attributes.getValue(i));
> 284:  }
> 285:}
> 286:bldrStack.getLast().append(' ');
> 287:  }
> {code}
> But this will contains lots of unwanted cruft: {{class}} and {{style}} tags, 
> etc.
> It would be much better if only attribute values containing addresses or 
> tooltip text, etc. were dumped into the catch-all field.  Here are a couple 
> of places where this kind of attribute are described:
> http://jericho.htmlparser.net/docs/javadoc/net/htmlparser/jericho/TextExtractor.html#includeAttribute(net.htmlparser.jericho.StartTag,%20net.htmlparser.jericho.Attribute)
> From Tika's {{HtmlHandler}} class:
> {code:java}
> // List of attributes that need to be resolved.
> private static final Set URI_ATTRIBUTES =
> new HashSet(Arrays.asList("src", "href", "longdesc", "cite"));
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9178) ExtractingRequestHandler doesn't strip HTML and adds metadata to content body

[JIRA]

[ 
https://issues.apache.org/jira/browse/SOLR-9178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765691#comment-15765691
 ] 

Ole Jørgen Brønner commented on SOLR-9178:
--

I think this (unfortunately) is expected behavior. If you look closer, the 
actually html tags have been removed, but the attribute values have been 
retained. 

You can get rid of the attribute values by adding {{captureAttr=true}}, but if 
you also want to capture some elements in separate fields ({{capture=p}}) 
you're out of luck (ie. you can't separate the captured attributes from the 
captured tag content)

Disclaimer: I'm no solr expert, but have recently spent a decent amount of time 
trying to bend cell/tika to my liking (unsuccessfully)

Related issue: SOLR-7027

> ExtractingRequestHandler doesn't strip HTML and adds metadata to content body
> -
>
> Key: SOLR-9178
> URL: https://issues.apache.org/jira/browse/SOLR-9178
> Project: Solr
>  Issue Type: Bug
>  Components: update
>Affects Versions: 5.0, 6.0.1
> Environment: java version "1.8.0_91" 64 bit
> Linux Mint 17, 64 bit
>Reporter: Simon Blandford
>
> Starting environment:
> solr-6.0.1.tgz is downloaded and extracted. We are in the solr-6.0.1 
> directory.
> The file, test.html, is downloaded from 
> https://wiki.apache.org/solr/UsingMailingLists.
> Affected versions: 4.10.3 is the last working version. 4.10.4 has some HTML 
> comments and Javascript breaking through. Versions >5.0 have full symptoms 
> described.
> Steps to reproduce:
> 1) bin/solr start
> 2) bin/solr create -c mycore
> 3) curl 
> "http://localhost:8983/solr/mycore/update/extract?literal.id=doc1=attr_=attr_content=true;
>  -F "content/tutorial=@test.html"
> 4) curl http://localhost:8983/solr/mycore/select?q=information
> Expected result: HTML->Text version of document indexed in  content 
> body.
> Actual result: Full HTML, but with anglebrackets removed, being indexed along 
> with other unwanted metadata in the content body including fragments of CSS 
> and Javascript that were in the source document. 
> Head of response body below...
> 
> 
> 0 name="QTime">0 name="q">information start="0">doc1 name="attr_stream_size">20440 name="attr_x_parsed_by">org.apache.tika.parser.DefaultParserorg.apache.tika.parser.html.HtmlParser  name="attr_stream_content_type">text/html name="attr_stream_name">test.html name="attr_stream_source_info">content/tutorial name="attr_dc_title">UsingMailingLists - Solr Wiki name="attr_content_encoding">UTF-8 name="attr_robots">index,nofollow name="attr_title">UsingMailingLists - Solr Wiki name="attr_content_type">text/html; charset=utf-8 name="attr_content"> 
>  
>  stylesheet text/css utf-8 all /wiki/modernized/css/common.css   stylesheet 
> text/css utf-8 screen /wiki/modernized/css/screen.css   stylesheet text/css 
> utf-8 print /wiki/modernized/css/print.css   stylesheet text/css utf-8 
> projection /wiki/modernized/css/projection.css   alternate Solr Wiki: 
> UsingMailingLists 
> /solr/UsingMailingLists?diffs=1show_att=1action=rss_rcunique=0page=UsingMailingListsddiffs=1
>  application/rss+xml   Start /solr/FrontPage   Alternate Wiki Markup 
> /solr/UsingMailingLists?action=raw   Alternate print Print View 
> /solr/UsingMailingLists?action=print   Search /solr/FindPage   Index 
> /solr/TitleIndex   Glossary /solr/WordIndex   Help /solr/HelpOnFormatting   
> stream_size 20440  
>  X-Parsed-By org.apache.tika.parser.DefaultParser  
>  X-Parsed-By org.apache.tika.parser.html.HtmlParser  
>  stream_content_type text/html  
>  stream_name test.html  
>  stream_source_info content/tutorial  
>  dc:title UsingMailingLists - Solr Wiki  
>  Content-Encoding UTF-8  
>  robots index,nofollow  
>  Content-Type text/html; charset=utf-8  
>  UsingMailingLists - Solr Wiki 
>  
>  
>  header 
>  application/x-www-form-urlencoded get searchform /solr/UsingMailingLists 
>  
>  hidden action fullsearch  
>  hidden context 180  
>  searchinput Search: 
>  text searchinput value  20 searchFocus(this) searchBlur(this) 
> searchChange(this) searchChange(this) Search  
>  submit titlesearch titlesearch Titles Search Titles  
>  submit fullsearch fullsearch Text Search Full Text  
>  
>  
>  text/javascript 
> !--// Initialize search form
> var f = document.getElementById('searchform');
> f.getElementsByTagName('label')[0].style.display = 'none';
> var e = document.getElementById('searchinput');
> searchChange(e);
> searchBlur(e);
> //--
>  
>  logo  rect /solr/FrontPage Solr Wiki  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (32bit/jdk1.8.0_112) - Build # 6299 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6299/
Java: 32bit/jdk1.8.0_112 -server -XX:+UseSerialGC

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.lucene.store.TestSimpleFSDirectory

Error Message:
Could not remove the following files (in the order of attempts):
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J1\temp\lucene.store.TestSimpleFSDirectory_B719BBD5FD1C59DA-001\testThreadSafety-001:
 java.nio.file.AccessDeniedException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J1\temp\lucene.store.TestSimpleFSDirectory_B719BBD5FD1C59DA-001\testThreadSafety-001

C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J1\temp\lucene.store.TestSimpleFSDirectory_B719BBD5FD1C59DA-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J1\temp\lucene.store.TestSimpleFSDirectory_B719BBD5FD1C59DA-001
 

Stack Trace:
java.io.IOException: Could not remove the following files (in the order of 
attempts):
   
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J1\temp\lucene.store.TestSimpleFSDirectory_B719BBD5FD1C59DA-001\testThreadSafety-001:
 java.nio.file.AccessDeniedException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J1\temp\lucene.store.TestSimpleFSDirectory_B719BBD5FD1C59DA-001\testThreadSafety-001
   
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J1\temp\lucene.store.TestSimpleFSDirectory_B719BBD5FD1C59DA-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J1\temp\lucene.store.TestSimpleFSDirectory_B719BBD5FD1C59DA-001

at __randomizedtesting.SeedInfo.seed([B719BBD5FD1C59DA]:0)
at org.apache.lucene.util.IOUtils.rm(IOUtils.java:323)
at 
org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:216)
at 
com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)


FAILED:  
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail

Error Message:
expected:<200> but was:<404>

Stack Trace:
java.lang.AssertionError: expected:<200> but was:<404>
at 
__randomizedtesting.SeedInfo.seed([C45BDE1A6C7F7AEB:ACE4EB30BCE56807]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.cancelDelegationToken(TestSolrCloudWithDelegationTokens.java:140)
at 
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail(TestSolrCloudWithDelegationTokens.java:294)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 

[jira] [Commented] (SOLR-9839) Use 'useFactory' in tests instead of setting manually

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765504#comment-15765504
 ] 

Erick Erickson commented on SOLR-9839:
--

Mike:

This patch doesn't seem necessary at this point in light of the restore rule.  
Although it does bring up the interesting point whether we should _remove_ the 
separate invocation of this rule in tests that extend SolrTestCaseJ4...

> Use 'useFactory' in tests instead of setting manually
> -
>
> Key: SOLR-9839
> URL: https://issues.apache.org/jira/browse/SOLR-9839
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Reporter: Mike Drob
>Priority: Minor
> Attachments: SOLR-9839.patch
>
>
> We have several tests that will explicitly set a directory factory via 
> SysProp, some of which forget to unset it.
> We should use {{useFactory}} so that we can benefit from the call to 
> {{resetFactory}} in {{SolrTestCaseJ4.teardownTestCases}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9883) example solr config files can lead to invalid tlog replays when using add-unknown-fields-to-schema updat chain

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765491#comment-15765491
 ] 

Erick Erickson commented on SOLR-9883:
--

There's quite a bit of discussion at SOLR-8030 that's relevant.

I don't quite know whether the simple expedient of putting the URPs before the 
DistribUpdateProcessorFactory is sufficient (or safe).

> example solr config files can lead to invalid tlog replays when using 
> add-unknown-fields-to-schema updat chain
> --
>
> Key: SOLR-9883
> URL: https://issues.apache.org/jira/browse/SOLR-9883
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.3, trunk
>Reporter: Erick Erickson
>
> The current basic_configs and data_driven_schema_configs try to create 
> unknown fields. The problem is that the date processing 
> "ParseDateFieldUpdateProcessorFactory" is not invoked if the doc is replayed 
> from the tlog. Whether there are other places this is a problem I don't know, 
> this is a concrete example that fails in the field.
> So say I have a pattern for dates that omits the trialing 'Z', as:
> -MM-dd'T'HH:mm:ss.SSS
> This work fine when the doc is initially indexed. Now say the doc must be 
> replayed from the tlog. The doc errors out with "unknown date format" since 
> (apparently) this doesn't go through the same update chain, perhaps due to 
> the sample configs defining ParseDateFieldUpdateProcessorFactory after  
> DistributedUpdateProcessorFactory?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9878) Code smell in if statement

[Jaechang Nam (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765487#comment-15765487
 ] 

Jaechang Nam commented on SOLR-9878:


I'm wondering why the second condition, "|| 
leadingWildcards.containsKey(fieldType)" still required.
The second condition can be reached only when fac is null. Can 
leadingWildcards.get(fieldType) have null when 
leadingWildcards.containsKey(fieldType) is true? If yes, this method will 
return null in this case. (Is this intended?) Since I don't have domain 
knowledge about this project, I just wanted to confirm this. If no, the second 
condition might not be necessary since when fac is null,  
leadingWildcards.containsKey(fieldType) is always false. It's quite confusing.

> Code smell in if statement
> --
>
> Key: SOLR-9878
> URL: https://issues.apache.org/jira/browse/SOLR-9878
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Jaechang Nam
>Priority: Trivial
> Attachments: SOLR-9878.patch
>
>
> In recent code snapshot (Github mirror's commit id: 
> c8542b2bd0470af9f8d64bb8133f31828b342604 as today), there is an illogical 
> condition that can be a code smell or a potential bug:
> {code}
> ReversedWildcardFilterFactory fac = leadingWildcards.get(fieldType);
> if (fac != null || leadingWildcards.containsKey(fac)) {
>   return fac;
> }
> {code}
> In SOLR-3492, it said there was a fix in SOLR-4093. However, the fix still 
> has an issue as above: containsKey will always have null in this if 
> statement. The second condition could be unnecessary. Does leadingWildcards 
> allow a null object as a key? If so, it will return null that might cause NPE 
> in some other locations.
> Patch could be just like in SOLR-3492?:
> {code}
> if (fac != null)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9883) example solr config files can lead to invalid tlog replays when using add-unknown-fields-to-schema updat chain

[Erick Erickson (JIRA)]

Erick Erickson created SOLR-9883:


 Summary: example solr config files can lead to invalid tlog 
replays when using add-unknown-fields-to-schema updat chain
 Key: SOLR-9883
 URL: https://issues.apache.org/jira/browse/SOLR-9883
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 6.3, trunk
Reporter: Erick Erickson


The current basic_configs and data_driven_schema_configs try to create unknown 
fields. The problem is that the date processing 
"ParseDateFieldUpdateProcessorFactory" is not invoked if the doc is replayed 
from the tlog. Whether there are other places this is a problem I don't know, 
this is a concrete example that fails in the field.

So say I have a pattern for dates that omits the trialing 'Z', as:
-MM-dd'T'HH:mm:ss.SSS

This work fine when the doc is initially indexed. Now say the doc must be 
replayed from the tlog. The doc errors out with "unknown date format" since 
(apparently) this doesn't go through the same update chain, perhaps due to the 
sample configs defining ParseDateFieldUpdateProcessorFactory after  
DistributedUpdateProcessorFactory?





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-7588) A parallel DrillSideways implementation

[Emmanuel Keller (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765461#comment-15765461
 ] 

Emmanuel Keller edited comment on LUCENE-7588 at 12/20/16 10:51 PM:


New patch:
1. In the DrillSideways.search method, if executor is non-null, we invoke the 
concurrent version.
2. The unit test tests effectively the new concurrent methods.

I work on the benchmark now. [~mikemccand] I will submit a new bench to your 
repo luceneutils.


was (Author: ekeller):
New patch:
1. In the DrillSideways.search method, if executor is non-null, we invoke the 
concurrent version.
2. The unit test tests effectively the new concurrent methods.


> A parallel DrillSideways implementation
> ---
>
> Key: LUCENE-7588
> URL: https://issues.apache.org/jira/browse/LUCENE-7588
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: master (7.0), 6.3.1
>Reporter: Emmanuel Keller
>Priority: Minor
>  Labels: facet, faceting
> Fix For: master (7.0), 6.3.1
>
> Attachments: LUCENE-7588.patch
>
>
> Currently DrillSideways implementation is based on the single threaded 
> IndexSearcher.search(Query query, Collector results).
> On large document set, the single threaded collection can be really slow.
> The ParallelDrillSideways implementation could:
> 1. Use the CollectionManager based method IndexSearcher.search(Query query, 
> CollectorManager collectorManager)  to get the benefits of multithreading on 
> index segments,
> 2. Compute each DrillSideway subquery on a single thread.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7588) A parallel DrillSideways implementation

[Emmanuel Keller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated LUCENE-7588:

Attachment: LUCENE-7588.patch

New patch:
1. In the DrillSideways.search method, if executor is non-null, we invoke the 
concurrent version.
2. The unit test tests effectively the new concurrent methods.


> A parallel DrillSideways implementation
> ---
>
> Key: LUCENE-7588
> URL: https://issues.apache.org/jira/browse/LUCENE-7588
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: master (7.0), 6.3.1
>Reporter: Emmanuel Keller
>Priority: Minor
>  Labels: facet, faceting
> Fix For: master (7.0), 6.3.1
>
> Attachments: LUCENE-7588.patch
>
>
> Currently DrillSideways implementation is based on the single threaded 
> IndexSearcher.search(Query query, Collector results).
> On large document set, the single threaded collection can be really slow.
> The ParallelDrillSideways implementation could:
> 1. Use the CollectionManager based method IndexSearcher.search(Query query, 
> CollectorManager collectorManager)  to get the benefits of multithreading on 
> index segments,
> 2. Compute each DrillSideway subquery on a single thread.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7588) A parallel DrillSideways implementation

[Emmanuel Keller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated LUCENE-7588:

Attachment: (was: LUCENE-7588.patch)

> A parallel DrillSideways implementation
> ---
>
> Key: LUCENE-7588
> URL: https://issues.apache.org/jira/browse/LUCENE-7588
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: master (7.0), 6.3.1
>Reporter: Emmanuel Keller
>Priority: Minor
>  Labels: facet, faceting
> Fix For: master (7.0), 6.3.1
>
>
> Currently DrillSideways implementation is based on the single threaded 
> IndexSearcher.search(Query query, Collector results).
> On large document set, the single threaded collection can be really slow.
> The ParallelDrillSideways implementation could:
> 1. Use the CollectionManager based method IndexSearcher.search(Query query, 
> CollectorManager collectorManager)  to get the benefits of multithreading on 
> index segments,
> 2. Compute each DrillSideway subquery on a single thread.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 233 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/233/

3 tests failed.
FAILED:  
org.apache.solr.cloud.CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap

Error Message:
Document mismatch on target after sync expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: Document mismatch on target after sync 
expected:<1> but was:<0>
at 
__randomizedtesting.SeedInfo.seed([76695358AB77535F:A1BE7C2F1F28CB18]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at 
org.apache.solr.cloud.CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap(CdcrBootstrapTest.java:134)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)


FAILED:  
org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI

Error Message:
Expected to see 

[jira] [Commented] (SOLR-9878) Code smell in if statement

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765346#comment-15765346
 ] 

ASF subversion and git services commented on SOLR-9878:
---

Commit 1e22dd25bf20844c71648c39028c48d2a9aa4e58 in lucene-solr's branch 
refs/heads/branch_6x from [~mkhludnev]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1e22dd2 ]

SOLR-9878: fix ReversedWildcardFilterFactory caching in query parser


> Code smell in if statement
> --
>
> Key: SOLR-9878
> URL: https://issues.apache.org/jira/browse/SOLR-9878
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Jaechang Nam
>Priority: Trivial
> Attachments: SOLR-9878.patch
>
>
> In recent code snapshot (Github mirror's commit id: 
> c8542b2bd0470af9f8d64bb8133f31828b342604 as today), there is an illogical 
> condition that can be a code smell or a potential bug:
> {code}
> ReversedWildcardFilterFactory fac = leadingWildcards.get(fieldType);
> if (fac != null || leadingWildcards.containsKey(fac)) {
>   return fac;
> }
> {code}
> In SOLR-3492, it said there was a fix in SOLR-4093. However, the fix still 
> has an issue as above: containsKey will always have null in this if 
> statement. The second condition could be unnecessary. Does leadingWildcards 
> allow a null object as a key? If so, it will return null that might cause NPE 
> in some other locations.
> Patch could be just like in SOLR-3492?:
> {code}
> if (fac != null)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9760) solr.cmd on Windows requires modify permissions in the current directory

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765326#comment-15765326
 ] 

ASF subversion and git services commented on SOLR-9760:
---

Commit b45293313be52127841dc01e84bcd66909d305aa in lucene-solr's branch 
refs/heads/branch_6x from [~mkhludnev]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b452933 ]

SOLR-9760: solr.cmd doesn't need write permission in current directory


> solr.cmd on Windows requires modify permissions in the current directory
> 
>
> Key: SOLR-9760
> URL: https://issues.apache.org/jira/browse/SOLR-9760
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Affects Versions: 6.3
> Environment: Windows
>Reporter: Alex Crome
>Assignee: Mikhail Khludnev
> Attachments: SOLR-9760.patch
>
>
> Currently starting solr fails if the user does not have permission to write 
> to the current directory.  This is caused by the resolve_java_vendor function 
> writing a temporary file to the current directory (javares). 
> {code}
> :resolve_java_vendor
> set "JAVA_VENDOR=Oracle"
> "%JAVA%" -version 2>&1 | findstr /i "IBM J9" > javares
> set /p JAVA_VENDOR_OUT= del javares
> if NOT "%JAVA_VENDOR_OUT%" == "" (
>   set "JAVA_VENDOR=IBM J9"
> )
> {code}
> Rather than writing this temporary file to disk, The exit code of findstr can 
> be used to determine if there is a match.  (0 == match, 1 == no match, 2 == 
> syntax error)
> {code}
> :resolve_java_vendor
> "%JAVA%" -version 2>&1 | findstr /i "IBM J9" > nul
> if %ERRORLEVEL% == 1 set "JAVA_VENDOR=Oracle" else set "JAVA_VENDOR=IBM J9"
> {code}
> By not writing this temp file, you can reduce the permissions solr needs.  As 
> a work around until this is fixed, you can start solr in a directory that has 
> the required permissions, 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9760) solr.cmd on Windows requires modify permissions in the current directory

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765319#comment-15765319
 ] 

ASF subversion and git services commented on SOLR-9760:
---

Commit febe0019a1e5ad2a0d38b8cf000bcf6f3abfa760 in lucene-solr's branch 
refs/heads/master from [~mkhludnev]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=febe001 ]

SOLR-9760: solr.cmd doesn't need write permission in current directory


> solr.cmd on Windows requires modify permissions in the current directory
> 
>
> Key: SOLR-9760
> URL: https://issues.apache.org/jira/browse/SOLR-9760
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Affects Versions: 6.3
> Environment: Windows
>Reporter: Alex Crome
>Assignee: Mikhail Khludnev
> Attachments: SOLR-9760.patch
>
>
> Currently starting solr fails if the user does not have permission to write 
> to the current directory.  This is caused by the resolve_java_vendor function 
> writing a temporary file to the current directory (javares). 
> {code}
> :resolve_java_vendor
> set "JAVA_VENDOR=Oracle"
> "%JAVA%" -version 2>&1 | findstr /i "IBM J9" > javares
> set /p JAVA_VENDOR_OUT= del javares
> if NOT "%JAVA_VENDOR_OUT%" == "" (
>   set "JAVA_VENDOR=IBM J9"
> )
> {code}
> Rather than writing this temporary file to disk, The exit code of findstr can 
> be used to determine if there is a match.  (0 == match, 1 == no match, 2 == 
> syntax error)
> {code}
> :resolve_java_vendor
> "%JAVA%" -version 2>&1 | findstr /i "IBM J9" > nul
> if %ERRORLEVEL% == 1 set "JAVA_VENDOR=Oracle" else set "JAVA_VENDOR=IBM J9"
> {code}
> By not writing this temp file, you can reduce the permissions solr needs.  As 
> a work around until this is fixed, you can start solr in a directory that has 
> the required permissions, 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9760) solr.cmd on Windows requires modify permissions in the current directory

[Mikhail Khludnev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated SOLR-9760:
---
Attachment: SOLR-9760.patch

[^SOLR-9760.patch] checked on both jvms. It works at least. 

> solr.cmd on Windows requires modify permissions in the current directory
> 
>
> Key: SOLR-9760
> URL: https://issues.apache.org/jira/browse/SOLR-9760
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Affects Versions: 6.3
> Environment: Windows
>Reporter: Alex Crome
>Assignee: Mikhail Khludnev
> Attachments: SOLR-9760.patch
>
>
> Currently starting solr fails if the user does not have permission to write 
> to the current directory.  This is caused by the resolve_java_vendor function 
> writing a temporary file to the current directory (javares). 
> {code}
> :resolve_java_vendor
> set "JAVA_VENDOR=Oracle"
> "%JAVA%" -version 2>&1 | findstr /i "IBM J9" > javares
> set /p JAVA_VENDOR_OUT= del javares
> if NOT "%JAVA_VENDOR_OUT%" == "" (
>   set "JAVA_VENDOR=IBM J9"
> )
> {code}
> Rather than writing this temporary file to disk, The exit code of findstr can 
> be used to determine if there is a match.  (0 == match, 1 == no match, 2 == 
> syntax error)
> {code}
> :resolve_java_vendor
> "%JAVA%" -version 2>&1 | findstr /i "IBM J9" > nul
> if %ERRORLEVEL% == 1 set "JAVA_VENDOR=Oracle" else set "JAVA_VENDOR=IBM J9"
> {code}
> By not writing this temp file, you can reduce the permissions solr needs.  As 
> a work around until this is fixed, you can start solr in a directory that has 
> the required permissions, 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9878) Code smell in if statement

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765293#comment-15765293
 ] 

ASF subversion and git services commented on SOLR-9878:
---

Commit f1e636f5611abea66efed896a95eebbb6d765300 in lucene-solr's branch 
refs/heads/master from [~mkhludnev]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f1e636f ]

SOLR-9878: fix ReversedWildcardFilterFactory caching in query parser


> Code smell in if statement
> --
>
> Key: SOLR-9878
> URL: https://issues.apache.org/jira/browse/SOLR-9878
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Jaechang Nam
>Priority: Trivial
> Attachments: SOLR-9878.patch
>
>
> In recent code snapshot (Github mirror's commit id: 
> c8542b2bd0470af9f8d64bb8133f31828b342604 as today), there is an illogical 
> condition that can be a code smell or a potential bug:
> {code}
> ReversedWildcardFilterFactory fac = leadingWildcards.get(fieldType);
> if (fac != null || leadingWildcards.containsKey(fac)) {
>   return fac;
> }
> {code}
> In SOLR-3492, it said there was a fix in SOLR-4093. However, the fix still 
> has an issue as above: containsKey will always have null in this if 
> statement. The second condition could be unnecessary. Does leadingWildcards 
> allow a null object as a key? If so, it will return null that might cause NPE 
> in some other locations.
> Patch could be just like in SOLR-3492?:
> {code}
> if (fac != null)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-9760) solr.cmd on Windows requires modify permissions in the current directory

[Mikhail Khludnev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev reassigned SOLR-9760:
--

Assignee: Mikhail Khludnev

> solr.cmd on Windows requires modify permissions in the current directory
> 
>
> Key: SOLR-9760
> URL: https://issues.apache.org/jira/browse/SOLR-9760
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Affects Versions: 6.3
> Environment: Windows
>Reporter: Alex Crome
>Assignee: Mikhail Khludnev
>
> Currently starting solr fails if the user does not have permission to write 
> to the current directory.  This is caused by the resolve_java_vendor function 
> writing a temporary file to the current directory (javares). 
> {code}
> :resolve_java_vendor
> set "JAVA_VENDOR=Oracle"
> "%JAVA%" -version 2>&1 | findstr /i "IBM J9" > javares
> set /p JAVA_VENDOR_OUT= del javares
> if NOT "%JAVA_VENDOR_OUT%" == "" (
>   set "JAVA_VENDOR=IBM J9"
> )
> {code}
> Rather than writing this temporary file to disk, The exit code of findstr can 
> be used to determine if there is a match.  (0 == match, 1 == no match, 2 == 
> syntax error)
> {code}
> :resolve_java_vendor
> "%JAVA%" -version 2>&1 | findstr /i "IBM J9" > nul
> if %ERRORLEVEL% == 1 set "JAVA_VENDOR=Oracle" else set "JAVA_VENDOR=IBM J9"
> {code}
> By not writing this temp file, you can reduce the permissions solr needs.  As 
> a work around until this is fixed, you can start solr in a directory that has 
> the required permissions, 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+147) - Build # 18576 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18576/
Java: 32bit/jdk-9-ea+147 -client -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.handler.admin.MetricsHandlerTest.test

Error Message:
Unable to make member of class com.sun.management.internal.OperatingSystemImpl 
accessible:  module jdk.management does not export com.sun.management.internal 
to unnamed module @1a5db8e

Stack Trace:
java.lang.reflect.InaccessibleObjectException: Unable to make member of class 
com.sun.management.internal.OperatingSystemImpl accessible:  module 
jdk.management does not export com.sun.management.internal to unnamed module 
@1a5db8e
at 
__randomizedtesting.SeedInfo.seed([DBD49BD6595B7890:5380A40CF7A71568]:0)
at 
java.base/jdk.internal.reflect.Reflection.throwInaccessibleObjectException(Reflection.java:424)
at 
java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:174)
at 
java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:192)
at java.base/java.lang.reflect.Method.setAccessible(Method.java:186)
at 
com.codahale.metrics.jvm.FileDescriptorRatioGauge.invoke(FileDescriptorRatioGauge.java:48)
at 
com.codahale.metrics.jvm.FileDescriptorRatioGauge.getRatio(FileDescriptorRatioGauge.java:35)
at com.codahale.metrics.RatioGauge.getValue(RatioGauge.java:64)
at com.codahale.metrics.RatioGauge.getValue(RatioGauge.java:11)
at 
org.apache.solr.util.stats.MetricUtils.gaugeToNamedList(MetricUtils.java:135)
at 
org.apache.solr.util.stats.MetricUtils.lambda$toNamedList$2(MetricUtils.java:87)
at 
java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at 
java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
at 
java.base/java.util.TreeMap$KeySpliterator.forEachRemaining(TreeMap.java:2739)
at 
java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at 
java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at 
java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at 
java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at 
java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at 
java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:430)
at 
org.apache.solr.util.stats.MetricUtils.toNamedList(MetricUtils.java:80)
at 
org.apache.solr.handler.admin.MetricsHandler.handleRequestBody(MetricsHandler.java:93)
at 
org.apache.solr.handler.admin.MetricsHandlerTest.test(MetricsHandlerTest.java:41)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:538)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
   

[jira] [Updated] (LUCENE-7585) Interface for common parameters used across analysis factories

[Ahmet Arslan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ahmet Arslan updated LUCENE-7585:
-
Attachment: LUCENE-7585.patch

Patch that adds javadocs. {{ant documentation-lint}} still fails for some 
reason that I cannot figure out.

> Interface for common parameters used across analysis factories
> --
>
> Key: LUCENE-7585
> URL: https://issues.apache.org/jira/browse/LUCENE-7585
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>Affects Versions: 6.3
>Reporter: Ahmet Arslan
>Assignee: David Smiley
>Priority: Minor
> Fix For: master (7.0)
>
> Attachments: LUCENE-7585.patch, LUCENE-7585.patch, LUCENE-7585.patch, 
> LUCENE-7585.patch
>
>
> Certain parameters (String constants) are same/common for multiple analysis 
> factories. Some examples are {{ignoreCase}}, {{dictionary}}, and 
> {{preserveOriginal}}. These string constants are handled inconsistently in 
> different factories. This is an effort to define most common constants in 
> ({{CommonAnalysisFactoryParams}}) interface and reuse them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7599) replace TestRandomChains.Predicate with java.util.function.Predicate

[Ahmet Arslan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ahmet Arslan updated LUCENE-7599:
-
Attachment: LUCENE-7599.patch

Patch that replaces ArgProducer with Function

> replace TestRandomChains.Predicate with java.util.function.Predicate
> 
>
> Key: LUCENE-7599
> URL: https://issues.apache.org/jira/browse/LUCENE-7599
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: general/test
>Affects Versions: 6.3
>Reporter: Ahmet Arslan
>Priority: Trivial
>  Labels: test
> Fix For: master (7.0)
>
> Attachments: LUCENE-7599.patch, LUCENE-7599.patch
>
>
> {{TestRandomChains}} has its own Predicate interface which can be replaced 
> with {{java.util.function.Predicate}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1021 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1021/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsRecoveryZkTest

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog] 
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
  at 
org.apache.solr.update.HdfsTransactionLog.(HdfsTransactionLog.java:130)  
at org.apache.solr.update.HdfsUpdateLog.init(HdfsUpdateLog.java:202)  at 
org.apache.solr.update.UpdateHandler.(UpdateHandler.java:137)  at 
org.apache.solr.update.UpdateHandler.(UpdateHandler.java:94)  at 
org.apache.solr.update.DirectUpdateHandler2.(DirectUpdateHandler2.java:102)
  at sun.reflect.GeneratedConstructorAccessor115.newInstance(Unknown Source)  
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  at java.lang.reflect.Constructor.newInstance(Constructor.java:423)  at 
org.apache.solr.core.SolrCore.createInstance(SolrCore.java:747)  at 
org.apache.solr.core.SolrCore.createUpdateHandler(SolrCore.java:809)  at 
org.apache.solr.core.SolrCore.initUpdateHandler(SolrCore.java:1060)  at 
org.apache.solr.core.SolrCore.(SolrCore.java:925)  at 
org.apache.solr.core.SolrCore.(SolrCore.java:817)  at 
org.apache.solr.core.CoreContainer.create(CoreContainer.java:902)  at 
org.apache.solr.core.CoreContainer.lambda$load$3(CoreContainer.java:547)  at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)  at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
 at java.lang.Thread.run(Thread.java:745)  

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not 
released!!! [HdfsTransactionLog]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at 
org.apache.solr.update.HdfsTransactionLog.(HdfsTransactionLog.java:130)
at org.apache.solr.update.HdfsUpdateLog.init(HdfsUpdateLog.java:202)
at org.apache.solr.update.UpdateHandler.(UpdateHandler.java:137)
at org.apache.solr.update.UpdateHandler.(UpdateHandler.java:94)
at 
org.apache.solr.update.DirectUpdateHandler2.(DirectUpdateHandler2.java:102)
at sun.reflect.GeneratedConstructorAccessor115.newInstance(Unknown 
Source)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:747)
at org.apache.solr.core.SolrCore.createUpdateHandler(SolrCore.java:809)
at org.apache.solr.core.SolrCore.initUpdateHandler(SolrCore.java:1060)
at org.apache.solr.core.SolrCore.(SolrCore.java:925)
at org.apache.solr.core.SolrCore.(SolrCore.java:817)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:902)
at 
org.apache.solr.core.CoreContainer.lambda$load$3(CoreContainer.java:547)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)


at __randomizedtesting.SeedInfo.seed([E23B63DE0CB75CFC]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:266)
at sun.reflect.GeneratedMethodAccessor20.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 

[jira] [Commented] (SOLR-9869) MiniSolrCloudCluster does not always remove jettys from running list after stopping them

[Mike Drob (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765149#comment-15765149
 ] 

Mike Drob commented on SOLR-9869:
-

Oh, good point. I just looked into this and found that the jettys list is 
already a COWList after your work in SOLR-8255. I think that means this change 
is safe?

> MiniSolrCloudCluster does not always remove jettys from running list after 
> stopping them
> 
>
> Key: SOLR-9869
> URL: https://issues.apache.org/jira/browse/SOLR-9869
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Reporter: Mike Drob
> Attachments: SOLR-9869.patch
>
>
> MiniSolrCloudCluster has two {{stopJettySolrRunner}} methods that behave 
> differently.
> The {{int}} version calls {{jettys.remove(index);}} to remove the now stopped 
> jetty from the list of running jettys.
> The version that takes a {{JettySolrRunner}}, however, does not modify the 
> running list.
> This can cause calls to {{getReplicaJetty}} to fail after a call to {{stop}} 
> because we will try to get the base url of a stopped jetty.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9839) Use 'useFactory' in tests instead of setting manually

[Mike Drob (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765086#comment-15765086
 ] 

Mike Drob commented on SOLR-9839:
-

[~erickerickson] - what do you think the path forward on this is? Good to 
commit? Close as "won't fix" due to SystemPropertiesRestoreRule?

> Use 'useFactory' in tests instead of setting manually
> -
>
> Key: SOLR-9839
> URL: https://issues.apache.org/jira/browse/SOLR-9839
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Reporter: Mike Drob
>Priority: Minor
> Attachments: SOLR-9839.patch
>
>
> We have several tests that will explicitly set a directory factory via 
> SysProp, some of which forget to unset it.
> We should use {{useFactory}} so that we can benefit from the call to 
> {{resetFactory}} in {{SolrTestCaseJ4.teardownTestCases}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7599) replace TestRandomChains.Predicate with java.util.function.Predicate

[Ahmet Arslan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ahmet Arslan updated LUCENE-7599:
-
Attachment: LUCENE-7599.patch

Patch that removes {{TestRandomChains.Predicate}} in favour of 
{{java.util.function.Predicate}. It simplifies code with lambda expressions or 
method references.

> replace TestRandomChains.Predicate with java.util.function.Predicate
> 
>
> Key: LUCENE-7599
> URL: https://issues.apache.org/jira/browse/LUCENE-7599
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: general/test
>Affects Versions: 6.3
>Reporter: Ahmet Arslan
>Priority: Trivial
>  Labels: test
> Fix For: master (7.0)
>
> Attachments: LUCENE-7599.patch
>
>
> {{TestRandomChains}} has its own Predicate interface which can be replaced 
> with {{java.util.function.Predicate}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9878) Code smell in if statement

[Mikhail Khludnev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated SOLR-9878:
---
Attachment: SOLR-9878.patch

what about [^SOLR-9878.patch]? this fixes caching of nulls. 

> Code smell in if statement
> --
>
> Key: SOLR-9878
> URL: https://issues.apache.org/jira/browse/SOLR-9878
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Jaechang Nam
>Priority: Trivial
> Attachments: SOLR-9878.patch
>
>
> In recent code snapshot (Github mirror's commit id: 
> c8542b2bd0470af9f8d64bb8133f31828b342604 as today), there is an illogical 
> condition that can be a code smell or a potential bug:
> {code}
> ReversedWildcardFilterFactory fac = leadingWildcards.get(fieldType);
> if (fac != null || leadingWildcards.containsKey(fac)) {
>   return fac;
> }
> {code}
> In SOLR-3492, it said there was a fix in SOLR-4093. However, the fix still 
> has an issue as above: containsKey will always have null in this if 
> statement. The second condition could be unnecessary. Does leadingWildcards 
> allow a null object as a key? If so, it will return null that might cause NPE 
> in some other locations.
> Patch could be just like in SOLR-3492?:
> {code}
> if (fac != null)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7599) replace TestRandomChains.Predicate with java.util.function.Predicate

[Ahmet Arslan (JIRA)]

Ahmet Arslan created LUCENE-7599:


 Summary: replace TestRandomChains.Predicate with 
java.util.function.Predicate
 Key: LUCENE-7599
 URL: https://issues.apache.org/jira/browse/LUCENE-7599
 Project: Lucene - Core
  Issue Type: Improvement
  Components: general/test
Affects Versions: 6.3
Reporter: Ahmet Arslan
Priority: Trivial
 Fix For: master (7.0)


{{TestRandomChains}} has its own Predicate interface which can be replaced with 
{{java.util.function.Predicate}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9882) ClassCastException: BasicResultContext cannot be cast to SolrDocumentList

[Yago Riveiro (JIRA)]

Yago Riveiro created SOLR-9882:
--

 Summary: ClassCastException: BasicResultContext cannot be cast to 
SolrDocumentList
 Key: SOLR-9882
 URL: https://issues.apache.org/jira/browse/SOLR-9882
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 6.3
Reporter: Yago Riveiro


After talk with [~yo...@apache.org] in the mailing list I open this Jira ticket

I'm hitting this bug in Solr 6.3.0.

null:java.lang.ClassCastException:
org.apache.solr.response.BasicResultContext cannot be cast to
org.apache.solr.common.SolrDocumentList
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:315)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2213)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at
org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:169)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:518)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(Thread.java:745)




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4146) Error handling 'status' action, cannot access GUI

[Cassandra Targett (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-4146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765019#comment-15765019
 ] 

Cassandra Targett commented on SOLR-4146:
-

Has anyone who has previously had this issue seen it with Solr 5.5.1 or higher? 
Alexey's earlier comment about SOLR-8587 led me to SOLR-8793 which was fixed in 
Solr 5.5.1. I'm wondering if this is resolved with that fix or if there is 
still something else going on.

> Error handling 'status' action, cannot access GUI
> -
>
> Key: SOLR-4146
> URL: https://issues.apache.org/jira/browse/SOLR-4146
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud, web gui
>Affects Versions: 6.0
>Reporter: Markus Jelsma
> Fix For: 6.0
>
> Attachments: 2016-04-26_1547.png, solr.png
>
>
> We sometimes see a node not responding to GUI requests. It then generates the 
> stack trace below. It does respond to search requests.
> {code}
> 2012-12-05 15:53:24,329 ERROR [solr.core.SolrCore] - [http-8080-exec-7] - : 
> org.apache.solr.common.SolrException: Error handling 'status' action 
> at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:725)
> at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:158)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:144)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:372)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:181)
> at 
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
> at 
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at 
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> at 
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> at 
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at 
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> at 
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> at 
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
> at 
> org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
> at 
> org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:744)
> at 
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2274)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: org.apache.solr.common.SolrException: 
> java.util.concurrent.RejectedExecutionException
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1674)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1330)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1265)
> at 
> org.apache.solr.handler.admin.CoreAdminHandler.getCoreStatus(CoreAdminHandler.java:997)
> at 
> org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:711)
> ... 18 more
> Caused by: java.util.concurrent.RejectedExecutionException
> at 
> java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1768)
> at 
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
> at 
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
> at 
> java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:92)
> at 
> java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:603)
> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1605)
> ... 22 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Announcing Marple, an index-structure visualiser for Apache Lucene

[Alan Woodward]

Fixed, thank you!

Alan Woodward
www.flax.co.uk


> On 20 Dec 2016, at 18:07, Adrien Grand  wrote:
> 
> It is pretty cool, I just found one minor issue when following the readme, 
> the command is run-server rather than run_server. Looking forward to the 
> panel for points!
> 
> Le mar. 20 déc. 2016 à 12:43, Alan Woodward  > a écrit :
> Hi all,
> 
> At the London and Boston hackdays a couple of months back, I started to work 
> on an upgrade/replacement for Luke.  My idea was to have something that would 
> provide a read-only view of an index, which would be useful both for 
> inspection and debugging of indexes, and as a training tool.  I also wanted 
> it to display via a browser, so that it could be used on remote servers, and 
> to use only Apache-licensed components, with a view to eventually 
> contributing it back to the lucene core project.
> 
> My colleague Tom Mortimer and I have been working on this on-and-off for a 
> few weeks now, and it’s in a sort of nearly-ready state - I’ve already found 
> it useful on a couple of client projects - so I thought the list would be 
> interested in trying it out.  It’s by no means finished, and there are quite 
> a few rough edges and probably some exciting bugs.  But please feel free to 
> poke around and open issues (and submit pull requests!).
> 
> The application divides up into two parts: a dropwizard-based webapp that 
> exposes the various parts of the index (fields, terms, postings, points, 
> docvalues, documents, etc) as json; and a React-based UI that makes this 
> navigable.  The first part is nearly feature-complete, although there are a 
> few missing bits (I haven’t done anything with termvectors yet, for example, 
> or index statistics).  The UI is still missing panels for postings lists and 
> for points, but everything else is mostly there.
> 
> The repository is here: https://github.com/flaxsearch/marple 
> 
> 
> Please download it, play around with it, and let me know what you think!
> 
> Thanks to Steve Rowe, Shinichiro Abe and Eric Pugh for some early 
> contributions, and to Andrzej Białecki for convincing me that it wasn’t a 
> dumb idea.
> 
> Alan Woodward
> www.flax.co.uk 
> 

[jira] [Commented] (SOLR-9869) MiniSolrCloudCluster does not always remove jettys from running list after stopping them

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15765014#comment-15765014
 ] 

Alan Woodward commented on SOLR-9869:
-

IIRC it's to avoid ConcurrentModificationExceptions during parallel shutdown, 
but we can avoid that I think by just calling jetty.stop() directly in the 
shutdown() lambdas rather than calling the stop method.

> MiniSolrCloudCluster does not always remove jettys from running list after 
> stopping them
> 
>
> Key: SOLR-9869
> URL: https://issues.apache.org/jira/browse/SOLR-9869
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Reporter: Mike Drob
> Attachments: SOLR-9869.patch
>
>
> MiniSolrCloudCluster has two {{stopJettySolrRunner}} methods that behave 
> differently.
> The {{int}} version calls {{jettys.remove(index);}} to remove the now stopped 
> jetty from the list of running jettys.
> The version that takes a {{JettySolrRunner}}, however, does not modify the 
> running list.
> This can cause calls to {{getReplicaJetty}} to fail after a call to {{stop}} 
> because we will try to get the base url of a stopped jetty.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9513) Introduce a generic authentication plugin which delegates all functionality to Hadoop authentication framework

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764992#comment-15764992
 ] 

ASF subversion and git services commented on SOLR-9513:
---

Commit b5cfb17bd0d56da03dbe1f179db0f03ea0acf735 in lucene-solr's branch 
refs/heads/master from [~ichattopadhyaya]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b5cfb17 ]

SOLR-9513: Fix test failure on Windows and Java9 by avoiding NPE in 
tearDownClass()


> Introduce a generic authentication plugin which delegates all functionality 
> to Hadoop authentication framework
> --
>
> Key: SOLR-9513
> URL: https://issues.apache.org/jira/browse/SOLR-9513
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hrishikesh Gadre
>Assignee: Ishan Chattopadhyaya
> Fix For: master (7.0), 6.4
>
> Attachments: SOLR-9513.patch, SOLR-9513_6x.patch
>
>
> Currently Solr kerberos authentication plugin delegates the core logic to 
> Hadoop authentication framework. But the configuration parameters required by 
> the Hadoop authentication framework are hardcoded in the plugin code itself. 
> https://github.com/apache/lucene-solr/blob/5b770b56d012279d334f41e4ef7fe652480fd3cf/solr/core/src/java/org/apache/solr/security/KerberosPlugin.java#L119
> The problem with this approach is that we need to make code changes in Solr 
> to expose new capabilities added in Hadoop authentication framework. e.g. 
> HADOOP-12082
> We should implement a generic Solr authentication plugin which will accept 
> configuration parameters via security.json (in Zookeeper) and delegate them 
> to Hadoop authentication framework. This will allow to utilize new features 
> in Hadoop without code changes in Solr.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9513) Introduce a generic authentication plugin which delegates all functionality to Hadoop authentication framework

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764991#comment-15764991
 ] 

ASF subversion and git services commented on SOLR-9513:
---

Commit 21a6fc3d763d2eedf665e13474b9f751301c738b in lucene-solr's branch 
refs/heads/branch_6x from [~ichattopadhyaya]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=21a6fc3 ]

SOLR-9513: Fix test failure on Windows and Java9 by avoiding NPE in 
tearDownClass()


> Introduce a generic authentication plugin which delegates all functionality 
> to Hadoop authentication framework
> --
>
> Key: SOLR-9513
> URL: https://issues.apache.org/jira/browse/SOLR-9513
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hrishikesh Gadre
>Assignee: Ishan Chattopadhyaya
> Fix For: master (7.0), 6.4
>
> Attachments: SOLR-9513.patch, SOLR-9513_6x.patch
>
>
> Currently Solr kerberos authentication plugin delegates the core logic to 
> Hadoop authentication framework. But the configuration parameters required by 
> the Hadoop authentication framework are hardcoded in the plugin code itself. 
> https://github.com/apache/lucene-solr/blob/5b770b56d012279d334f41e4ef7fe652480fd3cf/solr/core/src/java/org/apache/solr/security/KerberosPlugin.java#L119
> The problem with this approach is that we need to make code changes in Solr 
> to expose new capabilities added in Hadoop authentication framework. e.g. 
> HADOOP-12082
> We should implement a generic Solr authentication plugin which will accept 
> configuration parameters via security.json (in Zookeeper) and delegate them 
> to Hadoop authentication framework. This will allow to utilize new features 
> in Hadoop without code changes in Solr.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7579) Sorting on flushed segment

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764963#comment-15764963
 ] 

Michael McCandless commented on LUCENE-7579:


Wow this change gave a big jump in indexing throughput when index sorting is 
used: 
https://home.apache.org/~mikemccand/lucenebench/sparseResults.html#index_throughput

You can see the flush time went up but the merge time went way down.

> Sorting on flushed segment
> --
>
> Key: LUCENE-7579
> URL: https://issues.apache.org/jira/browse/LUCENE-7579
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ferenczi Jim
>
> Today flushed segments built by an index writer with an index sort specified 
> are not sorted. The merge is responsible of sorting these segments 
> potentially with others that are already sorted (resulted from another 
> merge). 
> I'd like to investigate the cost of sorting the segment directly during the 
> flush. This could make the merge faster since they are some cheap 
> optimizations that can be done only if all segments to be merged are sorted.
>  For instance the merge of the points could use the bulk merge instead of 
> rebuilding the points from scratch.
> I made a small prototype which sort the segment on flush here:
> https://github.com/apache/lucene-solr/compare/master...jimczi:flush_sort
> The idea is simple, for points, norms, docvalues and terms I use the 
> SortingLeafReader implementation to translate the values that we have in RAM 
> in a sorted enumeration for the writers.
> For stored fields I use a two pass scheme where the documents are first 
> written to disk unsorted and then copied to another file with the correct 
> sorting. I use the same stored field format for the two steps and just remove 
> the file produced by the first pass at the end of the process.
> This prototype has no implementation for index sorting that use term vectors 
> yet. I'll add this later if the tests are good enough.
> Speaking of testing, I tried this branch on [~mikemccand] benchmark scripts 
> and compared master with index sorting against my branch with index sorting 
> on flush. I tried with sparsetaxis and wikipedia and the first results are 
> weird. When I use the SerialScheduler and only one thread to write the docs,  
> index sorting on flush is slower. But when I use two threads the sorting on 
> flush is much faster even with the SerialScheduler. I'll continue to run the 
> tests in order to be able to share something more meaningful.
> The tests are passing except one about concurrent DV updates. I don't know 
> this part at all so I did not fix the test yet. I don't even know if we can 
> make it work with index sorting ;).
>  [~mikemccand] I would love to have your feedback about the prototype. Could 
> you please take a look ? I am sure there are plenty of bugs, ... but I think 
> it's a good start to evaluate the feasibility of this feature.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-6.x-Linux (64bit/jdk-9-ea+147) - Build # 2469 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/2469/
Java: 64bit/jdk-9-ea+147 -XX:-UseCompressedOops -XX:+UseParallelGC

2 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.security.hadoop.TestSolrCloudWithHadoopAuthPlugin

Error Message:
FIXME: SOLR-8182: This test fails under Java 9

Stack Trace:
com.carrotsearch.randomizedtesting.InternalAssumptionViolatedException: FIXME: 
SOLR-8182: This test fails under Java 9
at __randomizedtesting.SeedInfo.seed([905FE3FDA59F2192]:0)
at 
com.carrotsearch.randomizedtesting.RandomizedTest.assumeTrue(RandomizedTest.java:667)
at 
com.carrotsearch.randomizedtesting.RandomizedTest.assumeFalse(RandomizedTest.java:675)
at 
org.apache.lucene.util.LuceneTestCase.assumeFalse(LuceneTestCase.java:849)
at 
org.apache.solr.security.hadoop.TestSolrCloudWithHadoopAuthPlugin.setupClass(TestSolrCloudWithHadoopAuthPlugin.java:45)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:538)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:847)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  
junit.framework.TestSuite.org.apache.solr.security.hadoop.TestSolrCloudWithHadoopAuthPlugin

Error Message:


Stack Trace:
java.lang.NullPointerException
at __randomizedtesting.SeedInfo.seed([905FE3FDA59F2192]:0)
at 
org.apache.solr.security.hadoop.TestSolrCloudWithHadoopAuthPlugin.tearDownClass(TestSolrCloudWithHadoopAuthPlugin.java:62)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:538)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 

[jira] [Commented] (SOLR-6246) Core fails to reload when AnalyzingInfixSuggester is used as a Suggester

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764898#comment-15764898
 ] 

Erick Erickson commented on SOLR-6246:
--

Bah. The comments when linking other JIRAs don't have any obvious association, 
do they?

Perhaps related to SOLR-7747.

> Core fails to reload when AnalyzingInfixSuggester is used as a Suggester
> 
>
> Key: SOLR-6246
> URL: https://issues.apache.org/jira/browse/SOLR-6246
> Project: Solr
>  Issue Type: Sub-task
>  Components: SearchComponents - other
>Affects Versions: 4.8, 4.8.1, 4.9, 5.0, 5.1, 5.2, 5.3, 5.4
>Reporter: Varun Thacker
> Attachments: SOLR-6246-test.patch, SOLR-6246-test.patch, 
> SOLR-6246-test.patch, SOLR-6246.patch
>
>
> LUCENE-5477 - added near-real-time suggest building to 
> AnalyzingInfixSuggester. One of the changes that went in was a writer is 
> persisted now to support real time updates via the add() and update() methods.
> When we call Solr's reload command, a new instance of AnalyzingInfixSuggester 
> is created. When trying to create a new writer on the same Directory a lock 
> cannot be obtained and Solr fails to reload the core.
> Also when AnalyzingInfixLookupFactory throws a RuntimeException we should 
> pass along the original message.
> I am not sure what should be the approach to fix it. Should we have a 
> reloadHook where we close the writer?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6246) Core fails to reload when AnalyzingInfixSuggester is used as a Suggester

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764878#comment-15764878
 ] 

David Smiley commented on SOLR-6246:


bq. Perhaps related?

[~erickerickson] what is related to what?

> Core fails to reload when AnalyzingInfixSuggester is used as a Suggester
> 
>
> Key: SOLR-6246
> URL: https://issues.apache.org/jira/browse/SOLR-6246
> Project: Solr
>  Issue Type: Sub-task
>  Components: SearchComponents - other
>Affects Versions: 4.8, 4.8.1, 4.9, 5.0, 5.1, 5.2, 5.3, 5.4
>Reporter: Varun Thacker
> Attachments: SOLR-6246-test.patch, SOLR-6246-test.patch, 
> SOLR-6246-test.patch, SOLR-6246.patch
>
>
> LUCENE-5477 - added near-real-time suggest building to 
> AnalyzingInfixSuggester. One of the changes that went in was a writer is 
> persisted now to support real time updates via the add() and update() methods.
> When we call Solr's reload command, a new instance of AnalyzingInfixSuggester 
> is created. When trying to create a new writer on the same Directory a lock 
> cannot be obtained and Solr fails to reload the core.
> Also when AnalyzingInfixLookupFactory throws a RuntimeException we should 
> pass along the original message.
> I am not sure what should be the approach to fix it. Should we have a 
> reloadHook where we close the writer?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7598) SolrCore 'recs' is not available due to init failure

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764868#comment-15764868
 ] 

Erick Erickson commented on LUCENE-7598:


OK, you must give a lot more background here, including the sequence of events 
that lead up to this state. It _sounds_ like you manually manipulated the 
directories, moving things around maybe? Copying from one place to another? If 
so how? Why? What were you trying to accomplish if so? Were you copying while 
the index was actively getting documents added (assuming you were, indeed, 
copying)?

Because this is sounding like pilot error, nothing to do with Lucene or Solr 
really.

You can try using the CheckIndex tool, see: 
https://dzone.com/articles/lucene-and-solrs-checkindex for the background 
although you'll have to use up-to-date jars. Do be aware that the fix option 
will fix your index by removing everything it can't understand.

> SolrCore 'recs' is not available due to init failure
> 
>
> Key: LUCENE-7598
> URL: https://issues.apache.org/jira/browse/LUCENE-7598
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.5.1
> Environment: Windows Server 2008 R2 Standard, 64-bit OS, 8 GB ram, 2 
> core
>Reporter: Brian Kmetz
>Priority: Blocker
>
> Receiving the following error message:
> null:org.apache.solr.common.SolrException: SolrCore 'recs' is not available 
> due to init failure: Error opening new searcher
>  at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:783)
>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:287)
>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
>  at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
>  at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
>  at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
>  at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
>  at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
>  at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
>  at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
>  at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
>  at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
>  at org.eclipse.jetty.server.Server.handle(Server.java:368)
>  at 
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
>  at 
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
>  at 
> org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
>  at 
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
>  at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)
>  at 
> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
>  at 
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
>  at 
> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
>  at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
>  at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
>  at java.lang.Thread.run(Unknown Source)
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
>  at org.apache.solr.core.SolrCore.(SolrCore.java:834)
>  at org.apache.solr.core.SolrCore.(SolrCore.java:625)
>  at 
> org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:522)
>  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:557)
>  at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:247)
>  at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:239)
>  at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>  

[jira] [Comment Edited] (LUCENE-7598) SolrCore 'recs' is not available due to init failure

[Brian Kmetz (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764851#comment-15764851
 ] 

Brian Kmetz edited comment on LUCENE-7598 at 12/20/16 6:18 PM:
---

Originally it was pointing to the fact that it could not find solrconfig.xml. 
Now it appears that it cannot find a resource. What can I do to correct this 
issue?

Caused by: java.io.IOException: Can't find resource 'solrconfig.xml' in 
classpath or 'multicore\multicore\core0\conf/', cwd=C:\Apache\Solr\example


was (Author: brian.km...@hsn.net):
Originally it was pointing to the fact that it could not find solrconfig.xml. 
Now it appears that it cannot find the index_xzz7.si. What can I do to correct 
this issue?

> SolrCore 'recs' is not available due to init failure
> 
>
> Key: LUCENE-7598
> URL: https://issues.apache.org/jira/browse/LUCENE-7598
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.5.1
> Environment: Windows Server 2008 R2 Standard, 64-bit OS, 8 GB ram, 2 
> core
>Reporter: Brian Kmetz
>Priority: Blocker
>
> Receiving the following error message:
> null:org.apache.solr.common.SolrException: SolrCore 'recs' is not available 
> due to init failure: Error opening new searcher
>  at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:783)
>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:287)
>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
>  at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
>  at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
>  at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
>  at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
>  at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
>  at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
>  at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
>  at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
>  at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
>  at org.eclipse.jetty.server.Server.handle(Server.java:368)
>  at 
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
>  at 
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
>  at 
> org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
>  at 
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
>  at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)
>  at 
> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
>  at 
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
>  at 
> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
>  at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
>  at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
>  at java.lang.Thread.run(Unknown Source)
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
>  at org.apache.solr.core.SolrCore.(SolrCore.java:834)
>  at org.apache.solr.core.SolrCore.(SolrCore.java:625)
>  at 
> org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:522)
>  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:557)
>  at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:247)
>  at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:239)
>  at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>  at java.util.concurrent.FutureTask.run(Unknown Source)
>  at java.util.concurrent.Executors$RunnableAdapter.call(Unknown 
> Source)
>  at 

[jira] [Comment Edited] (LUCENE-7598) SolrCore 'recs' is not available due to init failure

[Brian Kmetz (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764851#comment-15764851
 ] 

Brian Kmetz edited comment on LUCENE-7598 at 12/20/16 6:12 PM:
---

Originally it was pointing to the fact that it could not find solrconfig.xml. 
Now it appears that it cannot find the index_xzz7.si. What can I do to correct 
this issue?


was (Author: brian.km...@hsn.net):
Originally it was pointing to the fact that it could not find solrconfig.xml. 
Now it appears that it cannot find the index_xzz7.si. What can I do to correct 
this issue.

> SolrCore 'recs' is not available due to init failure
> 
>
> Key: LUCENE-7598
> URL: https://issues.apache.org/jira/browse/LUCENE-7598
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.5.1
> Environment: Windows Server 2008 R2 Standard, 64-bit OS, 8 GB ram, 2 
> core
>Reporter: Brian Kmetz
>Priority: Blocker
>
> Receiving the following error message:
> null:org.apache.solr.common.SolrException: SolrCore 'recs' is not available 
> due to init failure: Error opening new searcher
>  at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:783)
>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:287)
>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
>  at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
>  at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
>  at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
>  at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
>  at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
>  at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
>  at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
>  at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
>  at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
>  at org.eclipse.jetty.server.Server.handle(Server.java:368)
>  at 
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
>  at 
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
>  at 
> org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
>  at 
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
>  at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)
>  at 
> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
>  at 
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
>  at 
> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
>  at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
>  at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
>  at java.lang.Thread.run(Unknown Source)
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
>  at org.apache.solr.core.SolrCore.(SolrCore.java:834)
>  at org.apache.solr.core.SolrCore.(SolrCore.java:625)
>  at 
> org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:522)
>  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:557)
>  at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:247)
>  at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:239)
>  at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>  at java.util.concurrent.FutureTask.run(Unknown Source)
>  at java.util.concurrent.Executors$RunnableAdapter.call(Unknown 
> Source)
>  at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>  at java.util.concurrent.FutureTask.run(Unknown Source)
>  at 

[jira] [Commented] (LUCENE-7598) SolrCore 'recs' is not available due to init failure

[Brian Kmetz (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764851#comment-15764851
 ] 

Brian Kmetz commented on LUCENE-7598:
-

Originally it was pointing to the fact that it could not find solrconfig.xml. 
Now it appears that it cannot find the index_xzz7.si. What can I do to correct 
this issue.

> SolrCore 'recs' is not available due to init failure
> 
>
> Key: LUCENE-7598
> URL: https://issues.apache.org/jira/browse/LUCENE-7598
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.5.1
> Environment: Windows Server 2008 R2 Standard, 64-bit OS, 8 GB ram, 2 
> core
>Reporter: Brian Kmetz
>Priority: Blocker
>
> Receiving the following error message:
> null:org.apache.solr.common.SolrException: SolrCore 'recs' is not available 
> due to init failure: Error opening new searcher
>  at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:783)
>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:287)
>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
>  at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
>  at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
>  at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
>  at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
>  at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
>  at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
>  at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
>  at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
>  at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
>  at org.eclipse.jetty.server.Server.handle(Server.java:368)
>  at 
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
>  at 
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
>  at 
> org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
>  at 
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
>  at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)
>  at 
> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
>  at 
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
>  at 
> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
>  at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
>  at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
>  at java.lang.Thread.run(Unknown Source)
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
>  at org.apache.solr.core.SolrCore.(SolrCore.java:834)
>  at org.apache.solr.core.SolrCore.(SolrCore.java:625)
>  at 
> org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:522)
>  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:557)
>  at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:247)
>  at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:239)
>  at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>  at java.util.concurrent.FutureTask.run(Unknown Source)
>  at java.util.concurrent.Executors$RunnableAdapter.call(Unknown 
> Source)
>  at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>  at java.util.concurrent.FutureTask.run(Unknown Source)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown 
> Source)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>  ... 1 more
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
>  at 

Re: Announcing Marple, an index-structure visualiser for Apache Lucene

[Adrien Grand]

It is pretty cool, I just found one minor issue when following the readme,
the command is run-server rather than run_server. Looking forward to the
panel for points!

Le mar. 20 déc. 2016 à 12:43, Alan Woodward  a écrit :

> Hi all,
>
> At the London and Boston hackdays a couple of months back, I started to
> work on an upgrade/replacement for Luke.  My idea was to have something
> that would provide a read-only view of an index, which would be useful both
> for inspection and debugging of indexes, and as a training tool.  I also
> wanted it to display via a browser, so that it could be used on remote
> servers, and to use only Apache-licensed components, with a view to
> eventually contributing it back to the lucene core project.
>
> My colleague Tom Mortimer and I have been working on this on-and-off for a
> few weeks now, and it’s in a sort of nearly-ready state - I’ve already
> found it useful on a couple of client projects - so I thought the list
> would be interested in trying it out.  It’s by no means finished, and there
> are quite a few rough edges and probably some exciting bugs.  But please
> feel free to poke around and open issues (and submit pull requests!).
>
> The application divides up into two parts: a dropwizard-based webapp that
> exposes the various parts of the index (fields, terms, postings, points,
> docvalues, documents, etc) as json; and a React-based UI that makes this
> navigable.  The first part is nearly feature-complete, although there are a
> few missing bits (I haven’t done anything with termvectors yet, for
> example, or index statistics).  The UI is still missing panels for postings
> lists and for points, but everything else is mostly there.
>
> The repository is here: https://github.com/flaxsearch/marple
>
> Please download it, play around with it, and let me know what you think!
>
> Thanks to Steve Rowe, Shinichiro Abe and Eric Pugh for some early
> contributions, and to Andrzej Białecki for convincing me that it wasn’t a
> dumb idea.
>
> Alan Woodward
> www.flax.co.uk
>
>

[jira] [Commented] (LUCENE-7598) SolrCore 'recs' is not available due to init failure

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764835#comment-15764835
 ] 

Erick Erickson commented on LUCENE-7598:


Why do you say it can't find solrconfig.xml? The error message is about one of 
the index files. Or is there more to the stack trace?

If it really is solrconfig.xml, that's often the result of the config being 
mal-formed XML. In that case though there's usually an explicit message in the 
solr log file.

And, FWIW there is very little likelihood that a patch will be produced for 
4.5, that's over 3 years old. Do you know if this can be reproduced on a more 
recent Solr, preferably 6x?

> SolrCore 'recs' is not available due to init failure
> 
>
> Key: LUCENE-7598
> URL: https://issues.apache.org/jira/browse/LUCENE-7598
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.5.1
> Environment: Windows Server 2008 R2 Standard, 64-bit OS, 8 GB ram, 2 
> core
>Reporter: Brian Kmetz
>Priority: Blocker
>
> Receiving the following error message:
> null:org.apache.solr.common.SolrException: SolrCore 'recs' is not available 
> due to init failure: Error opening new searcher
>  at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:783)
>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:287)
>  at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
>  at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
>  at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
>  at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
>  at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
>  at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
>  at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
>  at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
>  at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
>  at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
>  at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
>  at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
>  at org.eclipse.jetty.server.Server.handle(Server.java:368)
>  at 
> org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
>  at 
> org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
>  at 
> org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
>  at 
> org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
>  at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)
>  at 
> org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
>  at 
> org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
>  at 
> org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
>  at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
>  at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
>  at java.lang.Thread.run(Unknown Source)
> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
>  at org.apache.solr.core.SolrCore.(SolrCore.java:834)
>  at org.apache.solr.core.SolrCore.(SolrCore.java:625)
>  at 
> org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:522)
>  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:557)
>  at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:247)
>  at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:239)
>  at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>  at java.util.concurrent.FutureTask.run(Unknown Source)
>  at java.util.concurrent.Executors$RunnableAdapter.call(Unknown 
> Source)
>  at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>  at 

[jira] [Commented] (SOLR-8030) Transaction log does not store the update chain used for updates

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764834#comment-15764834
 ] 

David Smiley commented on SOLR-8030:


Yeah.  In general, put URPs *before* DistributedURP; only do after if you 
need/have to do otherwise, you know what you are doing, and furthermore 
actually test tlog replay somehow.

> Transaction log does not store the update chain used for updates
> 
>
> Key: SOLR-8030
> URL: https://issues.apache.org/jira/browse/SOLR-8030
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Affects Versions: 5.3
>Reporter: Ludovic Boutros
> Attachments: SOLR-8030.patch
>
>
> Transaction Log does not store the update chain used during updates.
> Therefore tLog uses the default update chain during log replay.
> If we implement custom update logic with multiple update chains, the log 
> replay could break this logic.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8030) Transaction log does not store the update chain used for updates

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764805#comment-15764805
 ] 

Erick Erickson commented on SOLR-8030:
--

There's another variant on this theme FWIW. With a 
ParseDateFieldUpdateProcessorFactory in the chain (after 
DistributedUpdateProcessorFactory), and indexing a date like 
2001-09-09T11:12:23 (not no 'Z') fails on tlog replay.

> Transaction log does not store the update chain used for updates
> 
>
> Key: SOLR-8030
> URL: https://issues.apache.org/jira/browse/SOLR-8030
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Affects Versions: 5.3
>Reporter: Ludovic Boutros
> Attachments: SOLR-8030.patch
>
>
> Transaction Log does not store the update chain used during updates.
> Therefore tLog uses the default update chain during log replay.
> If we implement custom update logic with multiple update chains, the log 
> replay could break this logic.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+147) - Build # 18575 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18575/
Java: 32bit/jdk-9-ea+147 -server -XX:+UseSerialGC

4 tests failed.
FAILED:  
org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication

Error Message:
expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<0>
at 
__randomizedtesting.SeedInfo.seed([A837F17EC34976C5:5F441F2605A1D923]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication(TestReplicationHandler.java:1329)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:538)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  

[jira] [Closed] (SOLR-9881) CSV DIH Sample

[Alexandre Rafalovitch (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexandre Rafalovitch closed SOLR-9881.
---
Resolution: Invalid

> CSV DIH Sample
> --
>
> Key: SOLR-9881
> URL: https://issues.apache.org/jira/browse/SOLR-9881
> Project: Solr
>  Issue Type: Wish
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - DataImportHandler
>Affects Versions: 6.3
> Environment: Solaris, Super Cluster
>Reporter: Moenieb
>  Labels: features, newbie, test
>
> Hi,
> I am struggling with DIH and a Pipe Separated file.
> Does anyone have sample code for pipe delimted file (schema and config file)
> With my current scenario, I run DIH but I am not getting anything that 
> directs me to where my issues are, like a nice error message or a log entry
> I want to use DIH to index the file as I want to treat certain exceptions and 
> data transformations
> Thanks in advance for the assistance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9881) CSV DIH Sample

[Moenieb (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764785#comment-15764785
 ] 

Moenieb commented on SOLR-9881:
---

Hi,

sent request to mail list

> CSV DIH Sample
> --
>
> Key: SOLR-9881
> URL: https://issues.apache.org/jira/browse/SOLR-9881
> Project: Solr
>  Issue Type: Wish
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - DataImportHandler
>Affects Versions: 6.3
> Environment: Solaris, Super Cluster
>Reporter: Moenieb
>  Labels: features, newbie, test
>
> Hi,
> I am struggling with DIH and a Pipe Separated file.
> Does anyone have sample code for pipe delimted file (schema and config file)
> With my current scenario, I run DIH but I am not getting anything that 
> directs me to where my issues are, like a nice error message or a log entry
> I want to use DIH to index the file as I want to treat certain exceptions and 
> data transformations
> Thanks in advance for the assistance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9881) CSV DIH Sample

[Alexandre Rafalovitch (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764773#comment-15764773
 ] 

Alexandre Rafalovitch commented on SOLR-9881:
-

This kind of issue is best asked on the Solr Users mailing list. Jira is for 
the issues against Solr code that are bug/problems/etc.

> CSV DIH Sample
> --
>
> Key: SOLR-9881
> URL: https://issues.apache.org/jira/browse/SOLR-9881
> Project: Solr
>  Issue Type: Wish
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: contrib - DataImportHandler
>Affects Versions: 6.3
> Environment: Solaris, Super Cluster
>Reporter: Moenieb
>  Labels: features, newbie, test
>
> Hi,
> I am struggling with DIH and a Pipe Separated file.
> Does anyone have sample code for pipe delimted file (schema and config file)
> With my current scenario, I run DIH but I am not getting anything that 
> directs me to where my issues are, like a nice error message or a log entry
> I want to use DIH to index the file as I want to treat certain exceptions and 
> data transformations
> Thanks in advance for the assistance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9881) CSV DIH Sample

[Moenieb (JIRA)]

Moenieb created SOLR-9881:
-

 Summary: CSV DIH Sample
 Key: SOLR-9881
 URL: https://issues.apache.org/jira/browse/SOLR-9881
 Project: Solr
  Issue Type: Wish
  Security Level: Public (Default Security Level. Issues are Public)
  Components: contrib - DataImportHandler
Affects Versions: 6.3
 Environment: Solaris, Super Cluster
Reporter: Moenieb


Hi,

I am struggling with DIH and a Pipe Separated file.
Does anyone have sample code for pipe delimted file (schema and config file)
With my current scenario, I run DIH but I am not getting anything that directs 
me to where my issues are, like a nice error message or a log entry
I want to use DIH to index the file as I want to treat certain exceptions and 
data transformations
Thanks in advance for the assistance.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-9513) Introduce a generic authentication plugin which delegates all functionality to Hadoop authentication framework

[Ishan Chattopadhyaya (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ishan Chattopadhyaya resolved SOLR-9513.

   Resolution: Fixed
 Assignee: Ishan Chattopadhyaya
Fix Version/s: 6.4

Thanks [~hgadre]!

There's a test failure [0], due to the test introduced here, on Java 9 and 
Windows. I don't think the test code is faulty, there could be some issue with 
the test runner setup. The mailing list thread is here [1] (FYI, [~thetaphi]).

[0] - https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18574/
[1] - 
http://lucene.472066.n3.nabble.com/JENKINS-EA-Lucene-Solr-master-Linux-32bit-jdk-9-ea-147-Build-18573-Still-Unstable-td4310502.html

> Introduce a generic authentication plugin which delegates all functionality 
> to Hadoop authentication framework
> --
>
> Key: SOLR-9513
> URL: https://issues.apache.org/jira/browse/SOLR-9513
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hrishikesh Gadre
>Assignee: Ishan Chattopadhyaya
> Fix For: master (7.0), 6.4
>
> Attachments: SOLR-9513.patch, SOLR-9513_6x.patch
>
>
> Currently Solr kerberos authentication plugin delegates the core logic to 
> Hadoop authentication framework. But the configuration parameters required by 
> the Hadoop authentication framework are hardcoded in the plugin code itself. 
> https://github.com/apache/lucene-solr/blob/5b770b56d012279d334f41e4ef7fe652480fd3cf/solr/core/src/java/org/apache/solr/security/KerberosPlugin.java#L119
> The problem with this approach is that we need to make code changes in Solr 
> to expose new capabilities added in Hadoop authentication framework. e.g. 
> HADOOP-12082
> We should implement a generic Solr authentication plugin which will accept 
> configuration parameters via security.json (in Zookeeper) and delegate them 
> to Hadoop authentication framework. This will allow to utilize new features 
> in Hadoop without code changes in Solr.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-9874) CREATEALIAS should fail if target collections don't exist

[JIRA]

 [ 
https://issues.apache.org/jira/browse/SOLR-9874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tomás Fernández Löbbe resolved SOLR-9874.
-
   Resolution: Fixed
Fix Version/s: 6.4
   master (7.0)

> CREATEALIAS should fail if target collections don't exist
> -
>
> Key: SOLR-9874
> URL: https://issues.apache.org/jira/browse/SOLR-9874
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Tomás Fernández Löbbe
>Assignee: Tomás Fernández Löbbe
>Priority: Minor
> Fix For: master (7.0), 6.4
>
> Attachments: SOLR-9874.patch
>
>
> As discussed 
> [here|http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201612.mbox/%3CCAMJgJxSoY9hovujET0V8D3ywyBf%3DrDZTz9WxZABx-wUYaO4jKg%40mail.gmail.com%3E],
>  we should fail requests to CREATEALIAS if the target collection doesnt't 
> exist



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8593) Integrate Apache Calcite into the SQLHandler

[Julian Hyde (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764709#comment-15764709
 ] 

Julian Hyde commented on SOLR-8593:
---

Yes, early January. I've logged CALCITE-1547 to track the release.

> Integrate Apache Calcite into the SQLHandler
> 
>
> Key: SOLR-8593
> URL: https://issues.apache.org/jira/browse/SOLR-8593
> Project: Solr
>  Issue Type: Improvement
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Attachments: SOLR-8593.patch, SOLR-8593.patch
>
>
>The Presto SQL Parser was perfect for phase one of the SQLHandler. It was 
> nicely split off from the larger Presto project and it did everything that 
> was needed for the initial implementation.
> Phase two of the SQL work though will require an optimizer. Here is where 
> Apache Calcite comes into play. It has a battle tested cost based optimizer 
> and has been integrated into Apache Drill and Hive.
> This work can begin in trunk following the 6.0 release. The final query plans 
> will continue to be translated to Streaming API objects (TupleStreams), so 
> continued work on the JDBC driver should plug in nicely with the Calcite work.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-9847) Deadlock on ManagedIndexSchema lock.

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe resolved SOLR-9847.
--
   Resolution: Fixed
 Assignee: Steve Rowe
Fix Version/s: 6.4
   master (7.0)

Thanks Mark.

> Deadlock on ManagedIndexSchema lock.
> 
>
> Key: SOLR-9847
> URL: https://issues.apache.org/jira/browse/SOLR-9847
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Steve Rowe
> Fix For: master (7.0), 6.4
>
> Attachments: 
> SOLR-9847-release-lock-before-replica-version-agreement.patch, SOLR-9847.patch
>
>
> Seems we hold the lock while in 
> ManagedIndexSchema.waitForSchemaZkVersionAgreement, so we may never see 
> agreement because are updates may also be waiting on that lock.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9847) Deadlock on ManagedIndexSchema lock.

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764692#comment-15764692
 ] 

ASF subversion and git services commented on SOLR-9847:
---

Commit cfbfc13209c57e1c76bd478c3b584135b5d109eb in lucene-solr's branch 
refs/heads/branch_6x from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=cfbfc13 ]

SOLR-9847: Stop blocking further schema updates while waiting for a pending 
update to propagate to other replicas.  This reduces the likelihood of a 
(time-limited) distributed deadlock during concurrent schema updates.


> Deadlock on ManagedIndexSchema lock.
> 
>
> Key: SOLR-9847
> URL: https://issues.apache.org/jira/browse/SOLR-9847
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
> Attachments: 
> SOLR-9847-release-lock-before-replica-version-agreement.patch, SOLR-9847.patch
>
>
> Seems we hold the lock while in 
> ManagedIndexSchema.waitForSchemaZkVersionAgreement, so we may never see 
> agreement because are updates may also be waiting on that lock.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9847) Deadlock on ManagedIndexSchema lock.

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764693#comment-15764693
 ] 

ASF subversion and git services commented on SOLR-9847:
---

Commit 04108d993574537d9623d5d1bf2658cafad12ef8 in lucene-solr's branch 
refs/heads/master from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=04108d9 ]

SOLR-9847: Stop blocking further schema updates while waiting for a pending 
update to propagate to other replicas.  This reduces the likelihood of a 
(time-limited) distributed deadlock during concurrent schema updates.


> Deadlock on ManagedIndexSchema lock.
> 
>
> Key: SOLR-9847
> URL: https://issues.apache.org/jira/browse/SOLR-9847
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
> Attachments: 
> SOLR-9847-release-lock-before-replica-version-agreement.patch, SOLR-9847.patch
>
>
> Seems we hold the lock while in 
> ManagedIndexSchema.waitForSchemaZkVersionAgreement, so we may never see 
> agreement because are updates may also be waiting on that lock.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-7580) Spans tree scoring

[Paul Elschot (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764653#comment-15764653
 ] 

Paul Elschot edited comment on LUCENE-7580 at 12/20/16 4:53 PM:


I have started using this in the tests for the surround query language, I'll 
open an issue for that later.

This found a bug in ConjunctionNearSpansDocScorer.recordMatch().
A -1 slop can occur when the same term is used twice in a SpanNearQuery, and 
this causes a division by zero in computing the slop factor in from 
recordMatch().
This can be easily avoided by using 0 slop in such cases.


was (Author: paul.elsc...@xs4all.nl):
I have started with on using this in the tests for the surround query language, 
I'll open an issue for that later.

This found a bug in ConjunctionNearSpansDocScorer.recordMatch().
A -1 slop can occur when the same term is used twice in a SpanNearQuery, and 
this causes a division by zero in computing the slop factor in from 
recordMatch().
This can be easily avoided by using 0 slop in such cases.

> Spans tree scoring
> --
>
> Key: LUCENE-7580
> URL: https://issues.apache.org/jira/browse/LUCENE-7580
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/search
>Affects Versions: master (7.0)
>Reporter: Paul Elschot
>Priority: Minor
> Fix For: 6.x
>
> Attachments: LUCENE-7580.patch, LUCENE-7580.patch
>
>
> Recurse the spans tree to compose a score based on the type of subqueries and 
> what matched



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7580) Spans tree scoring

[Paul Elschot (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764653#comment-15764653
 ] 

Paul Elschot commented on LUCENE-7580:
--

I have started with on using this in the tests for the surround query language, 
I'll open an issue for that later.

This found a bug in ConjunctionNearSpansDocScorer.recordMatch().
A -1 slop can occur when the same term is used twice in a SpanNearQuery, and 
this causes a division by zero in computing the slop factor in from 
recordMatch().
This can be easily avoided by using 0 slop in such cases.

> Spans tree scoring
> --
>
> Key: LUCENE-7580
> URL: https://issues.apache.org/jira/browse/LUCENE-7580
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/search
>Affects Versions: master (7.0)
>Reporter: Paul Elschot
>Priority: Minor
> Fix For: 6.x
>
> Attachments: LUCENE-7580.patch, LUCENE-7580.patch
>
>
> Recurse the spans tree to compose a score based on the type of subqueries and 
> what matched



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8593) Integrate Apache Calcite into the SQLHandler

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764633#comment-15764633
 ] 

Joel Bernstein commented on SOLR-8593:
--

Ok, that's great. Then I'll just continue working with solr-8593.

> Integrate Apache Calcite into the SQLHandler
> 
>
> Key: SOLR-8593
> URL: https://issues.apache.org/jira/browse/SOLR-8593
> Project: Solr
>  Issue Type: Improvement
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Attachments: SOLR-8593.patch, SOLR-8593.patch
>
>
>The Presto SQL Parser was perfect for phase one of the SQLHandler. It was 
> nicely split off from the larger Presto project and it did everything that 
> was needed for the initial implementation.
> Phase two of the SQL work though will require an optimizer. Here is where 
> Apache Calcite comes into play. It has a battle tested cost based optimizer 
> and has been integrated into Apache Drill and Hive.
> This work can begin in trunk following the 6.0 release. The final query plans 
> will continue to be translated to Streaming API objects (TupleStreams), so 
> continued work on the JDBC driver should plug in nicely with the Calcite work.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8593) Integrate Apache Calcite into the SQLHandler

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764613#comment-15764613
 ] 

Kevin Risden commented on SOLR-8593:


[~joel.bernstein] - the jira/solr-8593 branch is pretty up to date. I have 
merged master into it a few times. 

> Integrate Apache Calcite into the SQLHandler
> 
>
> Key: SOLR-8593
> URL: https://issues.apache.org/jira/browse/SOLR-8593
> Project: Solr
>  Issue Type: Improvement
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Attachments: SOLR-8593.patch, SOLR-8593.patch
>
>
>The Presto SQL Parser was perfect for phase one of the SQLHandler. It was 
> nicely split off from the larger Presto project and it did everything that 
> was needed for the initial implementation.
> Phase two of the SQL work though will require an optimizer. Here is where 
> Apache Calcite comes into play. It has a battle tested cost based optimizer 
> and has been integrated into Apache Drill and Hive.
> This work can begin in trunk following the 6.0 release. The final query plans 
> will continue to be translated to Streaming API objects (TupleStreams), so 
> continued work on the JDBC driver should plug in nicely with the Calcite work.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9847) Deadlock on ManagedIndexSchema lock.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764557#comment-15764557
 ] 

Mark Miller commented on SOLR-9847:
---

bq. Patch that narrows the period during which the schema update lock is held

That was my first inclination, but I don't know how all this works that well 
yet, so I tried to hack something sim with the wait.

bq. does it look good to commit?

+1. I'll report if I run into anything further.

> Deadlock on ManagedIndexSchema lock.
> 
>
> Key: SOLR-9847
> URL: https://issues.apache.org/jira/browse/SOLR-9847
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
> Attachments: 
> SOLR-9847-release-lock-before-replica-version-agreement.patch, SOLR-9847.patch
>
>
> Seems we hold the lock while in 
> ManagedIndexSchema.waitForSchemaZkVersionAgreement, so we may never see 
> agreement because are updates may also be waiting on that lock.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: SOLR 5.5 - Multithread in Dataimporthandler

[Shawn Heisey]

On 12/20/2016 3:43 AM, Vellaimary C wrote:
> My organization is using SOLR for search handling . As we need to
> index more volume of documents like 300 millions, we have moved to
> SOLR 5.5.1.
> To speed up the import, which takes more than three weeks now atleast
> to 1 week we need parallel data import handler triggered.
> Can anyone help me to implement multithreading in dataimport handler.

If it were easy to achieve this, it would have already been done.  DIH
actually used to have a parameter for the number of threads, but it
didn't work, so it was removed.  Implementing multi-threaded support is
*NOT* a trivial undertaking.  If you figure out how to do it, we welcome
patches.

The best option is for you to write your own indexing application that
pulls data from the original source and uses multiple threads to index
the data in parallel.

To achieve this with DIH requires that you create multiple handlers with
different URL paths for names, and start imports on them all that run at
the same time, with "clean=false" so that the imports won't wipe the
index when they start.  Each one needs to handle part of the data in the
source system.

FYI, your question belongs on the solr-user mailing list.  This list is
for discussions around the development of Lucene/Solr itself.

Thanks,
Shawn


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9869) MiniSolrCloudCluster does not always remove jettys from running list after stopping them

[Mike Drob (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764507#comment-15764507
 ] 

Mike Drob commented on SOLR-9869:
-

[~romseygeek] - it looks like you were the one to add this flavor of stop 
method in SOLR-7180. Do you have any thoughts on the patch?

> MiniSolrCloudCluster does not always remove jettys from running list after 
> stopping them
> 
>
> Key: SOLR-9869
> URL: https://issues.apache.org/jira/browse/SOLR-9869
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Reporter: Mike Drob
> Attachments: SOLR-9869.patch
>
>
> MiniSolrCloudCluster has two {{stopJettySolrRunner}} methods that behave 
> differently.
> The {{int}} version calls {{jettys.remove(index);}} to remove the now stopped 
> jetty from the list of running jettys.
> The version that takes a {{JettySolrRunner}}, however, does not modify the 
> running list.
> This can cause calls to {{getReplicaJetty}} to fail after a call to {{stop}} 
> because we will try to get the base url of a stopped jetty.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9513) Introduce a generic authentication plugin which delegates all functionality to Hadoop authentication framework

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764462#comment-15764462
 ] 

ASF subversion and git services commented on SOLR-9513:
---

Commit 791b8bc6bfde2197e52ab01d8ef0feea6e9838b7 in lucene-solr's branch 
refs/heads/branch_6x from [~ichattopadhyaya]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=791b8bc ]

SOLR-9513: Generic Hadoop authentication plugins, GenericHadoopAuthPlugin and 
ConfigurableInternodeAuthHadoopPlugin


> Introduce a generic authentication plugin which delegates all functionality 
> to Hadoop authentication framework
> --
>
> Key: SOLR-9513
> URL: https://issues.apache.org/jira/browse/SOLR-9513
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Hrishikesh Gadre
> Fix For: master (7.0)
>
> Attachments: SOLR-9513.patch, SOLR-9513_6x.patch
>
>
> Currently Solr kerberos authentication plugin delegates the core logic to 
> Hadoop authentication framework. But the configuration parameters required by 
> the Hadoop authentication framework are hardcoded in the plugin code itself. 
> https://github.com/apache/lucene-solr/blob/5b770b56d012279d334f41e4ef7fe652480fd3cf/solr/core/src/java/org/apache/solr/security/KerberosPlugin.java#L119
> The problem with this approach is that we need to make code changes in Solr 
> to expose new capabilities added in Hadoop authentication framework. e.g. 
> HADOOP-12082
> We should implement a generic Solr authentication plugin which will accept 
> configuration parameters via security.json (in Zookeeper) and delegate them 
> to Hadoop authentication framework. This will allow to utilize new features 
> in Hadoop without code changes in Solr.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7598) SolrCore 'recs' is not available due to init failure

[Brian Kmetz (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brian Kmetz updated LUCENE-7598:

Description: 
Receiving the following error message:

null:org.apache.solr.common.SolrException: SolrCore 'recs' is not available due 
to init failure: Error opening new searcher
 at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:783)
 at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:287)
 at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
 at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
 at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
 at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
 at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
 at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
 at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
 at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
 at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
 at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
 at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
 at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
 at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
 at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
 at org.eclipse.jetty.server.Server.handle(Server.java:368)
 at 
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
 at 
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
 at 
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
 at 
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
 at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)
 at 
org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
 at 
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
 at 
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
 at java.lang.Thread.run(Unknown Source)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
 at org.apache.solr.core.SolrCore.(SolrCore.java:834)
 at org.apache.solr.core.SolrCore.(SolrCore.java:625)
 at 
org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:522)
 at org.apache.solr.core.CoreContainer.create(CoreContainer.java:557)
 at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:247)
 at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:239)
 at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
 at java.util.concurrent.FutureTask.run(Unknown Source)
 at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
 at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
 at java.util.concurrent.FutureTask.run(Unknown Source)
 at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown 
Source)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
 ... 1 more
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
 at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1477)
 at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1589)
 at org.apache.solr.core.SolrCore.(SolrCore.java:821)
 ... 13 more
Caused by: java.io.FileNotFoundException: 
C:\Apache\Solr\example\multicore\core0\data\index\_xzz7.si (The system cannot 
find the file specified)
 at java.io.RandomAccessFile.open(Native Method)
 at java.io.RandomAccessFile.(Unknown Source)
 at 
org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:193)
 at 
org.apache.lucene.store.NRTCachingDirectory.openInput(NRTCachingDirectory.java:232)
 at 
org.apache.lucene.codecs.lucene40.Lucene40SegmentInfoReader.read(Lucene40SegmentInfoReader.java:50)
 at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:334)
 at 

[jira] [Commented] (LUCENE-7598) FileNotFoundException: _xzz7_Lucene41_0.tip

[Brian Kmetz (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764387#comment-15764387
 ] 

Brian Kmetz commented on LUCENE-7598:
-

It appears that the "recs" core could not be created - cannot read xml config.

null:org.apache.solr.common.SolrException: SolrCore 'recs' is not available due 
to init failure: Error opening new searcher
at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:783)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:287)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
at org.eclipse.jetty.server.Server.handle(Server.java:368)
at 
org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
at 
org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
at 
org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
at 
org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
at 
org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
at 
org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
at java.lang.Thread.run(Unknown Source)
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.(SolrCore.java:834)
at org.apache.solr.core.SolrCore.(SolrCore.java:625)
at 
org.apache.solr.core.CoreContainer.createFromLocal(CoreContainer.java:522)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:557)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:247)
at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:239)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown 
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
... 1 more
Caused by: org.apache.solr.common.SolrException: Error opening new searcher
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1477)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1589)
at org.apache.solr.core.SolrCore.(SolrCore.java:821)
... 13 more
Caused by: java.io.FileNotFoundException: 
C:\Apache\Solr\example\multicore\core0\data\index\_xzz7.si (The system cannot 
find the file specified)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.(Unknown Source)
at 
org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:193)
at 
org.apache.lucene.codecs.lucene40.Lucene40SegmentInfoReader.read(Lucene40SegmentInfoReader.java:50)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:334)
at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:380)
at 

[jira] [Updated] (LUCENE-7598) FileNotFoundException: _xzz7_Lucene41_0.tip

[Brian Kmetz (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brian Kmetz updated LUCENE-7598:

Priority: Blocker  (was: Critical)

> FileNotFoundException: _xzz7_Lucene41_0.tip
> ---
>
> Key: LUCENE-7598
> URL: https://issues.apache.org/jira/browse/LUCENE-7598
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 4.5.1
> Environment: Windows Server 2008 R2 Standard, 64-bit OS, 8 GB ram, 2 
> core
>Reporter: Brian Kmetz
>Priority: Blocker
>
> Preferable contact: 727-776-1897
> Receiving the following error message:
> FileNotFoundException: _xzz7_Lucene41_0.tip
> 16:46:09
> ERROR
> SolrDispatchFilter
> null:java.lang.RuntimeException: java.io.FileNotFoundException: 
> _xzz7_Lucene41_0.tip
> null:java.lang.RuntimeException: java.io.FileNotFoundException: 
> _xzz7_Lucene41_0.tip
>   at 
> org.apache.lucene.index.TieredMergePolicy$SegmentByteSizeDescending.compare(TieredMergePolicy.java:249)
>   at 
> org.apache.lucene.index.TieredMergePolicy$SegmentByteSizeDescending.compare(TieredMergePolicy.java:235)
>   at java.util.Arrays.mergeSort(Unknown Source)
>   at java.util.Arrays.mergeSort(Unknown Source)
>   at java.util.Arrays.mergeSort(Unknown Source)
>   at java.util.Arrays.mergeSort(Unknown Source)
>   at java.util.Arrays.mergeSort(Unknown Source)
>   at java.util.Arrays.mergeSort(Unknown Source)
>   at java.util.Arrays.mergeSort(Unknown Source)
>   at java.util.Arrays.mergeSort(Unknown Source)
>   at java.util.Arrays.mergeSort(Unknown Source)
>   at java.util.Arrays.sort(Unknown Source)
>   at java.util.Collections.sort(Unknown Source)
>   at 
> org.apache.lucene.index.TieredMergePolicy.findMerges(TieredMergePolicy.java:279)
>   at 
> org.apache.lucene.index.IndexWriter.updatePendingMerges(IndexWriter.java:1952)
>   at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1921)
>   at 
> org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2860)
>   at 
> org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2965)
>   at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2935)
>   at 
> org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:559)
>   at 
> org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95)
>   at 
> org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
>   at 
> org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1242)
>   at 
> org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1221)
>   at 
> org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:157)
>   at 
> org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:69)
>   at 
> org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
>   at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
>   at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:703)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:406)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:195)
>   at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
>   at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
>   at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
>   at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
>   at 
> 

[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-9-ea+147) - Build # 18574 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18574/
Java: 64bit/jdk-9-ea+147 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

3 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.security.hadoop.TestSolrCloudWithHadoopAuthPlugin

Error Message:
FIXME: SOLR-8182: This test fails under Java 9

Stack Trace:
com.carrotsearch.randomizedtesting.InternalAssumptionViolatedException: FIXME: 
SOLR-8182: This test fails under Java 9
at __randomizedtesting.SeedInfo.seed([9E9CE6EF38364F59]:0)
at 
com.carrotsearch.randomizedtesting.RandomizedTest.assumeTrue(RandomizedTest.java:667)
at 
com.carrotsearch.randomizedtesting.RandomizedTest.assumeFalse(RandomizedTest.java:675)
at 
org.apache.lucene.util.LuceneTestCase.assumeFalse(LuceneTestCase.java:850)
at 
org.apache.solr.security.hadoop.TestSolrCloudWithHadoopAuthPlugin.setupClass(TestSolrCloudWithHadoopAuthPlugin.java:45)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:538)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:847)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  
junit.framework.TestSuite.org.apache.solr.security.hadoop.TestSolrCloudWithHadoopAuthPlugin

Error Message:


Stack Trace:
java.lang.NullPointerException
at __randomizedtesting.SeedInfo.seed([9E9CE6EF38364F59]:0)
at 
org.apache.solr.security.hadoop.TestSolrCloudWithHadoopAuthPlugin.tearDownClass(TestSolrCloudWithHadoopAuthPlugin.java:62)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:538)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 

[jira] [Commented] (SOLR-3274) ZooKeeper related SolrCloud problems

[Yago Riveiro (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764336#comment-15764336
 ] 

Yago Riveiro commented on SOLR-3274:


I hitting this in 6.3.0 a lot and I don't know why, my TTL for zookeeper is 
120s and I had no log into the gc log with pauses higher than 100ms

Exists some configuration to see the reason for the failure talking with 
ZooKeeper? like connection timeout or something else?

org.apache.solr.common.SolrException: Cannot talk to ZooKeeper - Updates are 
disabled.
at 
org.apache.solr.update.processor.DistributedUpdateProcessor.zkCheck(DistributedUpdateProcessor.java:1508)
at 
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:696)
at 
org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:97)
at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:179)
at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readIterator(JavaBinUpdateRequestCodec.java:135)
at 
org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:275)
at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readNamedList(JavaBinUpdateRequestCodec.java:121)
at 
org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:240)
at 
org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:158)
at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:186)
at 
org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:107)
at 
org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:54)
at 
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)
at 
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2213)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:303)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:254)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at 
org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:169)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:518)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at 
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(Thread.java:745)

> ZooKeeper related SolrCloud problems
> 
>
> Key: SOLR-3274
>   

[jira] [Commented] (SOLR-8593) Integrate Apache Calcite into the SQLHandler

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764265#comment-15764265
 ] 

Joel Bernstein commented on SOLR-8593:
--

I saw on the Calcite dev list that the next release is coming pretty soon. So, 
I'm not going to move ahead creating another ticket for the HavingStream.

I'm planning to create a new branch from master locally and merge this branch 
into to it so I can test with the latest master code. Master has some 
significant changes both in Lucene and Solr that effects SQL. Specifically the 
/export handler has been refactored and there is new docValues iterator API to 
performance test. This will likely take up the rest of the year me. But 
hopefully after the new Calcite releases we will be ready to merge to master.





> Integrate Apache Calcite into the SQLHandler
> 
>
> Key: SOLR-8593
> URL: https://issues.apache.org/jira/browse/SOLR-8593
> Project: Solr
>  Issue Type: Improvement
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Attachments: SOLR-8593.patch, SOLR-8593.patch
>
>
>The Presto SQL Parser was perfect for phase one of the SQLHandler. It was 
> nicely split off from the larger Presto project and it did everything that 
> was needed for the initial implementation.
> Phase two of the SQL work though will require an optimizer. Here is where 
> Apache Calcite comes into play. It has a battle tested cost based optimizer 
> and has been integrated into Apache Drill and Hive.
> This work can begin in trunk following the 6.0 release. The final query plans 
> will continue to be translated to Streaming API objects (TupleStreams), so 
> continued work on the JDBC driver should plug in nicely with the Calcite work.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (32bit/jdk1.8.0_112) - Build # 6298 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6298/
Java: 32bit/jdk1.8.0_112 -client -XX:+UseG1GC

2 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.security.hadoop.TestSolrCloudWithHadoopAuthPlugin

Error Message:
Hadoop does not work on Windows

Stack Trace:
com.carrotsearch.randomizedtesting.InternalAssumptionViolatedException: Hadoop 
does not work on Windows
at __randomizedtesting.SeedInfo.seed([E8D9B080B39E6011]:0)
at 
com.carrotsearch.randomizedtesting.RandomizedTest.assumeTrue(RandomizedTest.java:667)
at 
com.carrotsearch.randomizedtesting.RandomizedTest.assumeFalse(RandomizedTest.java:675)
at 
org.apache.lucene.util.LuceneTestCase.assumeFalse(LuceneTestCase.java:850)
at 
org.apache.solr.security.hadoop.TestSolrCloudWithHadoopAuthPlugin.setupClass(TestSolrCloudWithHadoopAuthPlugin.java:44)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:847)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)


FAILED:  
junit.framework.TestSuite.org.apache.solr.security.hadoop.TestSolrCloudWithHadoopAuthPlugin

Error Message:


Stack Trace:
java.lang.NullPointerException
at __randomizedtesting.SeedInfo.seed([E8D9B080B39E6011]:0)
at 
org.apache.solr.security.hadoop.TestSolrCloudWithHadoopAuthPlugin.tearDownClass(TestSolrCloudWithHadoopAuthPlugin.java:62)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Commented] (SOLR-4735) Improve Solr metrics reporting

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764246#comment-15764246
 ] 

ASF subversion and git services commented on SOLR-4735:
---

Commit b37a72d941dba705b43c8868584bd9df775a1537 in lucene-solr's branch 
refs/heads/master from [~ab]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b37a72d ]

SOLR-4735 Use overridableRegistryName also for predefined shared registries.
Cleanup + javadocs.


> Improve Solr metrics reporting
> --
>
> Key: SOLR-4735
> URL: https://issues.apache.org/jira/browse/SOLR-4735
> Project: Solr
>  Issue Type: Improvement
>  Components: metrics
>Reporter: Alan Woodward
>Assignee: Andrzej Bialecki 
>Priority: Minor
> Fix For: master (7.0)
>
> Attachments: SOLR-4735.patch, SOLR-4735.patch, SOLR-4735.patch, 
> SOLR-4735.patch, SOLR-4735.patch, SOLR-4735.patch, screenshot-2.png
>
>
> Following on from a discussion on the mailing list:
> http://search-lucene.com/m/IO0EI1qdyJF1/codahale=Solr+metrics+in+Codahale+metrics+and+Graphite+
> It would be good to make Solr play more nicely with existing devops 
> monitoring systems, such as Graphite or Ganglia.  Stats monitoring at the 
> moment is poll-only, either via JMX or through the admin stats page.  I'd 
> like to refactor things a bit to make this more pluggable.
> This patch is a start.  It adds a new interface, InstrumentedBean, which 
> extends SolrInfoMBean to return a 
> [[Metrics|http://metrics.codahale.com/manual/core/]] MetricRegistry, and a 
> couple of MetricReporters (which basically just duplicate the JMX and admin 
> page reporting that's there at the moment, but which should be more 
> extensible).  The patch includes a change to RequestHandlerBase showing how 
> this could work.  The idea would be to eventually replace the getStatistics() 
> call on SolrInfoMBean with this instead.
> The next step would be to allow more MetricReporters to be defined in 
> solrconfig.xml.  The Metrics library comes with ganglia and graphite 
> reporting modules, and we can add contrib plugins for both of those.
> There's some more general cleanup that could be done around SolrInfoMBean 
> (we've got two plugin handlers at /mbeans and /plugins that basically do the 
> same thing, and the beans themselves have some weirdly inconsistent data on 
> them - getVersion() returns different things for different impls, and 
> getSource() seems pretty useless), but maybe that's for another issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7579) Sorting on flushed segment

[Ferenczi Jim (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764205#comment-15764205
 ] 

Ferenczi Jim commented on LUCENE-7579:
--

{quote}
Maybe you can work on the 6.x back port in the meantime 
{quote}

I am on it !

> Sorting on flushed segment
> --
>
> Key: LUCENE-7579
> URL: https://issues.apache.org/jira/browse/LUCENE-7579
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ferenczi Jim
>
> Today flushed segments built by an index writer with an index sort specified 
> are not sorted. The merge is responsible of sorting these segments 
> potentially with others that are already sorted (resulted from another 
> merge). 
> I'd like to investigate the cost of sorting the segment directly during the 
> flush. This could make the merge faster since they are some cheap 
> optimizations that can be done only if all segments to be merged are sorted.
>  For instance the merge of the points could use the bulk merge instead of 
> rebuilding the points from scratch.
> I made a small prototype which sort the segment on flush here:
> https://github.com/apache/lucene-solr/compare/master...jimczi:flush_sort
> The idea is simple, for points, norms, docvalues and terms I use the 
> SortingLeafReader implementation to translate the values that we have in RAM 
> in a sorted enumeration for the writers.
> For stored fields I use a two pass scheme where the documents are first 
> written to disk unsorted and then copied to another file with the correct 
> sorting. I use the same stored field format for the two steps and just remove 
> the file produced by the first pass at the end of the process.
> This prototype has no implementation for index sorting that use term vectors 
> yet. I'll add this later if the tests are good enough.
> Speaking of testing, I tried this branch on [~mikemccand] benchmark scripts 
> and compared master with index sorting against my branch with index sorting 
> on flush. I tried with sparsetaxis and wikipedia and the first results are 
> weird. When I use the SerialScheduler and only one thread to write the docs,  
> index sorting on flush is slower. But when I use two threads the sorting on 
> flush is much faster even with the SerialScheduler. I'll continue to run the 
> tests in order to be able to share something more meaningful.
> The tests are passing except one about concurrent DV updates. I don't know 
> this part at all so I did not fix the test yet. I don't even know if we can 
> make it work with index sorting ;).
>  [~mikemccand] I would love to have your feedback about the prototype. Could 
> you please take a look ? I am sure there are plenty of bugs, ... but I think 
> it's a good start to evaluate the feasibility of this feature.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-7115) UpdateLog can miss closing transaction log objects.

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764127#comment-15764127
 ] 

Yonik Seeley edited comment on SOLR-7115 at 12/20/16 12:44 PM:
---

I thought about that... but calling postCommit assumes a new searcher has been 
opened, so it doesn't seem right.
It seems like we should have explicit failure callbacks... commitFailed and 
openSearcherFailed (or add a boolean "success" flag to the post methods)?


was (Author: ysee...@gmail.com):
I thought about that... but calling postCommit assumes a new searcher has been 
opened, so it doesn't seem right.
It seems like we should have explicit failure callbacks... commitFailed and 
openSearcherFailed?

> UpdateLog can miss closing transaction log objects.
> ---
>
> Key: SOLR-7115
> URL: https://issues.apache.org/jira/browse/SOLR-7115
> Project: Solr
>  Issue Type: Bug
>Reporter: Mark Miller
>Assignee: Yonik Seeley
> Fix For: 6.x, master (7.0)
>
> Attachments: SOLR-7115-LargeVolumeEmbeddedTest-fail.txt, 
> SOLR-7115.patch, SOLR-7115.patch, tests-failures-7115.txt
>
>
> I've seen this happen on YourKit and in various tests - especially since 
> adding resource release tracking to the log objects. Now I've got a test that 
> catches it in SOLR-7113.
> It seems that in precommit, if prevTlog is not null, we need to close it 
> because we are going to overwrite prevTlog with a new log.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7115) UpdateLog can miss closing transaction log objects.

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764127#comment-15764127
 ] 

Yonik Seeley commented on SOLR-7115:


I thought about that... but calling postCommit assumes a new searcher has been 
opened, so it doesn't seem right.
It seems like we should have explicit failure callbacks... commitFailed and 
openSearcherFailed?

> UpdateLog can miss closing transaction log objects.
> ---
>
> Key: SOLR-7115
> URL: https://issues.apache.org/jira/browse/SOLR-7115
> Project: Solr
>  Issue Type: Bug
>Reporter: Mark Miller
>Assignee: Yonik Seeley
> Fix For: 6.x, master (7.0)
>
> Attachments: SOLR-7115-LargeVolumeEmbeddedTest-fail.txt, 
> SOLR-7115.patch, SOLR-7115.patch, tests-failures-7115.txt
>
>
> I've seen this happen on YourKit and in various tests - especially since 
> adding resource release tracking to the log objects. Now I've got a test that 
> catches it in SOLR-7113.
> It seems that in precommit, if prevTlog is not null, we need to close it 
> because we are going to overwrite prevTlog with a new log.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7115) UpdateLog can miss closing transaction log objects.

[Mikhail Khludnev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764087#comment-15764087
 ] 

Mikhail Khludnev commented on SOLR-7115:


[~ysee...@gmail.com]
bq. I don't know if the patch to DUH2 is needed for other reasons

SOLR-7912 prevents max Warming Searcher exception. But any other exception 
might lead to skipping line 
https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/update/DirectUpdateHandler2.java#L646
 and leaking transaction log. Isn't it's worth to be protected with finally?

> UpdateLog can miss closing transaction log objects.
> ---
>
> Key: SOLR-7115
> URL: https://issues.apache.org/jira/browse/SOLR-7115
> Project: Solr
>  Issue Type: Bug
>Reporter: Mark Miller
>Assignee: Yonik Seeley
> Fix For: 6.x, master (7.0)
>
> Attachments: SOLR-7115-LargeVolumeEmbeddedTest-fail.txt, 
> SOLR-7115.patch, SOLR-7115.patch, tests-failures-7115.txt
>
>
> I've seen this happen on YourKit and in various tests - especially since 
> adding resource release tracking to the log objects. Now I've got a test that 
> catches it in SOLR-7113.
> It seems that in precommit, if prevTlog is not null, we need to close it 
> because we are going to overwrite prevTlog with a new log.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9880) Add Ganglia and Graphite metrics reporters

[Andrzej Bialecki (JIRA)]

Andrzej Bialecki  created SOLR-9880:
---

 Summary: Add Ganglia and Graphite metrics reporters
 Key: SOLR-9880
 URL: https://issues.apache.org/jira/browse/SOLR-9880
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
  Components: metrics
Reporter: Andrzej Bialecki 
Assignee: Andrzej Bialecki 
Priority: Minor
 Fix For: master (7.0)


Originally SOLR-4735 provided implementations for these reporters (wrappers for 
Dropwizard components to use {{SolrMetricReporter}} API).

However, this functionality has been split into its own issue due to the 
additional transitive dependencies that these reporters bring:

* Ganglia:
** metrics-ganglia, ASL, 3kB
** gmetric4j (Ganglia RPC implementation), BSD, 29kB

* Graphite
** metrics-graphite, ASL, 10kB
** amqp-client (RabbitMQ Java client, marked optional in pom?), ASL/MIT/GPL2, 
190kB

IMHO these are not very large dependencies, and given the useful functionality 
they provide it's worth adding them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Git notifications for work-in-progress branches

[Michael McCandless]

+1 to not have silly notification friction for working on branches.

Maybe open a INFRA issue about not sending notification emails in the
"git merge master" case?

Mike McCandless

http://blog.mikemccandless.com


On Tue, Dec 20, 2016 at 6:36 AM, Andrzej Białecki  wrote:
> Hi devs,
>
> First, let me apologize for the tons of notification emails that the
> commits@ list has received recently due to my changes - I’m new to the git
> workflow at Apache, and it looks like I made a few missteps…
>
> Still, I think there’s some room for improvement to the current notification
> model, so that it accommodates for work in progress commits, and most
> importantly occasional merging / rebasing from other branches:
>
> * first, Shalin and I have used a ‘feature/metrics’ branch. As David Smiley
> kindly pointed out this branch doesn’t follow the ‘lucene-‘ or ‘solr-‘
> naming scheme so the git bot didn’t know to suppress the notifications on
> commits there. This should be fixed once INFRA-13133 is resolved (and
> probably a note should be added to this effect in the git guide for
> committers),
>
> * but even for a branch that follows this naming (jira/solr-9854) merging
> from master produced as many emails as there were commits between branches.
> Branching is a low cost operation in git (unlike in svn), so creating
> branches for even short-term work is attractive because it exposes work in
> progress to others and lets them collaborate. Longer-lived branches need
> occasional merging / rebasing so that they are reasonably up to date with
> master, but now each time this happens it will produce an avalanche of
> emails… The alternative would be to always merge —squash from master to a
> feature branch, so that it creates a single commit, but then it’s harder to
> merge it back and to track changes from master that may have affected your
> work.
>
> So, I don’t think this level of detail is useful for work in progress
> branches. Is there any way for the bot to eg. combine notifications from
> these branches into a single email, a la ‘git diff —stat’ ?
>
> ---
> Best regards,
>
> Andrzej Bialecki
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-4735) Improve Solr metrics reporting

[Andrzej Bialecki (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrzej Bialecki  updated SOLR-4735:

Attachment: SOLR-4735.patch

Uploaded final diff between the branch and master, for completeness.

> Improve Solr metrics reporting
> --
>
> Key: SOLR-4735
> URL: https://issues.apache.org/jira/browse/SOLR-4735
> Project: Solr
>  Issue Type: Improvement
>  Components: metrics
>Reporter: Alan Woodward
>Assignee: Andrzej Bialecki 
>Priority: Minor
> Fix For: master (7.0)
>
> Attachments: SOLR-4735.patch, SOLR-4735.patch, SOLR-4735.patch, 
> SOLR-4735.patch, SOLR-4735.patch, SOLR-4735.patch, screenshot-2.png
>
>
> Following on from a discussion on the mailing list:
> http://search-lucene.com/m/IO0EI1qdyJF1/codahale=Solr+metrics+in+Codahale+metrics+and+Graphite+
> It would be good to make Solr play more nicely with existing devops 
> monitoring systems, such as Graphite or Ganglia.  Stats monitoring at the 
> moment is poll-only, either via JMX or through the admin stats page.  I'd 
> like to refactor things a bit to make this more pluggable.
> This patch is a start.  It adds a new interface, InstrumentedBean, which 
> extends SolrInfoMBean to return a 
> [[Metrics|http://metrics.codahale.com/manual/core/]] MetricRegistry, and a 
> couple of MetricReporters (which basically just duplicate the JMX and admin 
> page reporting that's there at the moment, but which should be more 
> extensible).  The patch includes a change to RequestHandlerBase showing how 
> this could work.  The idea would be to eventually replace the getStatistics() 
> call on SolrInfoMBean with this instead.
> The next step would be to allow more MetricReporters to be defined in 
> solrconfig.xml.  The Metrics library comes with ganglia and graphite 
> reporting modules, and we can add contrib plugins for both of those.
> There's some more general cleanup that could be done around SolrInfoMBean 
> (we've got two plugin handlers at /mbeans and /plugins that basically do the 
> same thing, and the beans themselves have some weirdly inconsistent data on 
> them - getVersion() returns different things for different impls, and 
> getSource() seems pretty useless), but maybe that's for another issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-4735) Improve Solr metrics reporting

[Andrzej Bialecki (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-4735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrzej Bialecki  updated SOLR-4735:

Fix Version/s: master (7.0)

> Improve Solr metrics reporting
> --
>
> Key: SOLR-4735
> URL: https://issues.apache.org/jira/browse/SOLR-4735
> Project: Solr
>  Issue Type: Improvement
>  Components: metrics
>Reporter: Alan Woodward
>Assignee: Andrzej Bialecki 
>Priority: Minor
> Fix For: master (7.0)
>
> Attachments: SOLR-4735.patch, SOLR-4735.patch, SOLR-4735.patch, 
> SOLR-4735.patch, SOLR-4735.patch, SOLR-4735.patch, screenshot-2.png
>
>
> Following on from a discussion on the mailing list:
> http://search-lucene.com/m/IO0EI1qdyJF1/codahale=Solr+metrics+in+Codahale+metrics+and+Graphite+
> It would be good to make Solr play more nicely with existing devops 
> monitoring systems, such as Graphite or Ganglia.  Stats monitoring at the 
> moment is poll-only, either via JMX or through the admin stats page.  I'd 
> like to refactor things a bit to make this more pluggable.
> This patch is a start.  It adds a new interface, InstrumentedBean, which 
> extends SolrInfoMBean to return a 
> [[Metrics|http://metrics.codahale.com/manual/core/]] MetricRegistry, and a 
> couple of MetricReporters (which basically just duplicate the JMX and admin 
> page reporting that's there at the moment, but which should be more 
> extensible).  The patch includes a change to RequestHandlerBase showing how 
> this could work.  The idea would be to eventually replace the getStatistics() 
> call on SolrInfoMBean with this instead.
> The next step would be to allow more MetricReporters to be defined in 
> solrconfig.xml.  The Metrics library comes with ganglia and graphite 
> reporting modules, and we can add contrib plugins for both of those.
> There's some more general cleanup that could be done around SolrInfoMBean 
> (we've got two plugin handlers at /mbeans and /plugins that basically do the 
> same thing, and the beans themselves have some weirdly inconsistent data on 
> them - getVersion() returns different things for different impls, and 
> getSource() seems pretty useless), but maybe that's for another issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+147) - Build # 18573 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18573/
Java: 32bit/jdk-9-ea+147 -server -XX:+UseG1GC

3 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.security.hadoop.TestSolrCloudWithHadoopAuthPlugin

Error Message:
FIXME: SOLR-8182: This test fails under Java 9

Stack Trace:
com.carrotsearch.randomizedtesting.InternalAssumptionViolatedException: FIXME: 
SOLR-8182: This test fails under Java 9
at __randomizedtesting.SeedInfo.seed([74448FB1070B62E2]:0)
at 
com.carrotsearch.randomizedtesting.RandomizedTest.assumeTrue(RandomizedTest.java:667)
at 
com.carrotsearch.randomizedtesting.RandomizedTest.assumeFalse(RandomizedTest.java:675)
at 
org.apache.lucene.util.LuceneTestCase.assumeFalse(LuceneTestCase.java:850)
at 
org.apache.solr.security.hadoop.TestSolrCloudWithHadoopAuthPlugin.setupClass(TestSolrCloudWithHadoopAuthPlugin.java:45)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:538)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:847)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  
junit.framework.TestSuite.org.apache.solr.security.hadoop.TestSolrCloudWithHadoopAuthPlugin

Error Message:


Stack Trace:
java.lang.NullPointerException
at __randomizedtesting.SeedInfo.seed([74448FB1070B62E2]:0)
at 
org.apache.solr.security.hadoop.TestSolrCloudWithHadoopAuthPlugin.tearDownClass(TestSolrCloudWithHadoopAuthPlugin.java:62)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:538)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 

[jira] [Commented] (LUCENE-7579) Sorting on flushed segment

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764013#comment-15764013
 ] 

Michael McCandless commented on LUCENE-7579:


Thank you [~jim.ferenczi]!  I just pushed your last (squashed) commits to 
master ... let's let it bake for a while.  Maybe you can work on the 6.x back 
port in the meantime :)

> Sorting on flushed segment
> --
>
> Key: LUCENE-7579
> URL: https://issues.apache.org/jira/browse/LUCENE-7579
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ferenczi Jim
>
> Today flushed segments built by an index writer with an index sort specified 
> are not sorted. The merge is responsible of sorting these segments 
> potentially with others that are already sorted (resulted from another 
> merge). 
> I'd like to investigate the cost of sorting the segment directly during the 
> flush. This could make the merge faster since they are some cheap 
> optimizations that can be done only if all segments to be merged are sorted.
>  For instance the merge of the points could use the bulk merge instead of 
> rebuilding the points from scratch.
> I made a small prototype which sort the segment on flush here:
> https://github.com/apache/lucene-solr/compare/master...jimczi:flush_sort
> The idea is simple, for points, norms, docvalues and terms I use the 
> SortingLeafReader implementation to translate the values that we have in RAM 
> in a sorted enumeration for the writers.
> For stored fields I use a two pass scheme where the documents are first 
> written to disk unsorted and then copied to another file with the correct 
> sorting. I use the same stored field format for the two steps and just remove 
> the file produced by the first pass at the end of the process.
> This prototype has no implementation for index sorting that use term vectors 
> yet. I'll add this later if the tests are good enough.
> Speaking of testing, I tried this branch on [~mikemccand] benchmark scripts 
> and compared master with index sorting against my branch with index sorting 
> on flush. I tried with sparsetaxis and wikipedia and the first results are 
> weird. When I use the SerialScheduler and only one thread to write the docs,  
> index sorting on flush is slower. But when I use two threads the sorting on 
> flush is much faster even with the SerialScheduler. I'll continue to run the 
> tests in order to be able to share something more meaningful.
> The tests are passing except one about concurrent DV updates. I don't know 
> this part at all so I did not fix the test yet. I don't even know if we can 
> make it work with index sorting ;).
>  [~mikemccand] I would love to have your feedback about the prototype. Could 
> you please take a look ? I am sure there are plenty of bugs, ... but I think 
> it's a good start to evaluate the feasibility of this feature.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7579) Sorting on flushed segment

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764010#comment-15764010
 ] 

ASF subversion and git services commented on LUCENE-7579:
-

Commit 4ccb9fbd2bbc3afd075aa4bc2b6118f845ea4726 in lucene-solr's branch 
refs/heads/master from Mike McCandless
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4ccb9fb ]

LUCENE-7579: sort segments at flush too


> Sorting on flushed segment
> --
>
> Key: LUCENE-7579
> URL: https://issues.apache.org/jira/browse/LUCENE-7579
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ferenczi Jim
>
> Today flushed segments built by an index writer with an index sort specified 
> are not sorted. The merge is responsible of sorting these segments 
> potentially with others that are already sorted (resulted from another 
> merge). 
> I'd like to investigate the cost of sorting the segment directly during the 
> flush. This could make the merge faster since they are some cheap 
> optimizations that can be done only if all segments to be merged are sorted.
>  For instance the merge of the points could use the bulk merge instead of 
> rebuilding the points from scratch.
> I made a small prototype which sort the segment on flush here:
> https://github.com/apache/lucene-solr/compare/master...jimczi:flush_sort
> The idea is simple, for points, norms, docvalues and terms I use the 
> SortingLeafReader implementation to translate the values that we have in RAM 
> in a sorted enumeration for the writers.
> For stored fields I use a two pass scheme where the documents are first 
> written to disk unsorted and then copied to another file with the correct 
> sorting. I use the same stored field format for the two steps and just remove 
> the file produced by the first pass at the end of the process.
> This prototype has no implementation for index sorting that use term vectors 
> yet. I'll add this later if the tests are good enough.
> Speaking of testing, I tried this branch on [~mikemccand] benchmark scripts 
> and compared master with index sorting against my branch with index sorting 
> on flush. I tried with sparsetaxis and wikipedia and the first results are 
> weird. When I use the SerialScheduler and only one thread to write the docs,  
> index sorting on flush is slower. But when I use two threads the sorting on 
> flush is much faster even with the SerialScheduler. I'll continue to run the 
> tests in order to be able to share something more meaningful.
> The tests are passing except one about concurrent DV updates. I don't know 
> this part at all so I did not fix the test yet. I don't even know if we can 
> make it work with index sorting ;).
>  [~mikemccand] I would love to have your feedback about the prototype. Could 
> you please take a look ? I am sure there are plenty of bugs, ... but I think 
> it's a good start to evaluate the feasibility of this feature.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Announcing Marple, an index-structure visualiser for Apache Lucene

[Alan Woodward]

Hi all,

At the London and Boston hackdays a couple of months back, I started to work on 
an upgrade/replacement for Luke.  My idea was to have something that would 
provide a read-only view of an index, which would be useful both for inspection 
and debugging of indexes, and as a training tool.  I also wanted it to display 
via a browser, so that it could be used on remote servers, and to use only 
Apache-licensed components, with a view to eventually contributing it back to 
the lucene core project.

My colleague Tom Mortimer and I have been working on this on-and-off for a few 
weeks now, and it’s in a sort of nearly-ready state - I’ve already found it 
useful on a couple of client projects - so I thought the list would be 
interested in trying it out.  It’s by no means finished, and there are quite a 
few rough edges and probably some exciting bugs.  But please feel free to poke 
around and open issues (and submit pull requests!).

The application divides up into two parts: a dropwizard-based webapp that 
exposes the various parts of the index (fields, terms, postings, points, 
docvalues, documents, etc) as json; and a React-based UI that makes this 
navigable.  The first part is nearly feature-complete, although there are a few 
missing bits (I haven’t done anything with termvectors yet, for example, or 
index statistics).  The UI is still missing panels for postings lists and for 
points, but everything else is mostly there.

The repository is here: https://github.com/flaxsearch/marple 


Please download it, play around with it, and let me know what you think!

Thanks to Steve Rowe, Shinichiro Abe and Eric Pugh for some early 
contributions, and to Andrzej Białecki for convincing me that it wasn’t a dumb 
idea.

Alan Woodward
www.flax.co.uk 

[jira] [Commented] (LUCENE-7579) Sorting on flushed segment

[Ferenczi Jim (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15764001#comment-15764001
 ] 

Ferenczi Jim commented on LUCENE-7579:
--

Thanks [~jpountz] and [~mikemccand] !

> Sorting on flushed segment
> --
>
> Key: LUCENE-7579
> URL: https://issues.apache.org/jira/browse/LUCENE-7579
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ferenczi Jim
>
> Today flushed segments built by an index writer with an index sort specified 
> are not sorted. The merge is responsible of sorting these segments 
> potentially with others that are already sorted (resulted from another 
> merge). 
> I'd like to investigate the cost of sorting the segment directly during the 
> flush. This could make the merge faster since they are some cheap 
> optimizations that can be done only if all segments to be merged are sorted.
>  For instance the merge of the points could use the bulk merge instead of 
> rebuilding the points from scratch.
> I made a small prototype which sort the segment on flush here:
> https://github.com/apache/lucene-solr/compare/master...jimczi:flush_sort
> The idea is simple, for points, norms, docvalues and terms I use the 
> SortingLeafReader implementation to translate the values that we have in RAM 
> in a sorted enumeration for the writers.
> For stored fields I use a two pass scheme where the documents are first 
> written to disk unsorted and then copied to another file with the correct 
> sorting. I use the same stored field format for the two steps and just remove 
> the file produced by the first pass at the end of the process.
> This prototype has no implementation for index sorting that use term vectors 
> yet. I'll add this later if the tests are good enough.
> Speaking of testing, I tried this branch on [~mikemccand] benchmark scripts 
> and compared master with index sorting against my branch with index sorting 
> on flush. I tried with sparsetaxis and wikipedia and the first results are 
> weird. When I use the SerialScheduler and only one thread to write the docs,  
> index sorting on flush is slower. But when I use two threads the sorting on 
> flush is much faster even with the SerialScheduler. I'll continue to run the 
> tests in order to be able to share something more meaningful.
> The tests are passing except one about concurrent DV updates. I don't know 
> this part at all so I did not fix the test yet. I don't even know if we can 
> make it work with index sorting ;).
>  [~mikemccand] I would love to have your feedback about the prototype. Could 
> you please take a look ? I am sure there are plenty of bugs, ... but I think 
> it's a good start to evaluate the feasibility of this feature.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Git notifications for work-in-progress branches

[Andrzej Białecki]

Hi devs,

First, let me apologize for the tons of notification emails that the commits@ 
list has received recently due to my changes - I’m new to the git workflow at 
Apache, and it looks like I made a few missteps…

Still, I think there’s some room for improvement to the current notification 
model, so that it accommodates for work in progress commits, and most 
importantly occasional merging / rebasing from other branches:

* first, Shalin and I have used a ‘feature/metrics’ branch. As David Smiley 
kindly pointed out this branch doesn’t follow the ‘lucene-‘ or ‘solr-‘ naming 
scheme so the git bot didn’t know to suppress the notifications on commits 
there. This should be fixed once INFRA-13133 is resolved (and probably a note 
should be added to this effect in the git guide for committers),

* but even for a branch that follows this naming (jira/solr-9854) merging from 
master produced as many emails as there were commits between branches. 
Branching is a low cost operation in git (unlike in svn), so creating branches 
for even short-term work is attractive because it exposes work in progress to 
others and lets them collaborate. Longer-lived branches need occasional merging 
/ rebasing so that they are reasonably up to date with master, but now each 
time this happens it will produce an avalanche of emails… The alternative would 
be to always merge —squash from master to a feature branch, so that it creates 
a single commit, but then it’s harder to merge it back and to track changes 
from master that may have affected your work.

So, I don’t think this level of detail is useful for work in progress branches. 
Is there any way for the bot to eg. combine notifications from these branches 
into a single email, a la ‘git diff —stat’ ?

---
Best regards,

Andrzej Bialecki

SOLR 5.5 - Multithread in Dataimporthandler

[Vellaimary C]

Hi Team,

My organization is using SOLR for search handling . As we need to index
more volume of documents like 300 millions, we have moved to SOLR 5.5.1.

To speed up the import, which takes more than three weeks now atleast to 1
week we need parallel data import handler triggered.
Can anyone help me to implement multithreading in dataimport handler.

Please help me to achieve it.
Thanks in advance for any help.

-- 
Thanks & Regards,
C.Vellaimary

SOLR 5.5 - Multithread in Dataimporthandler

[Vellaimary C]

Hi Team,

I'm using SOLR 5.5.1 for document indexing.





I wish to import 300 million documents from database using paraller data
import handler triggered .
So that I wish to implement multithreading in dataimport handler. such as
threads=4 . Please help me to achieve it.



Thanks in advance for any help.


-- 
Thanks & Regards,
C.Vellaimary

[jira] [Commented] (LUCENE-7579) Sorting on flushed segment

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15763899#comment-15763899
 ] 

Adrien Grand commented on LUCENE-7579:
--

bq. I think (later, separate issue) we could use a more naive stored fields 
(and term vectors) format for the temp files written at flush ... a format that 
does no compression, just writes bytes to disk, maybe has simple in-memory 
array pointing to offset in the file for each document ...

+1

> Sorting on flushed segment
> --
>
> Key: LUCENE-7579
> URL: https://issues.apache.org/jira/browse/LUCENE-7579
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ferenczi Jim
>
> Today flushed segments built by an index writer with an index sort specified 
> are not sorted. The merge is responsible of sorting these segments 
> potentially with others that are already sorted (resulted from another 
> merge). 
> I'd like to investigate the cost of sorting the segment directly during the 
> flush. This could make the merge faster since they are some cheap 
> optimizations that can be done only if all segments to be merged are sorted.
>  For instance the merge of the points could use the bulk merge instead of 
> rebuilding the points from scratch.
> I made a small prototype which sort the segment on flush here:
> https://github.com/apache/lucene-solr/compare/master...jimczi:flush_sort
> The idea is simple, for points, norms, docvalues and terms I use the 
> SortingLeafReader implementation to translate the values that we have in RAM 
> in a sorted enumeration for the writers.
> For stored fields I use a two pass scheme where the documents are first 
> written to disk unsorted and then copied to another file with the correct 
> sorting. I use the same stored field format for the two steps and just remove 
> the file produced by the first pass at the end of the process.
> This prototype has no implementation for index sorting that use term vectors 
> yet. I'll add this later if the tests are good enough.
> Speaking of testing, I tried this branch on [~mikemccand] benchmark scripts 
> and compared master with index sorting against my branch with index sorting 
> on flush. I tried with sparsetaxis and wikipedia and the first results are 
> weird. When I use the SerialScheduler and only one thread to write the docs,  
> index sorting on flush is slower. But when I use two threads the sorting on 
> flush is much faster even with the SerialScheduler. I'll continue to run the 
> tests in order to be able to share something more meaningful.
> The tests are passing except one about concurrent DV updates. I don't know 
> this part at all so I did not fix the test yet. I don't even know if we can 
> make it work with index sorting ;).
>  [~mikemccand] I would love to have your feedback about the prototype. Could 
> you please take a look ? I am sure there are plenty of bugs, ... but I think 
> it's a good start to evaluate the feasibility of this feature.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7579) Sorting on flushed segment

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15763859#comment-15763859
 ] 

Michael McCandless commented on LUCENE-7579:


Patch looks great to me!

I think (later, separate issue) we could use a more naive stored fields (and 
term vectors) format for the temp files written at flush ... a format that does 
no compression, just writes bytes to disk, maybe has simple in-memory array 
pointing to offset in the file for each document ... this format would be 
package private to oal.index.  Later!  This patch is great, progress not 
perfection.

bq. I think we should look into forbidding that (in a different issue).

+1

I'll merge this soon to master so we can get it baking ...

> Sorting on flushed segment
> --
>
> Key: LUCENE-7579
> URL: https://issues.apache.org/jira/browse/LUCENE-7579
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ferenczi Jim
>
> Today flushed segments built by an index writer with an index sort specified 
> are not sorted. The merge is responsible of sorting these segments 
> potentially with others that are already sorted (resulted from another 
> merge). 
> I'd like to investigate the cost of sorting the segment directly during the 
> flush. This could make the merge faster since they are some cheap 
> optimizations that can be done only if all segments to be merged are sorted.
>  For instance the merge of the points could use the bulk merge instead of 
> rebuilding the points from scratch.
> I made a small prototype which sort the segment on flush here:
> https://github.com/apache/lucene-solr/compare/master...jimczi:flush_sort
> The idea is simple, for points, norms, docvalues and terms I use the 
> SortingLeafReader implementation to translate the values that we have in RAM 
> in a sorted enumeration for the writers.
> For stored fields I use a two pass scheme where the documents are first 
> written to disk unsorted and then copied to another file with the correct 
> sorting. I use the same stored field format for the two steps and just remove 
> the file produced by the first pass at the end of the process.
> This prototype has no implementation for index sorting that use term vectors 
> yet. I'll add this later if the tests are good enough.
> Speaking of testing, I tried this branch on [~mikemccand] benchmark scripts 
> and compared master with index sorting against my branch with index sorting 
> on flush. I tried with sparsetaxis and wikipedia and the first results are 
> weird. When I use the SerialScheduler and only one thread to write the docs,  
> index sorting on flush is slower. But when I use two threads the sorting on 
> flush is much faster even with the SerialScheduler. I'll continue to run the 
> tests in order to be able to share something more meaningful.
> The tests are passing except one about concurrent DV updates. I don't know 
> this part at all so I did not fix the test yet. I don't even know if we can 
> make it work with index sorting ;).
>  [~mikemccand] I would love to have your feedback about the prototype. Could 
> you please take a look ? I am sure there are plenty of bugs, ... but I think 
> it's a good start to evaluate the feasibility of this feature.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1020 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1020/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail

Error Message:
expected:<200> but was:<404>

Stack Trace:
java.lang.AssertionError: expected:<200> but was:<404>
at 
__randomizedtesting.SeedInfo.seed([A355A8D4037DC33A:CBEA9DFED3E7D1D6]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.cancelDelegationToken(TestSolrCloudWithDelegationTokens.java:140)
at 
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail(TestSolrCloudWithDelegationTokens.java:294)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Commented] (LUCENE-7579) Sorting on flushed segment

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15763808#comment-15763808
 ] 

Adrien Grand commented on LUCENE-7579:
--

Thanks, the diff looks good to me!

bq.  IndexWriter.addIndexes uses the merge to add indices that could be unsorted

I think we should look into forbidding that (in a different issue).

> Sorting on flushed segment
> --
>
> Key: LUCENE-7579
> URL: https://issues.apache.org/jira/browse/LUCENE-7579
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Ferenczi Jim
>
> Today flushed segments built by an index writer with an index sort specified 
> are not sorted. The merge is responsible of sorting these segments 
> potentially with others that are already sorted (resulted from another 
> merge). 
> I'd like to investigate the cost of sorting the segment directly during the 
> flush. This could make the merge faster since they are some cheap 
> optimizations that can be done only if all segments to be merged are sorted.
>  For instance the merge of the points could use the bulk merge instead of 
> rebuilding the points from scratch.
> I made a small prototype which sort the segment on flush here:
> https://github.com/apache/lucene-solr/compare/master...jimczi:flush_sort
> The idea is simple, for points, norms, docvalues and terms I use the 
> SortingLeafReader implementation to translate the values that we have in RAM 
> in a sorted enumeration for the writers.
> For stored fields I use a two pass scheme where the documents are first 
> written to disk unsorted and then copied to another file with the correct 
> sorting. I use the same stored field format for the two steps and just remove 
> the file produced by the first pass at the end of the process.
> This prototype has no implementation for index sorting that use term vectors 
> yet. I'll add this later if the tests are good enough.
> Speaking of testing, I tried this branch on [~mikemccand] benchmark scripts 
> and compared master with index sorting against my branch with index sorting 
> on flush. I tried with sparsetaxis and wikipedia and the first results are 
> weird. When I use the SerialScheduler and only one thread to write the docs,  
> index sorting on flush is slower. But when I use two threads the sorting on 
> flush is much faster even with the SerialScheduler. I'll continue to run the 
> tests in order to be able to share something more meaningful.
> The tests are passing except one about concurrent DV updates. I don't know 
> this part at all so I did not fix the test yet. I don't even know if we can 
> make it work with index sorting ;).
>  [~mikemccand] I would love to have your feedback about the prototype. Could 
> you please take a look ? I am sure there are plenty of bugs, ... but I think 
> it's a good start to evaluate the feasibility of this feature.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9859) replication.properties cannot be updated after being written and neither replication.properties or index.properties are durable in the face of a crash

[Pierre Alain (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15763791#comment-15763791
 ] 

Pierre Alain edited comment on SOLR-9859 at 12/20/16 9:49 AM:
--

Seems to be the same issue
* SOLR-9580 ?


was (Author: pierre alain):
Seems to be the same issue ?

> replication.properties cannot be updated after being written and neither 
> replication.properties or index.properties are durable in the face of a crash
> --
>
> Key: SOLR-9859
> URL: https://issues.apache.org/jira/browse/SOLR-9859
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 5.5.3, 6.3
>Reporter: Pushkar Raste
>Assignee: Mark Miller
>Priority: Minor
> Attachments: SOLR-9859.patch, SOLR-9859.patch, SOLR-9859.patch, 
> SOLR-9859.patch, SOLR-9859.patch
>
>
> If a shard recovers via replication (vs PeerSync) a file named 
> {{replication.properties}} gets created. If the same shard recovers once more 
> via replication, IndexFetcher fails to write latest replication information 
> as it tries to create {{replication.properties}} but as file already exists. 
> Here is the stack trace I saw 
> {code}
> java.nio.file.FileAlreadyExistsException: 
> \shard-3-001\cores\collection1\data\replication.properties
>   at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
>   at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
>   at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
>   at sun.nio.fs.WindowsFileSystemProvider.newByteChannel(Unknown Source)
>   at java.nio.file.spi.FileSystemProvider.newOutputStream(Unknown Source)
>   at java.nio.file.Files.newOutputStream(Unknown Source)
>   at 
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:413)
>   at 
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:409)
>   at 
> org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
>   at 
> org.apache.solr.handler.IndexFetcher.logReplicationTimeAndConfFiles(IndexFetcher.java:689)
>   at 
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:501)
>   at 
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:265)
>   at 
> org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
>   at 
> org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:157)
>   at 
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:409)
>   at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:222)
>   at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>   at java.util.concurrent.FutureTask.run(Unknown Source)
>   at 
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$0(ExecutorUtil.java:229)
>   at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>   at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>   at java.lang.Thread.run(Unknown Source)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9859) replication.properties cannot be updated after being written and neither replication.properties or index.properties are durable in the face of a crash

[Pierre Alain (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15763791#comment-15763791
 ] 

Pierre Alain commented on SOLR-9859:


Seems to be the same issue ?

> replication.properties cannot be updated after being written and neither 
> replication.properties or index.properties are durable in the face of a crash
> --
>
> Key: SOLR-9859
> URL: https://issues.apache.org/jira/browse/SOLR-9859
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 5.5.3, 6.3
>Reporter: Pushkar Raste
>Assignee: Mark Miller
>Priority: Minor
> Attachments: SOLR-9859.patch, SOLR-9859.patch, SOLR-9859.patch, 
> SOLR-9859.patch, SOLR-9859.patch
>
>
> If a shard recovers via replication (vs PeerSync) a file named 
> {{replication.properties}} gets created. If the same shard recovers once more 
> via replication, IndexFetcher fails to write latest replication information 
> as it tries to create {{replication.properties}} but as file already exists. 
> Here is the stack trace I saw 
> {code}
> java.nio.file.FileAlreadyExistsException: 
> \shard-3-001\cores\collection1\data\replication.properties
>   at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
>   at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
>   at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
>   at sun.nio.fs.WindowsFileSystemProvider.newByteChannel(Unknown Source)
>   at java.nio.file.spi.FileSystemProvider.newOutputStream(Unknown Source)
>   at java.nio.file.Files.newOutputStream(Unknown Source)
>   at 
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:413)
>   at 
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:409)
>   at 
> org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
>   at 
> org.apache.solr.handler.IndexFetcher.logReplicationTimeAndConfFiles(IndexFetcher.java:689)
>   at 
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:501)
>   at 
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:265)
>   at 
> org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
>   at 
> org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:157)
>   at 
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:409)
>   at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:222)
>   at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>   at java.util.concurrent.FutureTask.run(Unknown Source)
>   at 
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$0(ExecutorUtil.java:229)
>   at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>   at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>   at java.lang.Thread.run(Unknown Source)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org