[JENKINS-EA] Lucene-Solr-6.0-Linux (32bit/jdk-9-ea+114) - Build # 141 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.0-Linux/141/
Java: 32bit/jdk-9-ea+114 -server -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.handler.TestReqParamsAPI.test

Error Message:
Could not get expected value  'CY val' for path 'response/params/y/c' full 
output: {   "responseHeader":{ "status":0, "QTime":0},   "response":{   
  "znodeVersion":0, "params":{"x":{ "a":"A val", "b":"B 
val", "":{"v":0}

Stack Trace:
java.lang.AssertionError: Could not get expected value  'CY val' for path 
'response/params/y/c' full output: {
  "responseHeader":{
"status":0,
"QTime":0},
  "response":{
"znodeVersion":0,
"params":{"x":{
"a":"A val",
"b":"B val",
"":{"v":0}
at 
__randomizedtesting.SeedInfo.seed([445E801C87CA6172:CC0ABFC629360C8A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:458)
at 
org.apache.solr.handler.TestReqParamsAPI.testReqParams(TestReqParamsAPI.java:165)
at 
org.apache.solr.handler.TestReqParamsAPI.test(TestReqParamsAPI.java:67)
at sun.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native 
Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:531)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Commented] (SOLR-8716) Upgrade to Apache Tika 1.12

[Lewis John McGibbney (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251249#comment-15251249
 ] 

Lewis John McGibbney commented on SOLR-8716:


[~thetaphi] [~janhoy] out of curiosity how do patches in lucene-solr typically 
get reviewed? Do you have a pre-commit build or something set up? Thanks

> Upgrade to Apache Tika 1.12
> ---
>
> Key: SOLR-8716
> URL: https://issues.apache.org/jira/browse/SOLR-8716
> Project: Solr
>  Issue Type: Improvement
>  Components: contrib - Solr Cell (Tika extraction)
>Reporter: Lewis John McGibbney
>Assignee: Uwe Schindler
> Fix For: master
>
> Attachments: LUCENE-7041.patch
>
>
> We recently released Apache Tika 1.12. In order to use the fixes provided 
> within the Tika.translate API I propose to upgrade Tika from 1.7 --> 1.12 in 
> lucene/ivy-versions.properties.
> Patch coming up.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-master - Build # 1095 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/1095/

1 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.handler.TestSolrConfigHandlerCloud

Error Message:
2 threads leaked from SUITE scope at 
org.apache.solr.handler.TestSolrConfigHandlerCloud: 1) Thread[id=17747, 
name=Thread-6255, state=TIMED_WAITING, group=TGRP-TestSolrConfigHandlerCloud]   
  at java.lang.Thread.sleep(Native Method) at 
org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:101)
 at 
org.apache.solr.core.SolrResourceLoader.openSchema(SolrResourceLoader.java:333) 
at 
org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:48)
 at 
org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:75)
 at 
org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:108)
 at 
org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:79)   
  at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:937) 
at org.apache.solr.core.SolrCore.lambda$getConfListener$6(SolrCore.java:2488)   
  at org.apache.solr.core.SolrCore$$Lambda$17/579255287.run(Unknown Source) 
at org.apache.solr.cloud.ZkController$4.run(ZkController.java:2425)
2) Thread[id=18029, name=Thread-6291, state=TIMED_WAITING, 
group=TGRP-TestSolrConfigHandlerCloud] at java.lang.Thread.sleep(Native 
Method) at 
org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:101)
 at 
org.apache.solr.core.SolrResourceLoader.openSchema(SolrResourceLoader.java:333) 
at 
org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:48)
 at 
org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:75)
 at 
org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:108)
 at 
org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:79)   
  at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:937) 
at org.apache.solr.core.SolrCore.lambda$getConfListener$6(SolrCore.java:2488)   
  at org.apache.solr.core.SolrCore$$Lambda$17/579255287.run(Unknown Source) 
at org.apache.solr.cloud.ZkController$4.run(ZkController.java:2425)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE 
scope at org.apache.solr.handler.TestSolrConfigHandlerCloud: 
   1) Thread[id=17747, name=Thread-6255, state=TIMED_WAITING, 
group=TGRP-TestSolrConfigHandlerCloud]
at java.lang.Thread.sleep(Native Method)
at 
org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:101)
at 
org.apache.solr.core.SolrResourceLoader.openSchema(SolrResourceLoader.java:333)
at 
org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:48)
at 
org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:75)
at 
org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:108)
at 
org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:79)
at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:937)
at 
org.apache.solr.core.SolrCore.lambda$getConfListener$6(SolrCore.java:2488)
at org.apache.solr.core.SolrCore$$Lambda$17/579255287.run(Unknown 
Source)
at org.apache.solr.cloud.ZkController$4.run(ZkController.java:2425)
   2) Thread[id=18029, name=Thread-6291, state=TIMED_WAITING, 
group=TGRP-TestSolrConfigHandlerCloud]
at java.lang.Thread.sleep(Native Method)
at 
org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:101)
at 
org.apache.solr.core.SolrResourceLoader.openSchema(SolrResourceLoader.java:333)
at 
org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:48)
at 
org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:75)
at 
org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:108)
at 
org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:79)
at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:937)
at 
org.apache.solr.core.SolrCore.lambda$getConfListener$6(SolrCore.java:2488)
at org.apache.solr.core.SolrCore$$Lambda$17/579255287.run(Unknown 
Source)
at org.apache.solr.cloud.ZkController$4.run(ZkController.java:2425)
at __randomizedtesting.SeedInfo.seed([E32CDF0079306D84]:0)




Build Log:
[...truncated 12120 lines...]
   [junit4] Suite: org.apache.solr.handler.TestSolrConfigHandlerCloud
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud_E32CDF0079306D84-001/init-core-data-001
   [junit4]   2> 

[JENKINS] Lucene-Solr-6.0-Linux (64bit/jdk1.8.0_72) - Build # 140 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.0-Linux/140/
Java: 64bit/jdk1.8.0_72 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.cloud.TestMiniSolrCloudCluster.testStopAllStartAll

Error Message:
Address already in use

Stack Trace:
java.net.BindException: Address already in use
at 
__randomizedtesting.SeedInfo.seed([25DD20979590F124:53E33FE4D4A75C0B]:0)
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at 
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at 
org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:326)
at 
org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
at 
org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:244)
at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:384)
at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at 
org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:327)
at 
org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:356)
at 
org.apache.solr.cloud.TestMiniSolrCloudCluster.testStopAllStartAll(TestMiniSolrCloudCluster.java:441)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Resolved] (SOLR-9022) solr unable to handle/parse images when they are embedded in office docs(like word,xls,etc)

[Erick Erickson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson resolved SOLR-9022.
--
Resolution: Not A Problem

First of all, please raise this kind of question on the user's list first, we 
try to keep JIRAs for known code problems/improvements rather than usage 
questions.

In this case, this line:
 2016-04-20 16:55:13,342 ERROR [IPC Server handler 20 on 39376] 
org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: 
attempt_1460872551948_58233_m_000159_0 - exited : 
org.kitesdk.morphline.api.MorphlineRuntimeException: 
org.kitesdk.morphline.api.MorphlineRuntimeException: tryRules command found no 
successful rule for record: {_attachment_body=[TikaInputStream of 
java.io.BufferedInputStream@391f1777], 
_attachment_mimetype=[application/octet-stream], 
_attachment_name=[xl/media/image2.png], 
id=[185be63a-e527-4953-9a3d-ae957dc0fa51]}
at 
org.kitesdk.morphline.base.FaultTolerance.handleException(FaultTolerance.java:73)

implies that you have some "tryRules" in your morphline file and they do not 
handle the case that's being encountered when Morphlines is processing this 
file. IOW, the problem is your morphlines configuration, not Solr.

If that turns out not to be the case we can re-open this JIRA.

> solr unable to handle/parse images when they are embedded in office docs(like 
> word,xls,etc)
> ---
>
> Key: SOLR-9022
> URL: https://issues.apache.org/jira/browse/SOLR-9022
> Project: Solr
>  Issue Type: Bug
>  Components: SolrJ
>Affects Versions: 5.4.1
>Reporter: Bhanuprakash Prathap
>
> As we are trying to index multiple files, the solr throws below exception 
> whenever it encounters embedded images with other docs.
> The issues arises as embedded images files are read with MIME type of 
> binary(attachment_mimetype=[application/octet-stream])  though the attached 
> files are type png/txt etc.
> Full stack trace for this issue
> 2016-04-20 16:55:13,311 INFO [Thread-52] 
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Added 
> attempt_1460872551948_58233_m_000116_1 to list of failed maps
> 2016-04-20 16:55:13,329 INFO [IPC Server handler 18 on 39376] 
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt 
> attempt_1460872551948_58233_m_000159_0 is : 0.0
> 2016-04-20 16:55:13,342 ERROR [IPC Server handler 20 on 39376] 
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: 
> attempt_1460872551948_58233_m_000159_0 - exited : 
> org.kitesdk.morphline.api.MorphlineRuntimeException: 
> org.kitesdk.morphline.api.MorphlineRuntimeException: tryRules command found 
> no successful rule for record: {_attachment_body=[TikaInputStream of 
> java.io.BufferedInputStream@391f1777], 
> _attachment_mimetype=[application/octet-stream], 
> _attachment_name=[xl/media/image2.png], 
> id=[185be63a-e527-4953-9a3d-ae957dc0fa51]}
>   at 
> org.kitesdk.morphline.base.FaultTolerance.handleException(FaultTolerance.java:73)
>   at 
> org.apache.solr.hadoop.morphline.MorphlineMapRunner.map(MorphlineMapRunner.java:220)
>   at 
> org.apache.solr.hadoop.morphline.MorphlineMapper.map(MorphlineMapper.java:86)
>   at 
> org.apache.solr.hadoop.morphline.MorphlineMapper.map(MorphlineMapper.java:54)
>   at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
>   at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
>   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>   at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:415)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
>   at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.kitesdk.morphline.api.MorphlineRuntimeException: tryRules 
> command found no successful rule for record: 
> {_attachment_body=[TikaInputStream of java.io.BufferedInputStream@391f1777], 
> _attachment_mimetype=[application/octet-stream], 
> _attachment_name=[xl/media/image2.png], 
> id=[185be63a-e527-4953-9a3d-ae957dc0fa51]}
>   at 
> org.kitesdk.morphline.stdlib.TryRulesBuilder$TryRules.doProcess(TryRulesBuilder.java:132)
>   at 
> org.kitesdk.morphline.base.AbstractCommand.process(AbstractCommand.java:156)
>   at org.kitesdk.morphline.base.Connector.process(Connector.java:64)
>   at 
> org.kitesdk.morphline.base.AbstractCommand.doProcess(AbstractCommand.java:181)
>   at 
> org.kitesdk.morphline.tika.DetectMimeTypeBuilder$DetectMimeType.doProcess(DetectMimeTypeBuilder.java:166)
>   at 
> org.kitesdk.morphline.base.AbstractCommand.process(AbstractCommand.java:156)
>   at 

[JENKINS] Lucene-Solr-SmokeRelease-master - Build # 470 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/470/

No tests ran.

Build Log:
[...truncated 40520 lines...]
prepare-release-no-sign:
[mkdir] Created dir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/dist
 [copy] Copying 476 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/dist/lucene
 [copy] Copying 245 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/dist/solr
   [smoker] Java 1.8 
JAVA_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
   [smoker] NOTE: output encoding is UTF-8
   [smoker] 
   [smoker] Load release URL 
"file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/dist/"...
   [smoker] 
   [smoker] Test Lucene...
   [smoker]   test basics...
   [smoker]   get KEYS
   [smoker] 0.2 MB in 0.01 sec (15.9 MB/sec)
   [smoker]   check changes HTML...
   [smoker]   download lucene-7.0.0-src.tgz...
   [smoker] 28.6 MB in 0.03 sec (1104.6 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-7.0.0.tgz...
   [smoker] 62.9 MB in 0.06 sec (1088.6 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-7.0.0.zip...
   [smoker] 73.5 MB in 0.09 sec (774.4 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   unpack lucene-7.0.0.tgz...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.8...
   [smoker]   got 5997 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-7.0.0.zip...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.8...
   [smoker]   got 5997 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-7.0.0-src.tgz...
   [smoker] make sure no JARs/WARs in src dist...
   [smoker] run "ant validate"
   [smoker] run tests w/ Java 8 and testArgs='-Dtests.slow=false'...
   [smoker] test demo with 1.8...
   [smoker]   got 218 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] generate javadocs w/ Java 8...
   [smoker] 
   [smoker] Crawl/parse...
   [smoker] 
   [smoker] Verify...
   [smoker]   confirm all releases have coverage in TestBackwardsCompatibility
   [smoker] find all past Lucene releases...
   [smoker] run TestBackwardsCompatibility..
   [smoker] success!
   [smoker] 
   [smoker] Test Solr...
   [smoker]   test basics...
   [smoker]   get KEYS
   [smoker] 0.2 MB in 0.01 sec (24.2 MB/sec)
   [smoker]   check changes HTML...
   [smoker]   download solr-7.0.0-src.tgz...
   [smoker] 37.7 MB in 0.70 sec (53.6 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download solr-7.0.0.tgz...
   [smoker] 132.0 MB in 2.24 sec (59.0 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download solr-7.0.0.zip...
   [smoker] 140.5 MB in 2.68 sec (52.4 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   unpack solr-7.0.0.tgz...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] unpack lucene-7.0.0.tgz...
   [smoker]   **WARNING**: skipping check of 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/tmp/unpack/solr-7.0.0/contrib/dataimporthandler-extras/lib/javax.mail-1.5.1.jar:
 it has javax.* classes
   [smoker]   **WARNING**: skipping check of 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/tmp/unpack/solr-7.0.0/contrib/dataimporthandler-extras/lib/activation-1.1.1.jar:
 it has javax.* classes
   [smoker] copying unpacked distribution for Java 8 ...
   [smoker] test solr example w/ Java 8...
   [smoker]   start Solr instance 
(log=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/tmp/unpack/solr-7.0.0-java8/solr-example.log)...
   [smoker] No process found for Solr node running on port 8983
   [smoker]   Running techproducts example on port 8983 from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/tmp/unpack/solr-7.0.0-java8
   [smoker] Creating Solr home directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/tmp/unpack/solr-7.0.0-java8/example/techproducts/solr
   [smoker] 
   [smoker] Starting up Solr on port 8983 using command:
   [smoker] bin/solr start -p 8983 -s "example/techproducts/solr"
   [smoker] 
   [smoker] Waiting up to 30 seconds to see Solr running on port 8983 [|]  
 [/]   [-]   [\]   [|]   [/]   [-]   
[\]   [|]  

[jira] [Commented] (SOLR-9014) Audit all usages of ClusterState methods which may make calls to ZK via the lazy collection reference

[Shalin Shekhar Mangar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251031#comment-15251031
 ] 

Shalin Shekhar Mangar commented on SOLR-9014:
-

You are right. I was confused. I haven't dug into SOLR-8323 yet but if you say 
so :-)

> Audit all usages of ClusterState methods which may make calls to ZK via the 
> lazy collection reference
> -
>
> Key: SOLR-9014
> URL: https://issues.apache.org/jira/browse/SOLR-9014
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
> Fix For: master, 6.1
>
>
> ClusterState has a bunch of methods such as getSlice and getReplica which 
> internally call getCollectionOrNull that ends up making a call to ZK via the 
> lazy collection reference. Many classes use these methods even though a 
> DocCollection object is available. In such cases, multiple redundant calls to 
> ZooKeeper can happen if the collection is not watched locally. This is 
> especially true for Overseer classes which operate on all collections.
> We should audit all usages of these methods and replace them with calls to 
> appropriate DocCollection methods.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-8694) DistributedMap/Queue simplifications and fixes.

[Scott Blum (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Blum resolved SOLR-8694.
--
   Resolution: Fixed
Fix Version/s: 5.6
   5.5.1

> DistributedMap/Queue simplifications and fixes.
> ---
>
> Key: SOLR-8694
> URL: https://issues.apache.org/jira/browse/SOLR-8694
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Affects Versions: 5.4.1
>Reporter: Scott Blum
>Assignee: Scott Blum
>  Labels: patch, reliability
> Fix For: master, 6.0, 5.5.1, 5.6
>
> Attachments: SOLR-8694.patch
>
>
> Bugfix in DistributedQueue, it could add too many watchers since it assumed 
> the watcher was cleared on connection events.
> Huge simplification to DistributedMap; it implemented a lot of tricky stuff 
> that no one is actually using.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-6.x - Build # 156 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/156/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionStateFormat2Test.test

Error Message:
Could not find collection:.system

Stack Trace:
java.lang.AssertionError: Could not find collection:.system
at 
__randomizedtesting.SeedInfo.seed([9597C26453FD71F2:1DC3FDBEFD011C0A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at 
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:150)
at 
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:135)
at 
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:130)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:852)
at 
org.apache.solr.cloud.CollectionStateFormat2Test.testConfNameAndCollectionNameSame(CollectionStateFormat2Test.java:53)
at 
org.apache.solr.cloud.CollectionStateFormat2Test.test(CollectionStateFormat2Test.java:40)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Commented] (SOLR-8694) DistributedMap/Queue simplifications and fixes.

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251004#comment-15251004
 ] 

ASF subversion and git services commented on SOLR-8694:
---

Commit 6d3402188d3b7e2e12fc27d98d643a87feffa147 in lucene-solr's branch 
refs/heads/branch_5_5 from markrmiller
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6d34021 ]

SOLR-8694: DistributedMap/Queue can create too many Watchers and some code 
simplification.


> DistributedMap/Queue simplifications and fixes.
> ---
>
> Key: SOLR-8694
> URL: https://issues.apache.org/jira/browse/SOLR-8694
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Affects Versions: 5.4.1
>Reporter: Scott Blum
>Assignee: Scott Blum
>  Labels: patch, reliability
> Fix For: master, 6.0
>
> Attachments: SOLR-8694.patch
>
>
> Bugfix in DistributedQueue, it could add too many watchers since it assumed 
> the watcher was cleared on connection events.
> Huge simplification to DistributedMap; it implemented a lot of tricky stuff 
> that no one is actually using.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8694) DistributedMap/Queue simplifications and fixes.

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250999#comment-15250999
 ] 

ASF subversion and git services commented on SOLR-8694:
---

Commit bbae36aa92c58cdbe031ea447bcbd9ae66f5138c in lucene-solr's branch 
refs/heads/branch_5x from markrmiller
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=bbae36a ]

SOLR-8694: DistributedMap/Queue can create too many Watchers and some code 
simplification.


> DistributedMap/Queue simplifications and fixes.
> ---
>
> Key: SOLR-8694
> URL: https://issues.apache.org/jira/browse/SOLR-8694
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Affects Versions: 5.4.1
>Reporter: Scott Blum
>Assignee: Scott Blum
>  Labels: patch, reliability
> Fix For: master, 6.0
>
> Attachments: SOLR-8694.patch
>
>
> Bugfix in DistributedQueue, it could add too many watchers since it assumed 
> the watcher was cleared on connection events.
> Huge simplification to DistributedMap; it implemented a lot of tricky stuff 
> that no one is actually using.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8694) DistributedMap/Queue simplifications and fixes.

[Scott Blum (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250935#comment-15250935
 ] 

Scott Blum commented on SOLR-8694:
--

In progress.

> DistributedMap/Queue simplifications and fixes.
> ---
>
> Key: SOLR-8694
> URL: https://issues.apache.org/jira/browse/SOLR-8694
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Affects Versions: 5.4.1
>Reporter: Scott Blum
>Assignee: Scott Blum
>  Labels: patch, reliability
> Fix For: master, 6.0
>
> Attachments: SOLR-8694.patch
>
>
> Bugfix in DistributedQueue, it could add too many watchers since it assumed 
> the watcher was cleared on connection events.
> Huge simplification to DistributedMap; it implemented a lot of tricky stuff 
> that no one is actually using.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8694) DistributedMap/Queue simplifications and fixes.

[Scott Blum (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Blum updated SOLR-8694:
-
Fix Version/s: 6.0

> DistributedMap/Queue simplifications and fixes.
> ---
>
> Key: SOLR-8694
> URL: https://issues.apache.org/jira/browse/SOLR-8694
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Affects Versions: 5.4.1
>Reporter: Scott Blum
>Assignee: Scott Blum
>  Labels: patch, reliability
> Fix For: master, 6.0
>
> Attachments: SOLR-8694.patch
>
>
> Bugfix in DistributedQueue, it could add too many watchers since it assumed 
> the watcher was cleared on connection events.
> Huge simplification to DistributedMap; it implemented a lot of tricky stuff 
> that no one is actually using.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-8973) TX-frenzy on Zookeeper when collection is put to use

[Scott Blum (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Blum resolved SOLR-8973.
--
   Resolution: Fixed
Fix Version/s: 5.6
   5.5.1

> TX-frenzy on Zookeeper when collection is put to use
> 
>
> Key: SOLR-8973
> URL: https://issues.apache.org/jira/browse/SOLR-8973
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Affects Versions: 5.0, 5.1, 5.2, 5.3, 5.4, 5.5, master, 5.6
>Reporter: Janmejay Singh
>Assignee: Scott Blum
>  Labels: collections, patch-available, solrcloud, zookeeper
> Fix For: master, 5.5.1, 6.1, 5.6
>
> Attachments: SOLR-8973-ZkStateReader.patch, SOLR-8973.patch, 
> SOLR-8973.patch, SOLR-8973.patch
>
>
> This is to do with a distributed data-race. Core-creation happens at a time 
> when collection is not yet visible to the node. In this case a fallback 
> code-path is used which de-references collection-state lazily (on demand) as 
> opposed to setting a watch and keeping it cached locally.
> Due to this, as requests towards the core mount, it generates ZK fetch for 
> collection proportionately. On a large solr-cloud cluster, this generates 
> several Gbps of TX traffic on ZK nodes. This affects indexing 
> throughput(which floors) in addition to running ZK node out of network 
> bandwidth. 
> On smaller solr-cloud clusters its hard to run into, because probability of 
> this race materializing reduces.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8914) ZkStateReader's refreshLiveNodes(Watcher) is not thread safe

[Scott Blum (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Blum updated SOLR-8914:
-
Fix Version/s: 5.5.1

> ZkStateReader's refreshLiveNodes(Watcher) is not thread safe
> 
>
> Key: SOLR-8914
> URL: https://issues.apache.org/jira/browse/SOLR-8914
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 5.5, master, 6.0, 5.4.1
>Reporter: Hoss Man
>Assignee: Scott Blum
> Fix For: master, 5.5.1, 6.1, 5.6, 6.0.1
>
> Attachments: SOLR-8914.patch, SOLR-8914.patch, SOLR-8914.patch, 
> SOLR-8914.patch, jenkins.thetaphi.de_Lucene-Solr-6.x-Solaris_32.log.txt, 
> live_node_mentions_port56361_with_threadIds.log.txt, 
> live_nodes_mentions.log.txt
>
>
> Jenkin's encountered a failure in TestTolerantUpdateProcessorCloud over the 
> weekend
> {noformat}
> http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Solaris/32/consoleText
> Checking out Revision c46d7686643e7503304cb35dfe546bce9c6684e7 
> (refs/remotes/origin/branch_6x)
> Using Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC
> {noformat}
> The failure happened during the static setup of the test, when a 
> MiniSolrCloudCluster & several clients are initialized -- before any code 
> related to TolerantUpdateProcessor is ever used.
> I can't reproduce this, or really make sense of what i'm (not) seeing here in 
> the logs, so i'm filing this jira with my analysis in the hopes that someone 
> else can help make sense of it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-8914) ZkStateReader's refreshLiveNodes(Watcher) is not thread safe

[Scott Blum (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Blum resolved SOLR-8914.
--
Resolution: Fixed

> ZkStateReader's refreshLiveNodes(Watcher) is not thread safe
> 
>
> Key: SOLR-8914
> URL: https://issues.apache.org/jira/browse/SOLR-8914
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 5.5, master, 6.0, 5.4.1
>Reporter: Hoss Man
>Assignee: Scott Blum
> Fix For: master, 5.5.1, 6.1, 5.6, 6.0.1
>
> Attachments: SOLR-8914.patch, SOLR-8914.patch, SOLR-8914.patch, 
> SOLR-8914.patch, jenkins.thetaphi.de_Lucene-Solr-6.x-Solaris_32.log.txt, 
> live_node_mentions_port56361_with_threadIds.log.txt, 
> live_nodes_mentions.log.txt
>
>
> Jenkin's encountered a failure in TestTolerantUpdateProcessorCloud over the 
> weekend
> {noformat}
> http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Solaris/32/consoleText
> Checking out Revision c46d7686643e7503304cb35dfe546bce9c6684e7 
> (refs/remotes/origin/branch_6x)
> Using Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC
> {noformat}
> The failure happened during the static setup of the test, when a 
> MiniSolrCloudCluster & several clients are initialized -- before any code 
> related to TolerantUpdateProcessor is ever used.
> I can't reproduce this, or really make sense of what i'm (not) seeing here in 
> the logs, so i'm filing this jira with my analysis in the hopes that someone 
> else can help make sense of it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8914) ZkStateReader's refreshLiveNodes(Watcher) is not thread safe

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250912#comment-15250912
 ] 

ASF subversion and git services commented on SOLR-8914:
---

Commit f45fae5c81e68763c99826524bec8b9273dfefd4 in lucene-solr's branch 
refs/heads/branch_5_5 from markrmiller
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f45fae5 ]

SOLR-8914: ZkStateReader's refreshLiveNodes(Watcher) is not thread safe.


> ZkStateReader's refreshLiveNodes(Watcher) is not thread safe
> 
>
> Key: SOLR-8914
> URL: https://issues.apache.org/jira/browse/SOLR-8914
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 5.5, master, 6.0, 5.4.1
>Reporter: Hoss Man
>Assignee: Scott Blum
> Fix For: master, 6.1, 5.6, 6.0.1
>
> Attachments: SOLR-8914.patch, SOLR-8914.patch, SOLR-8914.patch, 
> SOLR-8914.patch, jenkins.thetaphi.de_Lucene-Solr-6.x-Solaris_32.log.txt, 
> live_node_mentions_port56361_with_threadIds.log.txt, 
> live_nodes_mentions.log.txt
>
>
> Jenkin's encountered a failure in TestTolerantUpdateProcessorCloud over the 
> weekend
> {noformat}
> http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Solaris/32/consoleText
> Checking out Revision c46d7686643e7503304cb35dfe546bce9c6684e7 
> (refs/remotes/origin/branch_6x)
> Using Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC
> {noformat}
> The failure happened during the static setup of the test, when a 
> MiniSolrCloudCluster & several clients are initialized -- before any code 
> related to TolerantUpdateProcessor is ever used.
> I can't reproduce this, or really make sense of what i'm (not) seeing here in 
> the logs, so i'm filing this jira with my analysis in the hopes that someone 
> else can help make sense of it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8973) TX-frenzy on Zookeeper when collection is put to use

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250913#comment-15250913
 ] 

ASF subversion and git services commented on SOLR-8973:
---

Commit 802ee6f8c90ba6462f29c963a760aa1dfeeb09b1 in lucene-solr's branch 
refs/heads/branch_5_5 from [~dragonsinth]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=802ee6f ]

SOLR-8973: Zookeeper frenzy when a core is first created.


> TX-frenzy on Zookeeper when collection is put to use
> 
>
> Key: SOLR-8973
> URL: https://issues.apache.org/jira/browse/SOLR-8973
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Affects Versions: 5.0, 5.1, 5.2, 5.3, 5.4, 5.5, master, 5.6
>Reporter: Janmejay Singh
>Assignee: Scott Blum
>  Labels: collections, patch-available, solrcloud, zookeeper
> Fix For: master, 6.1
>
> Attachments: SOLR-8973-ZkStateReader.patch, SOLR-8973.patch, 
> SOLR-8973.patch, SOLR-8973.patch
>
>
> This is to do with a distributed data-race. Core-creation happens at a time 
> when collection is not yet visible to the node. In this case a fallback 
> code-path is used which de-references collection-state lazily (on demand) as 
> opposed to setting a watch and keeping it cached locally.
> Due to this, as requests towards the core mount, it generates ZK fetch for 
> collection proportionately. On a large solr-cloud cluster, this generates 
> several Gbps of TX traffic on ZK nodes. This affects indexing 
> throughput(which floors) in addition to running ZK node out of network 
> bandwidth. 
> On smaller solr-cloud clusters its hard to run into, because probability of 
> this race materializing reduces.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8973) TX-frenzy on Zookeeper when collection is put to use

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250907#comment-15250907
 ] 

ASF subversion and git services commented on SOLR-8973:
---

Commit 9c7a031f988a0fa3a15d6fc23f7269d0c12fae16 in lucene-solr's branch 
refs/heads/branch_5x from [~dragonsinth]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9c7a031 ]

SOLR-8973: Zookeeper frenzy when a core is first created.


> TX-frenzy on Zookeeper when collection is put to use
> 
>
> Key: SOLR-8973
> URL: https://issues.apache.org/jira/browse/SOLR-8973
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Affects Versions: 5.0, 5.1, 5.2, 5.3, 5.4, 5.5, master, 5.6
>Reporter: Janmejay Singh
>Assignee: Scott Blum
>  Labels: collections, patch-available, solrcloud, zookeeper
> Fix For: master, 6.1
>
> Attachments: SOLR-8973-ZkStateReader.patch, SOLR-8973.patch, 
> SOLR-8973.patch, SOLR-8973.patch
>
>
> This is to do with a distributed data-race. Core-creation happens at a time 
> when collection is not yet visible to the node. In this case a fallback 
> code-path is used which de-references collection-state lazily (on demand) as 
> opposed to setting a watch and keeping it cached locally.
> Due to this, as requests towards the core mount, it generates ZK fetch for 
> collection proportionately. On a large solr-cloud cluster, this generates 
> several Gbps of TX traffic on ZK nodes. This affects indexing 
> throughput(which floors) in addition to running ZK node out of network 
> bandwidth. 
> On smaller solr-cloud clusters its hard to run into, because probability of 
> this race materializing reduces.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8929) Add an idea module for solr/server to enable launching start.jar

[Scott Blum (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250904#comment-15250904
 ] 

Scott Blum commented on SOLR-8929:
--

Follow up issue: SOLR-9023

> Add an idea module for solr/server to enable launching start.jar
> 
>
> Key: SOLR-8929
> URL: https://issues.apache.org/jira/browse/SOLR-8929
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 5.4.1
>Reporter: Scott Blum
>Assignee: Scott Blum
>Priority: Minor
> Fix For: master, 6.1
>
> Attachments: SOLR-8929-bin-solr-run-configuration.patch, 
> SOLR-8929.patch, SOLR-8929.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> Currently in IntelliJ, it's difficult to create a launch config to run Solr 
> in the same way that the bin/solr script would, because there aren't any 
> modules that reference the jetty start.jar that it uses.
> I want to create a simple solr/server IJ module that can be referenced from a 
> launch config.  I've created it manually in the past, but then I always lose 
> it when I have to regenerate idea on branch switch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9023) Improve idea solrcloud launch config

[Scott Blum (JIRA)]

Scott Blum created SOLR-9023:


 Summary: Improve idea solrcloud launch config
 Key: SOLR-9023
 URL: https://issues.apache.org/jira/browse/SOLR-9023
 Project: Solr
  Issue Type: Improvement
  Components: scripts and tools
Affects Versions: master
Reporter: Scott Blum
Priority: Minor


Two main problems:

1) The solrcloud launch config requires ant to have been run before it works.  
This is tolerable if not ideal.

2) It uses the precompiled jars in WEB-INF/lib instead of using the IntelliJ 
compiled classes that reflect up-to-date IDE edits.  Fixing this would be a be 
win, but may require some digging into setting up the jetty container properly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9014) Audit all usages of ClusterState methods which may make calls to ZK via the lazy collection reference

[Scott Blum (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250898#comment-15250898
 ] 

Scott Blum commented on SOLR-9014:
--

Hard to get the right context in mind in all these cases, eh?

1. If the overseer had set a watch on the collection it cared about, it would 
be more efficient to wait and loop.  Maybe we could fix this via SOLR-8323?  
Overseer could set a temporary watch on things it cared about.

One thing puzzles me in what you said tho; if the collection is lazy but 
existent, I think forceUpdateCollection() is basically exactly as efficient as 
just polling the lazy collection.  So going back to forceUpdateCollection() 
won't make it any better.  But in cases where the collection is watching, 
looping is far more efficient.

2) Agreed, maybe we should expose getCollectionStates() or getCollectionNames().


> Audit all usages of ClusterState methods which may make calls to ZK via the 
> lazy collection reference
> -
>
> Key: SOLR-9014
> URL: https://issues.apache.org/jira/browse/SOLR-9014
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
> Fix For: master, 6.1
>
>
> ClusterState has a bunch of methods such as getSlice and getReplica which 
> internally call getCollectionOrNull that ends up making a call to ZK via the 
> lazy collection reference. Many classes use these methods even though a 
> DocCollection object is available. In such cases, multiple redundant calls to 
> ZooKeeper can happen if the collection is not watched locally. This is 
> especially true for Overseer classes which operate on all collections.
> We should audit all usages of these methods and replace them with calls to 
> appropriate DocCollection methods.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-SmokeRelease-6.x - Build # 41 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-6.x/41/

No tests ran.

Build Log:
[...truncated 40519 lines...]
prepare-release-no-sign:
[mkdir] Created dir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-6.x/lucene/build/smokeTestRelease/dist
 [copy] Copying 476 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-6.x/lucene/build/smokeTestRelease/dist/lucene
 [copy] Copying 245 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-6.x/lucene/build/smokeTestRelease/dist/solr
   [smoker] Java 1.8 
JAVA_HOME=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8
   [smoker] NOTE: output encoding is UTF-8
   [smoker] 
   [smoker] Load release URL 
"file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-6.x/lucene/build/smokeTestRelease/dist/"...
   [smoker] 
   [smoker] Test Lucene...
   [smoker]   test basics...
   [smoker]   get KEYS
   [smoker] 0.2 MB in 0.02 sec (9.4 MB/sec)
   [smoker]   check changes HTML...
   [smoker]   download lucene-6.1.0-src.tgz...
   [smoker] 28.6 MB in 0.03 sec (1003.8 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-6.1.0.tgz...
   [smoker] 62.9 MB in 0.06 sec (1083.0 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-6.1.0.zip...
   [smoker] 73.4 MB in 0.07 sec (1098.7 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   unpack lucene-6.1.0.tgz...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.8...
   [smoker]   got 5993 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-6.1.0.zip...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.8...
   [smoker]   got 5993 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-6.1.0-src.tgz...
   [smoker] make sure no JARs/WARs in src dist...
   [smoker] run "ant validate"
   [smoker] run tests w/ Java 8 and testArgs='-Dtests.slow=false'...
   [smoker] test demo with 1.8...
   [smoker]   got 218 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] generate javadocs w/ Java 8...
   [smoker] 
   [smoker] Crawl/parse...
   [smoker] 
   [smoker] Verify...
   [smoker]   confirm all releases have coverage in TestBackwardsCompatibility
   [smoker] find all past Lucene releases...
   [smoker] run TestBackwardsCompatibility..
   [smoker] success!
   [smoker] 
   [smoker] Test Solr...
   [smoker]   test basics...
   [smoker]   get KEYS
   [smoker] 0.2 MB in 0.01 sec (21.7 MB/sec)
   [smoker]   check changes HTML...
   [smoker]   download solr-6.1.0-src.tgz...
   [smoker] 37.7 MB in 0.57 sec (65.8 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download solr-6.1.0.tgz...
   [smoker] 131.8 MB in 1.62 sec (81.4 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download solr-6.1.0.zip...
   [smoker] 140.3 MB in 1.45 sec (96.8 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   unpack solr-6.1.0.tgz...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] unpack lucene-6.1.0.tgz...
   [smoker]   **WARNING**: skipping check of 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-6.x/lucene/build/smokeTestRelease/tmp/unpack/solr-6.1.0/contrib/dataimporthandler-extras/lib/javax.mail-1.5.1.jar:
 it has javax.* classes
   [smoker]   **WARNING**: skipping check of 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-6.x/lucene/build/smokeTestRelease/tmp/unpack/solr-6.1.0/contrib/dataimporthandler-extras/lib/activation-1.1.1.jar:
 it has javax.* classes
   [smoker] copying unpacked distribution for Java 8 ...
   [smoker] test solr example w/ Java 8...
   [smoker]   start Solr instance 
(log=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-6.x/lucene/build/smokeTestRelease/tmp/unpack/solr-6.1.0-java8/solr-example.log)...
   [smoker] No process found for Solr node running on port 8983
   [smoker]   Running techproducts example on port 8983 from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-6.x/lucene/build/smokeTestRelease/tmp/unpack/solr-6.1.0-java8
   [smoker] Creating Solr home directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-6.x/lucene/build/smokeTestRelease/tmp/unpack/solr-6.1.0-java8/example/techproducts/solr
   [smoker] 
   [smoker] Starting up Solr on port 8983 using command:
   [smoker] bin/solr start -p 8983 -s "example/techproducts/solr"
   [smoker] 
   [smoker] Waiting up to 30 seconds to see Solr running on port 8983 [|]  
 [/]   [-]   [\]   [|]   [/]   [-]   
[\]   [|]   [/]  
   [smoker] Started 

[jira] [Updated] (SOLR-8914) ZkStateReader's refreshLiveNodes(Watcher) is not thread safe

[Scott Blum (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Blum updated SOLR-8914:
-
Fix Version/s: 5.6
   6.0.1

> ZkStateReader's refreshLiveNodes(Watcher) is not thread safe
> 
>
> Key: SOLR-8914
> URL: https://issues.apache.org/jira/browse/SOLR-8914
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 5.5, master, 6.0, 5.4.1
>Reporter: Hoss Man
>Assignee: Scott Blum
> Fix For: master, 6.1, 5.6, 6.0.1
>
> Attachments: SOLR-8914.patch, SOLR-8914.patch, SOLR-8914.patch, 
> SOLR-8914.patch, jenkins.thetaphi.de_Lucene-Solr-6.x-Solaris_32.log.txt, 
> live_node_mentions_port56361_with_threadIds.log.txt, 
> live_nodes_mentions.log.txt
>
>
> Jenkin's encountered a failure in TestTolerantUpdateProcessorCloud over the 
> weekend
> {noformat}
> http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Solaris/32/consoleText
> Checking out Revision c46d7686643e7503304cb35dfe546bce9c6684e7 
> (refs/remotes/origin/branch_6x)
> Using Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC
> {noformat}
> The failure happened during the static setup of the test, when a 
> MiniSolrCloudCluster & several clients are initialized -- before any code 
> related to TolerantUpdateProcessor is ever used.
> I can't reproduce this, or really make sense of what i'm (not) seeing here in 
> the logs, so i'm filing this jira with my analysis in the hopes that someone 
> else can help make sense of it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8914) ZkStateReader's refreshLiveNodes(Watcher) is not thread safe

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250886#comment-15250886
 ] 

ASF subversion and git services commented on SOLR-8914:
---

Commit 58f77fed2c45b8ce7da1831954f941dde39d98f9 in lucene-solr's branch 
refs/heads/branch_5x from markrmiller
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=58f77fe ]

SOLR-8914: ZkStateReader's refreshLiveNodes(Watcher) is not thread safe.


> ZkStateReader's refreshLiveNodes(Watcher) is not thread safe
> 
>
> Key: SOLR-8914
> URL: https://issues.apache.org/jira/browse/SOLR-8914
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 5.5, master, 6.0, 5.4.1
>Reporter: Hoss Man
>Assignee: Scott Blum
> Fix For: master, 6.1
>
> Attachments: SOLR-8914.patch, SOLR-8914.patch, SOLR-8914.patch, 
> SOLR-8914.patch, jenkins.thetaphi.de_Lucene-Solr-6.x-Solaris_32.log.txt, 
> live_node_mentions_port56361_with_threadIds.log.txt, 
> live_nodes_mentions.log.txt
>
>
> Jenkin's encountered a failure in TestTolerantUpdateProcessorCloud over the 
> weekend
> {noformat}
> http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Solaris/32/consoleText
> Checking out Revision c46d7686643e7503304cb35dfe546bce9c6684e7 
> (refs/remotes/origin/branch_6x)
> Using Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC
> {noformat}
> The failure happened during the static setup of the test, when a 
> MiniSolrCloudCluster & several clients are initialized -- before any code 
> related to TolerantUpdateProcessor is ever used.
> I can't reproduce this, or really make sense of what i'm (not) seeing here in 
> the logs, so i'm filing this jira with my analysis in the hopes that someone 
> else can help make sense of it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8914) ZkStateReader's refreshLiveNodes(Watcher) is not thread safe

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250882#comment-15250882
 ] 

ASF subversion and git services commented on SOLR-8914:
---

Commit b93cb2714b7a2ea43ae200d31cc5b04a0b0ecc7c in lucene-solr's branch 
refs/heads/branch_6_0 from [~dragonsinth]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b93cb27 ]

SOLR-8914: ZkStateReader's refreshLiveNodes(Watcher) is not thread safe.


> ZkStateReader's refreshLiveNodes(Watcher) is not thread safe
> 
>
> Key: SOLR-8914
> URL: https://issues.apache.org/jira/browse/SOLR-8914
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 5.5, master, 6.0, 5.4.1
>Reporter: Hoss Man
>Assignee: Scott Blum
> Fix For: master, 6.1
>
> Attachments: SOLR-8914.patch, SOLR-8914.patch, SOLR-8914.patch, 
> SOLR-8914.patch, jenkins.thetaphi.de_Lucene-Solr-6.x-Solaris_32.log.txt, 
> live_node_mentions_port56361_with_threadIds.log.txt, 
> live_nodes_mentions.log.txt
>
>
> Jenkin's encountered a failure in TestTolerantUpdateProcessorCloud over the 
> weekend
> {noformat}
> http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Solaris/32/consoleText
> Checking out Revision c46d7686643e7503304cb35dfe546bce9c6684e7 
> (refs/remotes/origin/branch_6x)
> Using Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC
> {noformat}
> The failure happened during the static setup of the test, when a 
> MiniSolrCloudCluster & several clients are initialized -- before any code 
> related to TolerantUpdateProcessor is ever used.
> I can't reproduce this, or really make sense of what i'm (not) seeing here in 
> the logs, so i'm filing this jira with my analysis in the hopes that someone 
> else can help make sense of it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8914) ZkStateReader's refreshLiveNodes(Watcher) is not thread safe

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250879#comment-15250879
 ] 

ASF subversion and git services commented on SOLR-8914:
---

Commit b93cb2714b7a2ea43ae200d31cc5b04a0b0ecc7c in lucene-solr's branch 
refs/heads/SOLR-8914-6_0 from [~dragonsinth]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b93cb27 ]

SOLR-8914: ZkStateReader's refreshLiveNodes(Watcher) is not thread safe.


> ZkStateReader's refreshLiveNodes(Watcher) is not thread safe
> 
>
> Key: SOLR-8914
> URL: https://issues.apache.org/jira/browse/SOLR-8914
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 5.5, master, 6.0, 5.4.1
>Reporter: Hoss Man
>Assignee: Scott Blum
> Fix For: master, 6.1
>
> Attachments: SOLR-8914.patch, SOLR-8914.patch, SOLR-8914.patch, 
> SOLR-8914.patch, jenkins.thetaphi.de_Lucene-Solr-6.x-Solaris_32.log.txt, 
> live_node_mentions_port56361_with_threadIds.log.txt, 
> live_nodes_mentions.log.txt
>
>
> Jenkin's encountered a failure in TestTolerantUpdateProcessorCloud over the 
> weekend
> {noformat}
> http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Solaris/32/consoleText
> Checking out Revision c46d7686643e7503304cb35dfe546bce9c6684e7 
> (refs/remotes/origin/branch_6x)
> Using Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC
> {noformat}
> The failure happened during the static setup of the test, when a 
> MiniSolrCloudCluster & several clients are initialized -- before any code 
> related to TolerantUpdateProcessor is ever used.
> I can't reproduce this, or really make sense of what i'm (not) seeing here in 
> the logs, so i'm filing this jira with my analysis in the hopes that someone 
> else can help make sense of it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9022) solr unable to handle/parse images when they are embedded in office docs(like word,xls,etc)

[Bhanuprakash Prathap (JIRA)]

Bhanuprakash Prathap created SOLR-9022:
--

 Summary: solr unable to handle/parse images when they are embedded 
in office docs(like word,xls,etc)
 Key: SOLR-9022
 URL: https://issues.apache.org/jira/browse/SOLR-9022
 Project: Solr
  Issue Type: Bug
  Components: SolrJ
Affects Versions: 5.4.1
Reporter: Bhanuprakash Prathap


As we are trying to index multiple files, the solr throws below exception 
whenever it encounters embedded images with other docs.

The issues arises as embedded images files are read with MIME type of 
binary(attachment_mimetype=[application/octet-stream])  though the attached 
files are type png/txt etc.

Full stack trace for this issue

2016-04-20 16:55:13,311 INFO [Thread-52] 
org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Added 
attempt_1460872551948_58233_m_000116_1 to list of failed maps
2016-04-20 16:55:13,329 INFO [IPC Server handler 18 on 39376] 
org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt 
attempt_1460872551948_58233_m_000159_0 is : 0.0
2016-04-20 16:55:13,342 ERROR [IPC Server handler 20 on 39376] 
org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: 
attempt_1460872551948_58233_m_000159_0 - exited : 
org.kitesdk.morphline.api.MorphlineRuntimeException: 
org.kitesdk.morphline.api.MorphlineRuntimeException: tryRules command found no 
successful rule for record: {_attachment_body=[TikaInputStream of 
java.io.BufferedInputStream@391f1777], 
_attachment_mimetype=[application/octet-stream], 
_attachment_name=[xl/media/image2.png], 
id=[185be63a-e527-4953-9a3d-ae957dc0fa51]}
at 
org.kitesdk.morphline.base.FaultTolerance.handleException(FaultTolerance.java:73)
at 
org.apache.solr.hadoop.morphline.MorphlineMapRunner.map(MorphlineMapRunner.java:220)
at 
org.apache.solr.hadoop.morphline.MorphlineMapper.map(MorphlineMapper.java:86)
at 
org.apache.solr.hadoop.morphline.MorphlineMapper.map(MorphlineMapper.java:54)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.kitesdk.morphline.api.MorphlineRuntimeException: tryRules 
command found no successful rule for record: {_attachment_body=[TikaInputStream 
of java.io.BufferedInputStream@391f1777], 
_attachment_mimetype=[application/octet-stream], 
_attachment_name=[xl/media/image2.png], 
id=[185be63a-e527-4953-9a3d-ae957dc0fa51]}
at 
org.kitesdk.morphline.stdlib.TryRulesBuilder$TryRules.doProcess(TryRulesBuilder.java:132)
at 
org.kitesdk.morphline.base.AbstractCommand.process(AbstractCommand.java:156)
at org.kitesdk.morphline.base.Connector.process(Connector.java:64)
at 
org.kitesdk.morphline.base.AbstractCommand.doProcess(AbstractCommand.java:181)
at 
org.kitesdk.morphline.tika.DetectMimeTypeBuilder$DetectMimeType.doProcess(DetectMimeTypeBuilder.java:166)
at 
org.kitesdk.morphline.base.AbstractCommand.process(AbstractCommand.java:156)
at org.kitesdk.morphline.base.Connector.process(Connector.java:64)
at 
org.kitesdk.morphline.base.AbstractCommand.doProcess(AbstractCommand.java:181)
at 
org.kitesdk.morphline.stdlib.SeparateAttachmentsBuilder$SeparateAttachments.doProcess(SeparateAttachmentsBuilder.java:79)
at 
org.kitesdk.morphline.base.AbstractCommand.process(AbstractCommand.java:156)
at 
org.kitesdk.morphline.base.AbstractCommand.doProcess(AbstractCommand.java:181)
at 
org.kitesdk.morphline.base.AbstractCommand.process(AbstractCommand.java:156)
at 
org.kitesdk.morphline.base.AbstractCommand.doProcess(AbstractCommand.java:181)
at 
org.kitesdk.morphline.base.AbstractCommand.process(AbstractCommand.java:156)
at org.kitesdk.morphline.base.Connector.process(Connector.java:64)
at 
org.kitesdk.morphline.base.AbstractCommand.doProcess(AbstractCommand.java:181)
at 
org.kitesdk.morphline.stdlib.GenerateUUIDBuilder$GenerateUUID.doProcess(GenerateUUIDBuilder.java:98)
at 
org.kitesdk.morphline.base.AbstractCommand.process(AbstractCommand.java:156)
at org.kitesdk.morphline.base.Connector.process(Connector.java:64)
at 
org.kitesdk.morphline.tika.decompress.EmbeddedExtractor.parseEmbedded(EmbeddedExtractor.java:57)
at 
org.kitesdk.morphline.tika.decompress.UnpackBuilder$Unpack.parseEntry(UnpackBuilder.java:138)
at 

[jira] [Commented] (SOLR-8716) Upgrade to Apache Tika 1.12

[Lewis John McGibbney (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250819#comment-15250819
 ] 

Lewis John McGibbney commented on SOLR-8716:


PR is updated to
1) ensure that the new dependencies involved in the above parsers are 
lexicographically ordered in lucene/ivy-versions.properties, and
2) that they are included within solr/contrib/extraction/ivy.xml

> Upgrade to Apache Tika 1.12
> ---
>
> Key: SOLR-8716
> URL: https://issues.apache.org/jira/browse/SOLR-8716
> Project: Solr
>  Issue Type: Improvement
>  Components: contrib - Solr Cell (Tika extraction)
>Reporter: Lewis John McGibbney
>Assignee: Uwe Schindler
> Fix For: master
>
> Attachments: LUCENE-7041.patch
>
>
> We recently released Apache Tika 1.12. In order to use the fixes provided 
> within the Tika.translate API I propose to upgrade Tika from 1.7 --> 1.12 in 
> lucene/ivy-versions.properties.
> Patch coming up.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-MacOSX (64bit/jdk1.8.0) - Build # 90 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-MacOSX/90/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  
org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigReplication

Error Message:
Error from server at http://127.0.0.1:54550/solr/collection1: Expected mime 
type application/octet-stream but got text/html.Error 
500HTTP ERROR: 500 Problem accessing 
/solr/collection1/select. Reason: 
{metadata={error-class=org.apache.solr.common.SolrException,root-error-class=org.xml.sax.SAXParseException},msg=SolrCore
 collection1 is not available due to init failure: Could not load 
conf for core collection1: Cant load schema 
/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E11700EF6E37A702-001/solr-instance-022/collection1/conf/schema.xml:
 org.xml.sax.SAXParseException; systemId: solrres:/schema.xml; lineNumber: 1; 
columnNumber: 1; Premature end of 
file.,trace=org.apache.solr.common.SolrException: SolrCore 
collection1 is not available due to init failure: Could not load 
conf for core collection1: Cant load schema 
/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E11700EF6E37A702-001/solr-instance-022/collection1/conf/schema.xml:
 org.xml.sax.SAXParseException; systemId: solrres:/schema.xml; lineNumber: 1; 
columnNumber: 1; Premature end of file.  at 
org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1067)  at 
org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:252)  at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:414)  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:229)
  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:184)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
  at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) 
 at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)  
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  
at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)  
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) 
 at org.eclipse.jetty.server.Server.handle(Server.java:518)  at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)  at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)  at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
  at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)  at 
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) 
 at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
  at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
  at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
  at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) 
 at java.lang.Thread.run(Thread.java:745) Caused by: 
org.apache.solr.common.SolrException: Could not load conf for core collection1: 
Cant load schema 
/Users/jenkins/workspace/Lucene-Solr-6.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E11700EF6E37A702-001/solr-instance-022/collection1/conf/schema.xml:
 org.xml.sax.SAXParseException; systemId: solrres:/schema.xml; lineNumber: 1; 
columnNumber: 1; Premature end of file.  at 
org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:86)  at 
org.apache.solr.core.CoreContainer.create(CoreContainer.java:810)  at 
org.apache.solr.core.CoreContainer.lambda$load$0(CoreContainer.java:466)  at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)  at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
 ... 1 more Caused by: org.apache.solr.common.SolrException: Cant load 
schema 

[JENKINS] Lucene-Solr-Tests-master - Build # 1094 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/1094/

1 tests failed.
FAILED:  
org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh

Error Message:
Could not find collection : c1

Stack Trace:
org.apache.solr.common.SolrException: Could not find collection : c1
at 
__randomizedtesting.SeedInfo.seed([106C213269B6897F:FD650C5B9D64FBA]:0)
at 
org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:170)
at 
org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdate(ZkStateReaderTest.java:135)
at 
org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh(ZkStateReaderTest.java:41)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11179 lines...]
   [junit4] Suite: org.apache.solr.cloud.overseer.ZkStateReaderTest
   [junit4]   2> Creating dataDir: 

[jira] [Updated] (SOLR-8845) SolrJ JDBC - Ensure that Spark works with SolrJ JDBC

[Kevin Risden (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Risden updated SOLR-8845:
---
Description: 
Ensure that Spark is able work with SolrJ JDBC.  
Currently, in Spark 1.6 there are 2 known issues:
1. SparkSQL query's via a "select *" query
2. SparkSQL query's via a select query with a 1=0 where clause

  was:
Ensure that Spark is able work with SolrJ JDBC.  
Currently, in Spark 1.6 there are 2 known issues:
1. SparkSQL uses a connection.prepareStatement() call, which returns null in 
the SolrJ implementation
2. SparkSQL query's via a "select *" query, which is currently not supported.


> SolrJ JDBC - Ensure that Spark works with SolrJ JDBC
> 
>
> Key: SOLR-8845
> URL: https://issues.apache.org/jira/browse/SOLR-8845
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master
>Reporter: Trey Cahill
>
> Ensure that Spark is able work with SolrJ JDBC.  
> Currently, in Spark 1.6 there are 2 known issues:
> 1. SparkSQL query's via a "select *" query
> 2. SparkSQL query's via a select query with a 1=0 where clause



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8845) SolrJ JDBC - Ensure that Spark works with SolrJ JDBC

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250710#comment-15250710
 ] 

Joel Bernstein commented on SOLR-8845:
--

I can take a look and see what needs to be done to support the 1=0 query. 

> SolrJ JDBC - Ensure that Spark works with SolrJ JDBC
> 
>
> Key: SOLR-8845
> URL: https://issues.apache.org/jira/browse/SOLR-8845
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master
>Reporter: Trey Cahill
>
> Ensure that Spark is able work with SolrJ JDBC.  
> Currently, in Spark 1.6 there are 2 known issues:
> 1. SparkSQL uses a connection.prepareStatement() call, which returns null in 
> the SolrJ implementation
> 2. SparkSQL query's via a "select *" query, which is currently not supported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8845) SolrJ JDBC - Ensure that Spark works with SolrJ JDBC

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250706#comment-15250706
 ] 

Kevin Risden commented on SOLR-8845:


Explanation via Stackoverflow 
http://stackoverflow.com/questions/9140606/why-would-you-use-where-1-0-statement-in-sql

Makes sure that no rows are sent back.

> SolrJ JDBC - Ensure that Spark works with SolrJ JDBC
> 
>
> Key: SOLR-8845
> URL: https://issues.apache.org/jira/browse/SOLR-8845
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master
>Reporter: Trey Cahill
>
> Ensure that Spark is able work with SolrJ JDBC.  
> Currently, in Spark 1.6 there are 2 known issues:
> 1. SparkSQL uses a connection.prepareStatement() call, which returns null in 
> the SolrJ implementation
> 2. SparkSQL query's via a "select *" query, which is currently not supported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8845) SolrJ JDBC - Ensure that Spark works with SolrJ JDBC

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250701#comment-15250701
 ] 

Joel Bernstein commented on SOLR-8845:
--

Why not just use the JDBC meta data queries anyway? Why this weirdness?

> SolrJ JDBC - Ensure that Spark works with SolrJ JDBC
> 
>
> Key: SOLR-8845
> URL: https://issues.apache.org/jira/browse/SOLR-8845
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master
>Reporter: Trey Cahill
>
> Ensure that Spark is able work with SolrJ JDBC.  
> Currently, in Spark 1.6 there are 2 known issues:
> 1. SparkSQL uses a connection.prepareStatement() call, which returns null in 
> the SolrJ implementation
> 2. SparkSQL query's via a "select *" query, which is currently not supported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8845) SolrJ JDBC - Ensure that Spark works with SolrJ JDBC

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250699#comment-15250699
 ] 

Kevin Risden commented on SOLR-8845:


Yea well here is the Apache Spark source for JDBCRDD for that method:

https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala#L109

> SolrJ JDBC - Ensure that Spark works with SolrJ JDBC
> 
>
> Key: SOLR-8845
> URL: https://issues.apache.org/jira/browse/SOLR-8845
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master
>Reporter: Trey Cahill
>
> Ensure that Spark is able work with SolrJ JDBC.  
> Currently, in Spark 1.6 there are 2 known issues:
> 1. SparkSQL uses a connection.prepareStatement() call, which returns null in 
> the SolrJ implementation
> 2. SparkSQL query's via a "select *" query, which is currently not supported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8845) SolrJ JDBC - Ensure that Spark works with SolrJ JDBC

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250695#comment-15250695
 ] 

Joel Bernstein commented on SOLR-8845:
--

Where 1=0 assumes a lot about what a SQL engine supports.

> SolrJ JDBC - Ensure that Spark works with SolrJ JDBC
> 
>
> Key: SOLR-8845
> URL: https://issues.apache.org/jira/browse/SOLR-8845
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master
>Reporter: Trey Cahill
>
> Ensure that Spark is able work with SolrJ JDBC.  
> Currently, in Spark 1.6 there are 2 known issues:
> 1. SparkSQL uses a connection.prepareStatement() call, which returns null in 
> the SolrJ implementation
> 2. SparkSQL query's via a "select *" query, which is currently not supported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8845) SolrJ JDBC - Ensure that Spark works with SolrJ JDBC

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250685#comment-15250685
 ] 

Joel Bernstein commented on SOLR-8845:
--

That's a weird query

> SolrJ JDBC - Ensure that Spark works with SolrJ JDBC
> 
>
> Key: SOLR-8845
> URL: https://issues.apache.org/jira/browse/SOLR-8845
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master
>Reporter: Trey Cahill
>
> Ensure that Spark is able work with SolrJ JDBC.  
> Currently, in Spark 1.6 there are 2 known issues:
> 1. SparkSQL uses a connection.prepareStatement() call, which returns null in 
> the SolrJ implementation
> 2. SparkSQL query's via a "select *" query, which is currently not supported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9014) Audit all usages of ClusterState methods which may make calls to ZK via the lazy collection reference

[Shalin Shekhar Mangar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250642#comment-15250642
 ] 

Shalin Shekhar Mangar edited comment on SOLR-9014 at 4/20/16 8:43 PM:
--

This is turning out to be interesting. We have to revisit a few assumptions:

# The Overseer has a wait loop to see a certain condition to be true in many 
places. The earlier assumption was that updateClusterState was expensive and 
therefore it was better to wait until you see the state. But not that we have 
lazy collections and a collection specific forceUpdateCollection, the wait loop 
is actually as expensive because it ends up reading the collection state from 
ZooKeeper -- sometime as frequently as 100ms. We should return the resolved 
reference from ZkStateReader#forceUpdateCollection and use it in such places.
# The ClusterState#getCollections was supposed to be lightweight i.e. it just 
read and returned the names of known collections from local cached state. This 
was changed in SOLR-6629 to resolve the reference. This means that it ends up 
going to ZK for each non-watched collection. So API calls like downnode etc are 
way more expensive than needed. It is better to start returning a 
List from this method instead.


was (Author: shalinmangar):
This is turning out to be interesting. We have to revisit a few assumptions:

# The Overseer has a wait loop to see a certain condition to be true in many 
places. The earlier assumption was that updateClusterState was expensive and 
therefore it was better to wait until you see the state. But not that we have 
lazy collections and a collection specific forceUpdateCollection, the wait loop 
is actually as expensive because it ends up reading the collection state from 
ZooKeeper -- sometime as frequently as 100ms. We should return the resolved 
reference from ZkStateReader#forceUpdateCollection and use it in such places.
# The ClusterState#getCollections was supposed to be lightweight i.e. it just 
read and returned the names of known collections from local cached state. This 
was changed in SOLR-6629 to resolve the reference. This means that it ends up 
going to ZK for each non-watched collection. So API calls like LIST, downnode 
etc have become way more expensive. It is better to start returning a 
List from this method instead.

> Audit all usages of ClusterState methods which may make calls to ZK via the 
> lazy collection reference
> -
>
> Key: SOLR-9014
> URL: https://issues.apache.org/jira/browse/SOLR-9014
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
> Fix For: master, 6.1
>
>
> ClusterState has a bunch of methods such as getSlice and getReplica which 
> internally call getCollectionOrNull that ends up making a call to ZK via the 
> lazy collection reference. Many classes use these methods even though a 
> DocCollection object is available. In such cases, multiple redundant calls to 
> ZooKeeper can happen if the collection is not watched locally. This is 
> especially true for Overseer classes which operate on all collections.
> We should audit all usages of these methods and replace them with calls to 
> appropriate DocCollection methods.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8845) SolrJ JDBC - Ensure that Spark works with SolrJ JDBC

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250673#comment-15250673
 ] 

Kevin Risden commented on SOLR-8845:


The query is:

{code}
SELECT * FROM test WHERE 1=0
{code}

What is happening is the where clause is being processed and since the 1 is a 
LongLiteral you get the class cast exception in the SQLHandler.

> SolrJ JDBC - Ensure that Spark works with SolrJ JDBC
> 
>
> Key: SOLR-8845
> URL: https://issues.apache.org/jira/browse/SOLR-8845
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master
>Reporter: Trey Cahill
>
> Ensure that Spark is able work with SolrJ JDBC.  
> Currently, in Spark 1.6 there are 2 known issues:
> 1. SparkSQL uses a connection.prepareStatement() call, which returns null in 
> the SolrJ implementation
> 2. SparkSQL query's via a "select *" query, which is currently not supported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9014) Audit all usages of ClusterState methods which may make calls to ZK via the lazy collection reference

[Shalin Shekhar Mangar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250642#comment-15250642
 ] 

Shalin Shekhar Mangar edited comment on SOLR-9014 at 4/20/16 8:28 PM:
--

This is turning out to be interesting. We have to revisit a few assumptions:

# The Overseer has a wait loop to see a certain condition to be true in many 
places. The earlier assumption was that updateClusterState was expensive and 
therefore it was better to wait until you see the state. But not that we have 
lazy collections and a collection specific forceUpdateCollection, the wait loop 
is actually as expensive because it ends up reading the collection state from 
ZooKeeper -- sometime as frequently as 100ms. We should return the resolved 
reference from ZkStateReader#forceUpdateCollection and use it in such places.
# The ClusterState#getCollections was supposed to be lightweight i.e. it just 
read and returned the names of known collections from local cached state. This 
was changed in SOLR-6629 to resolve the reference. This means that it ends up 
going to ZK for each non-watched collection. So API calls like LIST, downnode 
etc have become way more expensive. It is better to start returning a 
List from this method instead.


was (Author: shalinmangar):
This is turning out to be interesting. We have to revisit a few assumptions:

# The Overseer has a wait loop to see a certain condition to be true in many 
places. The earlier assumption was that updateClusterState was expensive and 
therefore it was better to wait until you see the state. But not that we have 
lazy collections and a collection specific forceUpdateCollection, the wait look 
is actually more expensive because it ends up reading the collection state from 
ZooKeeper -- sometime as frequently as 100ms.
# The ClusterState#getCollections was supposed to be lightweight i.e. it just 
read and returned the names of known collections from local cached state. This 
was changed in SOLR-6629 to resolve the reference. This means that it ends up 
going to ZK for each non-watched collection. So API calls like LIST, downnode 
etc have become way more expensive. It is better to start returning a 
List from this method instead.

> Audit all usages of ClusterState methods which may make calls to ZK via the 
> lazy collection reference
> -
>
> Key: SOLR-9014
> URL: https://issues.apache.org/jira/browse/SOLR-9014
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
> Fix For: master, 6.1
>
>
> ClusterState has a bunch of methods such as getSlice and getReplica which 
> internally call getCollectionOrNull that ends up making a call to ZK via the 
> lazy collection reference. Many classes use these methods even though a 
> DocCollection object is available. In such cases, multiple redundant calls to 
> ZooKeeper can happen if the collection is not watched locally. This is 
> especially true for Overseer classes which operate on all collections.
> We should audit all usages of these methods and replace them with calls to 
> appropriate DocCollection methods.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9014) Audit all usages of ClusterState methods which may make calls to ZK via the lazy collection reference

[Shalin Shekhar Mangar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250642#comment-15250642
 ] 

Shalin Shekhar Mangar commented on SOLR-9014:
-

This is turning out to be interesting. We have to revisit a few assumptions:

# The Overseer has a wait loop to see a certain condition to be true in many 
places. The earlier assumption was that updateClusterState was expensive and 
therefore it was better to wait until you see the state. But not that we have 
lazy collections and a collection specific forceUpdateCollection, the wait look 
is actually more expensive because it ends up reading the collection state from 
ZooKeeper -- sometime as frequently as 100ms.
# The ClusterState#getCollections was supposed to be lightweight i.e. it just 
read and returned the names of known collections from local cached state. This 
was changed in SOLR-6629 to resolve the reference. This means that it ends up 
going to ZK for each non-watched collection. So API calls like LIST, downnode 
etc have become way more expensive. It is better to start returning a 
List from this method instead.

> Audit all usages of ClusterState methods which may make calls to ZK via the 
> lazy collection reference
> -
>
> Key: SOLR-9014
> URL: https://issues.apache.org/jira/browse/SOLR-9014
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
> Fix For: master, 6.1
>
>
> ClusterState has a bunch of methods such as getSlice and getReplica which 
> internally call getCollectionOrNull that ends up making a call to ZK via the 
> lazy collection reference. Many classes use these methods even though a 
> DocCollection object is available. In such cases, multiple redundant calls to 
> ZooKeeper can happen if the collection is not watched locally. This is 
> especially true for Overseer classes which operate on all collections.
> We should audit all usages of these methods and replace them with calls to 
> appropriate DocCollection methods.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9017) Implement PreparedStatementImpl parameterization

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250639#comment-15250639
 ] 

Kevin Risden commented on SOLR-9017:


Some of the references I've been using to understand PreparedStatement usage 
and implementation:
* https://docs.oracle.com/javase/tutorial/jdbc/basics/prepared.html
* https://docs.oracle.com/javase/8/docs/api/java/sql/PreparedStatement.html
* https://www.owasp.org/index.php/Query_Parameterization_Cheat_Sheet

Should probably look at how MySQL does it.

> Implement PreparedStatementImpl parameterization
> 
>
> Key: SOLR-9017
> URL: https://issues.apache.org/jira/browse/SOLR-9017
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master
>Reporter: Kevin Risden
>Assignee: Kevin Risden
> Attachments: SOLR-9017.patch
>
>
> SOLR-8809 implemented prepared statements to avoid a NPE when clients were 
> connecting. The next step is to flesh out the rest of the class and implement 
> parameterization. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-8827) SolrJ JDBC - Ensure that SQuirrel SQL works with SolrJ JDBC

[Kevin Risden (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Risden reassigned SOLR-8827:
--

Assignee: Kevin Risden

> SolrJ JDBC - Ensure that SQuirrel SQL works with SolrJ JDBC
> ---
>
> Key: SOLR-8827
> URL: https://issues.apache.org/jira/browse/SOLR-8827
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master
>Reporter: Kevin Risden
>Assignee: Kevin Risden
> Attachments: SQuirreL_SQL_Client_Version_3_7.png
>
>
> There are a bunch of NPE exceptions that SolrJ JDBC causes in SquirrelSQL. 
> These need to be tracked down and fixed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-9011) SolrJ JDBC - Ensure that Python JayDeBeApi works with SolrJ JDBC

[Kevin Risden (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Risden reassigned SOLR-9011:
--

Assignee: Kevin Risden

> SolrJ JDBC - Ensure that Python JayDeBeApi works with SolrJ JDBC
> 
>
> Key: SOLR-9011
> URL: https://issues.apache.org/jira/browse/SOLR-9011
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master, 6.0
>Reporter: Kevin Risden
>Assignee: Kevin Risden
> Fix For: 6.1
>
>
> Python with JayDeBeApi enables connections to JDBC sources.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-9019) SolrJ JDBC - Ensure that R RJDBC works with SolrJ JDBC

[Kevin Risden (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Risden resolved SOLR-9019.

   Resolution: Fixed
 Assignee: Kevin Risden
Fix Version/s: 6.1
   master

Since SOLR-9020 was committed, R RJDBC now works.
{code}
/opt/scripts/test_solr.R
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
trying URL 'https://cran.rstudio.com/src/contrib/RJDBC_0.2-5.tar.gz'
Content type 'application/x-gzip' length 13879 bytes (13 KB)
==
downloaded 13 KB

* installing *source* package ‘RJDBC’ ...
** package ‘RJDBC’ successfully unpacked and MD5 sums checked
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (RJDBC)

The downloaded source packages are in
‘/tmp/RtmpnAFbYC/downloaded_packages’
Loading required package: methods
Loading required package: DBI
Loading required package: rJava
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further 
details.
SLF4J: Failed to load class "org.slf4j.impl.StaticMDCBinder".
SLF4J: Defaulting to no-operation MDCAdapter implementation.
SLF4J: See http://www.slf4j.org/codes.html#no_static_mdc_binder for further 
details.
  fielda fieldb fieldc fieldd_s fielde_i
1 a1 b1  1   d11
2 a2 b2  2   d12
3 a1 b3  3 1
4 a1 b4  4   d2   NA
5 a2 b2 NA   d22
[1] TRUE
{code}

> SolrJ JDBC - Ensure that R RJDBC works with SolrJ JDBC
> --
>
> Key: SOLR-9019
> URL: https://issues.apache.org/jira/browse/SOLR-9019
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Reporter: Kevin Risden
>Assignee: Kevin Risden
> Fix For: master, 6.1
>
>
> R has RJDBC (https://cran.r-project.org/web/packages/RJDBC/index.html) which 
> can connect to JDBC. Check that it works with SolrJ JDBC.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9017) Implement PreparedStatementImpl parameterization

[Kevin Risden (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Risden updated SOLR-9017:
---
Attachment: SOLR-9017.patch

First pass at implementation. This needs some more tests and another look at 
it. Currently there are 2 questions:

1. If not all parameters are set before execution what happens?
2. For strings, do quotes need to be added for the string replacements?

> Implement PreparedStatementImpl parameterization
> 
>
> Key: SOLR-9017
> URL: https://issues.apache.org/jira/browse/SOLR-9017
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master
>Reporter: Kevin Risden
>Assignee: Kevin Risden
> Attachments: SOLR-9017.patch
>
>
> SOLR-8809 implemented prepared statements to avoid a NPE when clients were 
> connecting. The next step is to flesh out the rest of the class and implement 
> parameterization. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-9020) Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper errors for traversal methods

[Kevin Risden (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Risden resolved SOLR-9020.

   Resolution: Fixed
Fix Version/s: 6.1
   master

> Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper 
> errors for traversal methods
> 
>
> Key: SOLR-9020
> URL: https://issues.apache.org/jira/browse/SOLR-9020
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master, 6.0
>Reporter: Kevin Risden
>Assignee: Kevin Risden
> Fix For: master, 6.1
>
> Attachments: SOLR-9020.patch
>
>
> There are 4 methods related to fetch in StatementImpl and 4 methods related 
> to fetch in ResultSetImpl. ResultSetImpl has some traversal methods that 
> don't make sense with the fetch direction. It would make sense to implement 
> them to support more SQL clients.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9020) Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper errors for traversal methods

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250583#comment-15250583
 ] 

ASF subversion and git services commented on SOLR-9020:
---

Commit d914ec4a67c42466f19e46169754965a7d5af35c in lucene-solr's branch 
refs/heads/branch_6x from [~risdenk]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d914ec4 ]

SOLR-9020: Implement StatementImpl/ResultSetImpl get/set fetch* methods and 
proper errors for traversal methods


> Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper 
> errors for traversal methods
> 
>
> Key: SOLR-9020
> URL: https://issues.apache.org/jira/browse/SOLR-9020
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master, 6.0
>Reporter: Kevin Risden
>Assignee: Kevin Risden
> Attachments: SOLR-9020.patch
>
>
> There are 4 methods related to fetch in StatementImpl and 4 methods related 
> to fetch in ResultSetImpl. ResultSetImpl has some traversal methods that 
> don't make sense with the fetch direction. It would make sense to implement 
> them to support more SQL clients.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9020) Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper errors for traversal methods

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250575#comment-15250575
 ] 

ASF subversion and git services commented on SOLR-9020:
---

Commit a9a842f05dcebb6b86a6f47354aa760a83763e6c in lucene-solr's branch 
refs/heads/master from [~risdenk]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a9a842f ]

SOLR-9020: Implement StatementImpl/ResultSetImpl get/set fetch* methods and 
proper errors for traversal methods


> Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper 
> errors for traversal methods
> 
>
> Key: SOLR-9020
> URL: https://issues.apache.org/jira/browse/SOLR-9020
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master, 6.0
>Reporter: Kevin Risden
>Assignee: Kevin Risden
> Attachments: SOLR-9020.patch
>
>
> There are 4 methods related to fetch in StatementImpl and 4 methods related 
> to fetch in ResultSetImpl. ResultSetImpl has some traversal methods that 
> don't make sense with the fetch direction. It would make sense to implement 
> them to support more SQL clients.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Term vs. token

[Jack Krupansky]

I gather that "term" is the proper technical term within the Vector Space
Model (TDIFS) and BM25 similarity, so it may simply be a question of where
the boundary is in Lucene between VSM processing and other stuff, like the
source for documents and queries.

-- Jack Krupansky

On Wed, Apr 20, 2016 at 1:51 PM, Ryan Josal  wrote:

> My understanding is a Term is comprised of a "token" and a field.  So then
> the documentation makes sense to me - return the count of tokens in a field
> for example.  But there were a couple of references you had there that
> don't match with that definition, like the number of tokens in a
> collection.  Although maybe a Term doesn't have a whole token because what
> about token attributes like payload.  I guess I've convinced myself I'm not
> entirely clear about it either, but I do feel good about the concept that
> tokens don't have fields.  You can tokenize a string without thinking about
> fields, and they become terms with fields when you query.
>
> Ryan
>
>
> On Wednesday, April 20, 2016, Jack Krupansky 
> wrote:
>
>> Looking at the Lucene Similarity Javadoc, I see some references to
>> tokens, but I am wondering if that is intentional or whether those should
>> really be references to terms.
>>
>> For example:
>>
>>  *lengthNorm - computed
>>  *when the document is added to the index in accordance with the
>> number of tokens
>>  *of this field in the document, so that shorter fields
>> contribute more to the score.
>>
>> I think that should be terms, not tokens.
>>
>> See:
>>
>> https://github.com/apache/lucene-solr/blob/releases/lucene-solr/5.5.0/lucene/core/src/java/org/apache/lucene/search/similarities/TFIDFSimilarity.java#L466
>>
>> And this:
>>
>>* Returns the total number of tokens in the field.
>>* @see Terms#getSumTotalTermFreq()
>>*/
>>   public long getNumberOfFieldTokens() {
>> return numberOfFieldTokens;
>>
>> I think that should be terms as well:
>>
>> See:
>>
>> https://github.com/apache/lucene-solr/blob/master/lucene/core/src/java/org/apache/lucene/search/similarities/BasicStats.java#L65
>>
>> And... this:
>>
>>   numberOfFieldTokens = sumTotalTermFreq;
>>
>> Where it is clearly starting with terms and treating them as tokens, but
>> as in the previous example, I think that should be terms as well.
>>
>> See:
>>
>> https://github.com/apache/lucene-solr/blob/master/lucene/core/src/java/org/apache/lucene/search/similarities/SimilarityBase.java#L128
>>
>> One last example:
>>
>>* Compute any collection-level weight (e.g. IDF, average document
>> length, etc) needed for scoring a query.
>>*
>>* @param collectionStats collection-level statistics, such as the
>> number of tokens in the collection.
>>* @param termStats term-level statistics, such as the document
>> frequency of a term across the collection.
>>* @return SimWeight object with the information this Similarity needs
>> to score a query.
>>*/
>>   public abstract SimWeight computeWeight(CollectionStatistics
>> collectionStats, TermStatistics... termStats);
>>
>> See:
>>
>> https://github.com/apache/lucene-solr/blob/master/lucene/core/src/java/org/apache/lucene/search/similarities/Similarity.java#L161
>>
>> In fact, CollectionStatistics uses term, not token:
>>
>>   /** returns the total number of tokens for this field
>>* @see Terms#getSumTotalTermFreq() */
>>   public final long sumTotalTermFreq() {
>> return sumTotalTermFreq;
>>
>> Oops... it uses both, emphasizing my point about the confusion.
>>
>> There are other examples as well.
>>
>> My understanding is that tokens are merely a temporary transition in
>> between the original raw source text for a field and then final terms to be
>> indexed (or query terms from a parsed and analyzed query.) Yes, during and
>> within TokenStream or the analyzer we speak of tokens and intermediate
>> string values are referred to as tokens, but once the final string value is
>> retrieved from the Token Stream (analyzer), it's a term.
>>
>> In any case, is there some distinction in any of these cited examples (or
>> other examples in this or related code) where "token" is an important
>> distinction to be made and "term" is not the proper... term... to be used?
>>
>> Unless the Lucene project fully intends that the terms token and term are
>> absolutely synonymous, a clear distinction should be drawn... I think. Or
>> at least the terms should be used consistently, which my last example
>> highlights.
>>
>> Thanks.
>>
>> -- Jack Krupansky
>>
>

[jira] [Commented] (SOLR-9020) Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper errors for traversal methods

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250548#comment-15250548
 ] 

Joel Bernstein commented on SOLR-9020:
--

Looks good to me.

> Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper 
> errors for traversal methods
> 
>
> Key: SOLR-9020
> URL: https://issues.apache.org/jira/browse/SOLR-9020
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master, 6.0
>Reporter: Kevin Risden
>Assignee: Kevin Risden
> Attachments: SOLR-9020.patch
>
>
> There are 4 methods related to fetch in StatementImpl and 4 methods related 
> to fetch in ResultSetImpl. ResultSetImpl has some traversal methods that 
> don't make sense with the fetch direction. It would make sense to implement 
> them to support more SQL clients.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9017) Implement PreparedStatementImpl parameterization

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250536#comment-15250536
 ] 

Kevin Risden commented on SOLR-9017:


Depends upon some of the cleanup for StatementImpl from SOLR-9020.

> Implement PreparedStatementImpl parameterization
> 
>
> Key: SOLR-9017
> URL: https://issues.apache.org/jira/browse/SOLR-9017
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master
>Reporter: Kevin Risden
>Assignee: Kevin Risden
>
> SOLR-8809 implemented prepared statements to avoid a NPE when clients were 
> connecting. The next step is to flesh out the rest of the class and implement 
> parameterization. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9020) Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper errors for traversal methods

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250534#comment-15250534
 ] 

Kevin Risden commented on SOLR-9020:


This should be good to go. [~joel.bernstein] - Can you take a look?

> Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper 
> errors for traversal methods
> 
>
> Key: SOLR-9020
> URL: https://issues.apache.org/jira/browse/SOLR-9020
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master, 6.0
>Reporter: Kevin Risden
>Assignee: Kevin Risden
> Attachments: SOLR-9020.patch
>
>
> There are 4 methods related to fetch in StatementImpl and 4 methods related 
> to fetch in ResultSetImpl. ResultSetImpl has some traversal methods that 
> don't make sense with the fetch direction. It would make sense to implement 
> them to support more SQL clients.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-9017) Implement PreparedStatementImpl parameterization

[Kevin Risden (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Risden reassigned SOLR-9017:
--

Assignee: Kevin Risden

> Implement PreparedStatementImpl parameterization
> 
>
> Key: SOLR-9017
> URL: https://issues.apache.org/jira/browse/SOLR-9017
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master
>Reporter: Kevin Risden
>Assignee: Kevin Risden
>
> SOLR-8809 implemented prepared statements to avoid a NPE when clients were 
> connecting. The next step is to flesh out the rest of the class and implement 
> parameterization. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-9020) Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper errors for traversal methods

[Kevin Risden (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Risden reassigned SOLR-9020:
--

Assignee: Kevin Risden

> Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper 
> errors for traversal methods
> 
>
> Key: SOLR-9020
> URL: https://issues.apache.org/jira/browse/SOLR-9020
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master, 6.0
>Reporter: Kevin Risden
>Assignee: Kevin Risden
> Attachments: SOLR-9020.patch
>
>
> There are 4 methods related to fetch in StatementImpl and 4 methods related 
> to fetch in ResultSetImpl. ResultSetImpl has some traversal methods that 
> don't make sense with the fetch direction. It would make sense to implement 
> them to support more SQL clients.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9020) Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper errors for traversal methods

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250482#comment-15250482
 ] 

Kevin Risden commented on SOLR-9020:


As part of this found that there is a SQLFeatureNotSupportedException that can 
be thrown for some methods. I'm going to create a separate JIRA to audit the 
SQL classes and replace the UnsupportedOperationExceptions with 
SQLFeatureNotSupportedException where applicable.

> Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper 
> errors for traversal methods
> 
>
> Key: SOLR-9020
> URL: https://issues.apache.org/jira/browse/SOLR-9020
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master, 6.0
>Reporter: Kevin Risden
> Attachments: SOLR-9020.patch
>
>
> There are 4 methods related to fetch in StatementImpl and 4 methods related 
> to fetch in ResultSetImpl. ResultSetImpl has some traversal methods that 
> don't make sense with the fetch direction. It would make sense to implement 
> them to support more SQL clients.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-5758) Extend SpanQueryParser with positional joins

[Paul Elschot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Elschot resolved LUCENE-5758.
--
Resolution: Won't Fix

> Extend SpanQueryParser with positional joins
> 
>
> Key: LUCENE-5758
> URL: https://issues.apache.org/jira/browse/LUCENE-5758
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/other, core/queryparser
>Reporter: Paul Elschot
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9020) Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper errors for traversal methods

[Kevin Risden (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Risden updated SOLR-9020:
---
Attachment: SOLR-9020.patch

Patch with tests.

> Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper 
> errors for traversal methods
> 
>
> Key: SOLR-9020
> URL: https://issues.apache.org/jira/browse/SOLR-9020
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master, 6.0
>Reporter: Kevin Risden
> Attachments: SOLR-9020.patch
>
>
> There are 4 methods related to fetch in StatementImpl and 4 methods related 
> to fetch in ResultSetImpl. ResultSetImpl has some traversal methods that 
> don't make sense with the fetch direction. It would make sense to implement 
> them to support more SQL clients.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-5627) Positional joins

[Paul Elschot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Elschot resolved LUCENE-5627.
--
Resolution: Won't Fix

> Positional joins
> 
>
> Key: LUCENE-5627
> URL: https://issues.apache.org/jira/browse/LUCENE-5627
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Paul Elschot
>Priority: Minor
> Attachments: LUCENE-5627-20141126.patch, LUCENE-5627-20150525.patch
>
>
> Prototype of analysis and search for labeled fragments



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9020) Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper errors for traversal methods

[Kevin Risden (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Risden updated SOLR-9020:
---
Description: There are 4 methods related to fetch in StatementImpl and 4 
methods related to fetch in ResultSetImpl. ResultSetImpl has some traversal 
methods that don't make sense with the fetch direction. It would make sense to 
implement them to support more SQL clients.  (was: There are 4 methods related 
to fetch in StatementImpl and 4 methods related to fetch in ResultSetImpl. It 
would make sense to implement them to support more SQL clients.)

> Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper 
> errors for traversal methods
> 
>
> Key: SOLR-9020
> URL: https://issues.apache.org/jira/browse/SOLR-9020
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master, 6.0
>Reporter: Kevin Risden
>
> There are 4 methods related to fetch in StatementImpl and 4 methods related 
> to fetch in ResultSetImpl. ResultSetImpl has some traversal methods that 
> don't make sense with the fetch direction. It would make sense to implement 
> them to support more SQL clients.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9020) Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper errors for traversal methods

[Kevin Risden (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Risden updated SOLR-9020:
---
Summary: Implement StatementImpl/ResultSetImpl get/set fetch* methods and 
proper errors for traversal methods  (was: Implement 
StatementImpl/ResultSetImpl get/set fetch* methods)

> Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper 
> errors for traversal methods
> 
>
> Key: SOLR-9020
> URL: https://issues.apache.org/jira/browse/SOLR-9020
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master, 6.0
>Reporter: Kevin Risden
>
> There are 4 methods related to fetch in StatementImpl and 4 methods related 
> to fetch in ResultSetImpl. It would make sense to implement them to support 
> more SQL clients.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9020) Implement StatementImpl/ResultSetImpl get/set fetch* methods

[Kevin Risden (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Risden updated SOLR-9020:
---
Description: There are 4 methods related to fetch in StatementImpl and 4 
methods related to fetch in ResultSetImpl. It would make sense to implement 
them to support more SQL clients.  (was: There are 4 methods related to fetch 
in ResultSetImpl. It would make sense to implement them to support more SQL 
clients.)

> Implement StatementImpl/ResultSetImpl get/set fetch* methods
> 
>
> Key: SOLR-9020
> URL: https://issues.apache.org/jira/browse/SOLR-9020
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master, 6.0
>Reporter: Kevin Risden
>
> There are 4 methods related to fetch in StatementImpl and 4 methods related 
> to fetch in ResultSetImpl. It would make sense to implement them to support 
> more SQL clients.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9020) Implement StatementImpl/ResultSetImpl get/set fetch* methods

[Kevin Risden (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Risden updated SOLR-9020:
---
Summary: Implement StatementImpl/ResultSetImpl get/set fetch* methods  
(was: Implement ResultSetImpl get/set fetch* methods)

> Implement StatementImpl/ResultSetImpl get/set fetch* methods
> 
>
> Key: SOLR-9020
> URL: https://issues.apache.org/jira/browse/SOLR-9020
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master, 6.0
>Reporter: Kevin Risden
>
> There are 4 methods related to fetch in ResultSetImpl. It would make sense to 
> implement them to support more SQL clients.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8973) TX-frenzy on Zookeeper when collection is put to use

[Scott Blum (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250457#comment-15250457
 ] 

Scott Blum commented on SOLR-8973:
--

Yeah it wouldn't surprise me if it applies clean.

> TX-frenzy on Zookeeper when collection is put to use
> 
>
> Key: SOLR-8973
> URL: https://issues.apache.org/jira/browse/SOLR-8973
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Affects Versions: 5.0, 5.1, 5.2, 5.3, 5.4, 5.5, master, 5.6
>Reporter: Janmejay Singh
>Assignee: Scott Blum
>  Labels: collections, patch-available, solrcloud, zookeeper
> Fix For: master, 6.1
>
> Attachments: SOLR-8973-ZkStateReader.patch, SOLR-8973.patch, 
> SOLR-8973.patch, SOLR-8973.patch
>
>
> This is to do with a distributed data-race. Core-creation happens at a time 
> when collection is not yet visible to the node. In this case a fallback 
> code-path is used which de-references collection-state lazily (on demand) as 
> opposed to setting a watch and keeping it cached locally.
> Due to this, as requests towards the core mount, it generates ZK fetch for 
> collection proportionately. On a large solr-cloud cluster, this generates 
> several Gbps of TX traffic on ZK nodes. This affects indexing 
> throughput(which floors) in addition to running ZK node out of network 
> bandwidth. 
> On smaller solr-cloud clusters its hard to run into, because probability of 
> this race materializing reduces.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 44 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/44/

4 tests failed.
FAILED:  org.apache.solr.TestDistributedSearch.test

Error Message:
Captured an uncaught exception in thread: Thread[id=32313, name=Thread-31915, 
state=RUNNABLE, group=TGRP-TestDistributedSearch]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=32313, name=Thread-31915, state=RUNNABLE, 
group=TGRP-TestDistributedSearch]
at 
__randomizedtesting.SeedInfo.seed([FF7B6F80A8BACE99:772F505A0646A361]:0)
Caused by: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:53032/gr_y/collection1: 
java.lang.NullPointerException
at 
org.apache.solr.search.grouping.distributed.responseprocessor.TopGroupsShardResponseProcessor.process(TopGroupsShardResponseProcessor.java:105)
at 
org.apache.solr.handler.component.QueryComponent.handleGroupedResponses(QueryComponent.java:753)
at 
org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:736)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:426)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2015)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:229)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:184)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:518)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at 
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
at 
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lang.Thread.run(Thread.java:745)

at __randomizedtesting.SeedInfo.seed([FF7B6F80A8BACE99]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:590)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:957)
at 
org.apache.solr.TestDistributedSearch$1.run(TestDistributedSearch.java:1114)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=107907, name=collection2, 
state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=107907, name=collection2, state=RUNNABLE, 
group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.common.SolrException: 

[JENKINS] Lucene-Solr-6.0-Windows (64bit/jdk1.8.0_72) - Build # 39 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.0-Windows/39/
Java: 64bit/jdk1.8.0_72 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.cloud.CloudExitableDirectoryReaderTest.test

Error Message:
No live SolrServers available to handle this 
request:[http://127.0.0.1:64816/collection1]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this request:[http://127.0.0.1:64816/collection1]
at 
__randomizedtesting.SeedInfo.seed([EADA544C72E8940F:628E6B96DC14F9F7]:0)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:352)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1100)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:870)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:957)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.queryServer(AbstractFullDistribZkTestBase.java:1397)
at 
org.apache.solr.cloud.CloudExitableDirectoryReaderTest.assertPartialResults(CloudExitableDirectoryReaderTest.java:99)
at 
org.apache.solr.cloud.CloudExitableDirectoryReaderTest.doTimeoutTests(CloudExitableDirectoryReaderTest.java:71)
at 
org.apache.solr.cloud.CloudExitableDirectoryReaderTest.test(CloudExitableDirectoryReaderTest.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Commented] (SOLR-8973) TX-frenzy on Zookeeper when collection is put to use

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250408#comment-15250408
 ] 

Erick Erickson commented on SOLR-8973:
--

[~dragonsinth] So I just applied this as-is to 5.5 for my own exploration and 
it applied cleanly. Would that be what you expect?

Thanks,
Erick

> TX-frenzy on Zookeeper when collection is put to use
> 
>
> Key: SOLR-8973
> URL: https://issues.apache.org/jira/browse/SOLR-8973
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Affects Versions: 5.0, 5.1, 5.2, 5.3, 5.4, 5.5, master, 5.6
>Reporter: Janmejay Singh
>Assignee: Scott Blum
>  Labels: collections, patch-available, solrcloud, zookeeper
> Fix For: master, 6.1
>
> Attachments: SOLR-8973-ZkStateReader.patch, SOLR-8973.patch, 
> SOLR-8973.patch, SOLR-8973.patch
>
>
> This is to do with a distributed data-race. Core-creation happens at a time 
> when collection is not yet visible to the node. In this case a fallback 
> code-path is used which de-references collection-state lazily (on demand) as 
> opposed to setting a watch and keeping it cached locally.
> Due to this, as requests towards the core mount, it generates ZK fetch for 
> collection proportionately. On a large solr-cloud cluster, this generates 
> several Gbps of TX traffic on ZK nodes. This affects indexing 
> throughput(which floors) in addition to running ZK node out of network 
> bandwidth. 
> On smaller solr-cloud clusters its hard to run into, because probability of 
> this race materializing reduces.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8327) SolrDispatchFilter is not caching new state format, which results in live fetch from ZK per request if node does not contain core from collection

[Shalin Shekhar Mangar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250384#comment-15250384
 ] 

Shalin Shekhar Mangar commented on SOLR-8327:
-

Varun, I am auditing all usages of ClusterState methods to ensure we don't end 
up going to ZK needlessly. See SOLR-9014

> SolrDispatchFilter is not caching new state format, which results in live 
> fetch from ZK per request if node does not contain core from collection
> -
>
> Key: SOLR-8327
> URL: https://issues.apache.org/jira/browse/SOLR-8327
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Affects Versions: 5.3
>Reporter: Jessica Cheng Mallet
>Assignee: Varun Thacker
>  Labels: solrcloud
> Attachments: SOLR-8327.patch
>
>
> While perf testing with non-solrj client (request can be sent to any solr 
> node), we noticed a huge amount of data from Zookeeper in our tcpdump (~1G 
> for 20 second dump). From the thread dump, we noticed this:
> java.lang.Object.wait (Native Method)
> java.lang.Object.wait (Object.java:503)
> org.apache.zookeeper.ClientCnxn.submitRequest (ClientCnxn.java:1309)
> org.apache.zookeeper.ZooKeeper.getData (ZooKeeper.java:1152)
> org.apache.solr.common.cloud.SolrZkClient$7.execute (SolrZkClient.java:345)
> org.apache.solr.common.cloud.SolrZkClient$7.execute (SolrZkClient.java:342)
> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation 
> (ZkCmdExecutor.java:61)
> org.apache.solr.common.cloud.SolrZkClient.getData (SolrZkClient.java:342)
> org.apache.solr.common.cloud.ZkStateReader.getCollectionLive 
> (ZkStateReader.java:841)
> org.apache.solr.common.cloud.ZkStateReader$7.get (ZkStateReader.java:515)
> org.apache.solr.common.cloud.ClusterState.getCollectionOrNull 
> (ClusterState.java:175)
> org.apache.solr.common.cloud.ClusterState.getLeader (ClusterState.java:98)
> org.apache.solr.servlet.HttpSolrCall.getCoreByCollection 
> (HttpSolrCall.java:784)
> org.apache.solr.servlet.HttpSolrCall.init (HttpSolrCall.java:272)
> org.apache.solr.servlet.HttpSolrCall.call (HttpSolrCall.java:417)
> org.apache.solr.servlet.SolrDispatchFilter.doFilter 
> (SolrDispatchFilter.java:210)
> org.apache.solr.servlet.SolrDispatchFilter.doFilter 
> (SolrDispatchFilter.java:179)
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter 
> (ServletHandler.java:1652)
> org.eclipse.jetty.servlet.ServletHandler.doHandle (ServletHandler.java:585)
> org.eclipse.jetty.server.handler.ScopedHandler.handle (ScopedHandler.java:143)
> org.eclipse.jetty.security.SecurityHandler.handle (SecurityHandler.java:577)
> org.eclipse.jetty.server.session.SessionHandler.doHandle 
> (SessionHandler.java:223)
> org.eclipse.jetty.server.handler.ContextHandler.doHandle 
> (ContextHandler.java:1127)
> org.eclipse.jetty.servlet.ServletHandler.doScope (ServletHandler.java:515)
> org.eclipse.jetty.server.session.SessionHandler.doScope 
> (SessionHandler.java:185)
> org.eclipse.jetty.server.handler.ContextHandler.doScope 
> (ContextHandler.java:1061)
> org.eclipse.jetty.server.handler.ScopedHandler.handle (ScopedHandler.java:141)
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle 
> (ContextHandlerCollection.java:215)
> org.eclipse.jetty.server.handler.HandlerCollection.handle 
> (HandlerCollection.java:110)
> org.eclipse.jetty.server.handler.HandlerWrapper.handle 
> (HandlerWrapper.java:97)
> org.eclipse.jetty.server.Server.handle (Server.java:499)
> org.eclipse.jetty.server.HttpChannel.handle (HttpChannel.java:310)
> org.eclipse.jetty.server.HttpConnection.onFillable (HttpConnection.java:257)
> org.eclipse.jetty.io.AbstractConnection$2.run (AbstractConnection.java:540)
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob 
> (QueuedThreadPool.java:635)
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run 
> (QueuedThreadPool.java:555)
> java.lang.Thread.run (Thread.java:745)
> Looks like SolrDispatchFilter doesn't have caching similar to the 
> collectionStateCache in CloudSolrClient, so if the node doesn't know about a 
> collection in the new state format, it just live-fetch it from Zookeeper on 
> every request.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8521) Add documentation for how to use Solr JDBC driver with SQL client like DB Visualizer

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250383#comment-15250383
 ] 

Kevin Risden commented on SOLR-8521:


Adding a note that the material here and related tickets need to be integrated 
with the Solr Reference Guide here: 
https://cwiki.apache.org/confluence/display/solr/Parallel+SQL+Interface#ParallelSQLInterface-SQLClientsandDatabaseVisualizationTools

> Add documentation for how to use Solr JDBC driver with SQL client like DB 
> Visualizer
> 
>
> Key: SOLR-8521
> URL: https://issues.apache.org/jira/browse/SOLR-8521
> Project: Solr
>  Issue Type: Sub-task
>  Components: documentation, SolrJ
>Affects Versions: master
>Reporter: Kevin Risden
> Attachments: solr_jdbc_dbvisualizer_20160203.pdf
>
>
> Currently this requires the following:
> * a JDBC SQL client program (like DBVisualizer or SQuirrelSQL)
> * all jars from solr/dist/solrj-lib/* to be on the SQL client classpath
> * solr/dist/solr-solrj-6.0.0-SNAPSHOT.jar on the SQL client classpath
> * a valid JDBC connection string (like 
> jdbc:solr://SOLR_ZK_CONNECTION_STRING?collection=COLLECTION_NAME)
> * without SOLR-8213, the username/password supplied by the SQL client will be 
> ignored.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9021) SolrJ JDBC - R RJDBC documentation

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250378#comment-15250378
 ] 

Kevin Risden commented on SOLR-9021:


The solrCP variable must be set to the SolrJ jars for the JDBC connection 
object. An example R script is below:
{code}
# https://www.rforge.net/RJDBC/

install.packages("RJDBC", dep=TRUE)

library("RJDBC")

solrCP <- c(list.files('/opt/solr/dist/solrj-lib', full.names=TRUE), 
list.files('/opt/solr/dist', pattern='solrj', full.names=TRUE, recursive = 
TRUE))

drv <- JDBC("org.apache.solr.client.solrj.io.sql.DriverImpl",
   solrCP,
   identifier.quote="`")
conn <- dbConnect(drv, "jdbc:solr://solr:9983?collection=test", "user", "pwd")
dbGetQuery(conn, "select fielda, fieldb, fieldc, fieldd_s, fielde_i from test 
limit 10")

#dbListTables(conn)
#data(iris)
#dbWriteTable(conn, "iris", iris, overwrite=TRUE)
#dbGetQuery(conn, "select count(*) from iris")
#d <- dbReadTable(conn, "iris")

dbDisconnect(conn)
{code}

> SolrJ JDBC - R RJDBC documentation
> --
>
> Key: SOLR-9021
> URL: https://issues.apache.org/jira/browse/SOLR-9021
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Reporter: Kevin Risden
>
> SOLR-9019 is to check that the Solr JDBC driver is usable from R RJDBC. Once 
> it works, it would be great to have this documented like SOLR-8521



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8327) SolrDispatchFilter is not caching new state format, which results in live fetch from ZK per request if node does not contain core from collection

[Varun Thacker (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250374#comment-15250374
 ] 

Varun Thacker commented on SOLR-8327:
-

Also the DistributedUpdateProcessor makes many calls to 
{{zkController.getClusterState().getCollection}} . So for non-solrJ users I 
guess we need to cache at the DitributedUpdateProcessor as well so that update 
requests don't make multiple calls to ZK 

> SolrDispatchFilter is not caching new state format, which results in live 
> fetch from ZK per request if node does not contain core from collection
> -
>
> Key: SOLR-8327
> URL: https://issues.apache.org/jira/browse/SOLR-8327
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Affects Versions: 5.3
>Reporter: Jessica Cheng Mallet
>Assignee: Varun Thacker
>  Labels: solrcloud
> Attachments: SOLR-8327.patch
>
>
> While perf testing with non-solrj client (request can be sent to any solr 
> node), we noticed a huge amount of data from Zookeeper in our tcpdump (~1G 
> for 20 second dump). From the thread dump, we noticed this:
> java.lang.Object.wait (Native Method)
> java.lang.Object.wait (Object.java:503)
> org.apache.zookeeper.ClientCnxn.submitRequest (ClientCnxn.java:1309)
> org.apache.zookeeper.ZooKeeper.getData (ZooKeeper.java:1152)
> org.apache.solr.common.cloud.SolrZkClient$7.execute (SolrZkClient.java:345)
> org.apache.solr.common.cloud.SolrZkClient$7.execute (SolrZkClient.java:342)
> org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation 
> (ZkCmdExecutor.java:61)
> org.apache.solr.common.cloud.SolrZkClient.getData (SolrZkClient.java:342)
> org.apache.solr.common.cloud.ZkStateReader.getCollectionLive 
> (ZkStateReader.java:841)
> org.apache.solr.common.cloud.ZkStateReader$7.get (ZkStateReader.java:515)
> org.apache.solr.common.cloud.ClusterState.getCollectionOrNull 
> (ClusterState.java:175)
> org.apache.solr.common.cloud.ClusterState.getLeader (ClusterState.java:98)
> org.apache.solr.servlet.HttpSolrCall.getCoreByCollection 
> (HttpSolrCall.java:784)
> org.apache.solr.servlet.HttpSolrCall.init (HttpSolrCall.java:272)
> org.apache.solr.servlet.HttpSolrCall.call (HttpSolrCall.java:417)
> org.apache.solr.servlet.SolrDispatchFilter.doFilter 
> (SolrDispatchFilter.java:210)
> org.apache.solr.servlet.SolrDispatchFilter.doFilter 
> (SolrDispatchFilter.java:179)
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter 
> (ServletHandler.java:1652)
> org.eclipse.jetty.servlet.ServletHandler.doHandle (ServletHandler.java:585)
> org.eclipse.jetty.server.handler.ScopedHandler.handle (ScopedHandler.java:143)
> org.eclipse.jetty.security.SecurityHandler.handle (SecurityHandler.java:577)
> org.eclipse.jetty.server.session.SessionHandler.doHandle 
> (SessionHandler.java:223)
> org.eclipse.jetty.server.handler.ContextHandler.doHandle 
> (ContextHandler.java:1127)
> org.eclipse.jetty.servlet.ServletHandler.doScope (ServletHandler.java:515)
> org.eclipse.jetty.server.session.SessionHandler.doScope 
> (SessionHandler.java:185)
> org.eclipse.jetty.server.handler.ContextHandler.doScope 
> (ContextHandler.java:1061)
> org.eclipse.jetty.server.handler.ScopedHandler.handle (ScopedHandler.java:141)
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle 
> (ContextHandlerCollection.java:215)
> org.eclipse.jetty.server.handler.HandlerCollection.handle 
> (HandlerCollection.java:110)
> org.eclipse.jetty.server.handler.HandlerWrapper.handle 
> (HandlerWrapper.java:97)
> org.eclipse.jetty.server.Server.handle (Server.java:499)
> org.eclipse.jetty.server.HttpChannel.handle (HttpChannel.java:310)
> org.eclipse.jetty.server.HttpConnection.onFillable (HttpConnection.java:257)
> org.eclipse.jetty.io.AbstractConnection$2.run (AbstractConnection.java:540)
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob 
> (QueuedThreadPool.java:635)
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run 
> (QueuedThreadPool.java:555)
> java.lang.Thread.run (Thread.java:745)
> Looks like SolrDispatchFilter doesn't have caching similar to the 
> collectionStateCache in CloudSolrClient, so if the node doesn't know about a 
> collection in the new state format, it just live-fetch it from Zookeeper on 
> every request.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Welcome Scott Blum as a Lucene/Solr committer!

[Anshum Gupta]

Congratulations and welcome Scott!

On Tue, Apr 19, 2016 at 2:21 AM, Shalin Shekhar Mangar 
wrote:

> I'm pleased to announce that Scott Blum has accepted the Lucene PMC's
> invitation to become a committer.
>
> Scott, it's tradition that you introduce yourself with a brief bio.
>
> Your handle "dragonsinth" has already added to the “lucene" LDAP group, so
> you now have commit privileges. Please test this by adding yourself to the
> committers section of the Who We Are page on the website: <
> http://lucene.apache.org/whoweare.html> (use the ASF CMS bookmarklet at
> the bottom of the page here:  - more
> info here ).
>
> The ASF dev page also has lots of useful links: <
> http://www.apache.org/dev/>.
>
> Congratulations and welcome!
>
> --
> Regards,
> Shalin Shekhar Mangar.
>



-- 
Anshum Gupta

[jira] [Commented] (SOLR-8323) Add CollectionWatcher API to ZkStateReader

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250369#comment-15250369
 ] 

Alan Woodward commented on SOLR-8323:
-

Pull request opened, review away!  I see that you've already committed some 
changes to the way legacy collections are dealt with, so we may well be able to 
remove the 'interestingcollections' list - will give it a go.

> Add CollectionWatcher API to ZkStateReader
> --
>
> Key: SOLR-8323
> URL: https://issues.apache.org/jira/browse/SOLR-8323
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: master
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-8323.patch, SOLR-8323.patch, SOLR-8323.patch, 
> SOLR-8323.patch
>
>
> An API to watch for changes to collection state would be a generally useful 
> thing, both internally and for client use.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8323) Add CollectionWatcher API to ZkStateReader

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250364#comment-15250364
 ] 

ASF GitHub Bot commented on SOLR-8323:
--

GitHub user romseygeek opened a pull request:

https://github.com/apache/lucene-solr/pull/32

SOLR-8323

Adds a CollectionStateWatcher API to listen for changes to collection state 
(SOLR-8323)

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/romseygeek/lucene-solr SOLR-8323

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/lucene-solr/pull/32.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #32






> Add CollectionWatcher API to ZkStateReader
> --
>
> Key: SOLR-8323
> URL: https://issues.apache.org/jira/browse/SOLR-8323
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: master
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-8323.patch, SOLR-8323.patch, SOLR-8323.patch, 
> SOLR-8323.patch
>
>
> An API to watch for changes to collection state would be a generally useful 
> thing, both internally and for client use.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr pull request: SOLR-8323

[romseygeek]

GitHub user romseygeek opened a pull request:

https://github.com/apache/lucene-solr/pull/32

SOLR-8323

Adds a CollectionStateWatcher API to listen for changes to collection state 
(SOLR-8323)

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/romseygeek/lucene-solr SOLR-8323

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/lucene-solr/pull/32.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #32






---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Term vs. token

[Ryan Josal]

My understanding is a Term is comprised of a "token" and a field.  So then
the documentation makes sense to me - return the count of tokens in a field
for example.  But there were a couple of references you had there that
don't match with that definition, like the number of tokens in a
collection.  Although maybe a Term doesn't have a whole token because what
about token attributes like payload.  I guess I've convinced myself I'm not
entirely clear about it either, but I do feel good about the concept that
tokens don't have fields.  You can tokenize a string without thinking about
fields, and they become terms with fields when you query.

Ryan

On Wednesday, April 20, 2016, Jack Krupansky 
wrote:

> Looking at the Lucene Similarity Javadoc, I see some references to tokens,
> but I am wondering if that is intentional or whether those should really be
> references to terms.
>
> For example:
>
>  *lengthNorm - computed
>  *when the document is added to the index in accordance with the
> number of tokens
>  *of this field in the document, so that shorter fields contribute
> more to the score.
>
> I think that should be terms, not tokens.
>
> See:
>
> https://github.com/apache/lucene-solr/blob/releases/lucene-solr/5.5.0/lucene/core/src/java/org/apache/lucene/search/similarities/TFIDFSimilarity.java#L466
>
> And this:
>
>* Returns the total number of tokens in the field.
>* @see Terms#getSumTotalTermFreq()
>*/
>   public long getNumberOfFieldTokens() {
> return numberOfFieldTokens;
>
> I think that should be terms as well:
>
> See:
>
> https://github.com/apache/lucene-solr/blob/master/lucene/core/src/java/org/apache/lucene/search/similarities/BasicStats.java#L65
>
> And... this:
>
>   numberOfFieldTokens = sumTotalTermFreq;
>
> Where it is clearly starting with terms and treating them as tokens, but
> as in the previous example, I think that should be terms as well.
>
> See:
>
> https://github.com/apache/lucene-solr/blob/master/lucene/core/src/java/org/apache/lucene/search/similarities/SimilarityBase.java#L128
>
> One last example:
>
>* Compute any collection-level weight (e.g. IDF, average document
> length, etc) needed for scoring a query.
>*
>* @param collectionStats collection-level statistics, such as the
> number of tokens in the collection.
>* @param termStats term-level statistics, such as the document
> frequency of a term across the collection.
>* @return SimWeight object with the information this Similarity needs
> to score a query.
>*/
>   public abstract SimWeight computeWeight(CollectionStatistics
> collectionStats, TermStatistics... termStats);
>
> See:
>
> https://github.com/apache/lucene-solr/blob/master/lucene/core/src/java/org/apache/lucene/search/similarities/Similarity.java#L161
>
> In fact, CollectionStatistics uses term, not token:
>
>   /** returns the total number of tokens for this field
>* @see Terms#getSumTotalTermFreq() */
>   public final long sumTotalTermFreq() {
> return sumTotalTermFreq;
>
> Oops... it uses both, emphasizing my point about the confusion.
>
> There are other examples as well.
>
> My understanding is that tokens are merely a temporary transition in
> between the original raw source text for a field and then final terms to be
> indexed (or query terms from a parsed and analyzed query.) Yes, during and
> within TokenStream or the analyzer we speak of tokens and intermediate
> string values are referred to as tokens, but once the final string value is
> retrieved from the Token Stream (analyzer), it's a term.
>
> In any case, is there some distinction in any of these cited examples (or
> other examples in this or related code) where "token" is an important
> distinction to be made and "term" is not the proper... term... to be used?
>
> Unless the Lucene project fully intends that the terms token and term are
> absolutely synonymous, a clear distinction should be drawn... I think. Or
> at least the terms should be used consistently, which my last example
> highlights.
>
> Thanks.
>
> -- Jack Krupansky
>

[jira] [Created] (SOLR-9021) SolrJ JDBC - R RJDBC documentation

[Kevin Risden (JIRA)]

Kevin Risden created SOLR-9021:
--

 Summary: SolrJ JDBC - R RJDBC documentation
 Key: SOLR-9021
 URL: https://issues.apache.org/jira/browse/SOLR-9021
 Project: Solr
  Issue Type: Sub-task
  Components: SolrJ
Reporter: Kevin Risden


SOLR-9019 is to check that the Solr JDBC driver is usable from R RJDBC. Once it 
works, it would be great to have this documented like SOLR-8521



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8716) Upgrade to Apache Tika 1.12

[Lewis John McGibbney (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250294#comment-15250294
 ] 

Lewis John McGibbney commented on SOLR-8716:


New parsers are
{code}
org.apache.tika.parser.image.WebPParser
org.apache.tika.parser.microsoft.JackcessParser
org.apache.tika.parser.pkg.RarParser
org.apache.tika.parser.dif.DIFParser
org.apache.tika.parser.gdal.GDALParser
org.apache.tika.parser.pot.PooledTimeSeriesParser
org.apache.tika.parser.grib.GribParser
org.apache.tika.parser.jdbc.SQLite3Parser
org.apache.tika.parser.isatab.ISArchiveParser
org.apache.tika.parser.geoinfo.GeographicInformationParser
org.apache.tika.parser.geo.topic.GeoParser
org.apache.tika.parser.external.CompositeExternalParser
org.apache.tika.parser.journal.JournalParser
{code}

> Upgrade to Apache Tika 1.12
> ---
>
> Key: SOLR-8716
> URL: https://issues.apache.org/jira/browse/SOLR-8716
> Project: Solr
>  Issue Type: Improvement
>  Components: contrib - Solr Cell (Tika extraction)
>Reporter: Lewis John McGibbney
>Assignee: Uwe Schindler
> Fix For: master
>
> Attachments: LUCENE-7041.patch
>
>
> We recently released Apache Tika 1.12. In order to use the fixes provided 
> within the Tika.translate API I propose to upgrade Tika from 1.7 --> 1.12 in 
> lucene/ivy-versions.properties.
> Patch coming up.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9020) Implement ResultSetImpl get/set fetch* methods

[Kevin Risden (JIRA)]

Kevin Risden created SOLR-9020:
--

 Summary: Implement ResultSetImpl get/set fetch* methods
 Key: SOLR-9020
 URL: https://issues.apache.org/jira/browse/SOLR-9020
 Project: Solr
  Issue Type: Sub-task
  Components: SolrJ
Affects Versions: master, 6.0
Reporter: Kevin Risden


There are 4 methods related to fetch in ResultSetImpl. It would make sense to 
implement them to support more SQL clients.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9019) SolrJ JDBC - Ensure that R RJDBC works with SolrJ JDBC

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250266#comment-15250266
 ] 

Kevin Risden commented on SOLR-9019:


Currently RJDBC is failing with the following error
{code}
Error in .jcall(rp, "I", "fetch", stride, block) : 
  java.lang.UnsupportedOperationException
Calls: dbGetQuery ... fetch -> fetch -> .local -> .jcall -> .jcheck -> .Call
{code}

This is caused by the following code in RJDBC
{code}
try { // run in a try since it's a hint, but some bad drivers fail on it anyway 
(see #11)
  rs.setFetchSize(fetchSize);
} catch (java.sql.SQLException e) { } // we can't use 
SQLFeatureNotSupportedException because that's 1.6+ only
{code}

This will require that ResultSetImpl implements setFetchSize or setFetchSize 
should throw a SQLException instead of UnsupportedOperationException.

> SolrJ JDBC - Ensure that R RJDBC works with SolrJ JDBC
> --
>
> Key: SOLR-9019
> URL: https://issues.apache.org/jira/browse/SOLR-9019
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Reporter: Kevin Risden
>
> R has RJDBC (https://cran.r-project.org/web/packages/RJDBC/index.html) which 
> can connect to JDBC. Check that it works with SolrJ JDBC.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9019) SolrJ JDBC - Ensure that R RJDBC works with SolrJ JDBC

[Kevin Risden (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Risden updated SOLR-9019:
---
Description: R has RJDBC 
(https://cran.r-project.org/web/packages/RJDBC/index.html) which can connect to 
JDBC. Check that it works with SolrJ JDBC.  (was: R has RJDBC which can connect 
to JDBC. Check that it works with SolrJ JDBC.)

> SolrJ JDBC - Ensure that R RJDBC works with SolrJ JDBC
> --
>
> Key: SOLR-9019
> URL: https://issues.apache.org/jira/browse/SOLR-9019
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Reporter: Kevin Risden
>
> R has RJDBC (https://cran.r-project.org/web/packages/RJDBC/index.html) which 
> can connect to JDBC. Check that it works with SolrJ JDBC.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9019) SolrJ JDBC - Ensure that R RJDBC works with SolrJ JDBC

[Kevin Risden (JIRA)]

Kevin Risden created SOLR-9019:
--

 Summary: SolrJ JDBC - Ensure that R RJDBC works with SolrJ JDBC
 Key: SOLR-9019
 URL: https://issues.apache.org/jira/browse/SOLR-9019
 Project: Solr
  Issue Type: Sub-task
  Components: SolrJ
Reporter: Kevin Risden


R has RJDBC which can connect to JDBC. Check that it works with SolrJ JDBC.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8845) SolrJ JDBC - Ensure that Spark works with SolrJ JDBC

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250259#comment-15250259
 ] 

Joel Bernstein commented on SOLR-8845:
--

Yeah, that would cause an exception of some sort.

> SolrJ JDBC - Ensure that Spark works with SolrJ JDBC
> 
>
> Key: SOLR-8845
> URL: https://issues.apache.org/jira/browse/SOLR-8845
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master
>Reporter: Trey Cahill
>
> Ensure that Spark is able work with SolrJ JDBC.  
> Currently, in Spark 1.6 there are 2 known issues:
> 1. SparkSQL uses a connection.prepareStatement() call, which returns null in 
> the SolrJ implementation
> 2. SparkSQL query's via a "select *" query, which is currently not supported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Windows (64bit/jdk1.8.0_72) - Build # 129 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/129/
Java: 64bit/jdk1.8.0_72 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.cloud.HttpPartitionTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=46555, 
name=SocketProxy-Response-53459:54006, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=46555, name=SocketProxy-Response-53459:54006, 
state=RUNNABLE, group=TGRP-HttpPartitionTest]
at 
__randomizedtesting.SeedInfo.seed([9D77F2067FE3E8AB:1523CDDCD11F8553]:0)
Caused by: java.lang.RuntimeException: java.net.SocketException: Socket is 
closed
at __randomizedtesting.SeedInfo.seed([9D77F2067FE3E8AB]:0)
at 
org.apache.solr.cloud.SocketProxy$Bridge$Pump.run(SocketProxy.java:347)
Caused by: java.net.SocketException: Socket is closed
at java.net.Socket.setSoTimeout(Socket.java:1137)
at 
org.apache.solr.cloud.SocketProxy$Bridge$Pump.run(SocketProxy.java:344)




Build Log:
[...truncated 11955 lines...]
   [junit4] Suite: org.apache.solr.cloud.HttpPartitionTest
   [junit4]   2> Creating dataDir: 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HttpPartitionTest_9D77F2067FE3E8AB-001\init-core-data-001
   [junit4]   2> 2563485 INFO  
(SUITE-HttpPartitionTest-seed#[9D77F2067FE3E8AB]-worker) [] 
o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 2563489 INFO  
(TEST-HttpPartitionTest.test-seed#[9D77F2067FE3E8AB]) [] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2563489 INFO  (Thread-5453) [] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2563489 INFO  (Thread-5453) [] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 2563590 INFO  
(TEST-HttpPartitionTest.test-seed#[9D77F2067FE3E8AB]) [] 
o.a.s.c.ZkTestServer start zk server on port:53406
   [junit4]   2> 2563590 INFO  
(TEST-HttpPartitionTest.test-seed#[9D77F2067FE3E8AB]) [] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2563592 INFO  
(TEST-HttpPartitionTest.test-seed#[9D77F2067FE3E8AB]) [] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2563596 INFO  (zkCallback-10788-thread-1) [] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@240cab4d 
name:ZooKeeperConnection Watcher:127.0.0.1:53406 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2563597 INFO  
(TEST-HttpPartitionTest.test-seed#[9D77F2067FE3E8AB]) [] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2563597 INFO  
(TEST-HttpPartitionTest.test-seed#[9D77F2067FE3E8AB]) [] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2563597 INFO  
(TEST-HttpPartitionTest.test-seed#[9D77F2067FE3E8AB]) [] 
o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2563600 INFO  
(TEST-HttpPartitionTest.test-seed#[9D77F2067FE3E8AB]) [] 
o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2563601 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [] 
o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from 
client sessionid 0x154348bea9c, likely client has closed socket
   [junit4]   2>at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2>at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2>at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2563601 INFO  
(TEST-HttpPartitionTest.test-seed#[9D77F2067FE3E8AB]) [] 
o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2563603 INFO  (zkCallback-10789-thread-1) [] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@1d16ff5 name:ZooKeeperConnection 
Watcher:127.0.0.1:53406/solr got event WatchedEvent state:SyncConnected 
type:None path:null path:null type:None
   [junit4]   2> 2563603 INFO  
(TEST-HttpPartitionTest.test-seed#[9D77F2067FE3E8AB]) [] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2563603 INFO  
(TEST-HttpPartitionTest.test-seed#[9D77F2067FE3E8AB]) [] 
o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2563603 INFO  
(TEST-HttpPartitionTest.test-seed#[9D77F2067FE3E8AB]) [] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 2563606 INFO  
(TEST-HttpPartitionTest.test-seed#[9D77F2067FE3E8AB]) [] 
o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 2563609 INFO  
(TEST-HttpPartitionTest.test-seed#[9D77F2067FE3E8AB]) [] 
o.a.s.c.c.SolrZkClient 

[jira] [Comment Edited] (SOLR-9018) SolrJ JDBC - Jython Documentation

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250169#comment-15250169
 ] 

Kevin Risden edited comment on SOLR-9018 at 4/20/16 4:15 PM:
-

Jython native example script:
{code}
#!/usr/bin/env jython

# http://www.jython.org/jythonbook/en/1.0/DatabasesAndJython.html
# https://wiki.python.org/jython/DatabaseExamples#SQLite_using_JDBC

import sys

from java.lang import Class
from java.sql  import DriverManager, SQLException

if __name__ == '__main__':
  jdbc_url = "jdbc:solr://solr:9983?collection=test"
  driverName = "org.apache.solr.client.solrj.io.sql.DriverImpl"
  statement = "select fielda, fieldb, fieldc, fieldd_s, fielde_i from test 
limit 10"

  dbConn = DriverManager.getConnection(jdbc_url)
  stmt = dbConn.createStatement()

  resultSet = stmt.executeQuery(statement)
  while resultSet.next():
print(resultSet.getString("fielda"))

  resultSet.close()
  stmt.close()
  dbConn.close()

  sys.exit(0)
{code}


was (Author: risdenk):
Jython Native Connection example script:
{code}
#!/usr/bin/env jython

# http://www.jython.org/jythonbook/en/1.0/DatabasesAndJython.html
# https://wiki.python.org/jython/DatabaseExamples#SQLite_using_JDBC

import sys

from java.lang import Class
from java.sql  import DriverManager, SQLException

if __name__ == '__main__':
  jdbc_url = "jdbc:solr://solr:9983?collection=test"
  driverName = "org.apache.solr.client.solrj.io.sql.DriverImpl"
  statement = "select fielda, fieldb, fieldc, fieldd_s, fielde_i from test 
limit 10"

  dbConn = DriverManager.getConnection(jdbc_url)
  stmt = dbConn.createStatement()

  resultSet = stmt.executeQuery(statement)
  while resultSet.next():
print(resultSet.getString("fielda"))

  resultSet.close()
  stmt.close()
  dbConn.close()

  sys.exit(0)
{code}

> SolrJ JDBC - Jython Documentation
> -
>
> Key: SOLR-9018
> URL: https://issues.apache.org/jira/browse/SOLR-9018
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Reporter: Kevin Risden
>
> Jython integrates both Python and Java. This makes connecting to JDBC 
> possible. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9018) SolrJ JDBC - Jython Documentation

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250169#comment-15250169
 ] 

Kevin Risden commented on SOLR-9018:


Jython Native Connection example script:
{code}
#!/usr/bin/env jython

# http://www.jython.org/jythonbook/en/1.0/DatabasesAndJython.html
# https://wiki.python.org/jython/DatabaseExamples#SQLite_using_JDBC

import sys

from java.lang import Class
from java.sql  import DriverManager, SQLException

if __name__ == '__main__':
  jdbc_url = "jdbc:solr://solr:9983?collection=test"
  driverName = "org.apache.solr.client.solrj.io.sql.DriverImpl"
  statement = "select fielda, fieldb, fieldc, fieldd_s, fielde_i from test 
limit 10"

  dbConn = DriverManager.getConnection(jdbc_url)
  stmt = dbConn.createStatement()

  resultSet = stmt.executeQuery(statement)
  while resultSet.next():
print(resultSet.getString("fielda"))

  resultSet.close()
  stmt.close()
  dbConn.close()

  sys.exit(0)
{code}

> SolrJ JDBC - Jython Documentation
> -
>
> Key: SOLR-9018
> URL: https://issues.apache.org/jira/browse/SOLR-9018
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Reporter: Kevin Risden
>
> Jython integrates both Python and Java. This makes connecting to JDBC 
> possible. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9018) SolrJ JDBC - Jython Documentation

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250171#comment-15250171
 ] 

Kevin Risden commented on SOLR-9018:


Jython zxJDBC example script:
{code}
#!/usr/bin/env jython

# http://www.jython.org/jythonbook/en/1.0/DatabasesAndJython.html
# https://wiki.python.org/jython/DatabaseExamples#SQLite_using_ziclix

import sys

from com.ziclix.python.sql import zxJDBC

if __name__ == '__main__':
  jdbc_url = "jdbc:solr://solr:9983?collection=test"
  driverName = "org.apache.solr.client.solrj.io.sql.DriverImpl"
  statement = "select fielda, fieldb, fieldc, fieldd_s, fielde_i from test 
limit 10"

  with zxJDBC.connect(jdbc_url, None, None, driverName) as conn:
with conn:
  with conn.cursor() as c:
c.execute(statement)
print(c.fetchall())

  sys.exit(0)
{code}

> SolrJ JDBC - Jython Documentation
> -
>
> Key: SOLR-9018
> URL: https://issues.apache.org/jira/browse/SOLR-9018
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Reporter: Kevin Risden
>
> Jython integrates both Python and Java. This makes connecting to JDBC 
> possible. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9018) SolrJ JDBC - Jython Documentation

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250168#comment-15250168
 ] 

Kevin Risden commented on SOLR-9018:


Requires that CLASSPATH environment variable be set to the SolrJ jars before 
running the Jython scripts.

> SolrJ JDBC - Jython Documentation
> -
>
> Key: SOLR-9018
> URL: https://issues.apache.org/jira/browse/SOLR-9018
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Reporter: Kevin Risden
>
> Jython integrates both Python and Java. This makes connecting to JDBC 
> possible. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-8913) When using a shared filesystem we should store data dir and tlog dir locations in the clusterstate.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller resolved SOLR-8913.
---
   Resolution: Fixed
Fix Version/s: 6.1
   master

> When using a shared filesystem we should store data dir and tlog dir 
> locations in the clusterstate.
> ---
>
> Key: SOLR-8913
> URL: https://issues.apache.org/jira/browse/SOLR-8913
> Project: Solr
>  Issue Type: Improvement
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: master, 6.1
>
>
> Spinning this out of SOLR-6237. I'll put up an initial patch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9018) SolrJ JDBC - Jython Documentation

[Kevin Risden (JIRA)]

Kevin Risden created SOLR-9018:
--

 Summary: SolrJ JDBC - Jython Documentation
 Key: SOLR-9018
 URL: https://issues.apache.org/jira/browse/SOLR-9018
 Project: Solr
  Issue Type: Sub-task
  Components: SolrJ
Reporter: Kevin Risden


Jython integrates both Python and Java. This makes connecting to JDBC possible. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8913) When using a shared filesystem we should store data dir and tlog dir locations in the clusterstate.

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250158#comment-15250158
 ] 

ASF subversion and git services commented on SOLR-8913:
---

Commit b111bfda429991b9db44ac0dbe50a2c14b076307 in lucene-solr's branch 
refs/heads/branch_6x from markrmiller
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b111bfd ]

SOLR-8913: When using a shared filesystem we should store data dir and tlog dir 
locations in the cluster state.


> When using a shared filesystem we should store data dir and tlog dir 
> locations in the clusterstate.
> ---
>
> Key: SOLR-8913
> URL: https://issues.apache.org/jira/browse/SOLR-8913
> Project: Solr
>  Issue Type: Improvement
>Reporter: Mark Miller
>Assignee: Mark Miller
>
> Spinning this out of SOLR-6237. I'll put up an initial patch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8659) Improve Solr JDBC Driver to support more SQL Clients

[Kevin Risden (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Risden updated SOLR-8659:
---
Description: 
SOLR-8502 was a great start to getting JDBC support to be more complete. This 
ticket is to track items that could further improve the JDBC support for more 
SQL clients and their features. A few SQL clients are:
* DbVisualizer
* SQuirrel SQL
* Apache Zeppelin (incubating)
* Spark
* Python & Jython
* IntelliJ IDEA Database Tool
* ODBC clients like Excel/Tableau

  was:
SOLR-8502 was a great start to getting JDBC support to be more complete. This 
ticket is to track items that could further improve the JDBC support for more 
SQL clients and their features. A few SQL clients are:
* DbVisualizer
* SQuirrel SQL
* Apache Zeppelin (incubating)
* IntelliJ IDEA Database Tool
* ODBC clients like Excel/Tableau


> Improve Solr JDBC Driver to support more SQL Clients
> 
>
> Key: SOLR-8659
> URL: https://issues.apache.org/jira/browse/SOLR-8659
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrJ
>Affects Versions: master
>Reporter: Kevin Risden
> Attachments: 
> iODBC_Demo__Unicode__-_Connected_to__remotesolr__and_Attach_screenshot_-_ASF_JIRA.png
>
>
> SOLR-8502 was a great start to getting JDBC support to be more complete. This 
> ticket is to track items that could further improve the JDBC support for more 
> SQL clients and their features. A few SQL clients are:
> * DbVisualizer
> * SQuirrel SQL
> * Apache Zeppelin (incubating)
> * Spark
> * Python & Jython
> * IntelliJ IDEA Database Tool
> * ODBC clients like Excel/Tableau



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8913) When using a shared filesystem we should store data dir and tlog dir locations in the clusterstate.

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250152#comment-15250152
 ] 

ASF subversion and git services commented on SOLR-8913:
---

Commit b44ca080d6967fce9f6689e38f2b52968bb96d81 in lucene-solr's branch 
refs/heads/master from markrmiller
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b44ca08 ]

SOLR-8913: When using a shared filesystem we should store data dir and tlog dir 
locations in the cluster state.


> When using a shared filesystem we should store data dir and tlog dir 
> locations in the clusterstate.
> ---
>
> Key: SOLR-8913
> URL: https://issues.apache.org/jira/browse/SOLR-8913
> Project: Solr
>  Issue Type: Improvement
>Reporter: Mark Miller
>Assignee: Mark Miller
>
> Spinning this out of SOLR-6237. I'll put up an initial patch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9013) SolrJ JDBC - Python JayDeBeApi documentation

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250150#comment-15250150
 ] 

Kevin Risden commented on SOLR-9013:


The CLASSPATH variable needs to be set to the SolrJ jars before running a 
Python script. An example of a Python JayDeBeApi script is below:

{code}
#!/usr/bin/env python

# https://pypi.python.org/pypi/JayDeBeApi/

import jaydebeapi
import sys

if __name__ == '__main__':
  jdbc_url = "jdbc:solr://solr:9983?collection=test"
  driverName = "org.apache.solr.client.solrj.io.sql.DriverImpl"
  statement = "select fielda, fieldb, fieldc, fieldd_s, fielde_i from test 
limit 10"

  conn = jaydebeapi.connect(driverName, jdbc_url)
  curs = conn.cursor()
  curs.execute(statement)
  print(curs.fetchall())

  conn.close()

  sys.exit(0)
{code}

> SolrJ JDBC - Python JayDeBeApi documentation
> 
>
> Key: SOLR-9013
> URL: https://issues.apache.org/jira/browse/SOLR-9013
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master, 6.0
>Reporter: Kevin Risden
>
> Like SOLR-8521, it would be great to document how Python JayDeBeApi can be 
> used with SolrJ JDBC.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Term vs. token

[Jack Krupansky]

Looking at the Lucene Similarity Javadoc, I see some references to tokens,
but I am wondering if that is intentional or whether those should really be
references to terms.

For example:

 *lengthNorm - computed
 *when the document is added to the index in accordance with the
number of tokens
 *of this field in the document, so that shorter fields contribute
more to the score.

I think that should be terms, not tokens.

See:
https://github.com/apache/lucene-solr/blob/releases/lucene-solr/5.5.0/lucene/core/src/java/org/apache/lucene/search/similarities/TFIDFSimilarity.java#L466

And this:

   * Returns the total number of tokens in the field.
   * @see Terms#getSumTotalTermFreq()
   */
  public long getNumberOfFieldTokens() {
return numberOfFieldTokens;

I think that should be terms as well:

See:
https://github.com/apache/lucene-solr/blob/master/lucene/core/src/java/org/apache/lucene/search/similarities/BasicStats.java#L65

And... this:

  numberOfFieldTokens = sumTotalTermFreq;

Where it is clearly starting with terms and treating them as tokens, but as
in the previous example, I think that should be terms as well.

See:
https://github.com/apache/lucene-solr/blob/master/lucene/core/src/java/org/apache/lucene/search/similarities/SimilarityBase.java#L128

One last example:

   * Compute any collection-level weight (e.g. IDF, average document
length, etc) needed for scoring a query.
   *
   * @param collectionStats collection-level statistics, such as the number
of tokens in the collection.
   * @param termStats term-level statistics, such as the document frequency
of a term across the collection.
   * @return SimWeight object with the information this Similarity needs to
score a query.
   */
  public abstract SimWeight computeWeight(CollectionStatistics
collectionStats, TermStatistics... termStats);

See:
https://github.com/apache/lucene-solr/blob/master/lucene/core/src/java/org/apache/lucene/search/similarities/Similarity.java#L161

In fact, CollectionStatistics uses term, not token:

  /** returns the total number of tokens for this field
   * @see Terms#getSumTotalTermFreq() */
  public final long sumTotalTermFreq() {
return sumTotalTermFreq;

Oops... it uses both, emphasizing my point about the confusion.

There are other examples as well.

My understanding is that tokens are merely a temporary transition in
between the original raw source text for a field and then final terms to be
indexed (or query terms from a parsed and analyzed query.) Yes, during and
within TokenStream or the analyzer we speak of tokens and intermediate
string values are referred to as tokens, but once the final string value is
retrieved from the Token Stream (analyzer), it's a term.

In any case, is there some distinction in any of these cited examples (or
other examples in this or related code) where "token" is an important
distinction to be made and "term" is not the proper... term... to be used?

Unless the Lucene project fully intends that the terms token and term are
absolutely synonymous, a clear distinction should be drawn... I think. Or
at least the terms should be used consistently, which my last example
highlights.

Thanks.

-- Jack Krupansky

[jira] [Resolved] (SOLR-9011) SolrJ JDBC - Ensure that Python JayDeBeApi works with SolrJ JDBC

[Kevin Risden (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Risden resolved SOLR-9011.

Resolution: Fixed

Resolving since SOLR-8809 was closed and manual testing shows Python with 
JayDeBeApi was able to connect to Solr.

> SolrJ JDBC - Ensure that Python JayDeBeApi works with SolrJ JDBC
> 
>
> Key: SOLR-9011
> URL: https://issues.apache.org/jira/browse/SOLR-9011
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master, 6.0
>Reporter: Kevin Risden
> Fix For: 6.1
>
>
> Python with JayDeBeApi enables connections to JDBC sources.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9011) SolrJ JDBC - Ensure that Python JayDeBeApi works with SolrJ JDBC

[Kevin Risden (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Risden updated SOLR-9011:
---
Fix Version/s: 6.1

> SolrJ JDBC - Ensure that Python JayDeBeApi works with SolrJ JDBC
> 
>
> Key: SOLR-9011
> URL: https://issues.apache.org/jira/browse/SOLR-9011
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master, 6.0
>Reporter: Kevin Risden
> Fix For: 6.1
>
>
> Python with JayDeBeApi enables connections to JDBC sources.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8845) SolrJ JDBC - Ensure that Spark works with SolrJ JDBC

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250136#comment-15250136
 ] 

Kevin Risden commented on SOLR-8845:


[~cahilltr] I think has some more insight. "Select *" is called by Spark when 
it first connects to get information about the field names. Could be related to 
that. I'll keep digging.

> SolrJ JDBC - Ensure that Spark works with SolrJ JDBC
> 
>
> Key: SOLR-8845
> URL: https://issues.apache.org/jira/browse/SOLR-8845
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master
>Reporter: Trey Cahill
>
> Ensure that Spark is able work with SolrJ JDBC.  
> Currently, in Spark 1.6 there are 2 known issues:
> 1. SparkSQL uses a connection.prepareStatement() call, which returns null in 
> the SolrJ implementation
> 2. SparkSQL query's via a "select *" query, which is currently not supported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8845) SolrJ JDBC - Ensure that Spark works with SolrJ JDBC

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250133#comment-15250133
 ] 

Joel Bernstein commented on SOLR-8845:
--

Interesting. I wonder what query generated this error.

> SolrJ JDBC - Ensure that Spark works with SolrJ JDBC
> 
>
> Key: SOLR-8845
> URL: https://issues.apache.org/jira/browse/SOLR-8845
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrJ
>Affects Versions: master
>Reporter: Trey Cahill
>
> Ensure that Spark is able work with SolrJ JDBC.  
> Currently, in Spark 1.6 there are 2 known issues:
> 1. SparkSQL uses a connection.prepareStatement() call, which returns null in 
> the SolrJ implementation
> 2. SparkSQL query's via a "select *" query, which is currently not supported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org