Re: Is it possible to use a sentence as a term?

[szzoli]

Btw.,
I work with Lucene 6.2. I found some solutions for older versions. These
din't work with 6.2.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Is-it-possible-to-use-a-sentence-as-a-term-tp4294737p4294738.html
Sent from the Lucene - Java Developer mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Is it possible to use a sentence as a term?

[szzoli]

Hi All,

Is it possible to use a sentence as a term? Can I search for sentences in
the index? How can I do it if possible?

Thank you in advance.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Is-it-possible-to-use-a-sentence-as-a-term-tp4294737.html
Sent from the Lucene - Java Developer mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [VOTE] Release Lucene/Solr 5.5.3 RC1

[Shalin Shekhar Mangar]

+1

SUCCESS! [1:07:01.319712]

On Fri, Sep 2, 2016 at 7:59 PM, Anshum Gupta  wrote:

> Please vote for release candidate 1 for Lucene/Solr 5.5.3
>
> The artifacts can be downloaded from:
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.5.3-RC1-
> rev8655b97b27d8da470c8235683af11a8b85a2b10f
>
> You can run the smoke tester directly with this command (on the release
> branch):
>
> python3 -u dev-tools/scripts/smokeTestRelease.py \
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.5.3-RC1-
> rev8655b97b27d8da470c8235683af11a8b85a2b10f
>
> Here's my +1
> SUCCESS! [0:36:59.036831]
>



-- 
Regards,
Shalin Shekhar Mangar.

[jira] [Comment Edited] (SOLR-9127) XLSX response writer - do we want it?

[Tony Moriarty (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15466493#comment-15466493
 ] 

Tony Moriarty edited comment on SOLR-9127 at 9/6/16 5:46 AM:
-

added 
https://issues.apache.org/jira/secure/attachment/12827123/9127-xlsxresponsewriter.patch
full patch as per PR 
https://github.com/apache/lucene-solr/pull/77

replaces first patch (SOLR-9127.patch)

all commits squashed into one for easy review


was (Author: desultir):
added 
https://issues.apache.org/jira/secure/attachment/12827123/9127-xlsxresponsewriter.patch
full patch as per PR 
https://github.com/apache/lucene-solr/pull/77

replaces first patch (SOLR-9127.patch)

> XLSX response writer - do we want it?
> -
>
> Key: SOLR-9127
> URL: https://issues.apache.org/jira/browse/SOLR-9127
> Project: Solr
>  Issue Type: Improvement
>  Components: Response Writers
>Reporter: Tony Moriarty
>Priority: Minor
> Fix For: 5.5, 6.0
>
> Attachments: 9127-xlsxresponsewriter.patch, SOLR-9127.patch
>
>
> I recently open sourced an XLSX response writer based on solr 4.6 and apache 
> poi.
> https://github.com/desultir/SolrXLSXResponseWriter
> Is this something the community would be interested in bringing into the solr 
> codebase? I'm willing to put the work into porting it to solr5 and solr6 if 
> the community is interested, happy to leave it as a plugin otherwise.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9127) XLSX response writer - do we want it?

[Tony Moriarty (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15466493#comment-15466493
 ] 

Tony Moriarty edited comment on SOLR-9127 at 9/6/16 5:45 AM:
-

added 
https://issues.apache.org/jira/secure/attachment/12827123/9127-xlsxresponsewriter.patch
full patch as per PR 
https://github.com/apache/lucene-solr/pull/77

replaces first patch (SOLR-9127.patch)


was (Author: desultir):
full patch as per PR 
https://github.com/apache/lucene-solr/pull/77

replaces first patch (SOLR-9127.patch)

> XLSX response writer - do we want it?
> -
>
> Key: SOLR-9127
> URL: https://issues.apache.org/jira/browse/SOLR-9127
> Project: Solr
>  Issue Type: Improvement
>  Components: Response Writers
>Reporter: Tony Moriarty
>Priority: Minor
> Fix For: 5.5, 6.0
>
> Attachments: 9127-xlsxresponsewriter.patch, SOLR-9127.patch
>
>
> I recently open sourced an XLSX response writer based on solr 4.6 and apache 
> poi.
> https://github.com/desultir/SolrXLSXResponseWriter
> Is this something the community would be interested in bringing into the solr 
> codebase? I'm willing to put the work into porting it to solr5 and solr6 if 
> the community is interested, happy to leave it as a plugin otherwise.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9127) XLSX response writer - do we want it?

[Tony Moriarty (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15466493#comment-15466493
 ] 

Tony Moriarty edited comment on SOLR-9127 at 9/6/16 5:44 AM:
-

full patch as per PR 
https://github.com/apache/lucene-solr/pull/77

replaces first patch (SOLR-9127.patch)


was (Author: desultir):
full patch as per PR 
https://github.com/apache/lucene-solr/pull/77

Incorporates first patch (SOLR-9127.patch)

> XLSX response writer - do we want it?
> -
>
> Key: SOLR-9127
> URL: https://issues.apache.org/jira/browse/SOLR-9127
> Project: Solr
>  Issue Type: Improvement
>  Components: Response Writers
>Reporter: Tony Moriarty
>Priority: Minor
> Fix For: 5.5, 6.0
>
> Attachments: 9127-xlsxresponsewriter.patch, SOLR-9127.patch
>
>
> I recently open sourced an XLSX response writer based on solr 4.6 and apache 
> poi.
> https://github.com/desultir/SolrXLSXResponseWriter
> Is this something the community would be interested in bringing into the solr 
> codebase? I'm willing to put the work into porting it to solr5 and solr6 if 
> the community is interested, happy to leave it as a plugin otherwise.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9127) XLSX response writer - do we want it?

[Tony Moriarty (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tony Moriarty updated SOLR-9127:

Attachment: 9127-xlsxresponsewriter.patch

full patch as per PR 
https://github.com/apache/lucene-solr/pull/77

Incorporates first patch (SOLR-9127.patch)

> XLSX response writer - do we want it?
> -
>
> Key: SOLR-9127
> URL: https://issues.apache.org/jira/browse/SOLR-9127
> Project: Solr
>  Issue Type: Improvement
>  Components: Response Writers
>Reporter: Tony Moriarty
>Priority: Minor
> Fix For: 5.5, 6.0
>
> Attachments: 9127-xlsxresponsewriter.patch, SOLR-9127.patch
>
>
> I recently open sourced an XLSX response writer based on solr 4.6 and apache 
> poi.
> https://github.com/desultir/SolrXLSXResponseWriter
> Is this something the community would be interested in bringing into the solr 
> codebase? I'm willing to put the work into porting it to solr5 and solr6 if 
> the community is interested, happy to leave it as a plugin otherwise.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9127) XLSX response writer - do we want it?

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15466488#comment-15466488
 ] 

ASF GitHub Bot commented on SOLR-9127:
--

GitHub user desultir opened a pull request:

https://github.com/apache/lucene-solr/pull/77

[SOLR-9127] XlsxResponseWriter

https://issues.apache.org/jira/browse/SOLR-9127

New response writer and test case (TextXLSXResponseWriter)

passes ant precommit
all work squashed into one commit for easy perusal

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/desultir/lucene-solr xlsx

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/lucene-solr/pull/77.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #77


commit 05363c347bb39efb24919eeb474280ca0edff753
Author: Tony Moriarty 
Date:   2016-09-06T05:34:46Z

full squashed patch




> XLSX response writer - do we want it?
> -
>
> Key: SOLR-9127
> URL: https://issues.apache.org/jira/browse/SOLR-9127
> Project: Solr
>  Issue Type: Improvement
>  Components: Response Writers
>Reporter: Tony Moriarty
>Priority: Minor
> Fix For: 5.5, 6.0
>
> Attachments: SOLR-9127.patch
>
>
> I recently open sourced an XLSX response writer based on solr 4.6 and apache 
> poi.
> https://github.com/desultir/SolrXLSXResponseWriter
> Is this something the community would be interested in bringing into the solr 
> codebase? I'm willing to put the work into porting it to solr5 and solr6 if 
> the community is interested, happy to leave it as a plugin otherwise.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr pull request #77: [SOLR-9127] XlsxResponseWriter

[desultir]

GitHub user desultir opened a pull request:

https://github.com/apache/lucene-solr/pull/77

[SOLR-9127] XlsxResponseWriter

https://issues.apache.org/jira/browse/SOLR-9127

New response writer and test case (TextXLSXResponseWriter)

passes ant precommit
all work squashed into one commit for easy perusal

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/desultir/lucene-solr xlsx

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/lucene-solr/pull/77.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #77


commit 05363c347bb39efb24919eeb474280ca0edff753
Author: Tony Moriarty 
Date:   2016-09-06T05:34:46Z

full squashed patch




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9127) XLSX response writer - do we want it?

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15466482#comment-15466482
 ] 

ASF GitHub Bot commented on SOLR-9127:
--

Github user desultir commented on the issue:

https://github.com/apache/lucene-solr/pull/59
  
closing to issue cleaner PR


> XLSX response writer - do we want it?
> -
>
> Key: SOLR-9127
> URL: https://issues.apache.org/jira/browse/SOLR-9127
> Project: Solr
>  Issue Type: Improvement
>  Components: Response Writers
>Reporter: Tony Moriarty
>Priority: Minor
> Fix For: 5.5, 6.0
>
> Attachments: SOLR-9127.patch
>
>
> I recently open sourced an XLSX response writer based on solr 4.6 and apache 
> poi.
> https://github.com/desultir/SolrXLSXResponseWriter
> Is this something the community would be interested in bringing into the solr 
> codebase? I'm willing to put the work into porting it to solr5 and solr6 if 
> the community is interested, happy to leave it as a plugin otherwise.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9127) XLSX response writer - do we want it?

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15466481#comment-15466481
 ] 

ASF GitHub Bot commented on SOLR-9127:
--

Github user desultir closed the pull request at:

https://github.com/apache/lucene-solr/pull/59


> XLSX response writer - do we want it?
> -
>
> Key: SOLR-9127
> URL: https://issues.apache.org/jira/browse/SOLR-9127
> Project: Solr
>  Issue Type: Improvement
>  Components: Response Writers
>Reporter: Tony Moriarty
>Priority: Minor
> Fix For: 5.5, 6.0
>
> Attachments: SOLR-9127.patch
>
>
> I recently open sourced an XLSX response writer based on solr 4.6 and apache 
> poi.
> https://github.com/desultir/SolrXLSXResponseWriter
> Is this something the community would be interested in bringing into the solr 
> codebase? I'm willing to put the work into porting it to solr5 and solr6 if 
> the community is interested, happy to leave it as a plugin otherwise.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr issue #59: SOLR-9127: XLSXResponseWriter

[desultir]

Github user desultir commented on the issue:

https://github.com/apache/lucene-solr/pull/59
  
closing to issue cleaner PR


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr pull request #59: SOLR-9127: XLSXResponseWriter

[desultir]

Github user desultir closed the pull request at:

https://github.com/apache/lucene-solr/pull/59


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9127) XLSX response writer - do we want it?

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15466416#comment-15466416
 ] 

Noble Paul commented on SOLR-9127:
--

yes. it's noisy and I don't know what is relevant and what is not. attach a 
real patch file to this ticket. 

> XLSX response writer - do we want it?
> -
>
> Key: SOLR-9127
> URL: https://issues.apache.org/jira/browse/SOLR-9127
> Project: Solr
>  Issue Type: Improvement
>  Components: Response Writers
>Reporter: Tony Moriarty
>Priority: Minor
> Fix For: 5.5, 6.0
>
> Attachments: SOLR-9127.patch
>
>
> I recently open sourced an XLSX response writer based on solr 4.6 and apache 
> poi.
> https://github.com/desultir/SolrXLSXResponseWriter
> Is this something the community would be interested in bringing into the solr 
> codebase? I'm willing to put the work into porting it to solr5 and solr6 if 
> the community is interested, happy to leave it as a plugin otherwise.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9480) Graph Traversal for Significantly Related Terms (Semantic Knowledge Graph)

[Trey Grainger (JIRA)]

Trey Grainger created SOLR-9480:
---

 Summary: Graph Traversal for Significantly Related Terms (Semantic 
Knowledge Graph)
 Key: SOLR-9480
 URL: https://issues.apache.org/jira/browse/SOLR-9480
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Trey Grainger


This issue is to track the contribution of the Semantic Knowledge Graph Solr 
Plugin (request handler), which exposes a graph-like interface for discovering 
and traversing significant relationships between entities within an inverted 
index.

This data model has been described in the following research paper: [The 
Semantic Knowledge Graph: A compact, auto-generated model for real-time 
traversal and ranking of any relationship within a 
domain|https://arxiv.org/abs/1609.00464], as well as in presentations I gave in 
October 2015 at [Lucene/Solr 
Revolution|http://www.slideshare.net/treygrainger/leveraging-lucenesolr-as-a-knowledge-graph-and-intent-engine]
 and November 2015 at the [Bay Area Search 
Meetup|http://www.treygrainger.com/posts/presentations/searching-on-intent-knowledge-graphs-personalization-and-contextual-disambiguation/].

The source code for this project is currently available at 
[https://github.com/careerbuilder/semantic-knowledge-graph], and the folks at 
CareerBuilder (where this was built) have given me the go-ahead to now 
contribute this back to the Apache Solr Project, as well.

Check out the Github repository, research paper, or presentations for a more 
detailed description of this contribution. Initial patch coming soon.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9127) XLSX response writer - do we want it?

[Tony Moriarty (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15466339#comment-15466339
 ] 

Tony Moriarty commented on SOLR-9127:
-

Apologies for not being clear: it's the same PR as originally linked in the 
thread at:
https://github.com/apache/lucene-solr/pull/59

The patch is noisy as it involves commits which were later undone insofar as i 
added things to solr core and then removed:
https://patch-diff.githubusercontent.com/raw/apache/lucene-solr/pull/59.patch

Will this noisy patch be an issue?

> XLSX response writer - do we want it?
> -
>
> Key: SOLR-9127
> URL: https://issues.apache.org/jira/browse/SOLR-9127
> Project: Solr
>  Issue Type: Improvement
>  Components: Response Writers
>Reporter: Tony Moriarty
>Priority: Minor
> Fix For: 5.5, 6.0
>
> Attachments: SOLR-9127.patch
>
>
> I recently open sourced an XLSX response writer based on solr 4.6 and apache 
> poi.
> https://github.com/desultir/SolrXLSXResponseWriter
> Is this something the community would be interested in bringing into the solr 
> codebase? I'm willing to put the work into porting it to solr5 and solr6 if 
> the community is interested, happy to leave it as a plugin otherwise.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (64bit/jdk1.8.0_102) - Build # 6102 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6102/
Java: 64bit/jdk1.8.0_102 -XX:+UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.CdcrVersionReplicationTest

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [InternalHttpClient]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not 
released!!! [InternalHttpClient]
at __randomizedtesting.SeedInfo.seed([309971CDF3C9F0FF]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:257)
at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11984 lines...]
   [junit4] Suite: org.apache.solr.cloud.CdcrVersionReplicationTest
   [junit4]   2> Creating dataDir: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrVersionReplicationTest_309971CDF3C9F0FF-001\init-core-data-001
   [junit4]   2> 1908979 INFO  
(SUITE-CdcrVersionReplicationTest-seed#[309971CDF3C9F0FF]-worker) [] 
o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: 
@org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 1908981 INFO  
(SUITE-CdcrVersionReplicationTest-seed#[309971CDF3C9F0FF]-worker) [] 
o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /a/
   [junit4]   2> 1908985 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[309971CDF3C9F0FF]) [ 
   ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1908985 INFO  (Thread-3827) [] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1908985 INFO  (Thread-3827) [] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 1909085 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[309971CDF3C9F0FF]) [ 
   ] o.a.s.c.ZkTestServer start zk server on port:61873
   [junit4]   2> 1909086 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[309971CDF3C9F0FF]) [ 
   ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1909087 INFO  
(TEST-CdcrVersionReplicationTest.testCdcrDocVersions-seed#[309971CDF3C9F0FF]) [ 
   ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1909091 INFO  (zkCallback-2666-thread-1) [] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@3f4dd6c2 
name:ZooKeeperConnection Watcher:127.0.0.1:61873 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1909092

[jira] [Updated] (SOLR-9477) UpdateRequestProcessors ignore child documents

[Alexandre Rafalovitch (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexandre Rafalovitch updated SOLR-9477:

Description: 
UpdateRequestProcessors completely ignore child documents. The only exception 
is AddSchemaFieldsUpdateProcessorFactory. The rest seem to be completely 
unaware that SolrInputDocument has getChildDocuments() or related methods.

Easy test (on Solr 6.2):

This works (with IDs auto-assigned and field names generated):
{code}
bin/solr create -c childtest
bin/post -c childtest -type application/json -format solr -d '[{"a":1,"b":2}]'
{code}

This fails as the second/third command, with "missing ID field":
{code}
bin/post -c childtest -type application/json -format solr -d 
'[{"a":1,"b":2,"_childDocuments_":[{"c":3,"d":4}]}]'
{code}

The message:
{noformat}
SimplePostTool version 5.0.0
POSTing args to http://localhost:8983/solr/childtest/update...
SimplePostTool: WARNING: Solr returned an error #400 (Bad Request) for url: 
http://localhost:8983/solr/childtest/update
SimplePostTool: WARNING: Response: 
{"responseHeader":{"status":400,"QTime":4},"error":{"metadata":["error-class","org.apache.solr.common.SolrException","root-error-class","org.apache.solr.common.SolrException"],"msg":"[doc=null]
 missing required field: id","code":400}}
SimplePostTool: WARNING: IOException while reading response: 
java.io.IOException: Server returned HTTP response code: 400 for URL: 
http://localhost:8983/solr/childtest/update
COMMITting Solr index changes to http://localhost:8983/solr/childtest/update...
Time spent: 0:00:00.042
{noformat}

I also verified it with BlankRemoving URP. I think this is a global problem.

  was:
UpdateRequestProcessors completely ignore child documents. The only exception 
is AddSchemaFieldsUpdateProcessorFactory. The rest seem to be completely 
unaware that SolrInputDocument has getChildDocuments() or related methods.

Easy test (on Solr 6.2):

This works (with IDs auto-assigned and field names generated):
{code}
bin/solr create -c childtest
bin/post -c childtest -type application/json -format solr -d "[{"a":1,"b":2}]"
{code}

This fails as the second/third command, with "missing ID field":
{code}
bin/post -c childtest -type application/json -format solr -d 
"[{"a":1,"b":2,"_childDocuments_":[{c:3,d:4}]}]"
{code}

The message:
{noformat}
SimplePostTool version 5.0.0
POSTing args to http://localhost:8983/solr/childtest/update...
SimplePostTool: WARNING: Solr returned an error #400 (Bad Request) for url: 
http://localhost:8983/solr/childtest/update
SimplePostTool: WARNING: Response: 
{"responseHeader":{"status":400,"QTime":4},"error":{"metadata":["error-class","org.apache.solr.common.SolrException","root-error-class","org.apache.solr.common.SolrException"],"msg":"[doc=null]
 missing required field: id","code":400}}
SimplePostTool: WARNING: IOException while reading response: 
java.io.IOException: Server returned HTTP response code: 400 for URL: 
http://localhost:8983/solr/childtest/update
COMMITting Solr index changes to http://localhost:8983/solr/childtest/update...
Time spent: 0:00:00.042
{noformat}

I also verified it with BlankRemoving URP. I think this is a global problem.


> UpdateRequestProcessors ignore child documents
> --
>
> Key: SOLR-9477
> URL: https://issues.apache.org/jira/browse/SOLR-9477
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2, master (7.0)
>Reporter: Alexandre Rafalovitch
>  Labels: UpdateProcessor
>
> UpdateRequestProcessors completely ignore child documents. The only exception 
> is AddSchemaFieldsUpdateProcessorFactory. The rest seem to be completely 
> unaware that SolrInputDocument has getChildDocuments() or related methods.
> Easy test (on Solr 6.2):
> This works (with IDs auto-assigned and field names generated):
> {code}
> bin/solr create -c childtest
> bin/post -c childtest -type application/json -format solr -d '[{"a":1,"b":2}]'
> {code}
> This fails as the second/third command, with "missing ID field":
> {code}
> bin/post -c childtest -type application/json -format solr -d 
> '[{"a":1,"b":2,"_childDocuments_":[{"c":3,"d":4}]}]'
> {code}
> The message:
> {noformat}
> SimplePostTool version 5.0.0
> POSTing args to http://localhost:8983/solr/childtest/update...
> SimplePostTool: WARNING: Solr returned an error #400 (Bad Request) for url: 
> http://localhost:8983/solr/childtest/update
> SimplePostTool: WARNING: Response: 
> {"responseHeader":{"status":400,"QTime":4},"error":{"metadata":["error-class","org.apache.solr.common.SolrException","root-error-class","org.apache.solr.common.SolrException"],"msg":"[doc=null]
>  missing required field: id","code":400}}
> SimplePostTool: WARNING: IOException while reading response: 
> java.io.IOException: Se

[jira] [Commented] (SOLR-9478) Do delta import will loss some documents, when the documents added in the duration of delta import.

[ted zhu (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15466102#comment-15466102
 ] 

ted zhu commented on SOLR-9478:
---

thank you very much, i 'll send a mail to solr-user

On Tue, Sep 6, 2016 at 3:25 AM, Erick Erickson (JIRA) 




-- 
*Ted Zhu 祝海潮*
* Software Development, China*


*Direct: +86 (21) 6032 7330**Mobile: +86 155 021 91092*

*Mintel Information Consulting (Shanghai) Co., Ltd *

*25th Floor | Broad Silver International Building | 398 Huaihai Zhong Road



-- 




Mintel Group Ltd | London | Chicago | New York | Sydney | Shanghai | Tokyo | 
Mumbai | Singapore | Kuala Lumpur | Sao Paulo | Belfast

Contact details for our offices can be found at 
http://www.mintel.com/office-locations.

This email and any attachments may include content that is confidential, 
privileged, or otherwise protected
under applicable law. Unauthorised disclosure, copying, distribution, or use of 
the contents is prohibited 
and may be unlawful. If you have received this email in error, including 
without appropriate authorisation, 
then please reply to the sender about the error and delete this email and any 
attachments.



> Do delta import will loss some documents, when the documents added in the 
> duration of delta import.
> ---
>
> Key: SOLR-9478
> URL: https://issues.apache.org/jira/browse/SOLR-9478
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 5.5, 5.5.2, 6.0
>Reporter: ted zhu
>  Labels: delta-import, solrcloud
> Fix For: 5.5.2
>
>
> hello guys,
> I met a problem when i using the solrcloud mode. When the solr instance run 
> delta-import, it may take 
> some time to be finished( my data source is mysql database). So during this 
> time, the new added documents
> will loss, the deltaQuery, i use SUBDATE($\{dih.last_index_time\}, INTERVAL 2 
> MINUTE), 
> let it run the delta-import 2 mins earlier than the last_index_time, if the 
> delta-import's duration is 5 mins, it will loss the records at the first 3 
> mins.
> Our servers doesn't use solr cloud mode before, we deal with this issue 
> is tring to rewrite dataimport.properties file, 
> query the max(sys_time_stamp), which will help to record the max time stamp, 
> and let the solr can run delta import standing 
> by the time found in the file, of course, it will never miss docuements. 
>But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
> and we may have multiple collections for the 
> same core.how can i update the dataimport.properties file now in colleciton 
> now? Do you have any solution to help record
>  the max(sys_time_stamp) in dataimport.properties, rather than using the time 
> of delta-import start to run?
> Cheers



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 830 - Still unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/830/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  
org.apache.solr.common.cloud.TestCollectionStateWatchers.testCanWaitForNonexistantCollection

Error Message:
waitForState was not triggered by collection creation

Stack Trace:
java.lang.AssertionError: waitForState was not triggered by collection creation
at 
__randomizedtesting.SeedInfo.seed([6E92450E1049C8D8:C5B2DEB87D170CF3]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.common.cloud.TestCollectionStateWatchers.testCanWaitForNonexistantCollection(TestCollectionStateWatchers.java:207)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 12959 lines...]
   [junit4] Suite: org.apache.solr.common.cloud.TestCollectionStateWatchers
   [junit4]   2> Creating dataDir: 
/export/home/jenkins

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3527 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3527/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC

3 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.handler.TestSolrConfigHandlerCloud

Error Message:
ObjectTracker found 5 object(s) that were not released!!! 
[MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, 
MDCAwareThreadPoolExecutor]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 5 object(s) that were not 
released!!! [MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, 
SolrCore, MDCAwareThreadPoolExecutor]
at __randomizedtesting.SeedInfo.seed([5035DE0607C64413]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:257)
at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)


FAILED:  
junit.framework.TestSuite.org.apache.solr.handler.TestSolrConfigHandlerCloud

Error Message:
1 thread leaked from SUITE scope at 
org.apache.solr.handler.TestSolrConfigHandlerCloud: 1) Thread[id=50734, 
name=searcherExecutor-12010-thread-1, state=WAITING, 
group=TGRP-TestSolrConfigHandlerCloud] at sun.misc.Unsafe.park(Native 
Method) at 
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
 at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) 
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)   
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.handler.TestSolrConfigHandlerCloud: 
   1) Thread[id=50734, name=searcherExecutor-12010-thread-1, state=WAITING, 
group=TGRP-TestSolrConfigHandlerCloud]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at 
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at 
java.ut

RE: Early Access builds of JDK 9 b134 are available on java.net

[Uwe Schindler]

Hi Rory,

 

Many thanks for the info! I installed build 134 earlier today. All looks fine. 
We are looking forward to see how the remaining bugs are fixed. Maybe the 
problems that other projects have with Jigsaw, are solved. For Lucene and Solr 
all seems fine with the module system.

 

Uwe

 

-

Uwe Schindler

H.-H.-Meier-Allee 63, D-28213 Bremen

  http://www.thetaphi.de

eMail: u...@thetaphi.de

 

From: Rory O'Donnell [mailto:rory.odonn...@oracle.com] 
Sent: Monday, September 5, 2016 7:56 PM
To: u...@thetaphi.de
Cc: rory.odonn...@oracle.com; Dalibor Topic ; 
Balchandra Vaidya ; Muneer Kolarkunnu 
; dawid.we...@cs.put.poznan.pl; 
dev@lucene.apache.org
Subject: Early Access builds of JDK 9 b134 are available on java.net

 

 

Hi Uwe & Dawid, 

Early Access b134   for JDK 9 is available on 
java.net, summary of  changes are listed here  
 .

 There have been a number of fixes , since the last availability email , to 
bugs reported by Open Source projects :

*   8156841sun.security.pkcs11.SunPKCS11 poller thread retains a strong 
reference to the context class loader 
*   8146961   Fix PermGen memory leaks caused by static final Exceptions 
*   8163353   NPE in ConcurrentHashMap.removeAll()
*   8160328   ClassCastException: sun.awt.image.BufImgSurfaceData cannot be 
cast to sun.java2d.xr.XRSurfaceData after xrandr change output

Secondly, there are a number of interesting items to bring to our attention

*   JDK 9 Rampdown Phase 1: Process proposal [1]
*   The Java team has published the “Oracle JRE and JDK Cryptographic 
Roadmap” [2]  java.com/cryptoroadmap   
*   The Quality Report for September 2016 is now available [3], thank you 
for your continued support!


Highlights from the Quality Report for September : 

*   21 new Open Source projects have joined the Outreach program
*   Projects filed 35 new issues in the JDK Bug System, this is almost 
double the number of bugs in the previous six months! 
*   Continuing to provide excellent feedback via the OpenJDK dev mailing 
lists

Thank you!

Rgds,Rory

[1] http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-August/004777.html
[2]  java.com/cryptoroadmap   
[3] 
https://wiki.openjdk.java.net/display/quality/Quality+Outreach+Report+September+2016



-- 
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland 

Re: Peer sync and overlapping "windows" heuristic may be causing unnecessary full syncs?

[Erick Erickson]

Pushkar:

I've been on vacation for a bit, I'll take a look later. Actually,
I'll probably just see if I can piggy-back on Noble's efforts ;).

The "in the field" situation I'm seeing, though, is in 5.x so
fingerprinting isn't in place so likely the real fix will require an
upgrade.

Thanks for pointing that JIRA out!

Best,
Erick

On Fri, Sep 2, 2016 at 7:37 AM, Pushkar Raste  wrote:
> Erick,
>
> If your updates are infrequent, you may benefit from a patch for
> https://issues.apache.org/jira/browse/SOLR-9446
> It seems like recovery always tries PeerSync first and would fall back on to
> replication if PeerSync fails.
>
> My fix for a different use case was to node trying PeerSync first checks
> fingerprint for the max version it has matches to the fingerprint of max
> version leader has. If checks out, PeerSync simply returns true.
>
> There may be a better way to do this. However, for you use case if updates
> are not that frequent nodes will not go into replication recovery.
>
> On Mon, Aug 22, 2016 at 4:23 PM, Erick Erickson 
> wrote:
>>
>> Looking at the peer sync code and I don't quite understand the
>> condition where we report " Our versions are too old." (about line 498
>> in PeerySync.java, 6x).
>>
>> Note that the "in the field" version is 5.3.1, but the code looks the
>> same in 6.x.
>>
>> I get that we're testing the overlap between the versions we have and
>> our peer has. But how was the 20% overlap number arrived at? What is
>> it intended to guarantee? And in a case where where the requested
>> number of updates is > the size of the returned list, is it valid to
>> return true if there is _any_ overlap?
>>
>>
>> Why do I care? I'm seeing a case in the field where a very large
>> document exceeds the timeout even though the document successfully
>> indexes on the follower, it just takes a while.  The Solr node is up
>> and accepting more updates etc. No updates have actually been missed
>> AFAICT.
>>
>> So the leader is telling the follower to sync due to the timeout. The
>> follower fails the test above and then goes into full sync
>> unnecessarily. Since this is a very large index this takes a very long
>> time, strains the system and the problem can cascade.
>>
>> I'm wondering if this test can be relaxed when the versions list
>> returned from the peer is smaller than requested to not fail if there
>> is any overlap. This feels like an incomplete fix though, because I'm
>> taking it on faith that if the list returned == numRecordsToKeep, then
>> this test wouldn't be as likely to be tripped. But there's no
>> guarantee there so a special test in this case would just kick the can
>> down the road I think.
>>
>> Can we do a different test perhaps (and I'm really reaching here into
>> unfamiliar code so this may be all wet)? Let's say the leader gets a
>> timeout. Would it be possible to rather than do a full peer sync have
>> the leader ask the follower "Hey, I sent you these versions and you
>> timed out, do you really have them or not?"? And if the follower was
>> still processing them not have to do any peer sync at all. Assuming we
>> could guarantee that the doc was in the replicas tlog when answering,
>> would that guarantee data integrity?
>>
>> I can raise a JIRA if any of this makes sense.
>>
>> Erick
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-9476) Can solrcloud recover a replica automatically when a shard goes down (not using hdfs)

[Erick Erickson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson resolved SOLR-9476.
--
Resolution: Invalid

Please raise questions like this on the user's list, we try to reserve JIRAs 
for known bugs/enhancements rather than usage questions.

See the "mailing list" section here: 
http://lucene.apache.org/solr/resources.html

> Can solrcloud  recover a replica automatically when a shard goes down (not 
> using hdfs)
> --
>
> Key: SOLR-9476
> URL: https://issues.apache.org/jira/browse/SOLR-9476
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 5.3.1
>Reporter: Dasiy 
>
> I know that the parameter "autoAddReplicas" is useful when using hdfs to 
> store index.
> I want to know if there is a way to make solrcloud to add replica 
> automatically when a shard goes down using  local file system to store index.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-9478) Do delta import will loss some documents, when the documents added in the duration of delta import.

[Erick Erickson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson resolved SOLR-9478.
--
Resolution: Not A Problem

Please raise issues like this (which appear to be usage, not a code problem) on 
the user's list before raising a JIRA. We try to reserve JIRAs for known code 
issues (bugs or enhancements), not usage questions. 

See the "mailing lists" section here: 
http://lucene.apache.org/solr/resources.html

> Do delta import will loss some documents, when the documents added in the 
> duration of delta import.
> ---
>
> Key: SOLR-9478
> URL: https://issues.apache.org/jira/browse/SOLR-9478
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 5.5, 5.5.2, 6.0
>Reporter: ted zhu
>  Labels: delta-import, solrcloud
> Fix For: 5.5.2
>
>
> hello guys,
> I met a problem when i using the solrcloud mode. When the solr instance run 
> delta-import, it may take 
> some time to be finished( my data source is mysql database). So during this 
> time, the new added documents
> will loss, the deltaQuery, i use SUBDATE($\{dih.last_index_time\}, INTERVAL 2 
> MINUTE), 
> let it run the delta-import 2 mins earlier than the last_index_time, if the 
> delta-import's duration is 5 mins, it will loss the records at the first 3 
> mins.
> Our servers doesn't use solr cloud mode before, we deal with this issue 
> is tring to rewrite dataimport.properties file, 
> query the max(sys_time_stamp), which will help to record the max time stamp, 
> and let the solr can run delta import standing 
> by the time found in the file, of course, it will never miss docuements. 
>But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
> and we may have multiple collections for the 
> same core.how can i update the dataimport.properties file now in colleciton 
> now? Do you have any solution to help record
>  the max(sys_time_stamp) in dataimport.properties, rather than using the time 
> of delta-import start to run?
> Cheers



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-5.5-Windows (64bit/jdk1.8.0_102) - Build # 115 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.5-Windows/115/
Java: 64bit/jdk1.8.0_102 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.morphlines.solr.SolrMorphlineZkAvroTest.test

Error Message:
Cannot download schema.xml from ZooKeeper near: { # String: 1 "solrUrl" 
: "", # String: 1 "collection" : "collection1", # String: 1 
"zkHost" : "127.0.0.1:50735/solr", # String: 1 "batchSize" : "1000",
 # String: 1 "solrHomeDir" : "" }

Stack Trace:
org.kitesdk.morphline.api.MorphlineCompilationException: Cannot download 
schema.xml from ZooKeeper near: {
# String: 1
"solrUrl" : "",
# String: 1
"collection" : "collection1",
# String: 1
"zkHost" : "127.0.0.1:50735/solr",
# String: 1
"batchSize" : "1000",
# String: 1
"solrHomeDir" : ""
}
at 
__randomizedtesting.SeedInfo.seed([2AB1CF3359C69947:A2E5F0E9F73AF4BF]:0)
at 
org.apache.solr.morphlines.solr.SolrLocator.getIndexSchema(SolrLocator.java:146)
at 
org.apache.solr.morphlines.solr.SanitizeUnknownSolrFieldsBuilder$SanitizeUnknownSolrFields.(SanitizeUnknownSolrFieldsBuilder.java:71)
at 
org.apache.solr.morphlines.solr.SanitizeUnknownSolrFieldsBuilder.build(SanitizeUnknownSolrFieldsBuilder.java:53)
at 
org.kitesdk.morphline.base.AbstractCommand.buildCommand(AbstractCommand.java:302)
at 
org.kitesdk.morphline.base.AbstractCommand.buildCommandChain(AbstractCommand.java:249)
at org.kitesdk.morphline.stdlib.Pipe.(Pipe.java:46)
at org.kitesdk.morphline.stdlib.PipeBuilder.build(PipeBuilder.java:40)
at 
org.apache.solr.morphlines.solr.AbstractSolrMorphlineZkTestBase.createMorphline(AbstractSolrMorphlineZkTestBase.java:116)
at 
org.apache.solr.morphlines.solr.AbstractSolrMorphlineZkTestBase.parse(AbstractSolrMorphlineZkTestBase.java:112)
at 
org.apache.solr.morphlines.solr.AbstractSolrMorphlineZkTestBase.parse(AbstractSolrMorphlineZkTestBase.java:101)
at 
org.apache.solr.morphlines.solr.SolrMorphlineZkAvroTest.test(SolrMorphlineZkAvroTest.java:66)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:996)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:971)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)

Early Access builds of JDK 9 b134 are available on java.net

[Rory O'Donnell]

Hi Uwe & Dawid,

Early Access b134  for JDK 9 is 
available on java.net, summary of  changes are listed here 
.


 There have been a number of fixes , since the last availability email 
, to bugs reported by Open Source projects :


 * 8156841sun.security.pkcs11.SunPKCS11 poller thread retains a
   strong reference to the context class loader
 * 8146961   Fix PermGen memory leaks caused by static final Exceptions
 * 8163353   NPE in ConcurrentHashMap.removeAll()
 * 8160328   ClassCastException: sun.awt.image.BufImgSurfaceData cannot
   be cast to sun.java2d.xr.XRSurfaceData after xrandr change output

Secondly, there are a number of interesting items to bring to our attention

 * JDK 9 Rampdown Phase 1: Process proposal [1]
 * The Java team has published the “Oracle JRE and JDK Cryptographic
   Roadmap” [2] java.com/cryptoroadmap 
 * The Quality Report for September 2016 is now available [3], thank
   you for your continued support!


Highlights from the Quality Report for September :

 * 21 new Open Source projects have joined the Outreach program
 * Projects filed 35 new issues in the JDK Bug System, this is almost
   double the number of bugs in the previous six months!
 * Continuing to provide excellent feedback via the OpenJDK dev mailing
   lists

Thank you!

Rgds,Rory

[1] http://mail.openjdk.java.net/pipermail/jdk9-dev/2016-August/004777.html
[2] java.com/cryptoroadmap 
[3] 
https://wiki.openjdk.java.net/display/quality/Quality+Outreach+Report+September+2016


--
Rgds,Rory O'Donnell
Quality Engineering Manager
Oracle EMEA , Dublin, Ireland

[jira] [Commented] (LUCENE-7417) Highlighting fails for MultiPhraseQuery's with one clause

[Thomas Kappler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15465472#comment-15465472
 ] 

Thomas Kappler commented on LUCENE-7417:


I got clearance from legal now and attached a patch, including test, against 
the current 5.x branch. Let me know if you need anything else, I'm happy to 
help.

> Highlighting fails for MultiPhraseQuery's with one clause
> -
>
> Key: LUCENE-7417
> URL: https://issues.apache.org/jira/browse/LUCENE-7417
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/search
>Affects Versions: 5.2.1, 5.x, 5.5.2
>Reporter: Thomas Kappler
> Attachments: multiphrasequery_singleclause_highlighting.patch
>
>
> This bug is the same issue as LUCENE-7231, just for MultiPhraseQuery instead 
> of PhraseQuery. The fix is the same as well. To reproduce, change the test 
> that was committed for LUCENE-7231 to use a MultiPhraseQuery. It results in 
> the same error
> {{java.lang.IllegalArgumentException: Less than 2 subSpans.size():1}}
> I have a patch including a test against branch_5.x, it just needs to go 
> through legal before I can post it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7417) Highlighting fails for MultiPhraseQuery's with one clause

[Thomas Kappler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Kappler updated LUCENE-7417:
---
Attachment: multiphrasequery_singleclause_highlighting.patch

Patch against 5.x as of 2016-09-05

> Highlighting fails for MultiPhraseQuery's with one clause
> -
>
> Key: LUCENE-7417
> URL: https://issues.apache.org/jira/browse/LUCENE-7417
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/search
>Affects Versions: 5.2.1, 5.x, 5.5.2
>Reporter: Thomas Kappler
> Attachments: multiphrasequery_singleclause_highlighting.patch
>
>
> This bug is the same issue as LUCENE-7231, just for MultiPhraseQuery instead 
> of PhraseQuery. The fix is the same as well. To reproduce, change the test 
> that was committed for LUCENE-7231 to use a MultiPhraseQuery. It results in 
> the same error
> {{java.lang.IllegalArgumentException: Less than 2 subSpans.size():1}}
> I have a patch including a test against branch_5.x, it just needs to go 
> through legal before I can post it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9474) MiniSolrCloudCluster should not reuse ports by default

[Shalin Shekhar Mangar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15465471#comment-15465471
 ] 

Shalin Shekhar Mangar commented on SOLR-9474:
-

I had forgotten that I had indeed saved one such failure at 
https://issues.apache.org/jira/browse/SOLR-9438?focusedCommentId=15450041&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15450041

> MiniSolrCloudCluster should not reuse ports by default
> --
>
> Key: SOLR-9474
> URL: https://issues.apache.org/jira/browse/SOLR-9474
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9474.patch
>
>
> Building on SOLR-9469, this just changes 
> MSCC.startJettySolrRunner(JettySolrRunner) to use a different port when it 
> restarts the passed-in Jetty.  This should stop the semi-frequent test 
> failures in CollectionStateWatchersTest.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9438) Shard split can lose data

[Shalin Shekhar Mangar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar updated SOLR-9438:

Attachment: SOLR-9438-split-data-loss.log

This log is from a run which reproduced this bug. A newly created replica logs 
the following:
{code}
38737 INFO  (parallelCoreAdminExecutor-8-thread-1-processing-n:127.0.0.1:42309_ 
9208de91-9c97-4a42-94f5-9e00e3b6189b388000949708638 CREATE) [n:127.0.0.1:42309_ 
c:collection1 s:shard1_1 r:core_node8 x:collection1_shard1_1_replica0] 
o.a.s.c.ShardLeaderElectionContext Was waiting for replicas to come up, but 
they are taking too long - assuming they won't come back till later
{code}

After this point, this replica becomes the leader (with 0 docs inside!) and 
eventually when the old replica comes back up, it syncs with this empty index 
and loses all data except for whatever was indexed after the split.

> Shard split can lose data
> -
>
> Key: SOLR-9438
> URL: https://issues.apache.org/jira/browse/SOLR-9438
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 4.10.4, 5.5.2, 6.1
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
>  Labels: difficulty-medium, impact-high
> Fix For: master (7.0), 6.3
>
> Attachments: SOLR-9438-split-data-loss.log, SOLR-9438.patch, 
> SOLR-9438.patch
>
>
> Solr’s shard split can lose documents if the parent/sub-shard leader is 
> killed (or crashes) between the time that the new sub-shard replica is 
> created and before it recovers. In such a case the slice has already been set 
> to ‘recovery’ state, the sub-shard replica comes up, finds that no other 
> replica is up, waits until the leader vote wait time and then proceeds to 
> become the leader as well as publish itself as active. Once that happens the 
> overseer seeing that all replicas of the sub-shard are now ‘active’, sets the 
> parent slice as ‘inactive’ and the new sub-shard as ‘active’.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9127) XLSX response writer - do we want it?

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15465446#comment-15465446
 ] 

Noble Paul edited comment on SOLR-9127 at 9/5/16 5:23 PM:
--

where is the pull request? if possible attach a patch


was (Author: noble.paul):
where is the pull request?

> XLSX response writer - do we want it?
> -
>
> Key: SOLR-9127
> URL: https://issues.apache.org/jira/browse/SOLR-9127
> Project: Solr
>  Issue Type: Improvement
>  Components: Response Writers
>Reporter: Tony Moriarty
>Priority: Minor
> Fix For: 5.5, 6.0
>
> Attachments: SOLR-9127.patch
>
>
> I recently open sourced an XLSX response writer based on solr 4.6 and apache 
> poi.
> https://github.com/desultir/SolrXLSXResponseWriter
> Is this something the community would be interested in bringing into the solr 
> codebase? I'm willing to put the work into porting it to solr5 and solr6 if 
> the community is interested, happy to leave it as a plugin otherwise.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9127) XLSX response writer - do we want it?

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15465446#comment-15465446
 ] 

Noble Paul commented on SOLR-9127:
--

where is the pull request?

> XLSX response writer - do we want it?
> -
>
> Key: SOLR-9127
> URL: https://issues.apache.org/jira/browse/SOLR-9127
> Project: Solr
>  Issue Type: Improvement
>  Components: Response Writers
>Reporter: Tony Moriarty
>Priority: Minor
> Fix For: 5.5, 6.0
>
> Attachments: SOLR-9127.patch
>
>
> I recently open sourced an XLSX response writer based on solr 4.6 and apache 
> poi.
> https://github.com/desultir/SolrXLSXResponseWriter
> Is this something the community would be interested in bringing into the solr 
> codebase? I'm willing to put the work into porting it to solr5 and solr6 if 
> the community is interested, happy to leave it as a plugin otherwise.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Windows (64bit/jdk1.8.0_102) - Build # 437 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/437/
Java: 64bit/jdk1.8.0_102 -XX:-UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail

Error Message:
expected:<200> but was:<404>

Stack Trace:
java.lang.AssertionError: expected:<200> but was:<404>
at 
__randomizedtesting.SeedInfo.seed([F9125357ED830138:91AD667D3D1913D4]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.cancelDelegationToken(TestSolrCloudWithDelegationTokens.java:141)
at 
org.apache.solr.cloud.TestSolrCloudWithDelegationTokens.testDelegationTokenCancelFail(TestSolrCloudWithDelegationTokens.java:295)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(Thr

Solr Reference Guide -- Is the PDF export stylesheet proprietary?

[Shawn Heisey]

Someone on the #solr IRC wants to obtain the PDF export stylesheet that
we use for publishing the reference guide as PDF.

I've managed to locate the stylesheet because apparently I do have the
necessary permission on Confluence, but I don't want to just give it to
them without making sure that we don't have any reason to keep it
internal-only.

How should I handle this request?

Thanks,
Shawn


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7398) Nested Span Queries are buggy

[Christoph Goller (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15465336#comment-15465336
 ] 

Christoph Goller commented on LUCENE-7398:
--

Good idea to try the nested tests from TestSpanCollection for the unordered 
case. The example from LUCENE-5331 shows the problems of incomplete 
backtracking (not comparing all combinations of span matches of all subspans) 
for the unordered case. In the ordered case we only have a problem with spans 
that have matches of different lenght, in the unorderd case we also see a 
problem with overlapping span-matches, even if they all have length 1.

> Nested Span Queries are buggy
> -
>
> Key: LUCENE-7398
> URL: https://issues.apache.org/jira/browse/LUCENE-7398
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/search
>Affects Versions: 5.5, 6.x
>Reporter: Christoph Goller
>Assignee: Alan Woodward
>Priority: Critical
> Attachments: LUCENE-7398-20160814.patch, LUCENE-7398.patch, 
> LUCENE-7398.patch, TestSpanCollection.java
>
>
> Example for a nested SpanQuery that is not working:
> Document: Human Genome Organization , HUGO , is trying to coordinate gene 
> mapping research worldwide.
> Query: spanNear([body:coordinate, spanOr([spanNear([body:gene, body:mapping], 
> 0, true), body:gene]), body:research], 0, true)
> The query should match "coordinate gene mapping research" as well as 
> "coordinate gene research". It does not match  "coordinate gene mapping 
> research" with Lucene 5.5 or 6.1, it did however match with Lucene 4.10.4. It 
> probably stopped working with the changes on SpanQueries in 5.3. I will 
> attach a unit test that shows the problem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-7398) Nested Span Queries are buggy

[Christoph Goller (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15465301#comment-15465301
 ] 

Christoph Goller edited comment on LUCENE-7398 at 9/5/16 4:07 PM:
--

Paul's  20160814 patch  almost convinced me. Unfortunately, it does not fix the 
case when an intermediate span has a longer match that reduces overall 
sloppyness but overlaps with a match of a subsequent span and consequently 
requires advancing the subsequent span. Here is an example 

Document: w1 w2 w3 w4 w5
near/0(w1, or(w2, near/0(w2, w3, w4)), or(w5, near/0(w4, w5)))

Add the following code to the end of TestSpanCollection.testNestedNearQuery()

{code}
SpanNearQuery q234 = new SpanNearQuery(new SpanQuery[]{q2, q3, q4}, 0, true);
SpanOrQuery q2234 = new SpanOrQuery(q2, q234);
SpanTermQuery p5 = new SpanTermQuery(new Term(FIELD, "w5"));
SpanNearQuery q45 = new SpanNearQuery(new SpanQuery[]{q4, p5}, 0, true);
SpanOrQuery q455 = new SpanOrQuery(q45, p5);

SpanNearQuery q1q2234q445 = new SpanNearQuery(new SpanQuery[]{q1, q2234, q455}, 
0, true);
spans = q1q2234q445.createWeight(searcher, false, 
1f).getSpans(searcher.getIndexReader().leaves().get(0),SpanWeight.Postings.POSITIONS);
assertEquals(0, spans.advance(0));
{code}

I think we can only fix it if we get give up lazy iteration. I don't think this 
is so bad for performance. If we implement a clever caching for positions in 
spans a complete backtracking would only consist of making a few additional 
int-comparisons. The expensive operation is iterating over all span positions 
(IO) and we do this already in advancePosition(Spans, int), aren't we. 


was (Author: gol...@detego-software.de):
Paul's fix almost convinced me. Unfortunately, it does not fix the case when an 
intermediate span has a longer match that reduces overall sloppyness but 
overlaps with a match of a subsequent span and consequently requires advancing 
the subsequent span. Here is an example 

Document: w1 w2 w3 w4 w5
near/0(w1, or(w2, near/0(w2, w3, w4)), or(w5, near/0(w4, w5)))

Add the following code to the end of TestSpanCollection.testNestedNearQuery()

{code}
SpanNearQuery q234 = new SpanNearQuery(new SpanQuery[]{q2, q3, q4}, 0, true);
SpanOrQuery q2234 = new SpanOrQuery(q2, q234);
SpanTermQuery p5 = new SpanTermQuery(new Term(FIELD, "w5"));
SpanNearQuery q45 = new SpanNearQuery(new SpanQuery[]{q4, p5}, 0, true);
SpanOrQuery q455 = new SpanOrQuery(q45, p5);

SpanNearQuery q1q2234q445 = new SpanNearQuery(new SpanQuery[]{q1, q2234, q455}, 
0, true);
spans = q1q2234q445.createWeight(searcher, false, 
1f).getSpans(searcher.getIndexReader().leaves().get(0),SpanWeight.Postings.POSITIONS);
assertEquals(0, spans.advance(0));
{code}

I think we can only fix it if we get give up lazy iteration. I don't think this 
is so bad for performance. If we implement a clever caching for positions in 
spans a complete backtracking would only consist of making a few additional 
int-comparisons. The expensive operation is iterating over all span positions 
(IO) and we do this already in advancePosition(Spans, int), aren't we. 

> Nested Span Queries are buggy
> -
>
> Key: LUCENE-7398
> URL: https://issues.apache.org/jira/browse/LUCENE-7398
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/search
>Affects Versions: 5.5, 6.x
>Reporter: Christoph Goller
>Assignee: Alan Woodward
>Priority: Critical
> Attachments: LUCENE-7398-20160814.patch, LUCENE-7398.patch, 
> LUCENE-7398.patch, TestSpanCollection.java
>
>
> Example for a nested SpanQuery that is not working:
> Document: Human Genome Organization , HUGO , is trying to coordinate gene 
> mapping research worldwide.
> Query: spanNear([body:coordinate, spanOr([spanNear([body:gene, body:mapping], 
> 0, true), body:gene]), body:research], 0, true)
> The query should match "coordinate gene mapping research" as well as 
> "coordinate gene research". It does not match  "coordinate gene mapping 
> research" with Lucene 5.5 or 6.1, it did however match with Lucene 4.10.4. It 
> probably stopped working with the changes on SpanQueries in 5.3. I will 
> attach a unit test that shows the problem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-5.5-Linux (64bit/jdk1.7.0_80) - Build # 387 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.5-Linux/387/
Java: 64bit/jdk1.7.0_80 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

2 tests failed.
FAILED:  
org.apache.solr.handler.TestReplicationHandler.doTestIndexFetchOnMasterRestart

Error Message:
expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
at 
__randomizedtesting.SeedInfo.seed([9124BD1B833E809A:49D379FF28E542C6]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.handler.TestReplicationHandler.doTestIndexFetchOnMasterRestart(TestReplicationHandler.java:624)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.schema.TestManagedSchemaAPI.test

Error Message:
Error fro

[jira] [Commented] (LUCENE-7398) Nested Span Queries are buggy

[Christoph Goller (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15465301#comment-15465301
 ] 

Christoph Goller commented on LUCENE-7398:
--

Paul's fix almost convinced me. Unfortunately, it does not fix the case when an 
intermediate span has a longer match that reduces overall sloppyness but 
overlaps with a match of a subsequent span and consequently requires advancing 
the subsequent span. Here is an example 

Document: w1 w2 w3 w4 w5
near/0(w1, or(w2, near/0(w2, w3, w4)), or(w5, near/0(w4, w5)))

Add the following code to the end of TestSpanCollection.testNestedNearQuery()

{code}
SpanNearQuery q234 = new SpanNearQuery(new SpanQuery[]{q2, q3, q4}, 0, true);
SpanOrQuery q2234 = new SpanOrQuery(q2, q234);
SpanTermQuery p5 = new SpanTermQuery(new Term(FIELD, "w5"));
SpanNearQuery q45 = new SpanNearQuery(new SpanQuery[]{q4, p5}, 0, true);
SpanOrQuery q455 = new SpanOrQuery(q45, p5);

SpanNearQuery q1q2234q445 = new SpanNearQuery(new SpanQuery[]{q1, q2234, q455}, 
0, true);
spans = q1q2234q445.createWeight(searcher, false, 
1f).getSpans(searcher.getIndexReader().leaves().get(0),SpanWeight.Postings.POSITIONS);
assertEquals(0, spans.advance(0));
{code}

I think we can only fix it if we get give up lazy iteration. I don't think this 
is so bad for performance. If we implement a clever caching for positions in 
spans a complete backtracking would only consist of making a few additional 
int-comparisons. The expensive operation is iterating over all span positions 
(IO) and we do this already in advancePosition(Spans, int), aren't we. 

> Nested Span Queries are buggy
> -
>
> Key: LUCENE-7398
> URL: https://issues.apache.org/jira/browse/LUCENE-7398
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/search
>Affects Versions: 5.5, 6.x
>Reporter: Christoph Goller
>Assignee: Alan Woodward
>Priority: Critical
> Attachments: LUCENE-7398-20160814.patch, LUCENE-7398.patch, 
> LUCENE-7398.patch, TestSpanCollection.java
>
>
> Example for a nested SpanQuery that is not working:
> Document: Human Genome Organization , HUGO , is trying to coordinate gene 
> mapping research worldwide.
> Query: spanNear([body:coordinate, spanOr([spanNear([body:gene, body:mapping], 
> 0, true), body:gene]), body:research], 0, true)
> The query should match "coordinate gene mapping research" as well as 
> "coordinate gene research". It does not match  "coordinate gene mapping 
> research" with Lucene 5.5 or 6.1, it did however match with Lucene 4.10.4. It 
> probably stopped working with the changes on SpanQueries in 5.3. I will 
> attach a unit test that shows the problem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (64bit/jdk1.8.0_102) - Build # 6101 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6101/
Java: 64bit/jdk1.8.0_102 -XX:+UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestLongPostings

Error Message:
Could not remove the following files (in the order of attempts):
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J0\temp\lucene.index.TestLongPostings_A00D58F21CAE2226-001\longpostings.5999876191789691164-001:
 java.nio.file.AccessDeniedException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J0\temp\lucene.index.TestLongPostings_A00D58F21CAE2226-001\longpostings.5999876191789691164-001

C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J0\temp\lucene.index.TestLongPostings_A00D58F21CAE2226-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J0\temp\lucene.index.TestLongPostings_A00D58F21CAE2226-001
 

Stack Trace:
java.io.IOException: Could not remove the following files (in the order of 
attempts):
   
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J0\temp\lucene.index.TestLongPostings_A00D58F21CAE2226-001\longpostings.5999876191789691164-001:
 java.nio.file.AccessDeniedException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J0\temp\lucene.index.TestLongPostings_A00D58F21CAE2226-001\longpostings.5999876191789691164-001
   
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J0\temp\lucene.index.TestLongPostings_A00D58F21CAE2226-001:
 java.nio.file.DirectoryNotEmptyException: 
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\build\core\test\J0\temp\lucene.index.TestLongPostings_A00D58F21CAE2226-001

at __randomizedtesting.SeedInfo.seed([A00D58F21CAE2226]:0)
at org.apache.lucene.util.IOUtils.rm(IOUtils.java:323)
at 
org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:216)
at 
com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 1002 lines...]
   [junit4] Suite: org.apache.lucene.index.TestLongPostings
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): 
{field=PostingsFormat(name=MockRandom)}, docValues:{}, maxPointsInLeafNode=135, 
maxMBSortInHeap=7.2891065784806806, sim=ClassicSimilarity, locale=ga-IE, 
timezone=America/Buenos_Aires
   [junit4]   2> NOTE: Windows 10 10.0 amd64/Oracle Corporation 1.8.0_102 
(64-bit)/cpus=3,threads=1,free=58989512,total=133836800
   [junit4]   2> NOTE: All tests run in this JVM: [Test2BPostings, 
TestIndexableField, TestRollingBuffer, TestIntroSorter, 
TestUsageTrackingFilterCachingPolicy, Test2BTerms, TestSimpleExplanations, 
TestLucene60PointsFormat, TestClassicSimilarity, 
TestLucene50StoredFieldsFormatHighCompression, TestAllFilesHaveCodecHeader, 
TestCrash, TestWildcardRandom, TestRAMDirectory, TestLockFactory, 
TestAtomicUpdate, TestScoreCachingWrappingScorer, TestSegmentTermDocs, 
TestRadixSelector, TestIndexSorting, TestPointQueries, TestParallelTermEnum, 
TestRegexpQuery, TestIsCurrent, TestBoolean2, TestIntBlockPool, 
TestBytesRefAttImpl, TestPointValues, TestExitableDirectoryReader, 
TestDeletionPolicy, TestIndexSearcher, TestRollingUpdates, 
TestSpanExplanations, TestByteSlices, TestVersion, TestDuelingCodecs, 
TestTieredMergePolicy, TestSimpleFSDirectory, TestDirectMonotonic, 
TestConsistentFieldNumbers, TestIndexWriter, TestSortedSetDocValues, 
TestEarlyTerminatingSortingCollector, TestDocsAndPositions, 
TestMutablePointsReaderUtils, TestLiveFieldValues, TestTryDelete, 
TestSloppyPhraseQuery2, TestSimpleAttributeImpl, TestSpanNotQuery, 
TestDisjunctionMaxQuery, TestForUtil, TestDocValuesIndexing, 
TestSpanContainQuery, TestMergeRateLimiter, TestMaxPosition, 
TestConstantScoreQuery, TestBufferedChecksum, TestFastDecompressionMode, 
TestFixedBitSet, TestSynonymQuery, TestDocument, TestToken, TestCharsRef, 

Re: [VOTE] Release Lucene/Solr 5.5.3 RC1

[Jan Høydahl]

SUCCESS! [1:17:49.294232]

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

> 2. sep. 2016 kl. 16.29 skrev Anshum Gupta :
> 
> Please vote for release candidate 1 for Lucene/Solr 5.5.3
> 
> The artifacts can be downloaded from:
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.5.3-RC1-rev8655b97b27d8da470c8235683af11a8b85a2b10f
>  
> 
> 
> You can run the smoke tester directly with this command (on the release 
> branch):
> 
> python3 -u dev-tools/scripts/smokeTestRelease.py \
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-5.5.3-RC1-rev8655b97b27d8da470c8235683af11a8b85a2b10f
>  
> 
> 
> Here's my +1
> SUCCESS! [0:36:59.036831]

[jira] [Updated] (SOLR-9479) Extends DIH with transformation on Solr document level

[Andrey Kudryavtsev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Kudryavtsev updated SOLR-9479:
-
Description: 
I use DIH to index nested documents.
I have couple of use cases where field values on parent document depend on 
children documents values. The simplest one - I need to "propagate" values from 
all children documents to new field on parent document. It could be a little 
bit tricky with current DIH architecture when you can apply transformation for 
"plain" documents which considered as a plain map. See 
_org.apache.solr.handler.dataimport.Transformer_ (You have to use some kind of 
"transient fields" in your data-config, this fields are populated from child 
sources, so sometimes you have to read child sources twice (or more times) for 
this. Maybe I do it wrong?)

I decided that maybe it makes sense to be able to apply transformation after 
nested documents were converted from collection of maps into good old 
hierarchical SolrInputDocument. 

So this initial patch was created: 
* It introduces concept of _DocumentTransformer_ with Java interface
{code:java}SolrInputDocument transform(SolrInputDocument solrDoc, Context 
context) {code}
* This interface should be implemented by clients transformers. One simple 
example of such transformer, _PropagationDocumentTransformer_, is implemented. 
It parametrised by two field names - child and parent, and copied values from 
children documents to parent document.  
* This kind of transformer should be added into data-config.xml to 
corresponding parents _entry_:

{code:xml}documentTransformer="org.apache.solr.handler.dataimport.PropagationDocumentTransformer"{code}

  was:
I use DIH to index nested documents.
I have couple of use cases where field values on parent document depend on 
children documents values. The simplest one - I need to "propagate" values from 
all children documents to new field on parent document. It could be a little 
bit tricky with current DIH architecture when you can apply transformation for 
"plain" documents which considered as a plain map. See 
_org.apache.solr.handler.dataimport.Transformer_ (You have to use some kind of 
"transient fields" in your data-config, this fields are populated by child 
sources, so sometimes you have to read child sources twice (more time) for 
this. Maybe I do it wrong?)

I decided that maybe it makes sense to be able to apply transformation after 
nested documents were converted from collection of map into good old 
SolrInputDocument. 

So this initial patch was created: 
* It introduces concept of _DocumentTransformer_ with Java interface
{code:java}SolrInputDocument transform(SolrInputDocument solrDoc, Context 
context) {code}
* This interface should be implemented by clients transformers. One simple 
example of such transformer, _PropagationDocumentTransformer_, is implemented. 
It parametrised by two field names - child and parent, and copied values from 
children documents to parent document.  
* This kind of transformer should be added into data-config.xml to 
corresponding parents _entry_:

{code:xml}documentTransformer="org.apache.solr.handler.dataimport.PropagationDocumentTransformer"{code}


> Extends DIH with transformation on Solr document level 
> ---
>
> Key: SOLR-9479
> URL: https://issues.apache.org/jira/browse/SOLR-9479
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Andrey Kudryavtsev
> Attachments: SOLR-9479.patch
>
>
> I use DIH to index nested documents.
> I have couple of use cases where field values on parent document depend on 
> children documents values. The simplest one - I need to "propagate" values 
> from all children documents to new field on parent document. It could be a 
> little bit tricky with current DIH architecture when you can apply 
> transformation for "plain" documents which considered as a plain map. See 
> _org.apache.solr.handler.dataimport.Transformer_ (You have to use some kind 
> of "transient fields" in your data-config, this fields are populated from 
> child sources, so sometimes you have to read child sources twice (or more 
> times) for this. Maybe I do it wrong?)
> I decided that maybe it makes sense to be able to apply transformation after 
> nested documents were converted from collection of maps into good old 
> hierarchical SolrInputDocument. 
> So this initial patch was created: 
> * It introduces concept of _DocumentTransformer_ with Java interface
> {code:java}SolrInputDocument transform(SolrInputDocument solrDoc, Context 
> context) {code}
> * This interface should be implemented by clients transformers. One simple 
> example of such transformer, _PropagationDocumentTransformer_, is 
> implemented. It parametrised by two field

[jira] [Updated] (SOLR-9479) Extends DIH with transformation on Solr document level

[Andrey Kudryavtsev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Kudryavtsev updated SOLR-9479:
-
Description: 
I use DIH to index nested documents.
I have couple of use cases where field values on parent document depend on 
children documents values. The simplest one - I need to "propagate" values from 
all children documents to new field on parent document. It could be a little 
bit tricky with current DIH architecture when you can apply transformation for 
"plain" documents which considered as a plain map. See 
_org.apache.solr.handler.dataimport.Transformer_ (You have to use some kind of 
"transient fields" in your data-config, this fields are populated by child 
sources, so sometimes you have to read child sources twice (more time) for 
this. Maybe I do it wrong?)

I decided that maybe it makes sense to be able to apply transformation after 
nested documents were converted from collection of map into good old 
SolrInputDocument. 

So this initial patch was created: 
* It introduces concept of _DocumentTransformer_ with Java interface
{code:java}SolrInputDocument transform(SolrInputDocument solrDoc, Context 
context) {code}
* This interface should be implemented by clients transformers. One simple 
example of such transformer, _PropagationDocumentTransformer_, is implemented. 
It parametrised by two field names - child and parent, and copied values from 
children documents to parent document.  
* This kind of transformer should be added into data-config.xml to 
corresponding parents _entry_:

{code:xml}documentTransformer="org.apache.solr.handler.dataimport.PropagationDocumentTransformer"{code}

  was:
I use DIH to index nested documents.
I have couple of use cases where field values on parent document depend on 
children documents values. The simplest one - I need to "propagate" values from 
all children documents to new field on parent document. It could be a little 
bit tricky with current DIH architecture when you can apply transformation for 
"plain" documents which considered as a plain map. See 
_org.apache.solr.handler.dataimport.Transformer_ (You have to use some kind of 
"transparent fields" in your config, this fields are populated by child 
sources, so sometimes you have to read child sources twice (more time) for 
this. Maybe I do it wrong?)

I decided that maybe it makes sense to be able to apply transformation after 
nested documents were converted from collection of map into good old 
SolrInputDocument. 

So this initial patch was created: 
* It introduces concept of _DocumentTransformer_ with Java interface
{code:java}SolrInputDocument transform(SolrInputDocument solrDoc, Context 
context) {code}
* This interface should be implemented by clients transformers. One simple 
example of such transformer, _PropagationDocumentTransformer_, is implemented. 
It parametrised by two field names - child and parent, and copied values from 
children documents to parent document.  
* This kind of transformer should be added into data-config.xml to 
corresponding parents _entry_:

{code:xml}documentTransformer="org.apache.solr.handler.dataimport.PropagationDocumentTransformer"{code}


> Extends DIH with transformation on Solr document level 
> ---
>
> Key: SOLR-9479
> URL: https://issues.apache.org/jira/browse/SOLR-9479
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Andrey Kudryavtsev
> Attachments: SOLR-9479.patch
>
>
> I use DIH to index nested documents.
> I have couple of use cases where field values on parent document depend on 
> children documents values. The simplest one - I need to "propagate" values 
> from all children documents to new field on parent document. It could be a 
> little bit tricky with current DIH architecture when you can apply 
> transformation for "plain" documents which considered as a plain map. See 
> _org.apache.solr.handler.dataimport.Transformer_ (You have to use some kind 
> of "transient fields" in your data-config, this fields are populated by child 
> sources, so sometimes you have to read child sources twice (more time) for 
> this. Maybe I do it wrong?)
> I decided that maybe it makes sense to be able to apply transformation after 
> nested documents were converted from collection of map into good old 
> SolrInputDocument. 
> So this initial patch was created: 
> * It introduces concept of _DocumentTransformer_ with Java interface
> {code:java}SolrInputDocument transform(SolrInputDocument solrDoc, Context 
> context) {code}
> * This interface should be implemented by clients transformers. One simple 
> example of such transformer, _PropagationDocumentTransformer_, is 
> implemented. It parametrised by two field names - child and parent, and 
> copied va

[jira] [Updated] (SOLR-9479) Extends DIH with transformation on Solr document level

[Andrey Kudryavtsev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Kudryavtsev updated SOLR-9479:
-
Description: 
I use DIH to index nested documents.
I have couple of use cases where field values on parent document depend on 
children documents values. The simplest one - I need to "propagate" values from 
all children documents to new field on parent document. It could be a little 
bit tricky with current DIH architecture when you can apply transformation for 
"plain" documents which considered as a plain map. See 
_org.apache.solr.handler.dataimport.Transformer_ (You have to use some kind of 
"transparent fields" in your config, this fields are populated by child 
sources, so sometimes you have to read child sources twice (more time) for 
this. Maybe I do it wrong?)

I decided that maybe it makes sense to be able to apply transformation after 
nested documents were converted from collection of map into good old 
SolrInputDocument. 

So this initial patch was created: 
* It introduces concept of _DocumentTransformer_ with Java interface
{code:java}SolrInputDocument transform(SolrInputDocument solrDoc, Context 
context) {code}
* This interface should be implemented by clients transformers. One simple 
example of such transformer, _PropagationDocumentTransformer_, is implemented. 
It parametrised by two field names - child and parent, and copied values from 
children documents to parent document.  
* This kind of transformer should be added into data-config.xml to 
corresponding parents _entry_:

{code:xml}documentTransformer="org.apache.solr.handler.dataimport.PropagationDocumentTransformer"{code}

  was:
I use DIH to index nested documents.
I have couple of use cases where field values on parent document depend on 
children documents values. The simplest one - I need to "propagate" values from 
all children documents to new field on parent document. It could be a little 
bit tricky with current DIH architecture when you can apply transformation for 
"plain" documents which considered as a plain map. See 
_org.apache.solr.handler.dataimport.Transformer_

I decided that maybe it makes sense to be able to apply transformation after 
nested documents were converted from collection of map into good old 
SolrInputDocument. 

So this initial patch was created: 
* It introduces concept of _DocumentTransformer_ with Java interface
{code:java}SolrInputDocument transform(SolrInputDocument solrDoc, Context 
context) {code}
* This interface should be implemented by clients transformers. One simple 
example of such transformer, _PropagationDocumentTransformer_, is implemented. 
It parametrised by two field names - child and parent, and copied values from 
children documents to parent document.  
* This kind of transformer should be added into data-config.xml to 
corresponding parents _entry_:

{code:xml}documentTransformer="org.apache.solr.handler.dataimport.PropagationDocumentTransformer"{code}


> Extends DIH with transformation on Solr document level 
> ---
>
> Key: SOLR-9479
> URL: https://issues.apache.org/jira/browse/SOLR-9479
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Andrey Kudryavtsev
> Attachments: SOLR-9479.patch
>
>
> I use DIH to index nested documents.
> I have couple of use cases where field values on parent document depend on 
> children documents values. The simplest one - I need to "propagate" values 
> from all children documents to new field on parent document. It could be a 
> little bit tricky with current DIH architecture when you can apply 
> transformation for "plain" documents which considered as a plain map. See 
> _org.apache.solr.handler.dataimport.Transformer_ (You have to use some kind 
> of "transparent fields" in your config, this fields are populated by child 
> sources, so sometimes you have to read child sources twice (more time) for 
> this. Maybe I do it wrong?)
> I decided that maybe it makes sense to be able to apply transformation after 
> nested documents were converted from collection of map into good old 
> SolrInputDocument. 
> So this initial patch was created: 
> * It introduces concept of _DocumentTransformer_ with Java interface
> {code:java}SolrInputDocument transform(SolrInputDocument solrDoc, Context 
> context) {code}
> * This interface should be implemented by clients transformers. One simple 
> example of such transformer, _PropagationDocumentTransformer_, is 
> implemented. It parametrised by two field names - child and parent, and 
> copied values from children documents to parent document.  
> * This kind of transformer should be added into data-config.xml to 
> corresponding parents _entry_:
> {code:xml}documentTransformer="org.apache.solr.handler.dat

[jira] [Updated] (SOLR-9479) Extends DIH with transformation on Solr document level

[Andrey Kudryavtsev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Kudryavtsev updated SOLR-9479:
-
Description: 
I use DIH to index nested documents.
I have couple of use cases where field values on parent document depend on 
children documents values. The simplest one - I need to "propagate" values from 
all children documents to new field on parent document. It could be a little 
bit tricky with current DIH architecture when you can apply transformation for 
"plain" documents which considered as a plain map. See 
_org.apache.solr.handler.dataimport.Transformer_

I decided that maybe it makes sense to be able to apply transformation after 
nested documents were converted from collection of map into good old 
SolrInputDocument. 

So this initial patch was created: 
* It introduces concept of _DocumentTransformer_ with Java interface
{code:java}SolrInputDocument transform(SolrInputDocument solrDoc, Context 
context) {code}
* This interface should be implemented by clients transformers. One simple 
example of such transformer, _PropagationDocumentTransformer_, is implemented. 
It parametrised by two field names - child and parent, and copied values from 
children documents to parent document.  
* This kind of transformer should be added into data-config.xml to 
corresponding parents _entry_:

{code:xml}documentTransformer="org.apache.solr.handler.dataimport.PropagationDocumentTransformer"{code}

  was:
I use DIH to index nested documents.
I have couple of use cases where fields values on parent document depend on 
children document values. The simplest one - I need to "propagate" values from 
all children documents to new field on parent document. It could be a little 
bit tricky with current DIH architecture when you can apply transformation for 
"plain" documents which considered as a  plain map. See 
_org.apache.solr.handler.dataimport.Transformer_

I decided that maybe it makes sense to be able to apply transformation after 
nested documents were converted from collection of map into good old 
SolrInputDocument. 

So this initial patch was created: 
* It introduces concept of _DocumentTransformer_ with Java interface
{code:java}SolrInputDocument transform(SolrInputDocument solrDoc, Context 
context) {code}
* This interface should be implemented by clients transformers. One simple 
example of such transformer, _PropagationDocumentTransformer_, is implemented. 
It parametrised by two field names - child and parent, and copied values from 
children documents to parent document.  
* This kind of transformer should be added into data-config.xml to 
corresponding parents _entry_:

{code:xml}documentTransformer="org.apache.solr.handler.dataimport.PropagationDocumentTransformer"{code}


> Extends DIH with transformation on Solr document level 
> ---
>
> Key: SOLR-9479
> URL: https://issues.apache.org/jira/browse/SOLR-9479
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Andrey Kudryavtsev
> Attachments: SOLR-9479.patch
>
>
> I use DIH to index nested documents.
> I have couple of use cases where field values on parent document depend on 
> children documents values. The simplest one - I need to "propagate" values 
> from all children documents to new field on parent document. It could be a 
> little bit tricky with current DIH architecture when you can apply 
> transformation for "plain" documents which considered as a plain map. See 
> _org.apache.solr.handler.dataimport.Transformer_
> I decided that maybe it makes sense to be able to apply transformation after 
> nested documents were converted from collection of map into good old 
> SolrInputDocument. 
> So this initial patch was created: 
> * It introduces concept of _DocumentTransformer_ with Java interface
> {code:java}SolrInputDocument transform(SolrInputDocument solrDoc, Context 
> context) {code}
> * This interface should be implemented by clients transformers. One simple 
> example of such transformer, _PropagationDocumentTransformer_, is 
> implemented. It parametrised by two field names - child and parent, and 
> copied values from children documents to parent document.  
> * This kind of transformer should be added into data-config.xml to 
> corresponding parents _entry_:
> {code:xml}documentTransformer="org.apache.solr.handler.dataimport.PropagationDocumentTransformer"{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9479) Extends DIH with transformation on Solr document level

[Andrey Kudryavtsev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Kudryavtsev updated SOLR-9479:
-
Summary: Extends DIH with transformation on Solr document level   (was: 
Extends DIH with transformation on Solr's document level )

> Extends DIH with transformation on Solr document level 
> ---
>
> Key: SOLR-9479
> URL: https://issues.apache.org/jira/browse/SOLR-9479
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Andrey Kudryavtsev
> Attachments: SOLR-9479.patch
>
>
> I use DIH to index nested documents.
> I have couple of use cases where fields values on parent document depend on 
> children document values. The simplest one - I need to "propagate" values 
> from all children documents to new field on parent document. It could be a 
> little bit tricky with current DIH architecture when you can apply 
> transformation for "plain" documents which considered as a  plain map. See 
> _org.apache.solr.handler.dataimport.Transformer_
> I decided that maybe it makes sense to be able to apply transformation after 
> nested documents were converted from collection of map into good old 
> SolrInputDocument. 
> So this initial patch was created: 
> * It introduces concept of _DocumentTransformer_ with Java interface
> {code:java}SolrInputDocument transform(SolrInputDocument solrDoc, Context 
> context) {code}
> * This interface should be implemented by clients transformers. One simple 
> example of such transformer, _PropagationDocumentTransformer_, is 
> implemented. It parametrised by two field names - child and parent, and 
> copied values from children documents to parent document.  
> * This kind of transformer should be added into data-config.xml to 
> corresponding parents _entry_:
> {code:xml}documentTransformer="org.apache.solr.handler.dataimport.PropagationDocumentTransformer"{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9479) Extends DIH with transformation on Solr's document level

[Andrey Kudryavtsev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrey Kudryavtsev updated SOLR-9479:
-
Attachment: SOLR-9479.patch

> Extends DIH with transformation on Solr's document level 
> -
>
> Key: SOLR-9479
> URL: https://issues.apache.org/jira/browse/SOLR-9479
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Andrey Kudryavtsev
> Attachments: SOLR-9479.patch
>
>
> I use DIH to index nested documents.
> I have couple of use cases where fields values on parent document depend on 
> children document values. The simplest one - I need to "propagate" values 
> from all children documents to new field on parent document. It could be a 
> little bit tricky with current DIH architecture when you can apply 
> transformation for "plain" documents which considered as a  plain map. See 
> _org.apache.solr.handler.dataimport.Transformer_
> I decided that maybe it makes sense to be able to apply transformation after 
> nested documents were converted from collection of map into good old 
> SolrInputDocument. 
> So this initial patch was created: 
> * It introduces concept of _DocumentTransformer_ with Java interface
> {code:java}SolrInputDocument transform(SolrInputDocument solrDoc, Context 
> context) {code}
> * This interface should be implemented by clients transformers. One simple 
> example of such transformer, _PropagationDocumentTransformer_, is 
> implemented. It parametrised by two field names - child and parent, and 
> copied values from children documents to parent document.  
> * This kind of transformer should be added into data-config.xml to 
> corresponding parents _entry_:
> {code:xml}documentTransformer="org.apache.solr.handler.dataimport.PropagationDocumentTransformer"{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9479) Extends DIH with transformation on Solr's document level

[Andrey Kudryavtsev (JIRA)]

Andrey Kudryavtsev created SOLR-9479:


 Summary: Extends DIH with transformation on Solr's document level 
 Key: SOLR-9479
 URL: https://issues.apache.org/jira/browse/SOLR-9479
 Project: Solr
  Issue Type: New Feature
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Andrey Kudryavtsev


I use DIH to index nested documents.
I have couple of use cases where fields values on parent document depend on 
children document values. The simplest one - I need to "propagate" values from 
all children documents to new field on parent document. It could be a little 
bit tricky with current DIH architecture when you can apply transformation for 
"plain" documents which considered as a  plain map. See 
_org.apache.solr.handler.dataimport.Transformer_

I decided that maybe it makes sense to be able to apply transformation after 
nested documents were converted from collection of map into good old 
SolrInputDocument. 

So this initial patch was created: 
* It introduces concept of _DocumentTransformer_ with Java interface
{code:java}SolrInputDocument transform(SolrInputDocument solrDoc, Context 
context) {code}
* This interface should be implemented by clients transformers. One simple 
example of such transformer, _PropagationDocumentTransformer_, is implemented. 
It parametrised by two field names - child and parent, and copied values from 
children documents to parent document.  
* This kind of transformer should be added into data-config.xml to 
corresponding parents _entry_:

{code:xml}documentTransformer="org.apache.solr.handler.dataimport.PropagationDocumentTransformer"{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9478) Do delta import will loss some documents, when the documents added in the duration of delta import.

[JIRA]

[ 
https://issues.apache.org/jira/browse/SOLR-9478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464638#comment-15464638
 ] 

Muḥammad T Jack commented on SOLR-9478:
---

Yeah, we got the  same problem. We have two solr instance running on the same 
shard. ( actually we just have only one shard).

We are not using dataimport.properties which stored in zookeeper. Since it is 
difficult to figure out which collection we are using when use collection alias.

We are using cronjob to run delta import every single minute, but when a delta 
import running on solr instance. another delta import must wait for it until 
server free. Just for example , Sometimes it may takes about 3 minutes to 
finish delta import. So the data changed during this period are lost.

Is there any better solutions to do delta import when solr running on cloud 
mode ?

Thanks
Muḥammad T Jack

> Do delta import will loss some documents, when the documents added in the 
> duration of delta import.
> ---
>
> Key: SOLR-9478
> URL: https://issues.apache.org/jira/browse/SOLR-9478
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 5.5, 5.5.2, 6.0
>Reporter: ted zhu
>  Labels: delta-import, solrcloud
> Fix For: 5.5.2
>
>
> hello guys,
> I met a problem when i using the solrcloud mode. When the solr instance run 
> delta-import, it may take 
> some time to be finished( my data source is mysql database). So during this 
> time, the new added documents
> will loss, the deltaQuery, i use SUBDATE($\{dih.last_index_time\}, INTERVAL 2 
> MINUTE), 
> let it run the delta-import 2 mins earlier than the last_index_time, if the 
> delta-import's duration is 5 mins, it will loss the records at the first 3 
> mins.
> Our servers doesn't use solr cloud mode before, we deal with this issue 
> is tring to rewrite dataimport.properties file, 
> query the max(sys_time_stamp), which will help to record the max time stamp, 
> and let the solr can run delta import standing 
> by the time found in the file, of course, it will never miss docuements. 
>But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
> and we may have multiple collections for the 
> same core.how can i update the dataimport.properties file now in colleciton 
> now? Do you have any solution to help record
>  the max(sys_time_stamp) in dataimport.properties, rather than using the time 
> of delta-import start to run?
> Cheers



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5331) nested SpanNearQuery with repeating groups does not find match

[Christoph Goller (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464619#comment-15464619
 ] 

Christoph Goller commented on LUCENE-5331:
--

As LUCENE-2861 the problem is caused by overlappiung matches for d, b, and c 
and an incomplete backtracking mechanism in unordered SpanQuery.

> nested SpanNearQuery with repeating groups does not find match
> --
>
> Key: LUCENE-5331
> URL: https://issues.apache.org/jira/browse/LUCENE-5331
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Jerry Zhou
> Attachments: NestedSpanNearTest.java, 
> NestedSpanNearTest_20160902.patch
>
>
> Nested spanNear queries do not work in some cases when repeating groups are 
> in the query.
> Test case is attached ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-2861) Search doesn't return document via query

[Christoph Goller (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-2861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464613#comment-15464613
 ] 

Christoph Goller commented on LUCENE-2861:
--

The problem is caused by overlapping matches within spanNear2. The first match 
for spanNear2 matches "intended message" and the second "message" to the same 
"message" in the text so that the match for "addressed" ist to far away. One 
possible fix would forbid overlapping matches or add aspecial very compley 
treatment like in SloppyPhraseScore. I think it would be better to give up lazy 
backtracking and implement a correct backtracking (see LUCENE-7398).

> Search doesn't return document via query
> 
>
> Key: LUCENE-2861
> URL: https://issues.apache.org/jira/browse/LUCENE-2861
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/search
>Affects Versions: 2.9.1, 2.9.4, 3.0.3
> Environment: Doesn't depend on enviroment
>Reporter: Zenoviy Veres
>
> The query doesn't return document that contain all words from query in 
> correct order.
> The issue might be within mechanism how do SpanQuerys actually match results 
> (http://www.lucidimagination.com/blog/2009/07/18/the-spanquery/)
> Please refer for details below. The example text wasn't passed through 
> snowball analyzer, however the issue exists after analyzing too
> Query:
> (intend within 3 of message) within 5 of message within 3 of addressed.  
> Text within document:
> The contents of this e-mail message and
> any attachments are intended solely for the
> addressee(s) and may contain confidential
> and/or legally privileged information. If you
> are not the intended recipient of this message
> or if this message has been addressed to you
> in error, please immediately alert the sender
>  by reply e-mail and then delete this message
> and any attachments
> Result query:
> SpanNearQuery spanNear = new SpanNearQuery(new SpanQuery[] {
> new SpanTermQuery(new Term(BODY, "intended")),
> new SpanTermQuery(new Term(BODY, "message"))},
> 4,
> false);
> SpanNearQuery spanNear2 = new SpanNearQuery(new SpanQuery[] 
> {spanNear, new SpanTermQuery(new Term(BODY, "message"))}, 5, false);
> SpanNearQuery spanNear3 = new SpanNearQuery(new SpanQuery[] 
> {spanNear2, new SpanTermQuery(new Term(BODY, "addressed"))}, 3, false);



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9478) Do delta import will loss some documents, when the documents added in the duration of delta import.

[ted zhu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ted zhu updated SOLR-9478:
--
Description: 
hello guys,
I met a problem when i using the solrcloud mode. When the solr instance run 
delta-import, it may take 
some time to be finished( my data source is mysql database). So during this 
time, the new added documents
will loss, the deltaQuery, i use SUBDATE($\{dih.last_index_time\}, INTERVAL 2 
MINUTE), 
let it run the delta-import 2 mins earlier than the last_index_time, if the 
delta-import's duration is 5 mins, it will loss the records at the first 3 mins.
Our servers doesn't use solr cloud mode before, we deal with this issue is 
tring to rewrite dataimport.properties file, 
query the max(sys_time_stamp), which will help to record the max time stamp, 
and let the solr can run delta import standing 
by the time found in the file, of course, it will never miss docuements. 
   But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
and we may have multiple collections for the 
same core.how can i update the dataimport.properties file now in colleciton 
now? Do you have any solution to help record
 the max(sys_time_stamp) in dataimport.properties, rather than using the time 
of delta-import start to run?

Cheers

  was:
hello guys,
I met a problem when i using the solrcloud mode. When the solr instance 
run delta-import, it may take 
some time to be finished( my data source is mysql database). So during this 
time, the new added documents
will loss, the deltaQuery, i use SUBDATE($\{dih.last_index_time\}, INTERVAL 2 
MINUTE), 
let it run the delta-import 2 mins earlier than the last_index_time, if the 
delta-import's duration is 5 mins, it will loss the records at the first 3 
mins. 
Our servers doesn't use solr cloud mode before, we deal with this issue is 
tring to rewrite dataimport.properties file, 
query the max(sys_time_stamp), which will help to record the max time stamp, 
and let the solr can run delta import standing 
by the time found in the file, of course, it will never miss docuements. 
   But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
and we may have multiple collections for the 
same core.how can i update the dataimport.properties file now in colleciton 
now? Do you have any solution to help record
 the max(sys_time_stamp) in dataimport.properties, rather than using the time 
of delta-import start to run?

Cheers


> Do delta import will loss some documents, when the documents added in the 
> duration of delta import.
> ---
>
> Key: SOLR-9478
> URL: https://issues.apache.org/jira/browse/SOLR-9478
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 5.5, 5.5.2, 6.0
>Reporter: ted zhu
>  Labels: delta-import, solrcloud
> Fix For: 5.5.2
>
>
> hello guys,
> I met a problem when i using the solrcloud mode. When the solr instance run 
> delta-import, it may take 
> some time to be finished( my data source is mysql database). So during this 
> time, the new added documents
> will loss, the deltaQuery, i use SUBDATE($\{dih.last_index_time\}, INTERVAL 2 
> MINUTE), 
> let it run the delta-import 2 mins earlier than the last_index_time, if the 
> delta-import's duration is 5 mins, it will loss the records at the first 3 
> mins.
> Our servers doesn't use solr cloud mode before, we deal with this issue 
> is tring to rewrite dataimport.properties file, 
> query the max(sys_time_stamp), which will help to record the max time stamp, 
> and let the solr can run delta import standing 
> by the time found in the file, of course, it will never miss docuements. 
>But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
> and we may have multiple collections for the 
> same core.how can i update the dataimport.properties file now in colleciton 
> now? Do you have any solution to help record
>  the max(sys_time_stamp) in dataimport.properties, rather than using the time 
> of delta-import start to run?
> Cheers



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9478) Do delta import will loss some documents, when the documents added in the duration of delta import.

[ted zhu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ted zhu updated SOLR-9478:
--
Description: 
hello guys,
I met a problem when i using the solrcloud mode. When the solr instance 
run delta-import, it may take 
some time to be finished( my data source is mysql database). So during this 
time, the new added documents
will loss, the deltaQuery, i use SUBDATE($\{dih.last_index_time\}, INTERVAL 2 
MINUTE), 
let it run the delta-import 2 mins earlier than the last_index_time, if the 
delta-import's duration is 5 mins, it will loss the records at the first 3 
mins. 
Our servers doesn't use solr cloud mode before, we deal with this issue is 
tring to rewrite dataimport.properties file, 
query the max(sys_time_stamp), which will help to record the max time stamp, 
and let the solr can run delta import standing 
by the time found in the file, of course, it will never miss docuements. 
   But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
and we may have multiple collections for the 
same core.how can i update the dataimport.properties file now in colleciton 
now? Do you have any solution to help record
 the max(sys_time_stamp) in dataimport.properties, rather than using the time 
of delta-import start to run?

Cheers

  was:
hello guys,
I met a problem when i using the solrcloud mode. When the solr instance run 
delta-import, it may take 
some time to be finished( my data source is mysql database). So during this 
time, the new added documents
will loss, the deltaQuery, i use SUBDATE($\{dih.last_index_time\}, INTERVAL 2 
MINUTE), 
let it run the delta-import 2 mins earlier than the last_index_time, if the 
delta-import's duration is 5 mins, it will loss the records at the first 3 
mins. 
Our servers doesn't use solr cloud mode before, we deal with this issue is 
tring to rewrite dataimport.properties file, 
query the max(sys_time_stamp), which will help to record the max time stamp, 
and let the solr can run delta import standing 
by the time found in the file, of course, it will never miss docuements. 
   But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
and we may have multiple collections for the 
same core.how can i update the dataimport.properties file now in colleciton 
now? Do you have any solution to help record
 the max(sys_time_stamp) in dataimport.properties, rather than using the time 
of delta-import start to run?

Cheers


> Do delta import will loss some documents, when the documents added in the 
> duration of delta import.
> ---
>
> Key: SOLR-9478
> URL: https://issues.apache.org/jira/browse/SOLR-9478
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 5.5, 5.5.2, 6.0
>Reporter: ted zhu
>  Labels: delta-import, solrcloud
> Fix For: 5.5.2
>
>
> hello guys,
> I met a problem when i using the solrcloud mode. When the solr 
> instance run delta-import, it may take 
> some time to be finished( my data source is mysql database). So during this 
> time, the new added documents
> will loss, the deltaQuery, i use SUBDATE($\{dih.last_index_time\}, INTERVAL 2 
> MINUTE), 
> let it run the delta-import 2 mins earlier than the last_index_time, if the 
> delta-import's duration is 5 mins, it will loss the records at the first 3 
> mins. 
> Our servers doesn't use solr cloud mode before, we deal with this issue 
> is tring to rewrite dataimport.properties file, 
> query the max(sys_time_stamp), which will help to record the max time stamp, 
> and let the solr can run delta import standing 
> by the time found in the file, of course, it will never miss docuements. 
>But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
> and we may have multiple collections for the 
> same core.how can i update the dataimport.properties file now in colleciton 
> now? Do you have any solution to help record
>  the max(sys_time_stamp) in dataimport.properties, rather than using the time 
> of delta-import start to run?
> Cheers



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9478) Do delta import will loss some documents, when the documents added in the duration of delta import.

[ted zhu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ted zhu updated SOLR-9478:
--
Description: 
hello guys,
I met a problem when i using the solrcloud mode. When the solr instance run 
delta-import, it may take 
some time to be finished( my data source is mysql database). So during this 
time, the new added documents
will loss, the deltaQuery, i use 
SUBDATE(${dih.last_index_time}, INTERVAL 2 MINUTE), 
let it run the delta-import 2 mins earlier than the last_index_time, if the 
delta-import's duration is 5 mins, it will loss the records at the first 3 
mins. 
Our servers doesn't use solr cloud mode before, we deal with this issue is 
tring to rewrite dataimport.properties file, 
query the max(sys_time_stamp), which will help to record the max time stamp, 
and let the solr can run delta import standing 
by the time found in the file, of course, it will never miss docuements. 
   But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
and we may have multiple collections for the 
same core.how can i update the dataimport.properties file now in colleciton 
now? Do you have any solution to help record
 the max(sys_time_stamp) in dataimport.properties, rather than using the time 
of delta-import start to run?

Cheers

  was:
hello guys,
I met a problem when i using the solrcloud mode. When the solr instance run 
delta-import, it may take 
some time to be finished( my data source is mysql database). So during this 
time, the new added documents
will loss, the deltaQuery, i use SUBDATE(${dih.last_index_time}, INTERVAL 2 
MINUTE), let it run the delta-import 
2 mins earlier than the last_index_time, if the delta-import's duration is 5 
mins, it will loss the records at the first 3 mins. 
Our servers doesn't use solr cloud mode before, we deal with this issue is 
tring to rewrite dataimport.properties file, 
query the max(sys_time_stamp), which will help to record the max time stamp, 
and let the solr can run delta import standing 
by the time found in the file, of course, it will never miss docuements. 
   But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
and we may have multiple collections for the 
same core.how can i update the dataimport.properties file now in colleciton 
now? Do you have any solution to help record
 the max(sys_time_stamp) in dataimport.properties, rather than using the time 
of delta-import start to run?

Cheers


> Do delta import will loss some documents, when the documents added in the 
> duration of delta import.
> ---
>
> Key: SOLR-9478
> URL: https://issues.apache.org/jira/browse/SOLR-9478
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 5.5, 5.5.2, 6.0
>Reporter: ted zhu
>  Labels: delta-import, solrcloud
> Fix For: 5.5.2
>
>
> hello guys,
> I met a problem when i using the solrcloud mode. When the solr instance 
> run delta-import, it may take 
> some time to be finished( my data source is mysql database). So during this 
> time, the new added documents
> will loss, the deltaQuery, i use 
> SUBDATE(${dih.last_index_time}, INTERVAL 2 MINUTE), 
> let it run the delta-import 2 mins earlier than the last_index_time, if the 
> delta-import's duration is 5 mins, it will loss the records at the first 3 
> mins. 
> Our servers doesn't use solr cloud mode before, we deal with this issue 
> is tring to rewrite dataimport.properties file, 
> query the max(sys_time_stamp), which will help to record the max time stamp, 
> and let the solr can run delta import standing 
> by the time found in the file, of course, it will never miss docuements. 
>But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
> and we may have multiple collections for the 
> same core.how can i update the dataimport.properties file now in colleciton 
> now? Do you have any solution to help record
>  the max(sys_time_stamp) in dataimport.properties, rather than using the time 
> of delta-import start to run?
> Cheers



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9478) Do delta import will loss some documents, when the documents added in the duration of delta import.

[ted zhu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ted zhu updated SOLR-9478:
--
Description: 
hello guys,
I met a problem when i using the solrcloud mode. When the solr instance run 
delta-import, it may take 
some time to be finished( my data source is mysql database). So during this 
time, the new added documents
will loss, the deltaQuery, i use SUBDATE(\${dih.last_index_time}, INTERVAL 2 
MINUTE), 
let it run the delta-import 2 mins earlier than the last_index_time, if the 
delta-import's duration is 5 mins, it will loss the records at the first 3 
mins. 
Our servers doesn't use solr cloud mode before, we deal with this issue is 
tring to rewrite dataimport.properties file, 
query the max(sys_time_stamp), which will help to record the max time stamp, 
and let the solr can run delta import standing 
by the time found in the file, of course, it will never miss docuements. 
   But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
and we may have multiple collections for the 
same core.how can i update the dataimport.properties file now in colleciton 
now? Do you have any solution to help record
 the max(sys_time_stamp) in dataimport.properties, rather than using the time 
of delta-import start to run?

Cheers

  was:
hello guys,
I met a problem when i using the solrcloud mode. When the solr instance run 
delta-import, it may take 
some time to be finished( my data source is mysql database). So during this 
time, the new added documents
will loss, the deltaQuery, i use SUBDATE(${dih.last_index_time}, INTERVAL 2 
MINUTE), 
let it run the delta-import 2 mins earlier than the last_index_time, if the 
delta-import's duration is 5 mins, it will loss the records at the first 3 
mins. 
Our servers doesn't use solr cloud mode before, we deal with this issue is 
tring to rewrite dataimport.properties file, 
query the max(sys_time_stamp), which will help to record the max time stamp, 
and let the solr can run delta import standing 
by the time found in the file, of course, it will never miss docuements. 
   But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
and we may have multiple collections for the 
same core.how can i update the dataimport.properties file now in colleciton 
now? Do you have any solution to help record
 the max(sys_time_stamp) in dataimport.properties, rather than using the time 
of delta-import start to run?

Cheers


> Do delta import will loss some documents, when the documents added in the 
> duration of delta import.
> ---
>
> Key: SOLR-9478
> URL: https://issues.apache.org/jira/browse/SOLR-9478
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 5.5, 5.5.2, 6.0
>Reporter: ted zhu
>  Labels: delta-import, solrcloud
> Fix For: 5.5.2
>
>
> hello guys,
> I met a problem when i using the solrcloud mode. When the solr instance 
> run delta-import, it may take 
> some time to be finished( my data source is mysql database). So during this 
> time, the new added documents
> will loss, the deltaQuery, i use SUBDATE(\${dih.last_index_time}, INTERVAL 2 
> MINUTE), 
> let it run the delta-import 2 mins earlier than the last_index_time, if the 
> delta-import's duration is 5 mins, it will loss the records at the first 3 
> mins. 
> Our servers doesn't use solr cloud mode before, we deal with this issue 
> is tring to rewrite dataimport.properties file, 
> query the max(sys_time_stamp), which will help to record the max time stamp, 
> and let the solr can run delta import standing 
> by the time found in the file, of course, it will never miss docuements. 
>But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
> and we may have multiple collections for the 
> same core.how can i update the dataimport.properties file now in colleciton 
> now? Do you have any solution to help record
>  the max(sys_time_stamp) in dataimport.properties, rather than using the time 
> of delta-import start to run?
> Cheers



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9478) Do delta import will loss some documents, when the documents added in the duration of delta import.

[ted zhu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ted zhu updated SOLR-9478:
--
Description: 
hello guys,
I met a problem when i using the solrcloud mode. When the solr instance run 
delta-import, it may take 
some time to be finished( my data source is mysql database). So during this 
time, the new added documents
will loss, the deltaQuery, i use SUBDATE(${dih.last_index_time}, INTERVAL 2 
MINUTE), 
let it run the delta-import 2 mins earlier than the last_index_time, if the 
delta-import's duration is 5 mins, it will loss the records at the first 3 
mins. 
Our servers doesn't use solr cloud mode before, we deal with this issue is 
tring to rewrite dataimport.properties file, 
query the max(sys_time_stamp), which will help to record the max time stamp, 
and let the solr can run delta import standing 
by the time found in the file, of course, it will never miss docuements. 
   But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
and we may have multiple collections for the 
same core.how can i update the dataimport.properties file now in colleciton 
now? Do you have any solution to help record
 the max(sys_time_stamp) in dataimport.properties, rather than using the time 
of delta-import start to run?

Cheers

  was:
hello guys,
I met a problem when i using the solrcloud mode. When the solr instance run 
delta-import, it may take 
some time to be finished( my data source is mysql database). So during this 
time, the new added documents
will loss, the deltaQuery, i use 
SUBDATE(${dih.last_index_time}, INTERVAL 2 MINUTE), 
let it run the delta-import 2 mins earlier than the last_index_time, if the 
delta-import's duration is 5 mins, it will loss the records at the first 3 
mins. 
Our servers doesn't use solr cloud mode before, we deal with this issue is 
tring to rewrite dataimport.properties file, 
query the max(sys_time_stamp), which will help to record the max time stamp, 
and let the solr can run delta import standing 
by the time found in the file, of course, it will never miss docuements. 
   But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
and we may have multiple collections for the 
same core.how can i update the dataimport.properties file now in colleciton 
now? Do you have any solution to help record
 the max(sys_time_stamp) in dataimport.properties, rather than using the time 
of delta-import start to run?

Cheers


> Do delta import will loss some documents, when the documents added in the 
> duration of delta import.
> ---
>
> Key: SOLR-9478
> URL: https://issues.apache.org/jira/browse/SOLR-9478
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 5.5, 5.5.2, 6.0
>Reporter: ted zhu
>  Labels: delta-import, solrcloud
> Fix For: 5.5.2
>
>
> hello guys,
> I met a problem when i using the solrcloud mode. When the solr instance 
> run delta-import, it may take 
> some time to be finished( my data source is mysql database). So during this 
> time, the new added documents
> will loss, the deltaQuery, i use SUBDATE(${dih.last_index_time}, INTERVAL 2 
> MINUTE), 
> let it run the delta-import 2 mins earlier than the last_index_time, if the 
> delta-import's duration is 5 mins, it will loss the records at the first 3 
> mins. 
> Our servers doesn't use solr cloud mode before, we deal with this issue 
> is tring to rewrite dataimport.properties file, 
> query the max(sys_time_stamp), which will help to record the max time stamp, 
> and let the solr can run delta import standing 
> by the time found in the file, of course, it will never miss docuements. 
>But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
> and we may have multiple collections for the 
> same core.how can i update the dataimport.properties file now in colleciton 
> now? Do you have any solution to help record
>  the max(sys_time_stamp) in dataimport.properties, rather than using the time 
> of delta-import start to run?
> Cheers



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9478) Do delta import will loss some documents, when the documents added in the duration of delta import.

[ted zhu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ted zhu updated SOLR-9478:
--
Description: 
hello guys,
I met a problem when i using the solrcloud mode. When the solr instance run 
delta-import, it may take 
some time to be finished( my data source is mysql database). So during this 
time, the new added documents
will loss, the deltaQuery, i use SUBDATE($\{dih.last_index_time\}, INTERVAL 2 
MINUTE), 
let it run the delta-import 2 mins earlier than the last_index_time, if the 
delta-import's duration is 5 mins, it will loss the records at the first 3 
mins. 
Our servers doesn't use solr cloud mode before, we deal with this issue is 
tring to rewrite dataimport.properties file, 
query the max(sys_time_stamp), which will help to record the max time stamp, 
and let the solr can run delta import standing 
by the time found in the file, of course, it will never miss docuements. 
   But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
and we may have multiple collections for the 
same core.how can i update the dataimport.properties file now in colleciton 
now? Do you have any solution to help record
 the max(sys_time_stamp) in dataimport.properties, rather than using the time 
of delta-import start to run?

Cheers

  was:
hello guys,
I met a problem when i using the solrcloud mode. When the solr instance run 
delta-import, it may take 
some time to be finished( my data source is mysql database). So during this 
time, the new added documents
will loss, the deltaQuery, i use SUBDATE(\${dih.last_index_time}, INTERVAL 2 
MINUTE), 
let it run the delta-import 2 mins earlier than the last_index_time, if the 
delta-import's duration is 5 mins, it will loss the records at the first 3 
mins. 
Our servers doesn't use solr cloud mode before, we deal with this issue is 
tring to rewrite dataimport.properties file, 
query the max(sys_time_stamp), which will help to record the max time stamp, 
and let the solr can run delta import standing 
by the time found in the file, of course, it will never miss docuements. 
   But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
and we may have multiple collections for the 
same core.how can i update the dataimport.properties file now in colleciton 
now? Do you have any solution to help record
 the max(sys_time_stamp) in dataimport.properties, rather than using the time 
of delta-import start to run?

Cheers


> Do delta import will loss some documents, when the documents added in the 
> duration of delta import.
> ---
>
> Key: SOLR-9478
> URL: https://issues.apache.org/jira/browse/SOLR-9478
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 5.5, 5.5.2, 6.0
>Reporter: ted zhu
>  Labels: delta-import, solrcloud
> Fix For: 5.5.2
>
>
> hello guys,
> I met a problem when i using the solrcloud mode. When the solr instance 
> run delta-import, it may take 
> some time to be finished( my data source is mysql database). So during this 
> time, the new added documents
> will loss, the deltaQuery, i use SUBDATE($\{dih.last_index_time\}, INTERVAL 2 
> MINUTE), 
> let it run the delta-import 2 mins earlier than the last_index_time, if the 
> delta-import's duration is 5 mins, it will loss the records at the first 3 
> mins. 
> Our servers doesn't use solr cloud mode before, we deal with this issue 
> is tring to rewrite dataimport.properties file, 
> query the max(sys_time_stamp), which will help to record the max time stamp, 
> and let the solr can run delta import standing 
> by the time found in the file, of course, it will never miss docuements. 
>But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
> and we may have multiple collections for the 
> same core.how can i update the dataimport.properties file now in colleciton 
> now? Do you have any solution to help record
>  the max(sys_time_stamp) in dataimport.properties, rather than using the time 
> of delta-import start to run?
> Cheers



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9478) Do delta import will loss some documents, when the documents added in the duration of delta import.

[ted zhu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ted zhu updated SOLR-9478:
--
Description: 
hello guys,
I met a problem when i using the solrcloud mode. When the solr instance run 
delta-import, it may take 
some time to be finished( my data source is mysql database). So during this 
time, the new added documents
will loss, the deltaQuery, i use `SUBDATE(${dih.last_index_time}, INTERVAL 2 
MINUTE)`, let it run the delta-import 
2 mins earlier than the last_index_time, if the delta-import's duration is 5 
mins, it will loss the records at the first 3 mins. 
Our servers doesn't use solr cloud mode before, we deal with this issue is 
tring to rewrite dataimport.properties file, 
query the max(sys_time_stamp), which will help to record the max time stamp, 
and let the solr can run delta import standing 
by the time found in the file, of course, it will never miss docuements. 
   But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
and we may have multiple collections for the 
same core.how can i update the dataimport.properties file now in colleciton 
now? Do you have any solution to help record
 the max(sys_time_stamp) in dataimport.properties, rather than using the time 
of delta-import start to run?

Cheers

  was:
hello guys,
I met a problem when i using the solrcloud mode. When the solr instance run 
delta-import, it may take 
some time to be finished( my data source is mysql database). So during this 
time, the new added documents
will loss, the deltaQuery, i use SUBDATE(${dih.last_index_time}, INTERVAL 2 
MINUTE), let it run the delta-import 
2 mins earlier than the last_index_time, if the delta-import's duration is 5 
mins, it will loss the records at the first 3 mins. 
Our servers doesn't use solr cloud mode before, we deal with this issue is 
tring to rewrite dataimport.properties file, 
query the max(sys_time_stamp), which will help to record the max time stamp, 
and let the solr can run delta import standing 
by the time found in the file, of course, it will never miss docuements. 
   But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
and we may have multiple collections for the 
same core.how can i update the dataimport.properties file now in colleciton 
now? Do you have any solution to help record
 the max(sys_time_stamp) in dataimport.properties, rather than using the time 
of delta-import start to run?

Cheers


> Do delta import will loss some documents, when the documents added in the 
> duration of delta import.
> ---
>
> Key: SOLR-9478
> URL: https://issues.apache.org/jira/browse/SOLR-9478
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 5.5, 5.5.2, 6.0
>Reporter: ted zhu
>  Labels: delta-import, solrcloud
> Fix For: 5.5.2
>
>
> hello guys,
> I met a problem when i using the solrcloud mode. When the solr instance 
> run delta-import, it may take 
> some time to be finished( my data source is mysql database). So during this 
> time, the new added documents
> will loss, the deltaQuery, i use `SUBDATE(${dih.last_index_time}, INTERVAL 2 
> MINUTE)`, let it run the delta-import 
> 2 mins earlier than the last_index_time, if the delta-import's duration is 5 
> mins, it will loss the records at the first 3 mins. 
> Our servers doesn't use solr cloud mode before, we deal with this issue 
> is tring to rewrite dataimport.properties file, 
> query the max(sys_time_stamp), which will help to record the max time stamp, 
> and let the solr can run delta import standing 
> by the time found in the file, of course, it will never miss docuements. 
>But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
> and we may have multiple collections for the 
> same core.how can i update the dataimport.properties file now in colleciton 
> now? Do you have any solution to help record
>  the max(sys_time_stamp) in dataimport.properties, rather than using the time 
> of delta-import start to run?
> Cheers



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9478) Do delta import will loss some documents, when the documents added in the duration of delta import.

[ted zhu (JIRA)]

ted zhu created SOLR-9478:
-

 Summary: Do delta import will loss some documents, when the 
documents added in the duration of delta import.
 Key: SOLR-9478
 URL: https://issues.apache.org/jira/browse/SOLR-9478
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
  Components: SolrCloud
Affects Versions: 6.0, 5.5.2, 5.5
Reporter: ted zhu
 Fix For: 5.5.2


hello guys,
I met a problem when i using the solrcloud mode. When the solr instance run 
delta-import, it may take 
some time to be finished( my data source is mysql database). So during this 
time, the new added documents
will loss, the deltaQuery, i use SUBDATE(${dih.last_index_time}, INTERVAL 2 
MINUTE), let it run the delta-import 
2 mins earlier than the last_index_time, if the delta-import's duration is 5 
mins, it will loss the records at the first 3 mins. 
Our servers doesn't use solr cloud mode before, we deal with this issue is 
tring to rewrite dataimport.properties file, 
query the max(sys_time_stamp), which will help to record the max time stamp, 
and let the solr can run delta import standing 
by the time found in the file, of course, it will never miss docuements. 
   But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
and we may have multiple collections for the 
same core.how can i update the dataimport.properties file now in colleciton 
now? Do you have any solution to help record
 the max(sys_time_stamp) in dataimport.properties, rather than using the time 
of delta-import start to run?

Cheers



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9344) BasicAuthIntegrationTest test failures on update

[Alan Woodward (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward updated SOLR-9344:

Attachment: SOLR-9344.patch

I'm pretty sure that this is due to stale connections.  Here's a patch that 
does the following:
* moves BasicAuthIntegrationTest to use SolrCloudTestCase
* removes the 'collectionCreateSearchDelete' part of the test, which is 
duplicated in MiniSolrCloudClusterTest, doesn't seem to do anything useful 
here, and is the source of any stale connections that are happening
* remove MiniSolrCloudClusterTestBase, which is now superfluous.

> BasicAuthIntegrationTest test failures on update
> 
>
> Key: SOLR-9344
> URL: https://issues.apache.org/jira/browse/SOLR-9344
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: security, Tests
>Affects Versions: trunk
>Reporter: Gregory Chanan
> Attachments: SOLR-9344.patch
>
>
> I've seen this a number of times while developing SOLR-9200 and SOLR-9324; 
> there's also a public failure here: 
> http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17372/
> {code}
> org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: IOException 
> occured when talking to server at: 
> http://127.0.0.1:45882/solr/testSolrCloudCollection_shard1_replica2
>   at 
> __randomizedtesting.SeedInfo.seed([99BB0D0378978FA8:A463A32F4079D1D8]:0)
>   at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:760)
>   at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1172)
>   at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
>   at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
>   at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
>   at 
> org.apache.solr.security.BasicAuthIntegrationTest.doExtraTests(BasicAuthIntegrationTest.java:193)
>   at 
> org.apache.solr.cloud.TestMiniSolrCloudClusterBase.testCollectionCreateSearchDelete(TestMiniSolrCloudClusterBase.java:196)
>   at 
> org.apache.solr.cloud.TestMiniSolrCloudClusterBase.testBasics(TestMiniSolrCloudClusterBase.java:79)
>   at 
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native 
> Method)
>   at 
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
>   at 
> jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:533)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
>   at 
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
>   at 
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
>   at 
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
>   at 
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
>   at 
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>   at 
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
>   at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>   at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
>   at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
>   at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
>   at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
>   at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>   at 
> com.carrotsearch.randomize

[jira] [Updated] (SOLR-9478) Do delta import will loss some documents, when the documents added in the duration of delta import.

[ted zhu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ted zhu updated SOLR-9478:
--
Description: 
hello guys,
I met a problem when i using the solrcloud mode. When the solr instance run 
delta-import, it may take 
some time to be finished( my data source is mysql database). So during this 
time, the new added documents
will loss, the deltaQuery, i use SUBDATE(${dih.last_index_time}, INTERVAL 2 
MINUTE), let it run the delta-import 
2 mins earlier than the last_index_time, if the delta-import's duration is 5 
mins, it will loss the records at the first 3 mins. 
Our servers doesn't use solr cloud mode before, we deal with this issue is 
tring to rewrite dataimport.properties file, 
query the max(sys_time_stamp), which will help to record the max time stamp, 
and let the solr can run delta import standing 
by the time found in the file, of course, it will never miss docuements. 
   But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
and we may have multiple collections for the 
same core.how can i update the dataimport.properties file now in colleciton 
now? Do you have any solution to help record
 the max(sys_time_stamp) in dataimport.properties, rather than using the time 
of delta-import start to run?

Cheers

  was:
hello guys,
I met a problem when i using the solrcloud mode. When the solr instance run 
delta-import, it may take 
some time to be finished( my data source is mysql database). So during this 
time, the new added documents
will loss, the deltaQuery, i use `SUBDATE(${dih.last_index_time}, INTERVAL 2 
MINUTE)`, let it run the delta-import 
2 mins earlier than the last_index_time, if the delta-import's duration is 5 
mins, it will loss the records at the first 3 mins. 
Our servers doesn't use solr cloud mode before, we deal with this issue is 
tring to rewrite dataimport.properties file, 
query the max(sys_time_stamp), which will help to record the max time stamp, 
and let the solr can run delta import standing 
by the time found in the file, of course, it will never miss docuements. 
   But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
and we may have multiple collections for the 
same core.how can i update the dataimport.properties file now in colleciton 
now? Do you have any solution to help record
 the max(sys_time_stamp) in dataimport.properties, rather than using the time 
of delta-import start to run?

Cheers


> Do delta import will loss some documents, when the documents added in the 
> duration of delta import.
> ---
>
> Key: SOLR-9478
> URL: https://issues.apache.org/jira/browse/SOLR-9478
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 5.5, 5.5.2, 6.0
>Reporter: ted zhu
>  Labels: delta-import, solrcloud
> Fix For: 5.5.2
>
>
> hello guys,
> I met a problem when i using the solrcloud mode. When the solr instance 
> run delta-import, it may take 
> some time to be finished( my data source is mysql database). So during this 
> time, the new added documents
> will loss, the deltaQuery, i use SUBDATE(${dih.last_index_time}, INTERVAL 2 
> MINUTE), let it run the delta-import 
> 2 mins earlier than the last_index_time, if the delta-import's duration is 5 
> mins, it will loss the records at the first 3 mins. 
> Our servers doesn't use solr cloud mode before, we deal with this issue 
> is tring to rewrite dataimport.properties file, 
> query the max(sys_time_stamp), which will help to record the max time stamp, 
> and let the solr can run delta import standing 
> by the time found in the file, of course, it will never miss docuements. 
>But now, we use solrcloud, the dataimport.properties is on the zookeeper, 
> and we may have multiple collections for the 
> same core.how can i update the dataimport.properties file now in colleciton 
> now? Do you have any solution to help record
>  the max(sys_time_stamp) in dataimport.properties, rather than using the time 
> of delta-import start to run?
> Cheers



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5396) SpanNearQuery returns single term spans

[Christoph Goller (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464535#comment-15464535
 ] 

Christoph Goller commented on LUCENE-5396:
--

Is this a bug or desired bahavior?

For me it is at least an acceptable behavior. I like the behavior of unordered 
SpanNearQuery to match if clauses overlap or match at the same position. and it 
would be quite difficult to find out if two clauses match at the same index 
term or only at the same position.

background: I am using a component for word decomposition. This might be a very 
rare case for English but it is a much more common phenomen for German and 
Dutch. The two compound parts of "wallpaper" (wall and paper) go into the same 
index position as wallpaper. I am using  spanNear([wall, paper], 0, false) to 
search for wallpaper and expect matches for "wallpaper" as well as for "wall 
paper". 

So far we do not have a proper definition of what SpanQueries should do and the 
only way to find out what they currently do is to look into the code. I think 
the current behavior is not very consistent. I will present some of my  
insights and ideas in LUCENE-7398.

> SpanNearQuery returns single term spans
> ---
>
> Key: LUCENE-5396
> URL: https://issues.apache.org/jira/browse/LUCENE-5396
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/search
>Reporter: Piotr Pęzik
>
> Let's assume we have an index with two documents:
> 1. contents: "test bunga bunga test"
> 2. contents: "test bunga test"
> We run two SpanNearQueries against this index:
> 1. spanNear([contents:bunga, contents:bunga], 0, true)
> 2. spanNear([contents:bunga, contents:bunga], 0, false)
> For the first query we get 1 hit. The first document in the example above 
> gets matched and the second one doesn't. This make sense, because we want the 
> term "bunga" followed by another "bunga" here.
> However, both documents get matched by the second query. This is also 
> problematic in cases where we have duplicate terms in longer (unordered) 
> spannear queries, e. g.: unordered 'A B A' will match spans such as 'A B' or 
> 'B A'.
> A complete example follows. 
> -
> import org.apache.lucene.analysis.Analyzer;
> import org.apache.lucene.analysis.standard.StandardAnalyzer;
> import org.apache.lucene.document.Document;
> import org.apache.lucene.document.TextField;
> import org.apache.lucene.index.DirectoryReader;
> import org.apache.lucene.index.IndexWriter;
> import org.apache.lucene.index.IndexWriterConfig;
> import org.apache.lucene.index.Term;
> import org.apache.lucene.search.IndexSearcher;
> import org.apache.lucene.search.TopDocs;
> import org.apache.lucene.search.spans.SpanNearQuery;
> import org.apache.lucene.search.spans.SpanQuery;
> import org.apache.lucene.search.spans.SpanTermQuery;
> import org.apache.lucene.store.Directory;
> import org.apache.lucene.store.FSDirectory;
> import org.apache.lucene.store.RAMDirectory;
> import org.apache.lucene.util.Version;
> import java.io.StringReader;
> import static org.junit.Assert.assertEquals;
> class SpansBug {
> public static void main(String [] args) throws Exception {
> Directory dir = new RAMDirectory();
> Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_45);
> IndexWriterConfig iwc = new IndexWriterConfig(Version.LUCENE_45, 
> analyzer);
> IndexWriter writer = new IndexWriter(dir, iwc);
> String contents = "contents";
> Document doc1 = new Document();
> doc1.add(new TextField(contents, new StringReader("test bunga bunga 
> test")));
> Document doc2 = new Document();
> doc2.add(new TextField(contents, new StringReader("test bunga 
> test")));
> writer.addDocument(doc1);
> writer.addDocument(doc2);
> writer.commit();
> IndexSearcher searcher = new IndexSearcher(DirectoryReader.open(dir));
> SpanQuery stq1 = new SpanTermQuery(new Term(contents,"bunga"));
> SpanQuery stq2 = new SpanTermQuery(new Term(contents,"bunga"));
> SpanQuery [] spqa = new SpanQuery[]{stq1,stq2};
> SpanNearQuery spanQ1 = new SpanNearQuery(spqa,0, true);
> SpanNearQuery spanQ2 = new SpanNearQuery(spqa,0, false);
> System.out.println(spanQ1);
> TopDocs tdocs1 = searcher.search(spanQ1,10);
> assertEquals(tdocs1.totalHits ,1);
> System.out.println(spanQ2);
> TopDocs tdocs2 = searcher.search(spanQ2,10);
> //I'd expect one hit here:
> assertEquals(tdocs2.totalHits ,1); // Assertion fails
> }
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.

[jira] [Created] (SOLR-9477) UpdateRequestProcessors ignore child documents

[Alexandre Rafalovitch (JIRA)]

Alexandre Rafalovitch created SOLR-9477:
---

 Summary: UpdateRequestProcessors ignore child documents
 Key: SOLR-9477
 URL: https://issues.apache.org/jira/browse/SOLR-9477
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 6.2, master (7.0)
Reporter: Alexandre Rafalovitch


UpdateRequestProcessors completely ignore child documents. The only exception 
is AddSchemaFieldsUpdateProcessorFactory. The rest seem to be completely 
unaware that SolrInputDocument has getChildDocuments() or related methods.

Easy test (on Solr 6.2):

This works (with IDs auto-assigned and field names generated):
{code}
bin/solr create -c childtest
bin/post -c childtest -type application/json -format solr -d "[{"a":1,"b":2}]"
{code}

This fails as the second/third command, with "missing ID field":
{code}
bin/post -c childtest -type application/json -format solr -d 
"[{"a":1,"b":2,"_childDocuments_":[{c:3,d:4}]}]"
{code}

The message:
{noformat}
SimplePostTool version 5.0.0
POSTing args to http://localhost:8983/solr/childtest/update...
SimplePostTool: WARNING: Solr returned an error #400 (Bad Request) for url: 
http://localhost:8983/solr/childtest/update
SimplePostTool: WARNING: Response: 
{"responseHeader":{"status":400,"QTime":4},"error":{"metadata":["error-class","org.apache.solr.common.SolrException","root-error-class","org.apache.solr.common.SolrException"],"msg":"[doc=null]
 missing required field: id","code":400}}
SimplePostTool: WARNING: IOException while reading response: 
java.io.IOException: Server returned HTTP response code: 400 for URL: 
http://localhost:8983/solr/childtest/update
COMMITting Solr index changes to http://localhost:8983/solr/childtest/update...
Time spent: 0:00:00.042
{noformat}

I also verified it with BlankRemoving URP. I think this is a global problem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9474) MiniSolrCloudCluster should not reuse ports by default

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464398#comment-15464398
 ] 

Alan Woodward commented on SOLR-9474:
-

OK, thanks Shalin.  I'll watch for some exciting new test failures, then :)

> MiniSolrCloudCluster should not reuse ports by default
> --
>
> Key: SOLR-9474
> URL: https://issues.apache.org/jira/browse/SOLR-9474
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9474.patch
>
>
> Building on SOLR-9469, this just changes 
> MSCC.startJettySolrRunner(JettySolrRunner) to use a different port when it 
> restarts the passed-in Jetty.  This should stop the semi-frequent test 
> failures in CollectionStateWatchersTest.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9127) XLSX response writer - do we want it?

[Tony Moriarty (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464352#comment-15464352
 ] 

Tony Moriarty edited comment on SOLR-9127 at 9/5/16 7:51 AM:
-

OK, i stripped out support for solrconfig init params and updated the tests 
accordingly. The PR is ready for review again

An additional note for the documentation that came up during testing:
The maximum value for a colwidth, as per poi limitations, is 255


was (Author: desultir):
OK, i stripped out support for solrconfig init params and updated the tests 
accordingly. The PR is ready for review again
Cheers

> XLSX response writer - do we want it?
> -
>
> Key: SOLR-9127
> URL: https://issues.apache.org/jira/browse/SOLR-9127
> Project: Solr
>  Issue Type: Improvement
>  Components: Response Writers
>Reporter: Tony Moriarty
>Priority: Minor
> Fix For: 5.5, 6.0
>
> Attachments: SOLR-9127.patch
>
>
> I recently open sourced an XLSX response writer based on solr 4.6 and apache 
> poi.
> https://github.com/desultir/SolrXLSXResponseWriter
> Is this something the community would be interested in bringing into the solr 
> codebase? I'm willing to put the work into porting it to solr5 and solr6 if 
> the community is interested, happy to leave it as a plugin otherwise.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Java function throws exception on JSP web application

[szzoli]

Now, I became angry with silly errors. I created a new web application,
copied the jars ini it, and now it works fine. 

But I have no idea, wat is the difference between the two applications.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Java-function-throws-exception-on-JSP-web-application-tp4294431p4294632.html
Sent from the Lucene - Java Developer mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9127) XLSX response writer - do we want it?

[Tony Moriarty (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15464352#comment-15464352
 ] 

Tony Moriarty commented on SOLR-9127:
-

OK, i stripped out support for solrconfig init params and updated the tests 
accordingly. The PR is ready for review again
Cheers

> XLSX response writer - do we want it?
> -
>
> Key: SOLR-9127
> URL: https://issues.apache.org/jira/browse/SOLR-9127
> Project: Solr
>  Issue Type: Improvement
>  Components: Response Writers
>Reporter: Tony Moriarty
>Priority: Minor
> Fix For: 5.5, 6.0
>
> Attachments: SOLR-9127.patch
>
>
> I recently open sourced an XLSX response writer based on solr 4.6 and apache 
> poi.
> https://github.com/desultir/SolrXLSXResponseWriter
> Is this something the community would be interested in bringing into the solr 
> codebase? I'm willing to put the work into porting it to solr5 and solr6 if 
> the community is interested, happy to leave it as a plugin otherwise.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org