[Lucene.Net] Barebones CI is setup.
A bare bones version for trunk CI builds have been setup. [Trunk] https://builds.apache.org/job/Lucene.Net-Trunk-Contrib-Poll-Changes/ https://builds.apache.org/job/Lucene.Net-Trunk-Contrib-Nightly/ https://builds.apache.org/job/Lucene.Net-Trunk-All-Poll-Changes/ https://builds.apache.org/job/Lucene.Net-Trunk-All-Nightly/ Lucene.Net-Trunk-All-Nightly Lucene.Net-Trunk-All-Poll-Changes will fail till the tests are in working order for gallio test runner invoking nunit. I think the fact that gallio is running as a 64 bit process might have something to do with it. I believe nunit tends to run as x86 by default. I put up a separate build for contrib projects because their test projects are smaller and the build for it runs quickly. So if you're working on a contrib project, the build update will not take as long. Those builds are currently successful. Poll-Changes suffix means: the build will poll SVN at every hour and 15 minutes. i.e 12:15, 1:15, etc for changes. build from scratch, run tests (and metrics when the tools for those become successfully installed) Nightly suffix means: the build will poll nightly at 1 am jenkins standard time. it will build from scratch, run tests (and metrics, packages, and documentation when those tools be come successfully installed) and archive the results. Make sure to thank Niklas Gustavsson for helping us to get this far. That last thing for the group to discuss is how would you all like to receive notifications. You can currently subscribe to build rss feeds. We could also setup notifications for e-mail, irc, or twitter. - Michael.
[Lucene.Net] [jira] [Created] (LUCENENET-448) GeoHashFilteredDocIdSet does not work at all
GeoHashFilteredDocIdSet does not work at all
Key: LUCENENET-448
URL: https://issues.apache.org/jira/browse/LUCENENET-448
Project: Lucene.Net
Issue Type: Bug
Components: Lucene.Net Contrib
Affects Versions: Lucene.Net 2.9.4
Environment: Windows 7 x64
Reporter: Jeff Johnson
Fix For: Lucene.Net 2.9.4
The GeoHashFilteredDocIdSet is assuming the values are always in the cache
which is wrong. A proposed fix for the method is listed here for
GeoHashDistanceFilter.cs:
public GeoHashFilteredDocIdSet(DocIdSet innerSet, string[] geoHashValues,
Dictionarystring, double distanceLookupCache, double lat, double lng, int
docBase, double distance, Dictionaryint, double distances)
: base(innerSet , (docid) = /* was: public override Match */
{
String geoHash = geoHashValues[docid];
double[] coords = GeoHashUtils.Decode(geoHash);
double x = coords[0];
double y = coords[1];
double cachedDistance;
distanceLookupCache.TryGetValue(geoHash, out cachedDistance);
double d;
if (cachedDistance 0)
{
d = cachedDistance;
}
else
{
d =
DistanceUtils.GetInstance().GetDistanceMi(lat, lng, x, y);
distanceLookupCache[geoHash] = d;
}
if (d distance)
{
distances[docid + docBase] = d;
return true;
}
return false;
})
{
_geoHashValues = geoHashValues;
_distances = distances;
_distance = distance;
_docBase = docBase;
_lng = lng;
_lat = lat;
_distanceLookupCache = distanceLookupCache;
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[JENKINS] Lucene-trunk - Build # 1700 - Still Failing
Build: https://builds.apache.org/job/Lucene-trunk/1700/
1 tests failed.
FAILED: org.apache.lucene.search.TestSearcherManager.testSearcherManager
Error Message:
MockDirectoryWrapper: cannot close: there are still open files: {_2j.cfs=14,
_2a.nrm=1, _2a.fdx=1, _2a.fdt=1, _2a.tvx=1, _2a.tvf=1, _2a.tvd=1, _2l.cfs=17}
Stack Trace:
java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still
open files: {_2j.cfs=14, _2a.nrm=1, _2a.fdx=1, _2a.fdt=1, _2a.tvx=1, _2a.tvf=1,
_2a.tvd=1, _2l.cfs=17}
at
org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:479)
at
org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase.runTest(ThreadedIndexingAndSearchingTestCase.java:636)
at
org.apache.lucene.search.TestSearcherManager.testSearcherManager(TestSearcherManager.java:48)
at
org.apache.lucene.util.LuceneTestCase$2$1.evaluate(LuceneTestCase.java:593)
at
org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:149)
at
org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:51)
Caused by: java.lang.RuntimeException: unclosed IndexInput: _2j.cfs
at
org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:418)
at
org.apache.lucene.store.MockDirectoryWrapper$1.openSlice(MockDirectoryWrapper.java:693)
at
org.apache.lucene.store.CompoundFileDirectory.openInput(CompoundFileDirectory.java:216)
at
org.apache.lucene.index.codecs.standard.StandardPostingsReader.init(StandardPostingsReader.java:64)
at
org.apache.lucene.index.codecs.mockrandom.MockRandomCodec.fieldsProducer(MockRandomCodec.java:306)
at
org.apache.lucene.index.PerFieldCodecWrapper$FieldsReader.init(PerFieldCodecWrapper.java:114)
at
org.apache.lucene.index.PerFieldCodecWrapper.fieldsProducer(PerFieldCodecWrapper.java:182)
at
org.apache.lucene.index.SegmentCoreReaders.init(SegmentCoreReaders.java:91)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:112)
at
org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:714)
at
org.apache.lucene.index.IndexWriter$ReaderPool.getReadOnlyClone(IndexWriter.java:686)
at
org.apache.lucene.index.IndexWriter$ReaderPool.getReadOnlyClone(IndexWriter.java:677)
at
org.apache.lucene.index.DirectoryReader.init(DirectoryReader.java:172)
at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:378)
at
org.apache.lucene.index.IndexReader.doOpenIfChanged(IndexReader.java:688)
at
org.apache.lucene.index.IndexReader.openIfChanged(IndexReader.java:670)
at
org.apache.lucene.search.SearcherManager$NRTSearcherManager.openIfChanged(SearcherManager.java:295)
at
org.apache.lucene.search.SearcherManager.maybeReopen(SearcherManager.java:106)
at
org.apache.lucene.search.TestSearcherManager$2.run(TestSearcherManager.java:99)
Build Log (for compile errors):
[...truncated 14207 lines...]
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Solr-3.x - Build # 482 - Failure
Build: https://builds.apache.org/job/Solr-3.x/482/ No tests ran. Build Log (for compile errors): [...truncated 10223 lines...] compile-test-framework: common.compile-test: [mkdir] Created dir: /usr/home/hudson/hudson-slave/workspace/Solr-3.x/checkout/solr/build/contrib/solr-cell/classes/test [javac] Compiling 1 source file to /usr/home/hudson/hudson-slave/workspace/Solr-3.x/checkout/solr/build/contrib/solr-cell/classes/test compile-test: [echo] Building solr-langid... compile-solr-test-framework: compile-solr-core: compile-solrj: check-lucene-core-uptodate: jar-lucene-core: check-analyzers-common-uptodate: jar-analyzers-common: check-spellchecker-uptodate: jar-spellchecker: check-highlighter-uptodate: jar-highlighter: check-memory-uptodate: jar-memory: check-misc-uptodate: jar-misc: check-spatial-uptodate: jar-spatial: check-grouping-uptodate: jar-grouping: check-queries-uptodate: jar-queries: prep-lucene-jars: common.init: compile-lucene-core: jflex-uptodate-check: jflex-notice: javacc-uptodate-check: javacc-notice: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: compile-core: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: [mkdir] Created dir: /usr/home/hudson/hudson-slave/workspace/Solr-3.x/checkout/solr/build/contrib/solr-langid/classes/java [javac] Compiling 7 source files to /usr/home/hudson/hudson-slave/workspace/Solr-3.x/checkout/solr/build/contrib/solr-langid/classes/java [javac] /usr/home/hudson/hudson-slave/workspace/Solr-3.x/checkout/solr/contrib/langid/src/java/org/apache/solr/update/processor/LangDetectLanguageIdentifierUpdateProcessor.java:27: cannot access com.cybozu.labs.langdetect.Detector [javac] bad class file: /usr/home/hudson/hudson-slave/workspace/Solr-3.x/checkout/solr/contrib/langid/lib/langdetect-r111.jar(com/cybozu/labs/langdetect/Detector.class) [javac] class file has wrong version 50.0, should be 49.0 [javac] Please remove or make sure it appears in the correct subdirectory of the classpath. [javac] import com.cybozu.labs.langdetect.Detector; [javac] ^ [javac] 1 error [...truncated 17 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS-MAVEN] Lucene-Solr-Maven-3.x #267: POMs out of sync
Build: https://builds.apache.org/job/Lucene-Solr-Maven-3.x/267/ No tests ran. Build Log (for compile errors): [...truncated 17674 lines...] init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: compile-core: check-lucene-core-uptodate: jar-lucene-core: check-analyzers-common-uptodate: jar-analyzers-common: check-spellchecker-uptodate: jar-spellchecker: check-highlighter-uptodate: jar-highlighter: check-memory-uptodate: jar-memory: check-misc-uptodate: jar-misc: check-spatial-uptodate: jar-spatial: check-grouping-uptodate: jar-grouping: check-queries-uptodate: jar-queries: prep-lucene-jars: common.init: compile-lucene-core: jflex-uptodate-check: jflex-notice: javacc-uptodate-check: javacc-notice: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: compile-core: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: [mkdir] Created dir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Maven-3.x/checkout/solr/build/contrib/solr-langid/classes/java [javac] Compiling 7 source files to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Maven-3.x/checkout/solr/build/contrib/solr-langid/classes/java [javac] /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Maven-3.x/checkout/solr/contrib/langid/src/java/org/apache/solr/update/processor/LangDetectLanguageIdentifierUpdateProcessor.java:27: cannot access com.cybozu.labs.langdetect.Detector [javac] bad class file: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Maven-3.x/checkout/solr/contrib/langid/lib/langdetect-r111.jar(com/cybozu/labs/langdetect/Detector.class) [javac] class file has wrong version 50.0, should be 49.0 [javac] Please remove or make sure it appears in the correct subdirectory of the classpath. [javac] import com.cybozu.labs.langdetect.Detector; [javac] ^ [javac] 1 error [...truncated 13 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS-MAVEN] Lucene-Solr-Maven-trunk #266: POMs out of sync
Build: https://builds.apache.org/job/Lucene-Solr-Maven-trunk/266/ No tests ran. Build Log (for compile errors): [...truncated 18924 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3520) If the NRT reader hasn't changed then IndexReader.openIfChanged should return null
[
https://issues.apache.org/jira/browse/LUCENE-3520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13128375#comment-13128375
]
Simon Willnauer commented on LUCENE-3520:
-
bq. New patch; I was able to simplify SearcherManager since both cases
(open-from-writer (= NRT case) and open-from-dir (= non-NRT case)) now call the
same IR.openIfChanged(oldReader).
yeah nice! looks good mike!
If the NRT reader hasn't changed then IndexReader.openIfChanged should return
null
--
Key: LUCENE-3520
URL: https://issues.apache.org/jira/browse/LUCENE-3520
Project: Lucene - Java
Issue Type: Bug
Components: core/index
Reporter: Michael McCandless
Assignee: Michael McCandless
Fix For: 3.5, 4.0
Attachments: LUCENE-3520.patch, LUCENE-3520.patch
I hit a failure in TestSearcherManager (NOTE: doesn't always fail):
{noformat}
ant test -Dtestcase=TestSearcherManager -Dtestmethod=testSearcherManager
-Dtests.seed=459ac99a4256789c:-29b8a7f52497c3b4:145ae632ae9e1ecf
{noformat}
It was tripping the assert inside SearcherLifetimeManager.record,
because two different IndexSearcher instances had different IR
instances sharing the same version. This was happening because
IW.getReader always returns a new reader even when there are no
changes. I think we should fix that...
Separately I found a deadlock in
TestSearcherManager.testIntermediateClose, if the test gets
SerialMergeScheduler and needs to merge during the second commit.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
RE: [JENKINS] Solr-3.x - Build # 482 - Failure
The JAR file is not Java 5 compatible, so cannot be used with Solr 3.x. - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Apache Jenkins Server [mailto:jenk...@builds.apache.org] Sent: Sunday, October 16, 2011 8:30 AM To: dev@lucene.apache.org Subject: [JENKINS] Solr-3.x - Build # 482 - Failure Build: https://builds.apache.org/job/Solr-3.x/482/ No tests ran. Build Log (for compile errors): [...truncated 10223 lines...] compile-test-framework: common.compile-test: [mkdir] Created dir: /usr/home/hudson/hudson-slave/workspace/Solr- 3.x/checkout/solr/build/contrib/solr-cell/classes/test [javac] Compiling 1 source file to /usr/home/hudson/hudson- slave/workspace/Solr-3.x/checkout/solr/build/contrib/solr-cell/classes/test compile-test: [echo] Building solr-langid... compile-solr-test-framework: compile-solr-core: compile-solrj: check-lucene-core-uptodate: jar-lucene-core: check-analyzers-common-uptodate: jar-analyzers-common: check-spellchecker-uptodate: jar-spellchecker: check-highlighter-uptodate: jar-highlighter: check-memory-uptodate: jar-memory: check-misc-uptodate: jar-misc: check-spatial-uptodate: jar-spatial: check-grouping-uptodate: jar-grouping: check-queries-uptodate: jar-queries: prep-lucene-jars: common.init: compile-lucene-core: jflex-uptodate-check: jflex-notice: javacc-uptodate-check: javacc-notice: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: compile-core: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: [mkdir] Created dir: /usr/home/hudson/hudson-slave/workspace/Solr- 3.x/checkout/solr/build/contrib/solr-langid/classes/java [javac] Compiling 7 source files to /usr/home/hudson/hudson- slave/workspace/Solr-3.x/checkout/solr/build/contrib/solr-langid/classes/java [javac] /usr/home/hudson/hudson-slave/workspace/Solr- 3.x/checkout/solr/contrib/langid/src/java/org/apache/solr/update/processor/L angDetectLanguageIdentifierUpdateProcessor.java:27: cannot access com.cybozu.labs.langdetect.Detector [javac] bad class file: /usr/home/hudson/hudson-slave/workspace/Solr- 3.x/checkout/solr/contrib/langid/lib/langdetect- r111.jar(com/cybozu/labs/langdetect/Detector.class) [javac] class file has wrong version 50.0, should be 49.0 [javac] Please remove or make sure it appears in the correct subdirectory of the classpath. [javac] import com.cybozu.labs.langdetect.Detector; [javac] ^ [javac] 1 error [...truncated 17 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Solr-3.x - Build # 482 - Failure
I recompiled it with -source/-target ancient and committed. On Sun, Oct 16, 2011 at 7:56 AM, Uwe Schindler u...@thetaphi.de wrote: The JAR file is not Java 5 compatible, so cannot be used with Solr 3.x. - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Apache Jenkins Server [mailto:jenk...@builds.apache.org] Sent: Sunday, October 16, 2011 8:30 AM To: dev@lucene.apache.org Subject: [JENKINS] Solr-3.x - Build # 482 - Failure Build: https://builds.apache.org/job/Solr-3.x/482/ No tests ran. Build Log (for compile errors): [...truncated 10223 lines...] compile-test-framework: common.compile-test: [mkdir] Created dir: /usr/home/hudson/hudson-slave/workspace/Solr- 3.x/checkout/solr/build/contrib/solr-cell/classes/test [javac] Compiling 1 source file to /usr/home/hudson/hudson- slave/workspace/Solr-3.x/checkout/solr/build/contrib/solr-cell/classes/test compile-test: [echo] Building solr-langid... compile-solr-test-framework: compile-solr-core: compile-solrj: check-lucene-core-uptodate: jar-lucene-core: check-analyzers-common-uptodate: jar-analyzers-common: check-spellchecker-uptodate: jar-spellchecker: check-highlighter-uptodate: jar-highlighter: check-memory-uptodate: jar-memory: check-misc-uptodate: jar-misc: check-spatial-uptodate: jar-spatial: check-grouping-uptodate: jar-grouping: check-queries-uptodate: jar-queries: prep-lucene-jars: common.init: compile-lucene-core: jflex-uptodate-check: jflex-notice: javacc-uptodate-check: javacc-notice: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: compile-core: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: [mkdir] Created dir: /usr/home/hudson/hudson-slave/workspace/Solr- 3.x/checkout/solr/build/contrib/solr-langid/classes/java [javac] Compiling 7 source files to /usr/home/hudson/hudson- slave/workspace/Solr-3.x/checkout/solr/build/contrib/solr-langid/classes/java [javac] /usr/home/hudson/hudson-slave/workspace/Solr- 3.x/checkout/solr/contrib/langid/src/java/org/apache/solr/update/processor/L angDetectLanguageIdentifierUpdateProcessor.java:27: cannot access com.cybozu.labs.langdetect.Detector [javac] bad class file: /usr/home/hudson/hudson-slave/workspace/Solr- 3.x/checkout/solr/contrib/langid/lib/langdetect- r111.jar(com/cybozu/labs/langdetect/Detector.class) [javac] class file has wrong version 50.0, should be 49.0 [javac] Please remove or make sure it appears in the correct subdirectory of the classpath. [javac] import com.cybozu.labs.langdetect.Detector; [javac] ^ [javac] 1 error [...truncated 17 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-2841) Scriptable UpdateRequestChain
Scriptable UpdateRequestChain - Key: SOLR-2841 URL: https://issues.apache.org/jira/browse/SOLR-2841 Project: Solr Issue Type: New Feature Components: update Reporter: Jan Høydahl UpdateProcessorChains must currently be defined with XML in solrconfig.xml. We should explore a scriptable chain implementation with a DSL that allows for full flexibility. The first step would be to make UpdateChain implementations pluggable in solrconfig.xml, for backward compat support. Benefits and possibilities with a Scriptable UpdateChain: * A compact DSL for defining Processors and Chains (Workflows would be a better, less limited term here) * Keeping update processor config separate from solrconfig.xml gives better separations of roles * Use this as an opportunity to natively support scripting language Processors (ideas from SOLR-1725) This issue is spun off from SOLR-2823. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2823) Re-use of UpdateProcessor configurations in multiple UpdateChains
[ https://issues.apache.org/jira/browse/SOLR-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13128397#comment-13128397 ] Jan Høydahl commented on SOLR-2823: --- The more I think about the scriptable chain/workflow option, the more I'd like to go that direction and gain full freedom, rather than patch the way of configuring via XML. I've created SOLR-2841 to continue that discussion. This JIRA can then be dedicated to improvements to the current XML config. Re-use of UpdateProcessor configurations in multiple UpdateChains - Key: SOLR-2823 URL: https://issues.apache.org/jira/browse/SOLR-2823 Project: Solr Issue Type: Improvement Components: update Reporter: Jan Høydahl Priority: Minor When dealing with multiple UpdateChains and Processors, you frequently need to re-use configuration. Two chains may be equal except for one config setting in one processor. I propose to allow named processor configs, which can be referenced by name in the chains. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-2842) Re-factor UpdateChain and UpdateProcessor interfaces
Re-factor UpdateChain and UpdateProcessor interfaces Key: SOLR-2842 URL: https://issues.apache.org/jira/browse/SOLR-2842 Project: Solr Issue Type: Improvement Components: update Reporter: Jan Høydahl The UpdateChain's main task is to send SolrInputDocuments through a chain of UpdateRequestProcessors in order to transform them in some way and then (typically) indexing them. This generic pipeline concept would also be useful on the client side (SolrJ), so that we could choose to do parts or all of the processing on the client. The most prominent use case is extracting text (Tika) from large binary documents, residing on local storage on the client(s). Streaming hundreds of Mb over to Solr for processing is not efficcient. See SOLR-1526. We're already implementing Tika as an UpdateProcessor in SOLR-1763, and what would be more natural than reusing this - and any other processor - on the client side? However, for this to be possible, some interfaces need to change slightly.. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-3.x-java7 - Build # 639 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-3.x-java7/639/ No tests ran. Build Log (for compile errors): [...truncated 4352 lines...] init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: compile-core: check-lucene-core-uptodate: jar-lucene-core: check-analyzers-common-uptodate: jar-analyzers-common: check-spellchecker-uptodate: jar-spellchecker: check-highlighter-uptodate: jar-highlighter: check-memory-uptodate: jar-memory: check-misc-uptodate: jar-misc: check-spatial-uptodate: jar-spatial: check-grouping-uptodate: jar-grouping: check-queries-uptodate: jar-queries: prep-lucene-jars: common.init: compile-lucene-core: jflex-uptodate-check: jflex-notice: javacc-uptodate-check: javacc-notice: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: compile-core: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: [mkdir] Created dir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-3.x-java7/checkout/solr/build/contrib/solr-langid/classes/java [javac] Compiling 7 source files to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-3.x-java7/checkout/solr/build/contrib/solr-langid/classes/java [javac] /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-3.x-java7/checkout/solr/contrib/langid/src/java/org/apache/solr/update/processor/LangDetectLanguageIdentifierUpdateProcessorFactory.java:64: method does not override a method from its superclass [javac] @Override [javac]^ [javac] /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-3.x-java7/checkout/solr/contrib/langid/src/java/org/apache/solr/update/processor/TikaLanguageIdentifierUpdateProcessorFactory.java:52: method does not override a method from its superclass [javac] @Override [javac]^ [javac] 2 errors [...truncated 13 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-3.x - Build # 10863 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-3.x/10863/ No tests ran. Build Log (for compile errors): [...truncated 4375 lines...] init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: compile-core: check-lucene-core-uptodate: jar-lucene-core: check-analyzers-common-uptodate: jar-analyzers-common: check-spellchecker-uptodate: jar-spellchecker: check-highlighter-uptodate: jar-highlighter: check-memory-uptodate: jar-memory: check-misc-uptodate: jar-misc: check-spatial-uptodate: jar-spatial: check-grouping-uptodate: jar-grouping: check-queries-uptodate: jar-queries: prep-lucene-jars: common.init: compile-lucene-core: jflex-uptodate-check: jflex-notice: javacc-uptodate-check: javacc-notice: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: compile-core: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: [mkdir] Created dir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-3.x/checkout/solr/build/contrib/solr-langid/classes/java [javac] Compiling 7 source files to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-3.x/checkout/solr/build/contrib/solr-langid/classes/java [javac] /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-3.x/checkout/solr/contrib/langid/src/java/org/apache/solr/update/processor/LangDetectLanguageIdentifierUpdateProcessorFactory.java:64: method does not override a method from its superclass [javac] @Override [javac]^ [javac] /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-3.x/checkout/solr/contrib/langid/src/java/org/apache/solr/update/processor/TikaLanguageIdentifierUpdateProcessorFactory.java:52: method does not override a method from its superclass [javac] @Override [javac]^ [javac] 2 errors [...truncated 13 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS-MAVEN] Lucene-Solr-Maven-3.x #268: POMs out of sync
Build: https://builds.apache.org/job/Lucene-Solr-Maven-3.x/268/ No tests ran. Build Log (for compile errors): [...truncated 17674 lines...] init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: compile-core: check-lucene-core-uptodate: jar-lucene-core: check-analyzers-common-uptodate: jar-analyzers-common: check-spellchecker-uptodate: jar-spellchecker: check-highlighter-uptodate: jar-highlighter: check-memory-uptodate: jar-memory: check-misc-uptodate: jar-misc: check-spatial-uptodate: jar-spatial: check-grouping-uptodate: jar-grouping: check-queries-uptodate: jar-queries: prep-lucene-jars: common.init: compile-lucene-core: jflex-uptodate-check: jflex-notice: javacc-uptodate-check: javacc-notice: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: compile-core: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: [mkdir] Created dir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Maven-3.x/checkout/solr/build/contrib/solr-langid/classes/java [javac] Compiling 7 source files to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Maven-3.x/checkout/solr/build/contrib/solr-langid/classes/java [javac] /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Maven-3.x/checkout/solr/contrib/langid/src/java/org/apache/solr/update/processor/LangDetectLanguageIdentifierUpdateProcessorFactory.java:64: method does not override a method from its superclass [javac] @Override [javac]^ [javac] /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Maven-3.x/checkout/solr/contrib/langid/src/java/org/apache/solr/update/processor/TikaLanguageIdentifierUpdateProcessorFactory.java:52: method does not override a method from its superclass [javac] @Override [javac]^ [javac] 2 errors [...truncated 13 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-3.x - Build # 10864 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-3.x/10864/ No tests ran. Build Log (for compile errors): [...truncated 4247 lines...] init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: compile-core: check-lucene-core-uptodate: jar-lucene-core: check-analyzers-common-uptodate: jar-analyzers-common: check-spellchecker-uptodate: jar-spellchecker: check-highlighter-uptodate: jar-highlighter: check-memory-uptodate: jar-memory: check-misc-uptodate: jar-misc: check-spatial-uptodate: jar-spatial: check-grouping-uptodate: jar-grouping: check-queries-uptodate: jar-queries: prep-lucene-jars: common.init: compile-lucene-core: jflex-uptodate-check: jflex-notice: javacc-uptodate-check: javacc-notice: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: compile-core: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: [mkdir] Created dir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-3.x/checkout/solr/build/contrib/solr-langid/classes/java [javac] Compiling 7 source files to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-3.x/checkout/solr/build/contrib/solr-langid/classes/java [javac] /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-3.x/checkout/solr/contrib/langid/src/java/org/apache/solr/update/processor/LangDetectLanguageIdentifierUpdateProcessorFactory.java:64: method does not override a method from its superclass [javac] @Override [javac]^ [javac] /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-3.x/checkout/solr/contrib/langid/src/java/org/apache/solr/update/processor/TikaLanguageIdentifierUpdateProcessorFactory.java:52: method does not override a method from its superclass [javac] @Override [javac]^ [javac] 2 errors [...truncated 13 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2842) Re-factor UpdateChain and UpdateProcessor interfaces
[ https://issues.apache.org/jira/browse/SOLR-2842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13128405#comment-13128405 ] Yonik Seeley commented on SOLR-2842: With all the libraries, configuration, and everything else one would need in this client, it starts looking very much like a Solr server again! I can even imagine once one has this fat client, that one would want to be able to accept requests from others to get the same processing. It almost seems preferable to just use solr instances as these special tika processors. It might make sense when setting up a cluster to have a bank of solr servers dedicated just to rich document processing, and then they will forward the processed document to the correct shard (assuming new solr cloud stuff). Custom code could somehow live in that bank of indexers to avoid an extra copy of large binary documents, or outside clients could use stream.url to make solr directly stream the large file from the source. Re-factor UpdateChain and UpdateProcessor interfaces Key: SOLR-2842 URL: https://issues.apache.org/jira/browse/SOLR-2842 Project: Solr Issue Type: Improvement Components: update Reporter: Jan Høydahl The UpdateChain's main task is to send SolrInputDocuments through a chain of UpdateRequestProcessors in order to transform them in some way and then (typically) indexing them. This generic pipeline concept would also be useful on the client side (SolrJ), so that we could choose to do parts or all of the processing on the client. The most prominent use case is extracting text (Tika) from large binary documents, residing on local storage on the client(s). Streaming hundreds of Mb over to Solr for processing is not efficcient. See SOLR-1526. We're already implementing Tika as an UpdateProcessor in SOLR-1763, and what would be more natural than reusing this - and any other processor - on the client side? However, for this to be possible, some interfaces need to change slightly.. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-1536) if a filter can support random access API, we should use it
[ https://issues.apache.org/jira/browse/LUCENE-1536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13128406#comment-13128406 ] Uwe Schindler commented on LUCENE-1536: --- Do we have a conclusion about the current patch, so we can commit and work in other issues to improve? I also want to open issues to remove broken SpanFilter from core and move to sandbox. if a filter can support random access API, we should use it --- Key: LUCENE-1536 URL: https://issues.apache.org/jira/browse/LUCENE-1536 Project: Lucene - Java Issue Type: Improvement Components: core/search Affects Versions: 2.4 Reporter: Michael McCandless Assignee: Michael McCandless Priority: Minor Labels: gsoc2011, lucene-gsoc-11, mentor Fix For: 4.0 Attachments: CachedFilterIndexReader.java, LUCENE-1536-rewrite.patch, LUCENE-1536-rewrite.patch, LUCENE-1536-rewrite.patch, LUCENE-1536-rewrite.patch, LUCENE-1536-rewrite.patch, LUCENE-1536-rewrite.patch, LUCENE-1536-rewrite.patch, LUCENE-1536-rewrite.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536_hack.patch, changes-yonik-uwe.patch, luceneutil.patch I ran some performance tests, comparing applying a filter via random-access API instead of current trunk's iterator API. This was inspired by LUCENE-1476, where we realized deletions should really be implemented just like a filter, but then in testing found that switching deletions to iterator was a very sizable performance hit. Some notes on the test: * Index is first 2M docs of Wikipedia. Test machine is Mac OS X 10.5.6, quad core Intel CPU, 6 GB RAM, java 1.6.0_07-b06-153. * I test across multiple queries. 1-X means an OR query, eg 1-4 means 1 OR 2 OR 3 OR 4, whereas +1-4 is an AND query, ie 1 AND 2 AND 3 AND 4. u s means united states (phrase search). * I test with multiple filter densities (0, 1, 2, 5, 10, 25, 75, 90, 95, 98, 99, 99.9 (filter is non-null but all bits are set), 100 (filter=null, control)). * Method high means I use random-access filter API in IndexSearcher's main loop. Method low means I use random-access filter API down in SegmentTermDocs (just like deleted docs today). * Baseline (QPS) is current trunk, where filter is applied as iterator up high (ie in IndexSearcher's search loop). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2839) add alternative language detection impl
[ https://issues.apache.org/jira/browse/SOLR-2839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13128407#comment-13128407 ] Jan Høydahl commented on SOLR-2839: --- Cool. The reasoning behind a list of detected languages was that a more advanced detector could go sentence by sentence and tag multi lingual documents correctly. FAST had that capability. How does this impl compare with the Tika one for short texts? And wouldn't it make more sense to add this on the Tika level letting the detection method be configurable? Then all Tika users would benefit from it. add alternative language detection impl --- Key: SOLR-2839 URL: https://issues.apache.org/jira/browse/SOLR-2839 Project: Solr Issue Type: Improvement Reporter: Robert Muir Assignee: Robert Muir Fix For: 3.5, 4.0 Attachments: SOLR-2839.patch based on http://code.google.com/p/language-detection (apache license), supports 53 languages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2839) add alternative language detection impl
[
https://issues.apache.org/jira/browse/SOLR-2839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13128409#comment-13128409
]
Robert Muir commented on SOLR-2839:
---
{quote}
How does this impl compare with the Tika one for short texts? And wouldn't it
make more sense to add this on the Tika level letting the detection method be
configurable? Then all Tika users would benefit from it.
{quote}
I have no idea, probably not that great? But i didnt compare to tika.
regarding short texts:
http://shuyo.wordpress.com/2011/09/29/langdetect-is-updatedadded-profiles-of-estonian-lithuanian-latvian-slovene-and-so-on/
{quote}
And wouldn't it make more sense to add this on the Tika level letting the
detection method be configurable? Then all Tika users would benefit from it.
{quote}
If someone wants to do this, then we can remove this implementation at that
time. But for lucene/solr, I am able to commit to this project, and I think
that its important for langid to be pluggable to different implementations.
For example, maybe someone ports google's detector
(http://src.chromium.org/viewvc/chrome/trunk/src/third_party/cld/) to java and
we expose that too, which might be interesting for short texts.
add alternative language detection impl
---
Key: SOLR-2839
URL: https://issues.apache.org/jira/browse/SOLR-2839
Project: Solr
Issue Type: Improvement
Reporter: Robert Muir
Assignee: Robert Muir
Fix For: 3.5, 4.0
Attachments: SOLR-2839.patch
based on http://code.google.com/p/language-detection (apache license),
supports 53 languages.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS-MAVEN] Lucene-Solr-Maven-3.x #269: POMs out of sync
Build: https://builds.apache.org/job/Lucene-Solr-Maven-3.x/269/ No tests ran. Build Log (for compile errors): [...truncated 17675 lines...] init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: compile-core: check-lucene-core-uptodate: jar-lucene-core: check-analyzers-common-uptodate: jar-analyzers-common: check-spellchecker-uptodate: jar-spellchecker: check-highlighter-uptodate: jar-highlighter: check-memory-uptodate: jar-memory: check-misc-uptodate: jar-misc: check-spatial-uptodate: jar-spatial: check-grouping-uptodate: jar-grouping: check-queries-uptodate: jar-queries: prep-lucene-jars: common.init: compile-lucene-core: jflex-uptodate-check: jflex-notice: javacc-uptodate-check: javacc-notice: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: compile-core: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: [mkdir] Created dir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Maven-3.x/checkout/solr/build/contrib/solr-langid/classes/java [javac] Compiling 7 source files to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Maven-3.x/checkout/solr/build/contrib/solr-langid/classes/java [javac] /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Maven-3.x/checkout/solr/contrib/langid/src/java/org/apache/solr/update/processor/LangDetectLanguageIdentifierUpdateProcessorFactory.java:64: method does not override a method from its superclass [javac] @Override [javac]^ [javac] /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Maven-3.x/checkout/solr/contrib/langid/src/java/org/apache/solr/update/processor/TikaLanguageIdentifierUpdateProcessorFactory.java:52: method does not override a method from its superclass [javac] @Override [javac]^ [javac] 2 errors [...truncated 13 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2842) Re-factor UpdateChain and UpdateProcessor interfaces
[ https://issues.apache.org/jira/browse/SOLR-2842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13128413#comment-13128413 ] Jan Høydahl commented on SOLR-2842: --- Yep, for distrib cloud stuff it would be cool to be able to have dedicated doc processor nodes. I don't think the client necessarily needs to be THAT fat or complex if this is done right. If we make the UpdateChain and the Processor itself more stand-alone, not depending on SolrCore, and make updateChains easily configurable outside of solrconfig.xml (see SOLR-2841), then it would be straight-forward to instansiate a chain on the client side, without the RunUpdateProcessor of course. Some processors use Schema, so we'd perhaps need a way to fetch the correct schema from the server, using admin/file or even better, ZK. Re-factor UpdateChain and UpdateProcessor interfaces Key: SOLR-2842 URL: https://issues.apache.org/jira/browse/SOLR-2842 Project: Solr Issue Type: Improvement Components: update Reporter: Jan Høydahl The UpdateChain's main task is to send SolrInputDocuments through a chain of UpdateRequestProcessors in order to transform them in some way and then (typically) indexing them. This generic pipeline concept would also be useful on the client side (SolrJ), so that we could choose to do parts or all of the processing on the client. The most prominent use case is extracting text (Tika) from large binary documents, residing on local storage on the client(s). Streaming hundreds of Mb over to Solr for processing is not efficcient. See SOLR-1526. We're already implementing Tika as an UpdateProcessor in SOLR-1763, and what would be more natural than reusing this - and any other processor - on the client side? However, for this to be possible, some interfaces need to change slightly.. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS-MAVEN] Lucene-Solr-Maven-trunk #267: POMs out of sync
Build: https://builds.apache.org/job/Lucene-Solr-Maven-trunk/267/ No tests ran. Build Log (for compile errors): [...truncated 18924 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-3.x - Build # 10865 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-3.x/10865/ No tests ran. Build Log (for compile errors): [...truncated 4248 lines...] init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: compile-core: check-lucene-core-uptodate: jar-lucene-core: check-analyzers-common-uptodate: jar-analyzers-common: check-spellchecker-uptodate: jar-spellchecker: check-highlighter-uptodate: jar-highlighter: check-memory-uptodate: jar-memory: check-misc-uptodate: jar-misc: check-spatial-uptodate: jar-spatial: check-grouping-uptodate: jar-grouping: check-queries-uptodate: jar-queries: prep-lucene-jars: common.init: compile-lucene-core: jflex-uptodate-check: jflex-notice: javacc-uptodate-check: javacc-notice: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: compile-core: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: [mkdir] Created dir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-3.x/checkout/solr/build/contrib/solr-langid/classes/java [javac] Compiling 7 source files to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-3.x/checkout/solr/build/contrib/solr-langid/classes/java [javac] /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-3.x/checkout/solr/contrib/langid/src/java/org/apache/solr/update/processor/LangDetectLanguageIdentifierUpdateProcessorFactory.java:64: method does not override a method from its superclass [javac] @Override [javac]^ [javac] /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-3.x/checkout/solr/contrib/langid/src/java/org/apache/solr/update/processor/TikaLanguageIdentifierUpdateProcessorFactory.java:52: method does not override a method from its superclass [javac] @Override [javac]^ [javac] 2 errors [...truncated 13 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2839) add alternative language detection impl
[ https://issues.apache.org/jira/browse/SOLR-2839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13128414#comment-13128414 ] Jan Høydahl commented on SOLR-2839: --- Sure, it's way better to get stuff done than debate on details :) Great work. Stuff can bubble down to Tika later just has stuff has bubbled down from Solr to Lucene.. add alternative language detection impl --- Key: SOLR-2839 URL: https://issues.apache.org/jira/browse/SOLR-2839 Project: Solr Issue Type: Improvement Reporter: Robert Muir Assignee: Robert Muir Fix For: 3.5, 4.0 Attachments: SOLR-2839.patch based on http://code.google.com/p/language-detection (apache license), supports 53 languages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-3.x-java7 - Build # 640 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-3.x-java7/640/ No tests ran. Build Log (for compile errors): [...truncated 4247 lines...] init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: compile-core: check-lucene-core-uptodate: jar-lucene-core: check-analyzers-common-uptodate: jar-analyzers-common: check-spellchecker-uptodate: jar-spellchecker: check-highlighter-uptodate: jar-highlighter: check-memory-uptodate: jar-memory: check-misc-uptodate: jar-misc: check-spatial-uptodate: jar-spatial: check-grouping-uptodate: jar-grouping: check-queries-uptodate: jar-queries: prep-lucene-jars: common.init: compile-lucene-core: jflex-uptodate-check: jflex-notice: javacc-uptodate-check: javacc-notice: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: compile-core: init: clover.setup: clover.info: [echo] [echo] Clover not found. Code coverage reports disabled. [echo] clover: common.compile-core: [mkdir] Created dir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-3.x-java7/checkout/solr/build/contrib/solr-langid/classes/java [javac] Compiling 7 source files to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-3.x-java7/checkout/solr/build/contrib/solr-langid/classes/java [javac] /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-3.x-java7/checkout/solr/contrib/langid/src/java/org/apache/solr/update/processor/LangDetectLanguageIdentifierUpdateProcessorFactory.java:64: method does not override a method from its superclass [javac] @Override [javac]^ [javac] /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-3.x-java7/checkout/solr/contrib/langid/src/java/org/apache/solr/update/processor/TikaLanguageIdentifierUpdateProcessorFactory.java:52: method does not override a method from its superclass [javac] @Override [javac]^ [javac] 2 errors [...truncated 13 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2839) add alternative language detection impl
[ https://issues.apache.org/jira/browse/SOLR-2839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13128416#comment-13128416 ] Robert Muir commented on SOLR-2839: --- Its not really the same in my opinion. Anyone can commit to both lucene and solr so we can always put things in the correct place. add alternative language detection impl --- Key: SOLR-2839 URL: https://issues.apache.org/jira/browse/SOLR-2839 Project: Solr Issue Type: Improvement Reporter: Robert Muir Assignee: Robert Muir Fix For: 3.5, 4.0 Attachments: SOLR-2839.patch based on http://code.google.com/p/language-detection (apache license), supports 53 languages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-trunk-java7 - Build # 635 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk-java7/635/ 1 tests failed. REGRESSION: org.apache.solr.analysis.TestICUTokenizerFactory.testMixedText Error Message: null Stack Trace: java.lang.ExceptionInInitializerError at org.apache.lucene.analysis.icu.segmentation.DefaultICUTokenizerConfig.clinit(DefaultICUTokenizerConfig.java:67) at org.apache.lucene.analysis.icu.segmentation.ICUTokenizer.init(ICUTokenizer.java:69) at org.apache.solr.analysis.ICUTokenizerFactory.create(ICUTokenizerFactory.java:30) at org.apache.solr.analysis.TestICUTokenizerFactory.testMixedText(TestICUTokenizerFactory.java:30) at org.apache.lucene.util.LuceneTestCase$2$1.evaluate(LuceneTestCase.java:593) at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:149) at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:51) Caused by: java.lang.NullPointerException at com.ibm.icu.impl.ICUResourceBundle.instantiateBundle(ICUResourceBundle.java:840) at com.ibm.icu.impl.ICUResourceBundle.getBundleInstance(ICUResourceBundle.java:815) at com.ibm.icu.util.UResourceBundle.getRootType(UResourceBundle.java:489) at com.ibm.icu.util.UResourceBundle.instantiateBundle(UResourceBundle.java:536) at com.ibm.icu.util.UResourceBundle.getBundleInstance(UResourceBundle.java:144) at com.ibm.icu.util.UResourceBundle.getBundleInstance(UResourceBundle.java:124) at com.ibm.icu.util.ULocale.bcp47ToLDMLKey(ULocale.java:3541) at com.ibm.icu.util.ULocale.access$400(ULocale.java:107) at com.ibm.icu.util.ULocale$JDKLocaleHelper.toULocale7(ULocale.java:3863) at com.ibm.icu.util.ULocale$JDKLocaleHelper.toULocale(ULocale.java:3738) at com.ibm.icu.util.ULocale.forLocale(ULocale.java:414) at com.ibm.icu.util.ULocale.clinit(ULocale.java:531) Build Log (for compile errors): [...truncated 12742 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-tests-only-trunk-java7 - Build # 635 - Failure
Don't tell me the bug isn't actually fixed! On Sun, Oct 16, 2011 at 11:21 AM, Apache Jenkins Server jenk...@builds.apache.org wrote: Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk-java7/635/ 1 tests failed. REGRESSION: org.apache.solr.analysis.TestICUTokenizerFactory.testMixedText Error Message: null Stack Trace: java.lang.ExceptionInInitializerError at org.apache.lucene.analysis.icu.segmentation.DefaultICUTokenizerConfig.clinit(DefaultICUTokenizerConfig.java:67) at org.apache.lucene.analysis.icu.segmentation.ICUTokenizer.init(ICUTokenizer.java:69) at org.apache.solr.analysis.ICUTokenizerFactory.create(ICUTokenizerFactory.java:30) at org.apache.solr.analysis.TestICUTokenizerFactory.testMixedText(TestICUTokenizerFactory.java:30) at org.apache.lucene.util.LuceneTestCase$2$1.evaluate(LuceneTestCase.java:593) at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:149) at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:51) Caused by: java.lang.NullPointerException at com.ibm.icu.impl.ICUResourceBundle.instantiateBundle(ICUResourceBundle.java:840) at com.ibm.icu.impl.ICUResourceBundle.getBundleInstance(ICUResourceBundle.java:815) at com.ibm.icu.util.UResourceBundle.getRootType(UResourceBundle.java:489) at com.ibm.icu.util.UResourceBundle.instantiateBundle(UResourceBundle.java:536) at com.ibm.icu.util.UResourceBundle.getBundleInstance(UResourceBundle.java:144) at com.ibm.icu.util.UResourceBundle.getBundleInstance(UResourceBundle.java:124) at com.ibm.icu.util.ULocale.bcp47ToLDMLKey(ULocale.java:3541) at com.ibm.icu.util.ULocale.access$400(ULocale.java:107) at com.ibm.icu.util.ULocale$JDKLocaleHelper.toULocale7(ULocale.java:3863) at com.ibm.icu.util.ULocale$JDKLocaleHelper.toULocale(ULocale.java:3738) at com.ibm.icu.util.ULocale.forLocale(ULocale.java:414) at com.ibm.icu.util.ULocale.clinit(ULocale.java:531) Build Log (for compile errors): [...truncated 12742 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2842) Re-factor UpdateChain and UpdateProcessor interfaces
[ https://issues.apache.org/jira/browse/SOLR-2842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13128423#comment-13128423 ] Mark Miller commented on SOLR-2842: --- bq. If we make the UpdateChain and the Processor itself more stand-alone, not depending on SolrCore, But the update processor should have access to SolrCore. I don't think this is something we want to drop. You do want access to the Request object and the SolrCore, as you have now. Re-factor UpdateChain and UpdateProcessor interfaces Key: SOLR-2842 URL: https://issues.apache.org/jira/browse/SOLR-2842 Project: Solr Issue Type: Improvement Components: update Reporter: Jan Høydahl The UpdateChain's main task is to send SolrInputDocuments through a chain of UpdateRequestProcessors in order to transform them in some way and then (typically) indexing them. This generic pipeline concept would also be useful on the client side (SolrJ), so that we could choose to do parts or all of the processing on the client. The most prominent use case is extracting text (Tika) from large binary documents, residing on local storage on the client(s). Streaming hundreds of Mb over to Solr for processing is not efficcient. See SOLR-1526. We're already implementing Tika as an UpdateProcessor in SOLR-1763, and what would be more natural than reusing this - and any other processor - on the client side? However, for this to be possible, some interfaces need to change slightly.. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Reopened] (LUCENE-3521) upgrade icu jar to 4.8.1.1 / remove lucenetestcase hack
[ https://issues.apache.org/jira/browse/LUCENE-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir reopened LUCENE-3521: - looks like the bug still exists, so we need the hack. ill see if i can make another test case upgrade icu jar to 4.8.1.1 / remove lucenetestcase hack --- Key: LUCENE-3521 URL: https://issues.apache.org/jira/browse/LUCENE-3521 Project: Lucene - Java Issue Type: Improvement Reporter: Robert Muir Fix For: 3.5, 4.0 Attachments: LUCENE-3521.patch This bug fix release fixes problems with icu under java7: http://bugs.icu-project.org/trac/ticket/8734 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS-MAVEN] Lucene-Solr-Maven-3.x #270: POMs out of sync
Build: https://builds.apache.org/job/Lucene-Solr-Maven-3.x/270/ No tests ran. Build Log (for compile errors): [...truncated 19697 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3521) upgrade icu jar to 4.8.1.1 / remove lucenetestcase hack
[ https://issues.apache.org/jira/browse/LUCENE-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13128427#comment-13128427 ] Robert Muir commented on LUCENE-3521: - my original test case submitted to ICU (http://bugs.icu-project.org/trac/ticket/8734) still fails, so this is still broken. I will add back the hack to lucenetestcase and let them know. upgrade icu jar to 4.8.1.1 / remove lucenetestcase hack --- Key: LUCENE-3521 URL: https://issues.apache.org/jira/browse/LUCENE-3521 Project: Lucene - Java Issue Type: Improvement Reporter: Robert Muir Fix For: 3.5, 4.0 Attachments: LUCENE-3521.patch This bug fix release fixes problems with icu under java7: http://bugs.icu-project.org/trac/ticket/8734 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-trunk - Build # 1700 - Still Failing
I committed a fix (in LUCENE-3520).
Mike McCandless
http://blog.mikemccandless.com
On Sun, Oct 16, 2011 at 2:23 AM, Apache Jenkins Server
jenk...@builds.apache.org wrote:
Build: https://builds.apache.org/job/Lucene-trunk/1700/
1 tests failed.
FAILED: org.apache.lucene.search.TestSearcherManager.testSearcherManager
Error Message:
MockDirectoryWrapper: cannot close: there are still open files: {_2j.cfs=14,
_2a.nrm=1, _2a.fdx=1, _2a.fdt=1, _2a.tvx=1, _2a.tvf=1, _2a.tvd=1, _2l.cfs=17}
Stack Trace:
java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are
still open files: {_2j.cfs=14, _2a.nrm=1, _2a.fdx=1, _2a.fdt=1, _2a.tvx=1,
_2a.tvf=1, _2a.tvd=1, _2l.cfs=17}
at
org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:479)
at
org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase.runTest(ThreadedIndexingAndSearchingTestCase.java:636)
at
org.apache.lucene.search.TestSearcherManager.testSearcherManager(TestSearcherManager.java:48)
at
org.apache.lucene.util.LuceneTestCase$2$1.evaluate(LuceneTestCase.java:593)
at
org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:149)
at
org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:51)
Caused by: java.lang.RuntimeException: unclosed IndexInput: _2j.cfs
at
org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:418)
at
org.apache.lucene.store.MockDirectoryWrapper$1.openSlice(MockDirectoryWrapper.java:693)
at
org.apache.lucene.store.CompoundFileDirectory.openInput(CompoundFileDirectory.java:216)
at
org.apache.lucene.index.codecs.standard.StandardPostingsReader.init(StandardPostingsReader.java:64)
at
org.apache.lucene.index.codecs.mockrandom.MockRandomCodec.fieldsProducer(MockRandomCodec.java:306)
at
org.apache.lucene.index.PerFieldCodecWrapper$FieldsReader.init(PerFieldCodecWrapper.java:114)
at
org.apache.lucene.index.PerFieldCodecWrapper.fieldsProducer(PerFieldCodecWrapper.java:182)
at
org.apache.lucene.index.SegmentCoreReaders.init(SegmentCoreReaders.java:91)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:112)
at
org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:714)
at
org.apache.lucene.index.IndexWriter$ReaderPool.getReadOnlyClone(IndexWriter.java:686)
at
org.apache.lucene.index.IndexWriter$ReaderPool.getReadOnlyClone(IndexWriter.java:677)
at
org.apache.lucene.index.DirectoryReader.init(DirectoryReader.java:172)
at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:378)
at
org.apache.lucene.index.IndexReader.doOpenIfChanged(IndexReader.java:688)
at
org.apache.lucene.index.IndexReader.openIfChanged(IndexReader.java:670)
at
org.apache.lucene.search.SearcherManager$NRTSearcherManager.openIfChanged(SearcherManager.java:295)
at
org.apache.lucene.search.SearcherManager.maybeReopen(SearcherManager.java:106)
at
org.apache.lucene.search.TestSearcherManager$2.run(TestSearcherManager.java:99)
Build Log (for compile errors):
[...truncated 14207 lines...]
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-3.x - Build # 10868 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-3.x/10868/ 1 tests failed. REGRESSION: org.apache.lucene.index.TestIndexWriterExceptions.testRandomExceptionsThreads Error Message: thread Indexer 2: hit unexpected failure Stack Trace: junit.framework.AssertionFailedError: thread Indexer 2: hit unexpected failure at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:147) at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:50) at org.apache.lucene.index.TestIndexWriterExceptions.testRandomExceptionsThreads(TestIndexWriterExceptions.java:237) at org.apache.lucene.util.LuceneTestCase$2$1.evaluate(LuceneTestCase.java:435) Build Log (for compile errors): [...truncated 7784 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-3520) If the NRT reader hasn't changed then IndexReader.openIfChanged should return null
[
https://issues.apache.org/jira/browse/LUCENE-3520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael McCandless resolved LUCENE-3520.
Resolution: Fixed
If the NRT reader hasn't changed then IndexReader.openIfChanged should return
null
--
Key: LUCENE-3520
URL: https://issues.apache.org/jira/browse/LUCENE-3520
Project: Lucene - Java
Issue Type: Bug
Components: core/index
Reporter: Michael McCandless
Assignee: Michael McCandless
Fix For: 3.5, 4.0
Attachments: LUCENE-3520.patch, LUCENE-3520.patch
I hit a failure in TestSearcherManager (NOTE: doesn't always fail):
{noformat}
ant test -Dtestcase=TestSearcherManager -Dtestmethod=testSearcherManager
-Dtests.seed=459ac99a4256789c:-29b8a7f52497c3b4:145ae632ae9e1ecf
{noformat}
It was tripping the assert inside SearcherLifetimeManager.record,
because two different IndexSearcher instances had different IR
instances sharing the same version. This was happening because
IW.getReader always returns a new reader even when there are no
changes. I think we should fix that...
Separately I found a deadlock in
TestSearcherManager.testIntermediateClose, if the test gets
SerialMergeScheduler and needs to merge during the second commit.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-tests-only-3.x - Build # 10868 - Failure
Hmm I can't repro this failure... can you? Mike McCandless http://blog.mikemccandless.com On Sun, Oct 16, 2011 at 1:49 PM, Apache Jenkins Server jenk...@builds.apache.org wrote: Build: https://builds.apache.org/job/Lucene-Solr-tests-only-3.x/10868/ 1 tests failed. REGRESSION: org.apache.lucene.index.TestIndexWriterExceptions.testRandomExceptionsThreads Error Message: thread Indexer 2: hit unexpected failure Stack Trace: junit.framework.AssertionFailedError: thread Indexer 2: hit unexpected failure at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:147) at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:50) at org.apache.lucene.index.TestIndexWriterExceptions.testRandomExceptionsThreads(TestIndexWriterExceptions.java:237) at org.apache.lucene.util.LuceneTestCase$2$1.evaluate(LuceneTestCase.java:435) Build Log (for compile errors): [...truncated 7784 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2842) Re-factor UpdateChain and UpdateProcessor interfaces
[ https://issues.apache.org/jira/browse/SOLR-2842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13128474#comment-13128474 ] Jan Høydahl commented on SOLR-2842: --- bq. But the update processor should have access to SolrCore. I don't think this is something we want to drop. You do want access to the Request object and the SolrCore, as you have now. The UpdateRequestProcessorChain now depends on SolrCore for getting config from solrconfig.xml, so we'd first need to separate updateChain config from solrconfig, e.g. through SOLR-2841 or similar. Although flexible to have full access for the Processors, it doesn't necessarily give the best APIs. Most processors will only need access to the input document and request params. In addition I think schema access for validating input and a resource loader to load own config from file are good candidates for what to provide to Processors. The resource loader on the client side could resolve resources locally, or even through the ZK loader. The remaining 5% of processors which really need SolrCore (such as RunUpdateProcessor) should implement SolrCoreAware, and UpdateChain should statically check and throw an exception if any of these are attempted loaded in a context where SolrCore is null. Re-factor UpdateChain and UpdateProcessor interfaces Key: SOLR-2842 URL: https://issues.apache.org/jira/browse/SOLR-2842 Project: Solr Issue Type: Improvement Components: update Reporter: Jan Høydahl The UpdateChain's main task is to send SolrInputDocuments through a chain of UpdateRequestProcessors in order to transform them in some way and then (typically) indexing them. This generic pipeline concept would also be useful on the client side (SolrJ), so that we could choose to do parts or all of the processing on the client. The most prominent use case is extracting text (Tika) from large binary documents, residing on local storage on the client(s). Streaming hundreds of Mb over to Solr for processing is not efficcient. See SOLR-1526. We're already implementing Tika as an UpdateProcessor in SOLR-1763, and what would be more natural than reusing this - and any other processor - on the client side? However, for this to be possible, some interfaces need to change slightly.. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2839) add alternative language detection impl
[ https://issues.apache.org/jira/browse/SOLR-2839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13128475#comment-13128475 ] Jan Høydahl commented on SOLR-2839: --- I meant to compare with the situation before the Solr+Lucene merge. It takes a whole lot longer time to wait for a dependency to get released before you can consume it, so then it's ok to add it higher up as a first step. add alternative language detection impl --- Key: SOLR-2839 URL: https://issues.apache.org/jira/browse/SOLR-2839 Project: Solr Issue Type: Improvement Reporter: Robert Muir Assignee: Robert Muir Fix For: 3.5, 4.0 Attachments: SOLR-2839.patch based on http://code.google.com/p/language-detection (apache license), supports 53 languages. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
unsubscribe
- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-trunk - Build # 10852 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/10852/
1 tests failed.
REGRESSION: org.apache.lucene.search.TestSearcherManager.testIntermediateClose
Error Message:
MockDirectoryWrapper: cannot close: there are still open files: {_2.fdt=1,
_2.fdx=1}
Stack Trace:
java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still
open files: {_2.fdt=1, _2.fdx=1}
at
org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:479)
at
org.apache.lucene.search.TestSearcherManager.testIntermediateClose(TestSearcherManager.java:248)
at
org.apache.lucene.util.LuceneTestCase$2$1.evaluate(LuceneTestCase.java:610)
at
org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:149)
at
org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:51)
Caused by: java.lang.RuntimeException: unclosed IndexInput: _2.fdt
at
org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:418)
at
org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:434)
at
org.apache.lucene.index.codecs.DefaultFieldsReader.init(DefaultFieldsReader.java:124)
at
org.apache.lucene.index.codecs.CodecProvider.fieldsReader(CodecProvider.java:116)
at
org.apache.lucene.index.SegmentCoreReaders.openDocStores(SegmentCoreReaders.java:168)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:114)
at
org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:714)
at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3779)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3315)
at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:379)
at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:447)
Build Log (for compile errors):
[...truncated 2873 lines...]
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-2711) DIH does not commit after delta-import task
[
https://issues.apache.org/jira/browse/SOLR-2711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexandre Sompheng resolved SOLR-2711.
--
Resolution: Fixed
Fix Version/s: 3.3
This is just crazy !
I finally found why it did not work... And kind of surprised that DIH does not
tell why nothing is updated, but knows that there are modified documents.
Anyway, the issue was located in the DIH config file (data/data-config.xml),
and it was 2 lowercase characters that did not allow DIH to commit.
instead of :
deltaImportQuery=select * from EVENT where ID='${dataimporter.delta.id}'
I wrote :
deltaImportQuery=select * from EVENT where ID='${dataimporter.delta.ID}'
to make it work !
And this is probably related to the column definition ID instead of id:
field column=ID name=id /
Regards,
Alexandre
DIH does not commit after delta-import task
---
Key: SOLR-2711
URL: https://issues.apache.org/jira/browse/SOLR-2711
Project: Solr
Issue Type: Bug
Components: contrib - DataImportHandler
Affects Versions: 3.3
Reporter: Alexandre Sompheng
Fix For: 3.3
I kind of have the same issue as SOLR-2492 but with delta-import that never
commits its updates...
I'm using solr-dataimporthandler-3.3.0.jar.
My log below says that there were 3 updates, but results data did not changed
in refreshed search results :
INFO: [] webapp=/apache-solr-3.3.0 path=/select
params={explainOther=indent=onhl.fl=wt=hl=onversion=2.2rows=100debugQuery=onfl=*,scorestart=0q=*:*qt=fq=}
hits=3 status=0 QTime=0
Aug 14, 2011 1:00:03 AM org.apache.solr.handler.dataimport.DataImporter
doDeltaImport
INFO: Starting Delta Import
Aug 14, 2011 1:00:03 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/apache-solr-3.3.0 path=/select
params={clean=falsecommit=truecommand=delta-importqt=/dataimport} status=0
QTime=0
Aug 14, 2011 1:00:03 AM org.apache.solr.handler.dataimport.SolrWriter
readIndexerProperties
INFO: Read dataimport.properties
Aug 14, 2011 1:00:03 AM org.apache.solr.handler.dataimport.DocBuilder doDelta
INFO: Starting delta collection.
Aug 14, 2011 1:00:03 AM org.apache.solr.handler.dataimport.DocBuilder
collectDelta
INFO: Running ModifiedRowKey() for Entity: event
Aug 14, 2011 1:00:03 AM org.apache.solr.handler.dataimport.JdbcDataSource$1
call
INFO: Creating a connection for entity event with URL:
jdbc:mysql://85.168.123.207:3306/AGENDA
Aug 14, 2011 1:00:04 AM org.apache.solr.handler.dataimport.JdbcDataSource$1
call
INFO: Time taken for getConnection(): 563
Aug 14, 2011 1:00:04 AM org.apache.solr.handler.dataimport.DocBuilder
collectDelta
INFO: Completed ModifiedRowKey for Entity: event rows obtained : 3
Aug 14, 2011 1:00:04 AM org.apache.solr.handler.dataimport.DocBuilder
collectDelta
INFO: Completed DeletedRowKey for Entity: event rows obtained : 0
Aug 14, 2011 1:00:04 AM org.apache.solr.handler.dataimport.DocBuilder
collectDelta
INFO: Completed parentDeltaQuery for Entity: event
Aug 14, 2011 1:00:04 AM org.apache.solr.handler.dataimport.DocBuilder doDelta
INFO: Delta Import completed successfully
Aug 14, 2011 1:00:04 AM org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {} 0 0
Aug 14, 2011 1:00:04 AM org.apache.solr.handler.dataimport.DocBuilder execute
INFO: Time taken = 0:0:0.745
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-3.x - Build # 10874 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-3.x/10874/
1 tests failed.
REGRESSION: org.apache.lucene.search.TestSearcherManager.testIntermediateClose
Error Message:
MockDirectoryWrapper: cannot close: there are still open files: {_0.cfs=1,
_1.cfs=1}
Stack Trace:
java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still
open files: {_0.cfs=1, _1.cfs=1}
at
org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:481)
at
org.apache.lucene.search.TestSearcherManager.testIntermediateClose(TestSearcherManager.java:245)
at
org.apache.lucene.util.LuceneTestCase$2$1.evaluate(LuceneTestCase.java:435)
at
org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:147)
at
org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:50)
Caused by: java.lang.RuntimeException: unclosed IndexInput: _0.cfs
at
org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:420)
at
org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:436)
at org.apache.lucene.store.Directory.openInput(Directory.java:143)
at
org.apache.lucene.index.CompoundFileReader.init(CompoundFileReader.java:64)
at
org.apache.lucene.index.SegmentCoreReaders.init(SegmentCoreReaders.java:68)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:115)
at
org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:710)
at
org.apache.lucene.index.IndexWriter$ReaderPool.getReadOnlyClone(IndexWriter.java:668)
at
org.apache.lucene.index.DirectoryReader.init(DirectoryReader.java:157)
at
org.apache.lucene.index.ReadOnlyDirectoryReader.init(ReadOnlyDirectoryReader.java:38)
at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:458)
at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:406)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:344)
at
org.apache.lucene.search.SearcherManager.init(SearcherManager.java:100)
at
org.apache.lucene.search.TestSearcherManager.testIntermediateClose(TestSearcherManager.java:198)
Build Log (for compile errors):
[...truncated 11331 lines...]
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-tests-only-3.x - Build # 10874 - Failure
I committed a fix!
Mike McCandless
http://blog.mikemccandless.com
On Sun, Oct 16, 2011 at 7:27 PM, Apache Jenkins Server
jenk...@builds.apache.org wrote:
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-3.x/10874/
1 tests failed.
REGRESSION:
org.apache.lucene.search.TestSearcherManager.testIntermediateClose
Error Message:
MockDirectoryWrapper: cannot close: there are still open files: {_0.cfs=1,
_1.cfs=1}
Stack Trace:
java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are
still open files: {_0.cfs=1, _1.cfs=1}
at
org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:481)
at
org.apache.lucene.search.TestSearcherManager.testIntermediateClose(TestSearcherManager.java:245)
at
org.apache.lucene.util.LuceneTestCase$2$1.evaluate(LuceneTestCase.java:435)
at
org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:147)
at
org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:50)
Caused by: java.lang.RuntimeException: unclosed IndexInput: _0.cfs
at
org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:420)
at
org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:436)
at org.apache.lucene.store.Directory.openInput(Directory.java:143)
at
org.apache.lucene.index.CompoundFileReader.init(CompoundFileReader.java:64)
at
org.apache.lucene.index.SegmentCoreReaders.init(SegmentCoreReaders.java:68)
at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:115)
at
org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:710)
at
org.apache.lucene.index.IndexWriter$ReaderPool.getReadOnlyClone(IndexWriter.java:668)
at
org.apache.lucene.index.DirectoryReader.init(DirectoryReader.java:157)
at
org.apache.lucene.index.ReadOnlyDirectoryReader.init(ReadOnlyDirectoryReader.java:38)
at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:458)
at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:406)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:344)
at
org.apache.lucene.search.SearcherManager.init(SearcherManager.java:100)
at
org.apache.lucene.search.TestSearcherManager.testIntermediateClose(TestSearcherManager.java:198)
Build Log (for compile errors):
[...truncated 11331 lines...]
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-trunk-java7 - Build # 640 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk-java7/640/ 1 tests failed. REGRESSION: org.apache.solr.client.solrj.embedded.LargeVolumeBinaryJettyTest.testMultiThreaded Error Message: java.lang.AssertionError: Some threads threw uncaught exceptions! Stack Trace: java.lang.RuntimeException: java.lang.AssertionError: Some threads threw uncaught exceptions! at org.apache.lucene.util.LuceneTestCase.tearDown(LuceneTestCase.java:739) at org.apache.solr.SolrTestCaseJ4.tearDown(SolrTestCaseJ4.java:89) at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:149) at org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:51) at org.apache.lucene.util.LuceneTestCase.checkUncaughtExceptionsAfter(LuceneTestCase.java:767) at org.apache.lucene.util.LuceneTestCase.tearDown(LuceneTestCase.java:711) Build Log (for compile errors): [...truncated 12225 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-3522) TermsFilter.getDocIdSet(context) NPE on missing field
TermsFilter.getDocIdSet(context) NPE on missing field - Key: LUCENE-3522 URL: https://issues.apache.org/jira/browse/LUCENE-3522 Project: Lucene - Java Issue Type: Bug Components: modules/other Affects Versions: 4.0 Reporter: Dan Climan Priority: Minor If the context does not contain the field for a term when calling TermsFilter.getDocIdSet(AtomicReaderContext context) then a NullPointerException is thrown due to not checking for null Terms before getting iterator. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3522) TermsFilter.getDocIdSet(context) NPE on missing field
[ https://issues.apache.org/jira/browse/LUCENE-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Climan updated LUCENE-3522: --- Attachment: LUCENE-3522.patch Proposed patch to TermsFilter and TermsFilterTest showing problem and demonstrating fix TermsFilter.getDocIdSet(context) NPE on missing field - Key: LUCENE-3522 URL: https://issues.apache.org/jira/browse/LUCENE-3522 Project: Lucene - Java Issue Type: Bug Components: modules/other Affects Versions: 4.0 Reporter: Dan Climan Priority: Minor Attachments: LUCENE-3522.patch If the context does not contain the field for a term when calling TermsFilter.getDocIdSet(AtomicReaderContext context) then a NullPointerException is thrown due to not checking for null Terms before getting iterator. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3522) TermsFilter.getDocIdSet(context) NPE on missing field
[ https://issues.apache.org/jira/browse/LUCENE-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Climan updated LUCENE-3522: --- Lucene Fields: New,Patch Available (was: New) TermsFilter.getDocIdSet(context) NPE on missing field - Key: LUCENE-3522 URL: https://issues.apache.org/jira/browse/LUCENE-3522 Project: Lucene - Java Issue Type: Bug Components: modules/other Affects Versions: 4.0 Reporter: Dan Climan Priority: Minor Attachments: LUCENE-3522.patch If the context does not contain the field for a term when calling TermsFilter.getDocIdSet(AtomicReaderContext context) then a NullPointerException is thrown due to not checking for null Terms before getting iterator. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2842) Re-factor UpdateChain and UpdateProcessor interfaces
[ https://issues.apache.org/jira/browse/SOLR-2842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13128682#comment-13128682 ] Chris Male commented on SOLR-2842: -- I really agree with both Yonik's here. I'm very wary of making the client API any more complex. There are many processing pipeline implementations out there, why should ours go client-side? It'd only benefit those using SolrJ and come at the cost of increased complexity. Having to check that the UpdateProcessor is running on on a client or server and then throwing Exceptions in certain circumstances... it all just feels a little messy! The Tika processing situation seems a different problem which Yonik's suggestion seems very reasonable - have a local Solr instance that replicates. I also agree with Mark. We shouldn't strip access to SolrCore, but I think we can reach a middle ground where the UpdateProcessor can define whether it wants a SolrCore reference? Bean setters anybody? The same goes for any Schemas / ResourceLoaders. We should make them all optional but definitely accessible. I don't want to seem like a downer, because I am fully for any refactorings and cleanups of these interfaces where possible. Re-factor UpdateChain and UpdateProcessor interfaces Key: SOLR-2842 URL: https://issues.apache.org/jira/browse/SOLR-2842 Project: Solr Issue Type: Improvement Components: update Reporter: Jan Høydahl The UpdateChain's main task is to send SolrInputDocuments through a chain of UpdateRequestProcessors in order to transform them in some way and then (typically) indexing them. This generic pipeline concept would also be useful on the client side (SolrJ), so that we could choose to do parts or all of the processing on the client. The most prominent use case is extracting text (Tika) from large binary documents, residing on local storage on the client(s). Streaming hundreds of Mb over to Solr for processing is not efficcient. See SOLR-1526. We're already implementing Tika as an UpdateProcessor in SOLR-1763, and what would be more natural than reusing this - and any other processor - on the client side? However, for this to be possible, some interfaces need to change slightly.. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Issue Comment Edited] (SOLR-2842) Re-factor UpdateChain and UpdateProcessor interfaces
[ https://issues.apache.org/jira/browse/SOLR-2842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13128682#comment-13128682 ] Chris Male edited comment on SOLR-2842 at 10/17/11 5:52 AM: I really agree with Yonik here. I'm very wary of making the client API any more complex. There are many processing pipeline implementations out there, why should ours go client-side? It'd only benefit those using SolrJ and come at the cost of increased complexity. Having to check that the UpdateProcessor is running on on a client or server and then throwing Exceptions in certain circumstances... it all just feels a little messy! The Tika processing situation seems a different problem which Yonik's suggestion seems very reasonable - have a local Solr instance that replicates. I also agree with Mark. We shouldn't strip access to SolrCore, but I think we can reach a middle ground where the UpdateProcessor can define whether it wants a SolrCore reference? Bean setters anybody? The same goes for any Schemas / ResourceLoaders. We should make them all optional but definitely accessible. I don't want to seem like a downer, because I am fully for any refactorings and cleanups of these interfaces where possible. was (Author: cmale): I really agree with both Yonik's here. I'm very wary of making the client API any more complex. There are many processing pipeline implementations out there, why should ours go client-side? It'd only benefit those using SolrJ and come at the cost of increased complexity. Having to check that the UpdateProcessor is running on on a client or server and then throwing Exceptions in certain circumstances... it all just feels a little messy! The Tika processing situation seems a different problem which Yonik's suggestion seems very reasonable - have a local Solr instance that replicates. I also agree with Mark. We shouldn't strip access to SolrCore, but I think we can reach a middle ground where the UpdateProcessor can define whether it wants a SolrCore reference? Bean setters anybody? The same goes for any Schemas / ResourceLoaders. We should make them all optional but definitely accessible. I don't want to seem like a downer, because I am fully for any refactorings and cleanups of these interfaces where possible. Re-factor UpdateChain and UpdateProcessor interfaces Key: SOLR-2842 URL: https://issues.apache.org/jira/browse/SOLR-2842 Project: Solr Issue Type: Improvement Components: update Reporter: Jan Høydahl The UpdateChain's main task is to send SolrInputDocuments through a chain of UpdateRequestProcessors in order to transform them in some way and then (typically) indexing them. This generic pipeline concept would also be useful on the client side (SolrJ), so that we could choose to do parts or all of the processing on the client. The most prominent use case is extracting text (Tika) from large binary documents, residing on local storage on the client(s). Streaming hundreds of Mb over to Solr for processing is not efficcient. See SOLR-1526. We're already implementing Tika as an UpdateProcessor in SOLR-1763, and what would be more natural than reusing this - and any other processor - on the client side? However, for this to be possible, some interfaces need to change slightly.. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
