about flexing ranking module in lucene
hi all, In current lucene versions(2.x/3.x) , we can hardly modify the scoring of documents because originally lucene adopt the VSM model and matching phase and ranking phase are integrated. But In many situation, we usually use complicated boolean query to filter out unrelated documents and score them by complicated business logic. http://wiki.apache.org/lucene-java/SummerOfCode2011ProjectRanking seems interesting. what's the status of this branch? will it be included in lucene4 release?
Board Report Sept 2011
hey folks, I just committed a board report draft to our svn. There is still a todo open for PyLucene since I am not on top of the development down there. Andy can you fill out that part please? Thanks, Simon - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-trunk - Build # 10399 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/10399/ No tests ran. Build Log (for compile errors): [...truncated 1189 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-2737) I get ClassCastException when staring tomcat server (Solr configured with Suggester option enabled)
I get ClassCastException when staring tomcat server (Solr configured with
Suggester option enabled)
---
Key: SOLR-2737
URL: https://issues.apache.org/jira/browse/SOLR-2737
Project: Solr
Issue Type: Bug
Components: spellchecker
Affects Versions: 3.3
Environment: Windows 7/Apache Tomcat 7/Solr 3.3/JDK 1.6
Reporter: Saurabh Kumar Singh
I get the following stack trace on server startup:
Aug 30, 2011 6:37:44 PM org.apache.solr.common.SolrException log
SEVERE: java.lang.ClassCastException: java.lang.String cannot be cast to
java.lang.Float
at org.apache.solr.spelling.suggest.Suggester.init(Suggester.java:84)
at
org.apache.solr.handler.component.SpellCheckComponent.inform(SpellCheckComponent.java:597)
at
org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:522)
at org.apache.solr.core.SolrCore.init(SolrCore.java:594)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:463)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:316)
at
org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:133)
at
org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:94)
at
org.apache.catalina.core.ApplicationFilterConfig.initFilter(ApplicationFilterConfig.java:273)
at
org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:254)
at
org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:372)
at
org.apache.catalina.core.ApplicationFilterConfig.init(ApplicationFilterConfig.java:98)
at
org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:4584)
at
org.apache.catalina.core.StandardContext$2.call(StandardContext.java:5262)
at
org.apache.catalina.core.StandardContext$2.call(StandardContext.java:5257)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Below is the patch for the fix (I did it locally. Please let me know the
criteria of how could I check-in the fix :) )
### Eclipse Workspace Patch 1.0
#P lucene_solr_3_3
Index: solr/src/java/org/apache/solr/spelling/suggest/Suggester.java
===
--- solr/src/java/org/apache/solr/spelling/suggest/Suggester.java
(revision 1163940)
+++ solr/src/java/org/apache/solr/spelling/suggest/Suggester.java
(working copy)
@@ -81,8 +81,22 @@
public String init(NamedList config, SolrCore core) {
LOG.info(init: + config);
String name = super.init(config, core);
-threshold = config.get(THRESHOLD_TOKEN_FREQUENCY) == null ? 0.0f
-: (Float)config.get(THRESHOLD_TOKEN_FREQUENCY);
+Object tokenFrequency = config.get(THRESHOLD_TOKEN_FREQUENCY);
+if ( config.get(THRESHOLD_TOKEN_FREQUENCY) == null)
+{
+ threshold = 0.0f;
+}
+else
+{
+ if (tokenFrequency instanceof Number)
+ {
+ threshold = (Float)tokenFrequency;
+ }
+ else if(tokenFrequency instanceof String)
+ {
+ threshold = Float.valueOf((String)tokenFrequency);
+ }
+}
sourceLocation = (String) config.get(LOCATION);
field = (String)config.get(FIELD);
lookupImpl = (String)config.get(LOOKUP_IMPL);
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Commented] (SOLR-2650) Empty docs array on response with grouping and result pagination
It's an svn co. This page has the info...
http://wiki.apache.org/solr/HowToContribute
NOTE: there's been some great work done
to allow you to get this all up and running
in IntelliJ or Eclipse. I'm not familiar
with the Maven stuff, but it's being maintained
too.
Thanks for volunteering!
Best
Erick
On Wed, Aug 31, 2011 at 5:04 PM, Des Lownds (JIRA) j...@apache.org wrote:
[
https://issues.apache.org/jira/browse/SOLR-2650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13094896#comment-13094896
]
Des Lownds commented on SOLR-2650:
--
I'd be happy to test a nightly, where do I download them from? or is it a svn
co?
Empty docs array on response with grouping and result pagination
Key: SOLR-2650
URL: https://issues.apache.org/jira/browse/SOLR-2650
Project: Solr
Issue Type: Bug
Components: search
Affects Versions: 3.3
Reporter: Massimo Schiavon
Attachments: grouping_patch.txt
Requesting a certain number of rows and setting start parameter to a greater
value returns 0 results with grouping enabled.
For example, requesting:
http://localhost:8080/solr/web/select/?q=*:*rows=1start=2
(grouping and highlighting are enabled by default)
I get this response:
[...]
response: {
numFound: 117852
start: 2
docs: [ ]
}
highlighting: {
0938630598: {
title: [ ... ]
content: [ ... ]
}
}
[...]
docs array is empty while the highlighted values of the document are present
Debugging the request in
org.apache.solr.search.Grouping.Command.createSimpleResponse() at row 534
[...]
int len = Math.min(numGroups, docsGathered);
if (offset len) {
len = 0;
}
[...]
The initial vars values are:
numGroups = 1
docsGathered = 3
offset = 2
so after the execution len = 0
I've tried commenting the if statement and this resolves the issue but could
introduce some other bugs.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Created] (SOLR-2737) I get ClassCastException when staring tomcat server (Solr configured with Suggester option enabled)
The way this usually works is that one of the committers
needs to pick this patch up and commit it to the repository.
To facilitate that, could you please attach the patch file
to the JIRA? There should be a more actions and
attach file.
And if you could, please call the file SOLR-2737.patch
Here is the web page for reference:
http://wiki.apache.org/solr/HowToContribute#Generating_a_patch
Thanks much for doing this work!
Best
Erick
On Thu, Sep 1, 2011 at 7:00 AM, Saurabh Kumar Singh (JIRA)
j...@apache.org wrote:
I get ClassCastException when staring tomcat server (Solr configured with
Suggester option enabled)
---
Key: SOLR-2737
URL: https://issues.apache.org/jira/browse/SOLR-2737
Project: Solr
Issue Type: Bug
Components: spellchecker
Affects Versions: 3.3
Environment: Windows 7/Apache Tomcat 7/Solr 3.3/JDK 1.6
Reporter: Saurabh Kumar Singh
I get the following stack trace on server startup:
Aug 30, 2011 6:37:44 PM org.apache.solr.common.SolrException log
SEVERE: java.lang.ClassCastException: java.lang.String cannot be cast to
java.lang.Float
at org.apache.solr.spelling.suggest.Suggester.init(Suggester.java:84)
at
org.apache.solr.handler.component.SpellCheckComponent.inform(SpellCheckComponent.java:597)
at
org.apache.solr.core.SolrResourceLoader.inform(SolrResourceLoader.java:522)
at org.apache.solr.core.SolrCore.init(SolrCore.java:594)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:463)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:316)
at
org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:133)
at
org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:94)
at
org.apache.catalina.core.ApplicationFilterConfig.initFilter(ApplicationFilterConfig.java:273)
at
org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:254)
at
org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:372)
at
org.apache.catalina.core.ApplicationFilterConfig.init(ApplicationFilterConfig.java:98)
at
org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:4584)
at
org.apache.catalina.core.StandardContext$2.call(StandardContext.java:5262)
at
org.apache.catalina.core.StandardContext$2.call(StandardContext.java:5257)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Below is the patch for the fix (I did it locally. Please let me know the
criteria of how could I check-in the fix :) )
### Eclipse Workspace Patch 1.0
#P lucene_solr_3_3
Index: solr/src/java/org/apache/solr/spelling/suggest/Suggester.java
===
--- solr/src/java/org/apache/solr/spelling/suggest/Suggester.java
(revision 1163940)
+++ solr/src/java/org/apache/solr/spelling/suggest/Suggester.java
(working copy)
@@ -81,8 +81,22 @@
public String init(NamedList config, SolrCore core) {
LOG.info(init: + config);
String name = super.init(config, core);
- threshold = config.get(THRESHOLD_TOKEN_FREQUENCY) == null ? 0.0f
- : (Float)config.get(THRESHOLD_TOKEN_FREQUENCY);
+ Object tokenFrequency = config.get(THRESHOLD_TOKEN_FREQUENCY);
+ if ( config.get(THRESHOLD_TOKEN_FREQUENCY) == null)
+ {
+ threshold = 0.0f;
+ }
+ else
+ {
+ if (tokenFrequency instanceof Number)
+ {
+ threshold = (Float)tokenFrequency;
+ }
+ else if(tokenFrequency instanceof String)
+ {
+ threshold = Float.valueOf((String)tokenFrequency);
+ }
+ }
sourceLocation = (String) config.get(LOCATION);
field = (String)config.get(FIELD);
lookupImpl = (String)config.get(LOOKUP_IMPL);
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Website API documentation Not Found
This link appears dead (404 Not found error), Is there anything we should do about it? http://lucene.apache.org/solr/api/index.html Erick - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (LUCENE-2599) Deprecate Spatial Contrib
[ https://issues.apache.org/jira/browse/LUCENE-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Male reassigned LUCENE-2599: -- Assignee: Chris Male Deprecate Spatial Contrib - Key: LUCENE-2599 URL: https://issues.apache.org/jira/browse/LUCENE-2599 Project: Lucene - Java Issue Type: Improvement Components: modules/spatial Affects Versions: 4.0 Reporter: Chris Male Assignee: Chris Male Attachments: LUCENE-2599.patch, LUCENE-2599.patch The spatial contrib is blighted by bugs. The latest series, found by Grant and discussed [here|http://search.lucidimagination.com/search/document/c32e81783642df47/spatial_rethinking_cartesian_tiers_implementation] shows that we need to re-think the cartesian tier implementation. Given the need to create a spatial module containing code taken from both lucene and Solr, it makes sense to deprecate the spatial contrib, and start from scratch in the new module. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2599) Deprecate Spatial Contrib
[ https://issues.apache.org/jira/browse/LUCENE-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095303#comment-13095303 ] Chris Male commented on LUCENE-2599: I'd like to come to an agreement here and finish this off as to make a clear statement about spatial in Lucene 4. It seems we have a few options in front of us now: - Deprecate in 3x, nuke fully in trunk - Deprecate in 3x and in trunk, moving the code to the sandbox - Deprecate in 3x and in trunk and leave the code where it is - Do nothing Robert's concern that removing the contrib entirely without providing an alternative seems very reasonable. But as I said in February, I'm very wary of continuing to release code which has so many problems. As such, I favour deprecating in 3x and in trunk and sandboxing the code wit big warnings that it'll go when we have a better solution. Any opinions? Robert? David? Deprecate Spatial Contrib - Key: LUCENE-2599 URL: https://issues.apache.org/jira/browse/LUCENE-2599 Project: Lucene - Java Issue Type: Improvement Components: modules/spatial Affects Versions: 4.0 Reporter: Chris Male Attachments: LUCENE-2599.patch, LUCENE-2599.patch The spatial contrib is blighted by bugs. The latest series, found by Grant and discussed [here|http://search.lucidimagination.com/search/document/c32e81783642df47/spatial_rethinking_cartesian_tiers_implementation] shows that we need to re-think the cartesian tier implementation. Given the need to create a spatial module containing code taken from both lucene and Solr, it makes sense to deprecate the spatial contrib, and start from scratch in the new module. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2599) Deprecate Spatial Contrib
[ https://issues.apache.org/jira/browse/LUCENE-2599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095309#comment-13095309 ] David Smiley commented on LUCENE-2599: -- Deprecate in 3x, absolutely. We know it's broken and we need to send a message to folks who don't know any better. I hope we can at least get consensus on this part. I don't see any point in this bad code moving forward into Lucene 4, whether there is a replacement or not. Deprecate Spatial Contrib - Key: LUCENE-2599 URL: https://issues.apache.org/jira/browse/LUCENE-2599 Project: Lucene - Java Issue Type: Improvement Components: modules/spatial Affects Versions: 4.0 Reporter: Chris Male Assignee: Chris Male Attachments: LUCENE-2599.patch, LUCENE-2599.patch The spatial contrib is blighted by bugs. The latest series, found by Grant and discussed [here|http://search.lucidimagination.com/search/document/c32e81783642df47/spatial_rethinking_cartesian_tiers_implementation] shows that we need to re-think the cartesian tier implementation. Given the need to create a spatial module containing code taken from both lucene and Solr, it makes sense to deprecate the spatial contrib, and start from scratch in the new module. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2675) Support core properties when creating cores via REST API (CoreAdminHandler)
[ https://issues.apache.org/jira/browse/SOLR-2675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095318#comment-13095318 ] Yury Kats commented on SOLR-2675: - Wiki updated. Support core properties when creating cores via REST API (CoreAdminHandler) --- Key: SOLR-2675 URL: https://issues.apache.org/jira/browse/SOLR-2675 Project: Solr Issue Type: Improvement Components: multicore Reporter: Yury Kats Assignee: Hoss Man Fix For: 3.4, 4.0 Attachments: SOLR-2675.patch, coreadmin.patch When crating cores through solr.xml, I am able to specify custom properties, to be referenced in solrconfig.xml. For example: cores adminPath=/admin/cores defaultCoreName=master core name=master instanceDir=core1 shard=shard1 collection=myconf property name=enable.master value=true / /core core name=slave instanceDir=core2 shard=shard2 collection=myconf property name=enable.slave value=true / property name=masterHost value=node2:8983 / /core /cores There does not seem a way to specify such properties when creating cores through the CoreAdminHandler request. CoreAdminHandler#handleCreateAction always calls dcore.setCoreProperties(null); -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-3411) TestIndexWriterDelete checkIndex failure
TestIndexWriterDelete checkIndex failure Key: LUCENE-3411 URL: https://issues.apache.org/jira/browse/LUCENE-3411 Project: Lucene - Java Issue Type: Bug Reporter: Robert Muir hit this on the flexscoring branch: no indexer code is changed here, but the random seeds won't work in trunk... ill supply a patch with my local mods so we can reproduce. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3411) TestIndexWriterDelete checkIndex failure
[
https://issues.apache.org/jira/browse/LUCENE-3411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095333#comment-13095333
]
Robert Muir commented on LUCENE-3411:
-
{noformat}
[junit] Testsuite: org.apache.lucene.index.TestIndexWriterDelete
[junit] Testcase:
testIndexingThenDeleting(org.apache.lucene.index.TestIndexWriterDelete):
Caused an ERROR
[junit] CheckIndex failed
[junit] java.lang.RuntimeException: CheckIndex failed
[junit] at
org.apache.lucene.util._TestUtil.checkIndex(_TestUtil.java:161)
[junit] at
org.apache.lucene.util._TestUtil.checkIndex(_TestUtil.java:147)
[junit] at
org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:488)
[junit] at
org.apache.lucene.index.TestIndexWriterDelete.testIndexingThenDeleting(TestIndexWriterDelete.java:941)
[junit] at
org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:148)
[junit] at
org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:50)
[junit]
[junit]
[junit] Tests run: 19, Failures: 0, Errors: 1, Time elapsed: 23.776 sec
[junit]
[junit] - Standard Output ---
[junit] CheckIndex failed
[junit] Segments file=segments_1 numSegments=2 version=4.0
format=FORMAT_4_0 [Lucene 4.0]
[junit] 1 of 2: name=_2 docCount=45482
[junit] codec=SegmentCodecs [codecs=[Memory],
provider=RandomCodecProvider: {field=Memory,
content=MockVariableIntBlock(baseBlockSize=20), id=Pulsing(freqCutoff=5
minBlockSize=92 maxBlockSize=263), body=MockSep, contents=MockDocValuesCodec,
value=Standard(minBlockSize=65 maxBlockSize=188), city=MockRandom}]
[junit] compound=true
[junit] hasProx=true
[junit] numFiles=2
[junit] size (MB)=2,256
[junit] diagnostics = {optimize=false, mergeFactor=2,
os.version=2.6.38-10-generic, os=Linux, lucene.version=4.0-SNAPSHOT,
source=merge, os.arch=amd64, java.version=1.6.0_24, java.vendor=Sun
Microsystems Inc.}
[junit] no deletions
[junit] test: open reader.OK
[junit] test: fields..OK [1 fields]
[junit] test: field norms.OK [1 fields]
[junit] test: terms, freq, prox...ERROR: java.lang.RuntimeException:
term [31 39]: doc 45482 = maxDoc 45482
[junit] java.lang.RuntimeException: term [31 39]: doc 45482 = maxDoc 45482
[junit] at
org.apache.lucene.index.CheckIndex.testTermIndex(CheckIndex.java:778)
[junit] at
org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:569)
[junit] at
org.apache.lucene.util._TestUtil.checkIndex(_TestUtil.java:157)
[junit] at
org.apache.lucene.util._TestUtil.checkIndex(_TestUtil.java:147)
[junit] at
org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:488)
[junit] at
org.apache.lucene.index.TestIndexWriterDelete.testIndexingThenDeleting(TestIndexWriterDelete.java:941)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit] at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
[junit] at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
[junit] at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
[junit] at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
[junit] at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48)
[junit] at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
[junit] at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
[junit] at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
[junit] at
org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:148)
[junit] at
org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:50)
[junit] at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
[junit] at
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
[junit] at
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
[junit] at
org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
[junit] at
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
[junit] at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
[junit] at
[jira] [Updated] (LUCENE-3411) TestIndexWriterDelete checkIndex failure
[ https://issues.apache.org/jira/browse/LUCENE-3411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir updated LUCENE-3411: Attachment: LUCENE-3411_localchanges.patch here are my local changes for reference, but i can't even reproduce this one locally TestIndexWriterDelete checkIndex failure Key: LUCENE-3411 URL: https://issues.apache.org/jira/browse/LUCENE-3411 Project: Lucene - Java Issue Type: Bug Reporter: Robert Muir Attachments: LUCENE-3411_localchanges.patch hit this on the flexscoring branch: no indexer code is changed here, but the random seeds won't work in trunk... ill supply a patch with my local mods so we can reproduce. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3178) Native MMapDir
[ https://issues.apache.org/jira/browse/LUCENE-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095341#comment-13095341 ] Varun Thacker commented on LUCENE-3178: --- If we pass down IOContext to NMapIndexInput and in the ctor use mmap and then use madvise with the appropriate flag ( depending on the Context). Is that the correct way to go about it ? Also slightly off topic but the the javadocs for NMapoDir#openInput still shows bufferSize as a parameter. In the java file nothing is specified as the @param values. Where is it coming from? It's probably my mistake from LUCENE-2793 but I would like to correct it here. Native MMapDir -- Key: LUCENE-3178 URL: https://issues.apache.org/jira/browse/LUCENE-3178 Project: Lucene - Java Issue Type: Improvement Components: core/store Reporter: Michael McCandless Spinoff from LUCENE-2793. Just like we will create native Dir impl (UnixDirectory) to pass the right OS level IO flags depending on the IOContext, we could in theory do something similar with MMapDir. The problem is MMap is apparently quite hairy... and to pass the flags the native code would need to invoke mmap (I think?), unlike UnixDir where the code only has to open the file handle. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3178) Native MMapDir
[ https://issues.apache.org/jira/browse/LUCENE-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095349#comment-13095349 ] Simon Willnauer commented on LUCENE-3178: - bq. Also slightly off topic but the the javadocs for NMapoDir#openInput still shows bufferSize as a parameter. In the java file nothing is specified as the @param values. Where is it coming from? It's probably my mistake from LUCENE-2793 but I would like to correct it here. I don't see the @param, which file r u referring to? Native MMapDir -- Key: LUCENE-3178 URL: https://issues.apache.org/jira/browse/LUCENE-3178 Project: Lucene - Java Issue Type: Improvement Components: core/store Reporter: Michael McCandless Spinoff from LUCENE-2793. Just like we will create native Dir impl (UnixDirectory) to pass the right OS level IO flags depending on the IOContext, we could in theory do something similar with MMapDir. The problem is MMap is apparently quite hairy... and to pass the flags the native code would need to invoke mmap (I think?), unlike UnixDir where the code only has to open the file handle. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3178) Native MMapDir
[ https://issues.apache.org/jira/browse/LUCENE-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095356#comment-13095356 ] Varun Thacker commented on LUCENE-3178: --- Just realized what was wrong. I was using the javadocs from version 3.3 on not from the trunk. Stupid mistake on my part. Sorry to bring that up. Native MMapDir -- Key: LUCENE-3178 URL: https://issues.apache.org/jira/browse/LUCENE-3178 Project: Lucene - Java Issue Type: Improvement Components: core/store Reporter: Michael McCandless Spinoff from LUCENE-2793. Just like we will create native Dir impl (UnixDirectory) to pass the right OS level IO flags depending on the IOContext, we could in theory do something similar with MMapDir. The problem is MMap is apparently quite hairy... and to pass the flags the native code would need to invoke mmap (I think?), unlike UnixDir where the code only has to open the file handle. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3390) Incorrect sort by Numeric values for documents missing the sorting field
[ https://issues.apache.org/jira/browse/LUCENE-3390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095409#comment-13095409 ] Doron Cohen commented on LUCENE-3390: - I think it may be useful to solve this also in 3x - without the cached-array-creators of the trunk, but with similar concept - i.e. an additional cache type will cache the docs missing values for certain field, and will allow to use the default value assigned by apps calling setMissingValue() as in trunk. Gilad and I looked at this, will post a patch shortly for review... Incorrect sort by Numeric values for documents missing the sorting field Key: LUCENE-3390 URL: https://issues.apache.org/jira/browse/LUCENE-3390 Project: Lucene - Java Issue Type: Bug Components: core/search Affects Versions: 3.3 Reporter: Gilad Barkai Priority: Minor Labels: double, float, int, long, numeric, sort Attachments: SortByDouble.java While sorting results over a numeric field, documents which do not contain a value for the sorting field seem to get 0 (ZERO) value in the sort. (Tested against Double, Float, Int Long numeric fields ascending and descending order). This behavior is unexpected, as zero is comparable to the rest of the values. A better solution would either be allowing the user to define such a non-value default, or always bring those document results as the last ones. Example scenario: Adding 3 documents, 1st with value 3.5d, 2nd with -10d, and 3rd without any value. Searching with MatchAllDocsQuery, with sort over that field in descending order yields the docid results of 0, 2, 1. Asking for the top 2 documents brings the document without any value as the 2nd result - which seems as a bug? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2312) Search on IndexWriter's RAM Buffer
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095412#comment-13095412 ] Jason Rutherglen commented on LUCENE-2312: -- I'll post a new patch shortly that fixes bugs and adds a bit more to the functionality. The benchmark results are interesting. Array copies are very fast, I don't see any problems with that, the median time is 2 ms. The concurrent skip list map is expensive to add numerous 10s of thousands of terms to. I think that is to be expected. The strategy of amortizing that cost by creating sorted by term int[]s will probably be more performant than CSLM. The sorted int[] terms can be merged just like segments, thus RT becomes a way to remove the [NRT] cost of merging [numerous] postings lists. The int[] terms can be merged in the background so that raw indexing speed is not affected. Search on IndexWriter's RAM Buffer -- Key: LUCENE-2312 URL: https://issues.apache.org/jira/browse/LUCENE-2312 Project: Lucene - Java Issue Type: New Feature Components: core/search Affects Versions: Realtime Branch Reporter: Jason Rutherglen Assignee: Michael Busch Fix For: Realtime Branch Attachments: LUCENE-2312-FC.patch, LUCENE-2312.patch, LUCENE-2312.patch, LUCENE-2312.patch In order to offer user's near realtime search, without incurring an indexing performance penalty, we can implement search on IndexWriter's RAM buffer. This is the buffer that is filled in RAM as documents are indexed. Currently the RAM buffer is flushed to the underlying directory (usually disk) before being made searchable. Todays Lucene based NRT systems must incur the cost of merging segments, which can slow indexing. Michael Busch has good suggestions regarding how to handle deletes using max doc ids. https://issues.apache.org/jira/browse/LUCENE-2293?focusedCommentId=12841923page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12841923 The area that isn't fully fleshed out is the terms dictionary, which needs to be sorted prior to queries executing. Currently IW implements a specialized hash table. Michael B has a suggestion here: https://issues.apache.org/jira/browse/LUCENE-2293?focusedCommentId=12841915page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12841915 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3390) Incorrect sort by Numeric values for documents missing the sorting field
[ https://issues.apache.org/jira/browse/LUCENE-3390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doron Cohen updated LUCENE-3390: Attachment: LUCENE-3390.patch Attached patch fixing this bug. TestSort was enhanced to test the new setMissingValue() method - actually merging the test from trunk r1002460 (LUCENE-2671). All search test passed (running the rest now..) Incorrect sort by Numeric values for documents missing the sorting field Key: LUCENE-3390 URL: https://issues.apache.org/jira/browse/LUCENE-3390 Project: Lucene - Java Issue Type: Bug Components: core/search Affects Versions: 3.3 Reporter: Gilad Barkai Priority: Minor Labels: double, float, int, long, numeric, sort Attachments: LUCENE-3390.patch, SortByDouble.java While sorting results over a numeric field, documents which do not contain a value for the sorting field seem to get 0 (ZERO) value in the sort. (Tested against Double, Float, Int Long numeric fields ascending and descending order). This behavior is unexpected, as zero is comparable to the rest of the values. A better solution would either be allowing the user to define such a non-value default, or always bring those document results as the last ones. Example scenario: Adding 3 documents, 1st with value 3.5d, 2nd with -10d, and 3rd without any value. Searching with MatchAllDocsQuery, with sort over that field in descending order yields the docid results of 0, 2, 1. Asking for the top 2 documents brings the document without any value as the 2nd result - which seems as a bug? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (LUCENE-3390) Incorrect sort by Numeric values for documents missing the sorting field
[ https://issues.apache.org/jira/browse/LUCENE-3390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doron Cohen reassigned LUCENE-3390: --- Assignee: Doron Cohen Incorrect sort by Numeric values for documents missing the sorting field Key: LUCENE-3390 URL: https://issues.apache.org/jira/browse/LUCENE-3390 Project: Lucene - Java Issue Type: Bug Components: core/search Affects Versions: 3.3 Reporter: Gilad Barkai Assignee: Doron Cohen Priority: Minor Labels: double, float, int, long, numeric, sort Attachments: LUCENE-3390.patch, SortByDouble.java While sorting results over a numeric field, documents which do not contain a value for the sorting field seem to get 0 (ZERO) value in the sort. (Tested against Double, Float, Int Long numeric fields ascending and descending order). This behavior is unexpected, as zero is comparable to the rest of the values. A better solution would either be allowing the user to define such a non-value default, or always bring those document results as the last ones. Example scenario: Adding 3 documents, 1st with value 3.5d, 2nd with -10d, and 3rd without any value. Searching with MatchAllDocsQuery, with sort over that field in descending order yields the docid results of 0, 2, 1. Asking for the top 2 documents brings the document without any value as the 2nd result - which seems as a bug? -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Website API documentation Not Found
: This link appears dead (404 Not found error), Is there anything we : should do about it? : : http://lucene.apache.org/solr/api/index.html it looks like this was broken when Jan updated the website the other day ... he moved the old ./solr dir (that had a ./solr/api dir) out of the way to ./solr.old.janhoy and created a new one from SVN (but the api docs aren't in svn). Not really Jan's fault though when you consider that it looks like he didn't have much choice -- the docs on updating the website say you can do an svn up but by the looks of it the dir that was there when he started wasn't actually an svn working copy, see the note he added to the wiki... https://wiki.apache.org/solr/Website_Update_HOWTO ...i havne't really been following the website process that closely -- did it change at some point? is it no longer suppose to be updated manually using svn up? is the wiki out of date? (The wiki also says the docs come from solr trunk, but last i heard it was being published from the 3x branch) I just copied the api dir from ./solr.old.janhoy to ./solr/ so we should have a workarround once the site syncs. A bigger question is short term maintence (until we're ready for the new CMS based site to go live). we now have 2 backups of the solr dir... drwxr-sr-x 5 rmuir lucene 22 Jul 7 18:16 solr.old.rmuir.2 drwxrwsr-x 5 rmuir lucene 22 Jul 28 20:56 solr.old.janhoy drwxrwsr-x 5 janhoy lucene 22 Sep 1 17:06 solr ...neither of which are svn working copies, so it's not clear when/why things were changed so svn up no longer. ./solr.old.janhoy has a mix of files owned by rmuir and uschindler suggesting that rmuir created the dir (when publishing the site as a 3.x probably?) and then uwe updated it (when publishing the java7 warning most likelye) but it's not clear how uwe updated the site if it's not a subversion working copy. -Hoss - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-2675) Support core properties when creating cores via REST API (CoreAdminHandler)
[ https://issues.apache.org/jira/browse/SOLR-2675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hoss Man resolved SOLR-2675. Resolution: Fixed Thanks Yury! Support core properties when creating cores via REST API (CoreAdminHandler) --- Key: SOLR-2675 URL: https://issues.apache.org/jira/browse/SOLR-2675 Project: Solr Issue Type: Improvement Components: multicore Reporter: Yury Kats Assignee: Hoss Man Fix For: 3.4, 4.0 Attachments: SOLR-2675.patch, coreadmin.patch When crating cores through solr.xml, I am able to specify custom properties, to be referenced in solrconfig.xml. For example: cores adminPath=/admin/cores defaultCoreName=master core name=master instanceDir=core1 shard=shard1 collection=myconf property name=enable.master value=true / /core core name=slave instanceDir=core2 shard=shard2 collection=myconf property name=enable.slave value=true / property name=masterHost value=node2:8983 / /core /cores There does not seem a way to specify such properties when creating cores through the CoreAdminHandler request. CoreAdminHandler#handleCreateAction always calls dcore.setCoreProperties(null); -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: Website API documentation Not Found
For me the last time svn up worked (when I posted the big warning message about Java 7). Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Chris Hostetter [mailto:hossman_luc...@fucit.org] Sent: Thursday, September 01, 2011 7:18 PM To: Lucene Dev Subject: Re: Website API documentation Not Found : This link appears dead (404 Not found error), Is there anything we : should do about it? : : http://lucene.apache.org/solr/api/index.html it looks like this was broken when Jan updated the website the other day ... he moved the old ./solr dir (that had a ./solr/api dir) out of the way to ./solr.old.janhoy and created a new one from SVN (but the api docs aren't in svn). Not really Jan's fault though when you consider that it looks like he didn't have much choice -- the docs on updating the website say you can do an svn up but by the looks of it the dir that was there when he started wasn't actually an svn working copy, see the note he added to the wiki... https://wiki.apache.org/solr/Website_Update_HOWTO ...i havne't really been following the website process that closely -- did it change at some point? is it no longer suppose to be updated manually using svn up? is the wiki out of date? (The wiki also says the docs come from solr trunk, but last i heard it was being published from the 3x branch) I just copied the api dir from ./solr.old.janhoy to ./solr/ so we should have a workarround once the site syncs. A bigger question is short term maintence (until we're ready for the new CMS based site to go live). we now have 2 backups of the solr dir... drwxr-sr-x 5 rmuir lucene 22 Jul 7 18:16 solr.old.rmuir.2 drwxrwsr-x 5 rmuir lucene 22 Jul 28 20:56 solr.old.janhoy drwxrwsr-x 5 janhoy lucene 22 Sep 1 17:06 solr ...neither of which are svn working copies, so it's not clear when/why things were changed so svn up no longer. ./solr.old.janhoy has a mix of files owned by rmuir and uschindler suggesting that rmuir created the dir (when publishing the site as a 3.x probably?) and then uwe updated it (when publishing the java7 warning most likelye) but it's not clear how uwe updated the site if it's not a subversion working copy. -Hoss - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-2734) MoreLikeThisHandler does not include debugging info
[
https://issues.apache.org/jira/browse/SOLR-2734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man updated SOLR-2734:
---
Fix Version/s: 3.4
Assignee: Koji Sekiguchi
This looks like a minor mistake introduced when SOLR-860 was backported to 3x
... simple enough to fix i suppose, but what alarms me is that we have a
MoreLikeThisHandlerTest that explicitly tests if it gets debug output when
using the MLT Handler, and it seems to be working on the 3x branch.
I'm hoping Koji can chime in here with some idea as to why the test is working
even though the code is clearly a typo?
MoreLikeThisHandler does not include debugging info
---
Key: SOLR-2734
URL: https://issues.apache.org/jira/browse/SOLR-2734
Project: Solr
Issue Type: Bug
Components: MoreLikeThis
Affects Versions: 3.1, 3.2, 3.3
Environment: All
Reporter: Andrés Cobas
Assignee: Koji Sekiguchi
Fix For: 3.4
Original Estimate: 1h
Remaining Estimate: 1h
After upgrading to solr 3.3.0 from 1.4.0, I noticed that I couldn't get the
MoreLikeThisHandler to return debugging data. I tried the debug parameters
debugQuery and debug, but all I got was:
bool name=debugtrue/bool
I took a look at the code for the MoreLikeThisHandler, and noted in the
debbuging part that the handler is adding the variable dbg to the response
(line 211):
{quote} rsp.add(debug, dbg);{quote}
Such variable is created at line 197:
{quote}boolean dbg = req.getParams().getBool(CommonParams.DEBUG_QUERY,
false);{quote}
I suppose the correct variable to add to the response would be dbgInfo:
{quote}NamedListObject dbgInfo = SolrPluginUtils.doStandardDebug(req, q,
mlt.getRawMLTQuery(), mltDocs.docList);
if (null != dbgInfo) {
if (null != filters) {
dbgInfo.add(filter_queries,req.getParams().getParams(CommonParams.FQ));
ListString fqs = new ArrayListString(filters.size());
for (Query fq : filters) {
fqs.add(QueryParsing.toString(fq, req.getSchema()));
}
dbgInfo.add(parsed_filter_queries,fqs);
}
{quote}
Summarizing, i believe line 211 should be changed to:
{quote}
rsp.add(debug, dbgInfo);
{quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-2585) Context-Sensitive Spelling Suggestions Collations
[ https://issues.apache.org/jira/browse/SOLR-2585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Dyer updated SOLR-2585: - Attachment: SOLR-2585.patch sync with current Trunk. Context-Sensitive Spelling Suggestions Collations --- Key: SOLR-2585 URL: https://issues.apache.org/jira/browse/SOLR-2585 Project: Solr Issue Type: Improvement Components: spellchecker Affects Versions: 4.0 Reporter: James Dyer Priority: Minor Attachments: SOLR-2585.patch, SOLR-2585.patch Solr currently cannot offer what I'm calling here a context-sensitive spelling suggestion. That is, if a user enters one or more words that have docFrequency 0, but nevertheless are misspelled, then no suggestions are offered. Currently, Solr will always consider a word correctly spelled if it is in the index and/or dictionary, regardless of context. This issue patch add support for context-sensitive spelling suggestions. See SpellCheckCollatorTest.testContextSensitiveCollate() for a the typical use case for this functionality. This tests both using IndexBasedSepllChecker and DirectSolrSpellChecker. Two new Spelling Parameters were added: - spellcheck.alternativeTermCount - The count of suggestions to return for each query term existing in the index and/or dictionary. Presumably, users will want fewer suggestions for words with docFrequency0. Also setting this value turns on context-sensitive spell suggestions. - spellcheck.maxResultsForSuggest - The maximum number of hits the request can return in order to both generate spelling suggestions and set the correctlySpelled element to false. For example, if this is set to 5 and the user's query returns 5 or fewer results, the spellchecker will report correctlySpelled=false and also offer suggestions (and collations if requested). Setting this greater than zero is useful for creating did-you-mean suggestions for queries that return a low number of hits. I have also included a test using shards. See additions to DistributedSpellCheckComponentTest. In Lucene, SpellChecker.java can already support this functionality (by passing a null IndexReader and field-name). The DirectSpellChecker, however, needs a minor enhancement. This gives the option to allow DirectSpellChecker to return suggestions for all query terms regardless of frequency. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Board Report Sept 2011
thanks andi! Simon On Thu, Sep 1, 2011 at 8:11 PM, Andi Vajda va...@apache.org wrote: On Thu, 1 Sep 2011, Simon Willnauer wrote: hey folks, I just committed a board report draft to our svn. There is still a todo open for PyLucene since I am not on top of the development down there. Andy can you fill out that part please? Done in rev 1164193. Andi..
[jira] [Created] (SOLR-2738) Allow wt=velocity to work without a VM_global_library.vm requirement
Allow wt=velocity to work without a VM_global_library.vm requirement Key: SOLR-2738 URL: https://issues.apache.org/jira/browse/SOLR-2738 Project: Solr Issue Type: Improvement Reporter: Erik Hatcher Assignee: Erik Hatcher Currently wt=velocity requires a VM_global_library file (by default in /conf/velocity). No reason Velocity really needs this, so we should be able to live fine without it. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-2739) TestSqlEntityProcessorDelta.testNonWritablePersistFile failures on some systems
TestSqlEntityProcessorDelta.testNonWritablePersistFile failures on some systems --- Key: SOLR-2739 URL: https://issues.apache.org/jira/browse/SOLR-2739 Project: Solr Issue Type: Bug Affects Versions: 3.3 Reporter: Shawn Heisey Assignee: Hoss Man Fix For: 3.4, 4.0 Shawn Heisey noted on the mailing list that he was getting consistent failures from TestSqlEntityProcessorDelta.testNonWritablePersistFile on his machine. I can't reproduce his exact failures, but the test is hinky enough that i want to try and clean it up. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2739) TestSqlEntityProcessorDelta.testNonWritablePersistFile failures on some systems
[
https://issues.apache.org/jira/browse/SOLR-2739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095635#comment-13095635
]
Hoss Man commented on SOLR-2739:
Failure Shawn reported seeing consistently with all seeds...
{noformat}
[junit] 08/08/2011 10:05:13 م org.apache.solr.SolrTestCaseJ4 assertQ
[junit] SEVERE: REQUEST FAILED: xpath=//*[@numFound='0']
[junit] xml response was: ?xml version=1.0 encoding=UTF-8?
[junit] response
[junit] lst name=responseHeaderint name=status0/intint
name=QTime1/intlst name=paramsstr name=start0/strstr
name=qid:1/strstr name=qtstandard/strstr name=rows20/strstr
name=version2.2/str/lst/lstresult name=response numFound=1
start=0docarr name=descstrhello/str/arrstr
name=id1/strdate
name=timestamp2011-08-08T19:05:13.353Z/date/doc/result
[junit] /response
[junit]
[junit] request was:start=0q=id:1qt=standardrows=20version=2.2
[junit] 08/08/2011 10:05:13 م org.apache.solr.common.SolrException log
[junit] SEVERE: REQUEST FAILED:
start=0q=id:1qt=standardrows=20version=2.2:java.lang.RuntimeException:
REQUEST FAILED: xpath=//*[@numFound='0']
[junit] xml response was: ?xml version=1.0 encoding=UTF-8?
[junit] response
[junit] lst name=responseHeaderint name=status0/intint
name=QTime1/intlst name=paramsstr name=start0/strstr
name=qid:1/strstr name=qtstandard/strstr name=rows20/strstr
name=version2.2/str/lst/lstresult name=response numFound=1
start=0docarr name=descstrhello/str/arrstr
name=id1/strdate
name=timestamp2011-08-08T19:05:13.353Z/date/doc/result
[junit] /response
[junit]
[junit] request was:start=0q=id:1qt=standardrows=20version=2.2
[junit] at
org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:377)
[junit] at
org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:351)
[junit] at
org.apache.solr.handler.dataimport.TestSqlEntityProcessorDelta.testNonWritablePersistFile(TestSqlEntityProcessorDelta.java:121)
[junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit] at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
[junit] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
[junit] at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
[junit] at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
[junit] at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
[junit] at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48)
[junit] at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
[junit] at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
[junit] at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
[junit] at
org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1339)
[junit] at
org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1241)
[junit] at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
[junit] at
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
[junit] at
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
[junit] at
org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
[junit] at
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
[junit] at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
[junit] at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
[junit] at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
[junit] at
junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
[junit] at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:743)
[junit]
[junit] NOTE: reproduce with: ant test
-Dtestcase=TestSqlEntityProcessorDelta -Dtestmethod=testNonWritablePersistFile
-Dtests.seed=-792d6c110a4b6e96:1c3eeb109a03bb0b:-1ac8e3d1d169d356
[junit] NOTE: test params are: locale=ar_SA, timezone=Asia/Amman
[junit] NOTE: all tests run in this JVM:
[junit] [TestCachedSqlEntityProcessor, TestClobTransformer,
TestContentStreamDataSource, TestDataConfig,
[jira] [Updated] (LUCENE-2312) Search on IndexWriter's RAM Buffer
[ https://issues.apache.org/jira/browse/LUCENE-2312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Rutherglen updated LUCENE-2312: - Fix Version/s: (was: Realtime Branch) Affects Version/s: (was: Realtime Branch) 4.0 Search on IndexWriter's RAM Buffer -- Key: LUCENE-2312 URL: https://issues.apache.org/jira/browse/LUCENE-2312 Project: Lucene - Java Issue Type: New Feature Components: core/search Affects Versions: 4.0 Reporter: Jason Rutherglen Assignee: Michael Busch Attachments: LUCENE-2312-FC.patch, LUCENE-2312.patch, LUCENE-2312.patch, LUCENE-2312.patch In order to offer user's near realtime search, without incurring an indexing performance penalty, we can implement search on IndexWriter's RAM buffer. This is the buffer that is filled in RAM as documents are indexed. Currently the RAM buffer is flushed to the underlying directory (usually disk) before being made searchable. Todays Lucene based NRT systems must incur the cost of merging segments, which can slow indexing. Michael Busch has good suggestions regarding how to handle deletes using max doc ids. https://issues.apache.org/jira/browse/LUCENE-2293?focusedCommentId=12841923page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12841923 The area that isn't fully fleshed out is the terms dictionary, which needs to be sorted prior to queries executing. Currently IW implements a specialized hash table. Michael B has a suggestion here: https://issues.apache.org/jira/browse/LUCENE-2293?focusedCommentId=12841915page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12841915 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2739) TestSqlEntityProcessorDelta.testNonWritablePersistFile failures on some systems
[ https://issues.apache.org/jira/browse/SOLR-2739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095642#comment-13095642 ] Hoss Man commented on SOLR-2739: Committed revision 1164287 - trunk Committed revision 1164289 - 3x This should make the test sane enough that it either works, or correctly skips because of the assumption, on all platforms. leaving open to see if Shawn chimes in with any followup comments TestSqlEntityProcessorDelta.testNonWritablePersistFile failures on some systems --- Key: SOLR-2739 URL: https://issues.apache.org/jira/browse/SOLR-2739 Project: Solr Issue Type: Bug Affects Versions: 3.3 Reporter: Shawn Heisey Assignee: Hoss Man Fix For: 3.4, 4.0 Shawn Heisey noted on the mailing list that he was getting consistent failures from TestSqlEntityProcessorDelta.testNonWritablePersistFile on his machine. I can't reproduce his exact failures, but the test is hinky enough that i want to try and clean it up. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: about flexing ranking module in lucene
On Thu, Sep 1, 2011 at 4:17 AM, Li Li fancye...@gmail.com wrote: hi all, In current lucene versions(2.x/3.x) , we can hardly modify the scoring of documents because originally lucene adopt the VSM model and matching phase and ranking phase are integrated. But In many situation, we usually use complicated boolean query to filter out unrelated documents and score them by complicated business logic. http://wiki.apache.org/lucene-java/SummerOfCode2011ProjectRanking seems interesting. what's the status of this branch? will it be included in lucene4 release? Hi, its very close. there are some nocommits still in the branch right now, once these are fixed we will look at merging to trunk. -- lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2739) TestSqlEntityProcessorDelta.testNonWritablePersistFile failures on some systems
[
https://issues.apache.org/jira/browse/SOLR-2739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095718#comment-13095718
]
Shawn Heisey commented on SOLR-2739:
I continue to get a failure. It looks like it's checking to make sure numFound
is 0, but it's seeing 1, which is what it did before. The exception does look
different, and now it's caused by another exception. Failure text from the new
branch_3x:
{noformat}
[junit] - Standard Error -
[junit] NOTE: reproduce with: ant test
-Dtestcase=TestSqlEntityProcessorDelta -Dtestmethod=testNonWritablePersistFile
-Dtests.seed=-5706afed1ddb3d06:65ec54c2cd2efa63:3ba1f077e387d65
[junit] NOTE: test params are: locale=en_IE, timezone=Europe/Berlin
[junit] NOTE: all tests run in this JVM:
[junit] [TestCachedSqlEntityProcessor, TestClobTransformer,
TestContentStreamDataSource, TestDataConfig, TestDateFormatTransformer,
TestDocBuilder, TestDocBuilder2, TestDocBuilderThreaded,
TestEntityProcessorBase, TestErrorHandling, TestEvaluatorBag, TestFieldReader,
TestFileListEntityProcessor, TestJdbcDataSource, TestLineEntityProcessor,
TestNumberFormatTransformer, TestPlainTextEntityProcessor,
TestRegexTransformer, TestScriptTransformer, TestSqlEntityProcessor,
TestSqlEntityProcessor2, TestSqlEntityProcessorDelta]
[junit] NOTE: Linux 2.6.18-238.12.1.el5.centos.plusxen amd64/Sun
Microsystems Inc. 1.6.0_26
(64-bit)/cpus=3,threads=4,free=193825632,total=247136256
[junit] - ---
[junit] Testcase:
testNonWritablePersistFile(org.apache.solr.handler.dataimport.TestSqlEntityProcessorDelta):
Caused an ERROR
[junit] Exception during query
[junit] java.lang.RuntimeException: Exception during query
[junit] at
org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:385)
[junit] at
org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:352)
[junit] at
org.apache.solr.handler.dataimport.TestSqlEntityProcessorDelta.testNonWritablePersistFile(TestSqlEntityProcessorDelta.java:124)
[junit] at
org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:147)
[junit] at
org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:50)
[junit] Caused by: java.lang.RuntimeException: REQUEST FAILED:
xpath=//*[@numFound='0']
[junit] xml response was: ?xml version=1.0 encoding=UTF-8?
[junit] response
[junit] lst name=responseHeaderint name=status0/intint
name=QTime0/intlst name=paramsstr name=start0/strstr
name=qid:1/strstr name=qtstandard/strstr name=rows20/strstr
name=version2.2/str/lst/lstresult name=response numFound=1
start=0docarr name=descstrhello/str/arrstr
name=id1/strdate
name=timestamp2011-09-02T01:45:21.014Z/date/doc/result
[junit] /response
[junit]
[junit] request was:start=0q=id:1qt=standardrows=20version=2.2
[junit] at
org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:378)
{noformat}
TestSqlEntityProcessorDelta.testNonWritablePersistFile failures on some
systems
---
Key: SOLR-2739
URL: https://issues.apache.org/jira/browse/SOLR-2739
Project: Solr
Issue Type: Bug
Affects Versions: 3.3
Reporter: Shawn Heisey
Assignee: Hoss Man
Fix For: 3.4, 4.0
Shawn Heisey noted on the mailing list that he was getting consistent
failures from TestSqlEntityProcessorDelta.testNonWritablePersistFile on his
machine.
I can't reproduce his exact failures, but the test is hinky enough that i
want to try and clean it up.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-2740) debug info test in MLT handler should be moved to MLT component
debug info test in MLT handler should be moved to MLT component --- Key: SOLR-2740 URL: https://issues.apache.org/jira/browse/SOLR-2740 Project: Solr Issue Type: Test Components: MoreLikeThis, SearchComponents - other Affects Versions: 3.3, 3.2, 3.1, 4.0 Reporter: Koji Sekiguchi Priority: Minor In SOLR-860, I wrote a test for debug info that is produced by MLT *component*. But I put it in MLT *handler* test case. As it confuses them, it should be moved to MLT component test case. Also, adding a test for debug info (for handler) in handler test case is nice to have. It avoids introducing regression like that was reported in SOLR-2734. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-2734) MoreLikeThisHandler does not include debugging info
[
https://issues.apache.org/jira/browse/SOLR-2734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Koji Sekiguchi resolved SOLR-2734.
--
Resolution: Fixed
Thanks Andrés and Hoss!
MoreLikeThisHandler does not include debugging info
---
Key: SOLR-2734
URL: https://issues.apache.org/jira/browse/SOLR-2734
Project: Solr
Issue Type: Bug
Components: MoreLikeThis
Affects Versions: 3.1, 3.2, 3.3
Environment: All
Reporter: Andrés Cobas
Assignee: Koji Sekiguchi
Fix For: 3.4
Original Estimate: 1h
Remaining Estimate: 1h
After upgrading to solr 3.3.0 from 1.4.0, I noticed that I couldn't get the
MoreLikeThisHandler to return debugging data. I tried the debug parameters
debugQuery and debug, but all I got was:
bool name=debugtrue/bool
I took a look at the code for the MoreLikeThisHandler, and noted in the
debbuging part that the handler is adding the variable dbg to the response
(line 211):
{quote} rsp.add(debug, dbg);{quote}
Such variable is created at line 197:
{quote}boolean dbg = req.getParams().getBool(CommonParams.DEBUG_QUERY,
false);{quote}
I suppose the correct variable to add to the response would be dbgInfo:
{quote}NamedListObject dbgInfo = SolrPluginUtils.doStandardDebug(req, q,
mlt.getRawMLTQuery(), mltDocs.docList);
if (null != dbgInfo) {
if (null != filters) {
dbgInfo.add(filter_queries,req.getParams().getParams(CommonParams.FQ));
ListString fqs = new ArrayListString(filters.size());
for (Query fq : filters) {
fqs.add(QueryParsing.toString(fq, req.getSchema()));
}
dbgInfo.add(parsed_filter_queries,fqs);
}
{quote}
Summarizing, i believe line 211 should be changed to:
{quote}
rsp.add(debug, dbgInfo);
{quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2734) MoreLikeThisHandler does not include debugging info
[
https://issues.apache.org/jira/browse/SOLR-2734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13095725#comment-13095725
]
Koji Sekiguchi commented on SOLR-2734:
--
I committed the Andrés's fix in r1164331. Thanks Andrés!
bq. what alarms me is that we have a MoreLikeThisHandlerTest that explicitly
tests if it gets debug output when using the MLT Handler, and it seems to be
working on the 3x branch.
The test was also introduced in SOLR-860, but it confuses them. I've opened
SOLR-2740.
MoreLikeThisHandler does not include debugging info
---
Key: SOLR-2734
URL: https://issues.apache.org/jira/browse/SOLR-2734
Project: Solr
Issue Type: Bug
Components: MoreLikeThis
Affects Versions: 3.1, 3.2, 3.3
Environment: All
Reporter: Andrés Cobas
Assignee: Koji Sekiguchi
Fix For: 3.4
Original Estimate: 1h
Remaining Estimate: 1h
After upgrading to solr 3.3.0 from 1.4.0, I noticed that I couldn't get the
MoreLikeThisHandler to return debugging data. I tried the debug parameters
debugQuery and debug, but all I got was:
bool name=debugtrue/bool
I took a look at the code for the MoreLikeThisHandler, and noted in the
debbuging part that the handler is adding the variable dbg to the response
(line 211):
{quote} rsp.add(debug, dbg);{quote}
Such variable is created at line 197:
{quote}boolean dbg = req.getParams().getBool(CommonParams.DEBUG_QUERY,
false);{quote}
I suppose the correct variable to add to the response would be dbgInfo:
{quote}NamedListObject dbgInfo = SolrPluginUtils.doStandardDebug(req, q,
mlt.getRawMLTQuery(), mltDocs.docList);
if (null != dbgInfo) {
if (null != filters) {
dbgInfo.add(filter_queries,req.getParams().getParams(CommonParams.FQ));
ListString fqs = new ArrayListString(filters.size());
for (Query fq : filters) {
fqs.add(QueryParsing.toString(fq, req.getSchema()));
}
dbgInfo.add(parsed_filter_queries,fqs);
}
{quote}
Summarizing, i believe line 211 should be changed to:
{quote}
rsp.add(debug, dbgInfo);
{quote}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3199) Add non-desctructive sort to BytesRefHash
[ https://issues.apache.org/jira/browse/LUCENE-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Rutherglen updated LUCENE-3199: - Attachment: LUCENE-3199.patch Here's a version of this issue. Added are a couple of new methods to TestBytesRefHash to test the new frozen compact and sorting functionality of BytesRefHash. This is being posted now because it's useful in relation to LUCENE-2312 and a terms dictionary that is composed of sorted by term[id]s int[]s. Add non-desctructive sort to BytesRefHash - Key: LUCENE-3199 URL: https://issues.apache.org/jira/browse/LUCENE-3199 Project: Lucene - Java Issue Type: Improvement Components: core/index Affects Versions: 4.0 Reporter: Jason Rutherglen Priority: Minor Attachments: LUCENE-3199.patch Currently the BytesRefHash is destructive. We can add a method that returns a non-destructively generated int[]. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
