Re: SIGSEGV in JCCEnv::setClassPath

2011-10-25 Thread Andi Vajda


 Hi,

On Tue, 25 Oct 2011, Stein, Ruben wrote:


I am using the ReviewBoard software, which internally uses PyLucene for
its search function. Almost every time I use the search functionality
however, I get a segmentation fault, which gets logged by apache:

# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x7fab1f777874, pid=16468, tid=140373040924416
#
# JRE version: 6.0_26-b03
# Java VM: Java HotSpot(TM) 64-Bit Server VM (20.1-b02 mixed mode
linux-amd64 compressed oops)
# Problematic frame:
# C  [libjcc.so+0x7874]  JCCEnv::setClassPath(char const*)+0x24

I recently built the current PyLucene (3.4.0-1) manually and swapped the
open-jdk with sun's jdk without any better result. As for other users
everything works fine, I guess the error might be hidden in my setup. The
reason I ask this here (also posted to ReviewBoard list), is that they had
no solution for a similar error and suggested posting here. The rest of
the apache's error log is listed below.


The usual problem with embedding a Python extension built with JCC in an 
Apache process - which I guess is what you are doing - is that the process 
and threads where JCC is used in not properly initialized:

 - initVM() must be called from the main thread before any Java call can be
   made.
 - other threads must call attachCurrentThread() before they can make any
   Java call.

Andi..



I would be grateful for any pointer why this is crashing.

Regards, Ruben

Pylucene: 3.4.0-1 (also tried 2.3.1-1.1u from aptitude)
Python: 2.6.5
Apache: 2.2.14-5ub
OS: Ubuntu 10.04.3 LTS
ReviewBoard: 1.6.1



---  T H R E A D  ---

Current thread is native thread

siginfo:si_signo=SIGSEGV: si_errno=0, si_code=1 (SEGV_MAPERR),
si_addr=0x

Registers:
RAX=0x, RBX=0x, RCX=0x7fab1f77c12b,
RDX=0x7fab253d8a40
RSP=0x7fab253d5a60, RBP=0x031351a0, RSI=0x037e57e4,
RDI=0x0003
R8 =0x037e57e4, R9 =0x0200, R10=0x,
R11=0x02084ef0
R12=0x, R13=0x03389d29, R14=0x,
R15=0x
RIP=0x7fab1f777874, EFLAGS=0x00010246,
CSGSFS=0x0033, ERR=0x0004
 TRAPNO=0x000e

Top of Stack: (sp=0x7fab253d5a60)
0x7fab253d5a60:   02f5b840 037e57e4
0x7fab253d5a70:   0029 7fab1f97f640
0x7fab253d5a80:   031351a0 
0x7fab253d5a90:   03389d29 
0x7fab253d5aa0:    7fab1f7768c8
0x7fab253d5ab0:   7fab253d5d18 7fab253d5d10
0x7fab253d5ac0:   7fab253d5d08 0055497f
0x7fab253d5ad0:   00554958 7fab253d5b80
0x7fab253d5ae0:   0029 0055497f
0x7fab253d5af0:   7fab253d5b80 004661f5
0x7fab253d5b00:   68a5be142498d9cb 0190
0x7fab253d5b10:   00554980 7fab253d5c80
0x7fab253d5b20:   00300010 7fab253d5c80
0x7fab253d5b30:   7fab253d5ba0 02fbe640
0x7fab253d5b40:   007f 0081f560
0x7fab253d5b50:    0082af20
0x7fab253d5b60:   7fab276add50 
0x7fab253d5b70:    004b5d62
0x7fab253d5b80:   00300020 7fab253d5c80
0x7fab253d5b90:   7fab253d5ba0 02085878
0x7fab253d5ba0:   02f13830 01e88510
0x7fab253d5bb0:   00550f74 7fab27765ad4
0x7fab253d5bc0:   7fab253d59d0 0200
0x7fab253d5bd0:   7fab2778e0e0 004c4e41
0x7fab253d5be0:   03389c8e 0030
0x7fab253d5bf0:    7fab276add74
0x7fab253d5c00:   7fab253d5c48 004c51b2
0x7fab253d5c10:   000a 7fab276add74
0x7fab253d5c20:   7fab253d5ca8 7fab253d86a8
0x7fab253d5c30:    0043d8db
0x7fab253d5c40:   7fab253d5d20 7fab276add75
0x7fab253d5c50:   7fab276add50 7fab276add74

Instructions: (pc=0x7fab1f777874)
0x7fab1f777854:   41 55 41 54 55 53 48 83 ec 18 48 8b 05 33 77 20
0x7fab1f777864:   00 48 89 74 24 08 8b 38 e8 97 de ff ff 48 89 c3
0x7fab1f777874:   48 8b 00 48 8d 35 52 4a 00 00 48 89 df ff 50 30
0x7fab1f777884:   49 89 c7 48 8b 03 48 8d 35 ca 4a 00 00 48 89 df

Register to memory mapping:

RAX=0x is an unknown value
RBX=0x is an unknown value
RCX=0x7fab1f77c12b: offset 0xc12b in
/usr/local/lib/python2.6/dist-packages/JCC-2.11-py2.6-linux-x86_64.egg/libj
cc.so at 0x7fab1f77
RDX=0x7fab253d8a40 is an unknown value
RSP=0x7fab253d5a60 is an unknown value
RBP=0x031351a0 is an unknown value
RSI=0x037e57e4 is an unknown value
RDI=0x0003 is an unknown value
R8 =0x037e57e4 is an unknown value
R9 

Re: Request for Feedback for Patch to Allow DIH to Archive Files

2011-10-25 Thread Martijn v Groningen
Hi Josh,

I think this functionality is useful. I'd create an Jira issue and
attach your code as a patch. I think that the functionality should be
added to the FileListEntityProcessor since it seems to be a more
natural place for it. Maybe we need something more generic, like a
post action if a file has been processed.

Martijn

On 24 October 2011 21:31, Josh Harness josh.harn...@jtv.com wrote:
 Hi -

  We are using SOLR to process XML input files using the Data Import
 Handler. I didn't see a way to move the xml files out of the way after
 processing, so I wrote a small extension to allow this. The How to
 Contribute page says to pitch the request to the developer list in order to
 decide whether or not to submit a patch. As such, here goes:

  The new code basically extends FileDataSource and wraps the underlying
 reader such that when the close method on the input stream is called, the
 file is moved to a configurable archive directory. It is unclear to me
 whether this is the correct place to put it (I pondered changing the
 FileListEntityProcessor but this somehow felt safer). I realize that a more
 robust implementation would consider the success status of the file being
 processed and would also allow for configurable policies rather than a
 concrete implementation. Nonetheless, I didn't want the perfect to be the
 enemy of the good.

  Please peruse the attached source code file and provide feedback as to
 the merit of the idea, whether I ought to submit a JIRA ticket/patch and if
 my approach is correct.

 Thanks!

 Josh Harness


 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org




-- 
Met vriendelijke groet,

Martijn van Groningen

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



Re: Patch submission for DataImportHandler's FileListEntityProcessor to sort files

2011-10-25 Thread Martijn v Groningen
Hi Gabriel,

I'm not an expert FileEntityProcessor user, but I'd expect a
consistent process order. Your code seems kosher to me. You use the
last modified date as order, which seems ok to me. So create a Jira
issue and attach your patch!

Martijn

On 24 October 2011 21:49, Gabriel Cooper gabriel.coo...@jtv.com wrote:
 Hello,

 I noticed what appears to be a bug in DataImportHandler's
 FileListEntityProcessor. Specifically, it relies on Java's File.list()
 method to retrieve a list of files from the configured dataimport directory,
 but list() does not guarantee a sort order. This means that if you have two
 files that update the same record, the results are non-deterministic.
 Typically, list() does in fact return them lexigraphically sorted, but this
 is not guaranteed.

 An example of how you can get into trouble is to imagine the following:

 xyz.xml -- Created one hour ago. Contains updates to records Foo and
 Bar.
 abc.xml -- Created one minute ago. Contains updates to records Bar and
 Baz.

 In this case, the newest file, in abc.xml, would (likely, but not
 guaranteed) be run first, updating the Bar and Baz records. Next, the
 older file, xyz.xml, would update Foo and overwrite Bar with outdated
 changes.

 The HowToContribute wiki page suggested I send my request here before
 opening an actual bug ticket, so please let me know if there's anything else
 I can or should do to get this patch submitted and approved. I've attached a
 patch of FileListEntityProcessor, along with an updated test, please let me
 know if it's kosher.

 Thank you,

 Gabriel.


 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org




-- 
Met vriendelijke groet,

Martijn van Groningen

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-3508) Decompounders based on CompoundWordTokenFilterBase cannot be used with custom attributes

2011-10-25 Thread Uwe Schindler (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134786#comment-13134786
 ] 

Uwe Schindler commented on LUCENE-3508:
---

Robert: I agree. I think, I will do this in a separate issue, to get those 
changes first into 3.x (by merging) and then deprecate/remove all this later. 
Do we have other StopFilters and similar classes that have those String[] ctors?

I will also add a javadocs info to the ctor: This filter does many lookups on 
the dictionary, so it is recommended for optimal performance to add a 
LowercaseFilter *before* this filter and make the CharArraySet passed as 
dictionary case insensitive.

I would like to add this as a hint, as this filter does per term lots of 
lookups in the Set.

 Decompounders based on CompoundWordTokenFilterBase cannot be used with custom 
 attributes
 

 Key: LUCENE-3508
 URL: https://issues.apache.org/jira/browse/LUCENE-3508
 Project: Lucene - Java
  Issue Type: Bug
  Components: modules/analysis
Affects Versions: 3.4, 4.0
Reporter: Spyros Kapnissis
Assignee: Uwe Schindler
 Fix For: 3.5, 4.0

 Attachments: LUCENE-3508.patch, LUCENE-3508.patch, LUCENE-3508.patch, 
 LUCENE-3508.patch


 The CompoundWordTokenFilterBase.setToken method will call clearAttributes() 
 and then will reset only the default Token attributes (term, position, flags, 
 etc) resulting in any custom attributes losing their value. Commenting out 
 clearAttributes() seems to do the trick, but will fail the 
 TestCompoundWordTokenFilter tests..

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-2804) Logging error causes entire DIH process to fail

2011-10-25 Thread Adam Neal (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-2804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134800#comment-13134800
 ] 

Adam Neal commented on SOLR-2804:
-

Just re-read my comment, I mean the threads attribute not maxthreads

 Logging error causes entire DIH process to fail
 ---

 Key: SOLR-2804
 URL: https://issues.apache.org/jira/browse/SOLR-2804
 Project: Solr
  Issue Type: Bug
  Components: contrib - DataImportHandler
Affects Versions: 4.0
 Environment: java version 1.6.0_26
 Java(TM) SE Runtime Environment (build 1.6.0_26-b03-384-10M3425)
 Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02-384, mixed mode)
 Model Name:   MacBook Pro
   Model Identifier:   MacBookPro8,2
   Processor Name: Intel Core i7
   Processor Speed:2.2 GHz
   Number of Processors:   1
   Total Number of Cores:  4
   L2 Cache (per Core):256 KB
   L3 Cache:   6 MB
   Memory: 4 GB
 System Software Overview:
   System Version: Mac OS X 10.6.8 (10K549)
   Kernel Version: Darwin 10.8.0
Reporter: Pulkit Singhal
  Labels: dih
   Original Estimate: 48h
  Remaining Estimate: 48h

 SEVERE: Full Import failed:java.lang.ClassCastException:
 java.util.ArrayList cannot be cast to java.lang.String
at org.apache.solr.common.util.NamedList.getName(NamedList.java:127)
at org.apache.solr.common.util.NamedList.toString(NamedList.java:263)
at java.lang.String.valueOf(String.java:2826)
at java.lang.StringBuilder.append(StringBuilder.java:115)
at 
 org.apache.solr.update.processor.LogUpdateProcessor.finish(LogUpdateProcessorFactory.java:188)
at 
 org.apache.solr.handler.dataimport.SolrWriter.close(SolrWriter.java:57)
at 
 org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:265)
at 
 org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:372)
at 
 org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:440)
at 
 org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:421)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



unsubscribe

2011-10-25 Thread Marc Tinnemeyer

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



Re: unsubscribe

2011-10-25 Thread Simon Willnauer
Hey Marc,

to unsubscribe send a mail to dev-unsubscr...@lucene.apache.org from
the mail address you used to subscribe to the list.

thanks,

Simon
On Tue, Oct 25, 2011 at 9:32 AM, Marc Tinnemeyer marc-...@gmx.net wrote:

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-3339) TestNRTThreads hangs in nightly 3.x builds

2011-10-25 Thread Commented

[ 
https://issues.apache.org/jira/browse/LUCENE-3339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134870#comment-13134870
 ] 

Olve Sæther Hansen commented on LUCENE-3339:


I am testing Lucene for my project now, and it seems that I run into this (or 
something like this) in 3.3.0. I try to index three different sources 
concurrently and all three processes get stuck here.

The indexation happens as cause of a scheduled event, and at startup time I try 
to debug and since my breakpoints synchronize the events, they continue in 
perfect sync om my four cores. 
I do not see that issue when I startup with my breakpoints disabled.

The stacktrace for my three processes are exactly the same. Is there a 
workaround for this issue or do I have to wait for 3.5 (that is if it is the 
same problem)?
{code}
minokWorkerExecutor-1@5444, prio=5, in group 'main', status: 'waiting'
  java.lang.Thread.State: WAITING
  at java.lang.Object.wait(Object.java:-1)
  at java.lang.Object.wait(Object.java:485)
  at 
org.apache.lucene.index.DocumentsWriter.waitForWaitQueue(DocumentsWriter.java:1057)
  at 
org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:875)
  at 
org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:2163)
  at 
org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:2121)
  at 
org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:2105)
{code}

 TestNRTThreads hangs in nightly 3.x builds
 --

 Key: LUCENE-3339
 URL: https://issues.apache.org/jira/browse/LUCENE-3339
 Project: Lucene - Java
  Issue Type: Bug
Reporter: Robert Muir
Assignee: Michael McCandless
 Fix For: 3.5

 Attachments: LUCENE-3339.patch


 Maybe we have a problem, maybe its a bug in the test.
 But its strange that lately the 3.x nightlies have been hanging here.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (LUCENE-3508) Decompounders based on CompoundWordTokenFilterBase cannot be used with custom attributes

2011-10-25 Thread Uwe Schindler (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-3508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-3508:
--

Attachment: LUCENE-3508.patch

Final trunk patch, which gets committed now:
- Changed the CompoundToken class to use CharSequence for holding the slices.
- Added javadocs with recommendations for performance.
- Deprecated all String[] methods.

Once this is committed, I will backport and then in a second step remove the 
deprecated methods in trunk.

 Decompounders based on CompoundWordTokenFilterBase cannot be used with custom 
 attributes
 

 Key: LUCENE-3508
 URL: https://issues.apache.org/jira/browse/LUCENE-3508
 Project: Lucene - Java
  Issue Type: Bug
  Components: modules/analysis
Affects Versions: 3.4, 4.0
Reporter: Spyros Kapnissis
Assignee: Uwe Schindler
 Fix For: 3.5, 4.0

 Attachments: LUCENE-3508.patch, LUCENE-3508.patch, LUCENE-3508.patch, 
 LUCENE-3508.patch, LUCENE-3508.patch


 The CompoundWordTokenFilterBase.setToken method will call clearAttributes() 
 and then will reset only the default Token attributes (term, position, flags, 
 etc) resulting in any custom attributes losing their value. Commenting out 
 clearAttributes() seems to do the trick, but will fail the 
 TestCompoundWordTokenFilter tests..

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-3508) Decompounders based on CompoundWordTokenFilterBase cannot be used with custom attributes

2011-10-25 Thread Uwe Schindler (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134926#comment-13134926
 ] 

Uwe Schindler commented on LUCENE-3508:
---

Committed trunk revision: 1188597

 Decompounders based on CompoundWordTokenFilterBase cannot be used with custom 
 attributes
 

 Key: LUCENE-3508
 URL: https://issues.apache.org/jira/browse/LUCENE-3508
 Project: Lucene - Java
  Issue Type: Bug
  Components: modules/analysis
Affects Versions: 3.4, 4.0
Reporter: Spyros Kapnissis
Assignee: Uwe Schindler
 Fix For: 3.5, 4.0

 Attachments: LUCENE-3508.patch, LUCENE-3508.patch, LUCENE-3508.patch, 
 LUCENE-3508.patch, LUCENE-3508.patch


 The CompoundWordTokenFilterBase.setToken method will call clearAttributes() 
 and then will reset only the default Token attributes (term, position, flags, 
 etc) resulting in any custom attributes losing their value. Commenting out 
 clearAttributes() seems to do the trick, but will fail the 
 TestCompoundWordTokenFilter tests..

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Created] (LUCENE-3530) Remove deprecated methods in CompoundTokenFilters

2011-10-25 Thread Uwe Schindler (Created) (JIRA)
Remove deprecated methods in CompoundTokenFilters
-

 Key: LUCENE-3530
 URL: https://issues.apache.org/jira/browse/LUCENE-3530
 Project: Lucene - Java
  Issue Type: Sub-task
Affects Versions: 4.0
Reporter: Uwe Schindler
Assignee: Uwe Schindler
 Fix For: 4.0




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (LUCENE-3530) Remove deprecated methods in CompoundTokenFilters

2011-10-25 Thread Uwe Schindler (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-3530:
--

Attachment: LUCENE-3530.patch

Patch

 Remove deprecated methods in CompoundTokenFilters
 -

 Key: LUCENE-3530
 URL: https://issues.apache.org/jira/browse/LUCENE-3530
 Project: Lucene - Java
  Issue Type: Sub-task
  Components: modules/analysis
Affects Versions: 4.0
Reporter: Uwe Schindler
Assignee: Uwe Schindler
 Fix For: 4.0

 Attachments: LUCENE-3530.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Resolved] (LUCENE-3530) Remove deprecated methods in CompoundTokenFilters

2011-10-25 Thread Uwe Schindler (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-3530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler resolved LUCENE-3530.
---

   Resolution: Fixed
Lucene Fields: New,Patch Available  (was: New)

Committed trunk revision: 1188613

 Remove deprecated methods in CompoundTokenFilters
 -

 Key: LUCENE-3530
 URL: https://issues.apache.org/jira/browse/LUCENE-3530
 Project: Lucene - Java
  Issue Type: Sub-task
  Components: modules/analysis
Affects Versions: 4.0
Reporter: Uwe Schindler
Assignee: Uwe Schindler
 Fix For: 4.0

 Attachments: LUCENE-3530.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Assigned] (LUCENE-1536) if a filter can support random access API, we should use it

2011-10-25 Thread Uwe Schindler (Assigned) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-1536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler reassigned LUCENE-1536:
-

Assignee: Uwe Schindler  (was: Michael McCandless)

 if a filter can support random access API, we should use it
 ---

 Key: LUCENE-1536
 URL: https://issues.apache.org/jira/browse/LUCENE-1536
 Project: Lucene - Java
  Issue Type: Improvement
  Components: core/search
Affects Versions: 2.4
Reporter: Michael McCandless
Assignee: Uwe Schindler
Priority: Minor
  Labels: gsoc2011, lucene-gsoc-11, mentor
 Fix For: 4.0

 Attachments: CachedFilterIndexReader.java, LUCENE-1536-rewrite.patch, 
 LUCENE-1536-rewrite.patch, LUCENE-1536-rewrite.patch, 
 LUCENE-1536-rewrite.patch, LUCENE-1536-rewrite.patch, 
 LUCENE-1536-rewrite.patch, LUCENE-1536-rewrite.patch, 
 LUCENE-1536-rewrite.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536_hack.patch, changes-yonik-uwe.patch, luceneutil.patch


 I ran some performance tests, comparing applying a filter via
 random-access API instead of current trunk's iterator API.
 This was inspired by LUCENE-1476, where we realized deletions should
 really be implemented just like a filter, but then in testing found
 that switching deletions to iterator was a very sizable performance
 hit.
 Some notes on the test:
   * Index is first 2M docs of Wikipedia.  Test machine is Mac OS X
 10.5.6, quad core Intel CPU, 6 GB RAM, java 1.6.0_07-b06-153.
   * I test across multiple queries.  1-X means an OR query, eg 1-4
 means 1 OR 2 OR 3 OR 4, whereas +1-4 is an AND query, ie 1 AND 2
 AND 3 AND 4.  u s means united states (phrase search).
   * I test with multiple filter densities (0, 1, 2, 5, 10, 25, 75, 90,
 95, 98, 99, 99.9 (filter is non-null but all bits are set),
 100 (filter=null, control)).
   * Method high means I use random-access filter API in
 IndexSearcher's main loop.  Method low means I use random-access
 filter API down in SegmentTermDocs (just like deleted docs
 today).
   * Baseline (QPS) is current trunk, where filter is applied as iterator up
 high (ie in IndexSearcher's search loop).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Resolved] (LUCENE-1536) if a filter can support random access API, we should use it

2011-10-25 Thread Uwe Schindler (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-1536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler resolved LUCENE-1536.
---

Resolution: Fixed

Committed trunk revision: 1188624

Thanks to all!

 if a filter can support random access API, we should use it
 ---

 Key: LUCENE-1536
 URL: https://issues.apache.org/jira/browse/LUCENE-1536
 Project: Lucene - Java
  Issue Type: Improvement
  Components: core/search
Affects Versions: 2.4
Reporter: Michael McCandless
Assignee: Uwe Schindler
Priority: Minor
  Labels: gsoc2011, lucene-gsoc-11, mentor
 Fix For: 4.0

 Attachments: CachedFilterIndexReader.java, LUCENE-1536-rewrite.patch, 
 LUCENE-1536-rewrite.patch, LUCENE-1536-rewrite.patch, 
 LUCENE-1536-rewrite.patch, LUCENE-1536-rewrite.patch, 
 LUCENE-1536-rewrite.patch, LUCENE-1536-rewrite.patch, 
 LUCENE-1536-rewrite.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536_hack.patch, changes-yonik-uwe.patch, luceneutil.patch


 I ran some performance tests, comparing applying a filter via
 random-access API instead of current trunk's iterator API.
 This was inspired by LUCENE-1476, where we realized deletions should
 really be implemented just like a filter, but then in testing found
 that switching deletions to iterator was a very sizable performance
 hit.
 Some notes on the test:
   * Index is first 2M docs of Wikipedia.  Test machine is Mac OS X
 10.5.6, quad core Intel CPU, 6 GB RAM, java 1.6.0_07-b06-153.
   * I test across multiple queries.  1-X means an OR query, eg 1-4
 means 1 OR 2 OR 3 OR 4, whereas +1-4 is an AND query, ie 1 AND 2
 AND 3 AND 4.  u s means united states (phrase search).
   * I test with multiple filter densities (0, 1, 2, 5, 10, 25, 75, 90,
 95, 98, 99, 99.9 (filter is non-null but all bits are set),
 100 (filter=null, control)).
   * Method high means I use random-access filter API in
 IndexSearcher's main loop.  Method low means I use random-access
 filter API down in SegmentTermDocs (just like deleted docs
 today).
   * Baseline (QPS) is current trunk, where filter is applied as iterator up
 high (ie in IndexSearcher's search loop).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-1536) if a filter can support random access API, we should use it

2011-10-25 Thread Chris Male (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-1536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134967#comment-13134967
 ] 

Chris Male commented on LUCENE-1536:


Yay!

 if a filter can support random access API, we should use it
 ---

 Key: LUCENE-1536
 URL: https://issues.apache.org/jira/browse/LUCENE-1536
 Project: Lucene - Java
  Issue Type: Improvement
  Components: core/search
Affects Versions: 2.4
Reporter: Michael McCandless
Assignee: Uwe Schindler
Priority: Minor
  Labels: gsoc2011, lucene-gsoc-11, mentor
 Fix For: 4.0

 Attachments: CachedFilterIndexReader.java, LUCENE-1536-rewrite.patch, 
 LUCENE-1536-rewrite.patch, LUCENE-1536-rewrite.patch, 
 LUCENE-1536-rewrite.patch, LUCENE-1536-rewrite.patch, 
 LUCENE-1536-rewrite.patch, LUCENE-1536-rewrite.patch, 
 LUCENE-1536-rewrite.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536.patch, LUCENE-1536.patch, LUCENE-1536.patch, 
 LUCENE-1536_hack.patch, changes-yonik-uwe.patch, luceneutil.patch


 I ran some performance tests, comparing applying a filter via
 random-access API instead of current trunk's iterator API.
 This was inspired by LUCENE-1476, where we realized deletions should
 really be implemented just like a filter, but then in testing found
 that switching deletions to iterator was a very sizable performance
 hit.
 Some notes on the test:
   * Index is first 2M docs of Wikipedia.  Test machine is Mac OS X
 10.5.6, quad core Intel CPU, 6 GB RAM, java 1.6.0_07-b06-153.
   * I test across multiple queries.  1-X means an OR query, eg 1-4
 means 1 OR 2 OR 3 OR 4, whereas +1-4 is an AND query, ie 1 AND 2
 AND 3 AND 4.  u s means united states (phrase search).
   * I test with multiple filter densities (0, 1, 2, 5, 10, 25, 75, 90,
 95, 98, 99, 99.9 (filter is non-null but all bits are set),
 100 (filter=null, control)).
   * Method high means I use random-access filter API in
 IndexSearcher's main loop.  Method low means I use random-access
 filter API down in SegmentTermDocs (just like deleted docs
 today).
   * Baseline (QPS) is current trunk, where filter is applied as iterator up
 high (ie in IndexSearcher's search loop).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Created] (LUCENE-3531) Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical to liveDocs

2011-10-25 Thread Uwe Schindler (Created) (JIRA)
Improve CachingWrapperFilter to optionally also cache acceptDocs, if identical 
to liveDocs
--

 Key: LUCENE-3531
 URL: https://issues.apache.org/jira/browse/LUCENE-3531
 Project: Lucene - Java
  Issue Type: Improvement
  Components: core/search
Affects Versions: 4.0
Reporter: Uwe Schindler


Spinoff from LUCENE-1536: This issue removed the different cache modes 
completely and always applies the acceptDocs using BitsFilteredDocIdSet.wrap(), 
the cache only contains raw DocIdSet without any deletions/acceptDocs. For 
IndexReaders that are seldom reopened, this might not be as performant as it 
could be. If the acceptDocs==IR.liveDocs, those DocIdSet could also be cached 
with liveDocs applied.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Assigned] (LUCENE-3424) Return sequence ids from IW update/delete/add/commit to allow total ordering outside of IW

2011-10-25 Thread Simon Willnauer (Assigned) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-3424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Simon Willnauer reassigned LUCENE-3424:
---

Assignee: Simon Willnauer

 Return sequence ids from IW update/delete/add/commit to allow total ordering 
 outside of IW
 --

 Key: LUCENE-3424
 URL: https://issues.apache.org/jira/browse/LUCENE-3424
 Project: Lucene - Java
  Issue Type: Improvement
  Components: core/index
Affects Versions: 4.0
Reporter: Simon Willnauer
Assignee: Simon Willnauer
 Fix For: 4.0


 Based on the discussion on the [mailing 
 list|http://mail-archives.apache.org/mod_mbox/lucene-dev/201109.mbox/%3CCAAHmpki-h7LUZGCUX_rfFx=q5-YkLJei+piRG=oic8d1pnr...@mail.gmail.com%3E]
  IW should return sequence ids from update/delete/add and commit to allow 
 ordering of events for consistent transaction logs and recovery.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Created] (LUCENE-3532) Improve Weight.scorer() API to enforce consistent topScorer/outOfOrder parameters across segments

2011-10-25 Thread Uwe Schindler (Created) (JIRA)
Improve Weight.scorer() API to enforce consistent topScorer/outOfOrder 
parameters across segments
-

 Key: LUCENE-3532
 URL: https://issues.apache.org/jira/browse/LUCENE-3532
 Project: Lucene - Java
  Issue Type: Improvement
  Components: core/search
Affects Versions: 4.0
Reporter: Uwe Schindler


Spinoff from LUCENE-1536: In the past, when filters were applied, all scorers 
were forced to score in order. With random access DocIdSets, this is no longer 
needed. Some Weights (BooleanWeight) unfortunately return different scorers for 
in-order/out-of-order, leading to incompatible scores between segments.

For now we enforce in-order execution of scorers for FilteredQuery (as we do in 
3.x), but once we fix BooleanWeight or have some other good way to produce 
compatible scores, we can reenable random access. Maybe we should nuke 
BooleanScorer2... - Robert and Mike have some ideas how to do that :)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (LUCENE-3424) Return sequence ids from IW update/delete/add/commit to allow total ordering outside of IW

2011-10-25 Thread Simon Willnauer (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-3424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Simon Willnauer updated LUCENE-3424:


Attachment: LUCENE-3424.patch

here is a first patch to add sequence ids to the IndexWriter. Add, Update and 
Delete methods return a long sequence id which is incremented for each 
operation. For updates and deletes the sequence ids introduce a small overhead 
in the DeleteQueue since I have to add a long value to each item . However, for 
addDocument I now have to add an empty Item in the queue to allow increasing 
seq ids even when you add a document. Since those queue items are very short 
living I think this is feasible. 

if that is too much of an overhead we can also disable this by default via IWC 
and make it optional, this is actually very straight forward.

reviews  comments are very appreciated.

 Return sequence ids from IW update/delete/add/commit to allow total ordering 
 outside of IW
 --

 Key: LUCENE-3424
 URL: https://issues.apache.org/jira/browse/LUCENE-3424
 Project: Lucene - Java
  Issue Type: Improvement
  Components: core/index
Affects Versions: 4.0
Reporter: Simon Willnauer
Assignee: Simon Willnauer
 Fix For: 4.0

 Attachments: LUCENE-3424.patch


 Based on the discussion on the [mailing 
 list|http://mail-archives.apache.org/mod_mbox/lucene-dev/201109.mbox/%3CCAAHmpki-h7LUZGCUX_rfFx=q5-YkLJei+piRG=oic8d1pnr...@mail.gmail.com%3E]
  IW should return sequence ids from update/delete/add and commit to allow 
 ordering of events for consistent transaction logs and recovery.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (LUCENE-3506) tests for verifying that assertions are enabled do nothing since they ignore AssertionError

2011-10-25 Thread Doron Cohen (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doron Cohen updated LUCENE-3506:


Attachment: LUCENE-3506.patch

Attached fix for this:

- assertionsEnabled() method added to LTC. 

- tests that were no op were fixed to actually test that the assertion failed.

- after the fix, in trunk, analyzer's final'ness assertion tests failed because 
being final (class or method) is no longer needed in trunk. So these tests were 
removed in TestAssertions.
-- note: should not remove these tests when merging to 3x.

- TestSegmentMerger also failed with this fix - because it used the stale IW's 
SegmentInfos to create a compound segment. Fixed by reading a fresh SIS.

- only one test (TestAssertions.testbasics()) fails if assertions are not 
enabled. The other tests do not fail (though they do execute). I think that 
this was intended in the code, thought since it did not work it is hard to 
tell...

This is ready to commit.

 tests for verifying that assertions are enabled do nothing since they ignore 
 AssertionError
 ---

 Key: LUCENE-3506
 URL: https://issues.apache.org/jira/browse/LUCENE-3506
 Project: Lucene - Java
  Issue Type: Bug
  Components: general/test
Reporter: Doron Cohen
Assignee: Doron Cohen
Priority: Minor
 Attachments: LUCENE-3506.patch


 Follow-up from LUCENE-3501

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-3533) Nuke SpanFilters and CachingSpanFilter (maybe move to sandbox)

2011-10-25 Thread Grant Ingersoll (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134981#comment-13134981
 ] 

Grant Ingersoll commented on LUCENE-3533:
-

+1

 Nuke SpanFilters and CachingSpanFilter (maybe move to sandbox)
 --

 Key: LUCENE-3533
 URL: https://issues.apache.org/jira/browse/LUCENE-3533
 Project: Lucene - Java
  Issue Type: Task
Reporter: Uwe Schindler
Assignee: Uwe Schindler

 SpanFilters are inefficient and OOM easily (they don't scale at all: Create 
 large Lists of Objects for every match, also filtering deleted docs is a 
 pain). Some talks with Grant on Eurocon and also the fact that caching of 
 them is still broken in 3.x (but fixed on trunk) - I assume nobody uses them, 
 so let's nuke them. They are also in wrong package, so standard statement: 
 Die, SpanFilters, die!

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[JENKINS] Lucene-trunk - Build # 1710 - Still Failing

2011-10-25 Thread Apache Jenkins Server
Build: https://builds.apache.org/job/Lucene-trunk/1710/

All tests passed

Build Log (for compile errors):
[...truncated 19239 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Created] (SOLR-2851) Allow Data Import Handler to Archive Files After Processing

2011-10-25 Thread Josh Harness (Created) (JIRA)
Allow Data Import Handler to Archive Files After Processing
---

 Key: SOLR-2851
 URL: https://issues.apache.org/jira/browse/SOLR-2851
 Project: Solr
  Issue Type: New Feature
  Components: contrib - DataImportHandler
Affects Versions: 3.3
Reporter: Josh Harness
Priority: Minor
 Fix For: 3.3


It would be great if, after processing flat files, DIH could place them in an 
archive folder so that other system processes could compress them, delete them, 
etc. We needed this feature, so I extended FileDataSource and decorated the 
InputStreamReader such that when the close method is called, the file is moved 
out of the way. This concrete implementation did exactly what I needed. 
However, a more correct textbook approach would involve refactoring 
FileListEntityProcessor such that it could take a configurable list of 
post-processing action policy/command classes so that the user could provide 
custom behavior that suits nearly any scenario (in a Chain of Responsibility 
style pattern). Nonetheless, I wanted to provide the suggestion, a basic 
one-off concrete implementation of archiving and start the discussion for where 
things might head in the future. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-2205) Rework of the TermInfosReader class to remove the Terms[], TermInfos[], and the index pointer long[] and create a more memory efficient data structure.

2011-10-25 Thread Michael McCandless (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134982#comment-13134982
 ] 

Michael McCandless commented on LUCENE-2205:


Well thank you Aaron for doing all the hard parts, and, persisting!  Sorry this 
took so long.

This will be an enormous improvement for 3.5.0.

 Rework of the TermInfosReader class to remove the Terms[], TermInfos[], and 
 the index pointer long[] and create a more memory efficient data structure.
 ---

 Key: LUCENE-2205
 URL: https://issues.apache.org/jira/browse/LUCENE-2205
 Project: Lucene - Java
  Issue Type: Improvement
  Components: core/index
 Environment: Java5
Reporter: Aaron McCurry
Assignee: Michael McCandless
 Fix For: 3.5

 Attachments: LUCENE-2205.patch, RandomAccessTest.java, 
 TermInfosReader.java, TermInfosReaderIndex.java, 
 TermInfosReaderIndexDefault.java, TermInfosReaderIndexSmall.java, 
 lowmemory_w_utf8_encoding.patch, lowmemory_w_utf8_encoding.v4.patch, 
 patch-final.txt, rawoutput.txt


 Basically packing those three arrays into a byte array with an int array as 
 an index offset.  
 The performance benefits are stagering on my test index (of size 6.2 GB, with 
 ~1,000,000 documents and ~175,000,000 terms), the memory needed to load the 
 terminfos into memory were reduced to 17% of there original size.  From 291.5 
 MB to 49.7 MB.  The random access speed has been made better by 1-2%, load 
 time of the segments are ~40% faster as well, and full GC's on my JVM were 
 made 7 times faster.
 I have already performed the work and am offering this code as a patch.  
 Currently all test in the trunk pass with this new code enabled.  I did write 
 a system property switch to allow for the original implementation to be used 
 as well.
 -Dorg.apache.lucene.index.TermInfosReader=default or small
 I have also written a blog about this patch here is the link.
 http://www.nearinfinity.com/blogs/aaron_mccurry/my_first_lucene_patch.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (SOLR-2851) Allow Data Import Handler to Archive Files After Processing

2011-10-25 Thread Josh Harness (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-2851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Harness updated SOLR-2851:
---

Attachment: ArchivingFileDataSource.java

Custom file data source that archives files when the close method on the input 
stream reader is called. Java package name is still specific to my company and 
needs to be modified to reflect the apache structure. Also, tests are needed. 

 Allow Data Import Handler to Archive Files After Processing
 ---

 Key: SOLR-2851
 URL: https://issues.apache.org/jira/browse/SOLR-2851
 Project: Solr
  Issue Type: New Feature
  Components: contrib - DataImportHandler
Affects Versions: 3.3
Reporter: Josh Harness
Priority: Minor
  Labels: dataimport, solr
 Fix For: 3.3

 Attachments: ArchivingFileDataSource.java


 It would be great if, after processing flat files, DIH could place them in an 
 archive folder so that other system processes could compress them, delete 
 them, etc. We needed this feature, so I extended FileDataSource and decorated 
 the InputStreamReader such that when the close method is called, the file is 
 moved out of the way. This concrete implementation did exactly what I needed. 
 However, a more correct textbook approach would involve refactoring 
 FileListEntityProcessor such that it could take a configurable list of 
 post-processing action policy/command classes so that the user could provide 
 custom behavior that suits nearly any scenario (in a Chain of Responsibility 
 style pattern). Nonetheless, I wanted to provide the suggestion, a basic 
 one-off concrete implementation of archiving and start the discussion for 
 where things might head in the future. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



Re: Request for Feedback for Patch to Allow DIH to Archive Files

2011-10-25 Thread Josh Harness
Martijn,

 Thanks for the feedback. I have created
SOLR-2851https://issues.apache.org/jira/browse/SOLR-2851for this
feature request. Please let me know if there's anything else you'd
like me to do.

Thanks!

Josh

On Tue, Oct 25, 2011 at 2:29 AM, Martijn v Groningen 
martijn.v.gronin...@gmail.com wrote:

 Hi Josh,

 I think this functionality is useful. I'd create an Jira issue and
 attach your code as a patch. I think that the functionality should be
 added to the FileListEntityProcessor since it seems to be a more
 natural place for it. Maybe we need something more generic, like a
 post action if a file has been processed.

 Martijn

 On 24 October 2011 21:31, Josh Harness josh.harn...@jtv.com wrote:
  Hi -
 
   We are using SOLR to process XML input files using the Data Import
  Handler. I didn't see a way to move the xml files out of the way after
  processing, so I wrote a small extension to allow this. The How to
  Contribute page says to pitch the request to the developer list in order
 to
  decide whether or not to submit a patch. As such, here goes:
 
   The new code basically extends FileDataSource and wraps the
 underlying
  reader such that when the close method on the input stream is called,
 the
  file is moved to a configurable archive directory. It is unclear to me
  whether this is the correct place to put it (I pondered changing the
  FileListEntityProcessor but this somehow felt safer). I realize that a
 more
  robust implementation would consider the success status of the file being
  processed and would also allow for configurable policies rather than a
  concrete implementation. Nonetheless, I didn't want the perfect to be the
  enemy of the good.
 
   Please peruse the attached source code file and provide feedback as
 to
  the merit of the idea, whether I ought to submit a JIRA ticket/patch and
 if
  my approach is correct.
 
  Thanks!
 
  Josh Harness
 
 
  -
  To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
  For additional commands, e-mail: dev-h...@lucene.apache.org
 



 --
 Met vriendelijke groet,

 Martijn van Groningen

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org




[jira] [Commented] (LUCENE-3506) tests for verifying that assertions are enabled do nothing since they ignore AssertionError

2011-10-25 Thread Chris Male (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134987#comment-13134987
 ] 

Chris Male commented on LUCENE-3506:


Thanks for catching the Analyzer final assertion test issue.

 tests for verifying that assertions are enabled do nothing since they ignore 
 AssertionError
 ---

 Key: LUCENE-3506
 URL: https://issues.apache.org/jira/browse/LUCENE-3506
 Project: Lucene - Java
  Issue Type: Bug
  Components: general/test
Reporter: Doron Cohen
Assignee: Doron Cohen
Priority: Minor
 Attachments: LUCENE-3506.patch


 Follow-up from LUCENE-3501

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Created] (LUCENE-3534) Backport FilteredQuery/IndexSearcher changes to 3.x: Remove filter logic from IndexSearcher and delegate to FilteredQuery

2011-10-25 Thread Uwe Schindler (Created) (JIRA)
Backport FilteredQuery/IndexSearcher changes to 3.x: Remove filter logic from 
IndexSearcher and delegate to FilteredQuery
-

 Key: LUCENE-3534
 URL: https://issues.apache.org/jira/browse/LUCENE-3534
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Uwe Schindler


Spinoff from LUCENE-1536: We simplified the code in IndexSearcher to no longer 
do the filtering there, instead wrap all Query with FilteredQuery, if a 
non-null filter is given. The conjunction code would then only exist in 
FilteredQuery which makes it easier to maintain. Currently both implementations 
differ in 3.x, in trunk we used the more optimized IndexSearcher variant with 
addition of a simplified in-order conjunction code.

This issue will backport those changes (without random access bits).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Assigned] (LUCENE-3534) Backport FilteredQuery/IndexSearcher changes to 3.x: Remove filter logic from IndexSearcher and delegate to FilteredQuery

2011-10-25 Thread Uwe Schindler (Assigned) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler reassigned LUCENE-3534:
-

Assignee: Uwe Schindler

I will start merging IndexSearcher and FilteredQuery changes to 3.x soon.

 Backport FilteredQuery/IndexSearcher changes to 3.x: Remove filter logic from 
 IndexSearcher and delegate to FilteredQuery
 -

 Key: LUCENE-3534
 URL: https://issues.apache.org/jira/browse/LUCENE-3534
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Uwe Schindler
Assignee: Uwe Schindler

 Spinoff from LUCENE-1536: We simplified the code in IndexSearcher to no 
 longer do the filtering there, instead wrap all Query with FilteredQuery, if 
 a non-null filter is given. The conjunction code would then only exist in 
 FilteredQuery which makes it easier to maintain. Currently both 
 implementations differ in 3.x, in trunk we used the more optimized 
 IndexSearcher variant with addition of a simplified in-order conjunction code.
 This issue will backport those changes (without random access bits).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



RE: [JENKINS] Lucene-trunk - Build # 1710 - Still Failing

2011-10-25 Thread Uwe Schindler
This build was hanging in the final cleanup and artifact copy.

-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de


 -Original Message-
 From: Apache Jenkins Server [mailto:jenk...@builds.apache.org]
 Sent: Tuesday, October 25, 2011 2:37 PM
 To: dev@lucene.apache.org
 Subject: [JENKINS] Lucene-trunk - Build # 1710 - Still Failing
 
 Build: https://builds.apache.org/job/Lucene-trunk/1710/
 
 All tests passed
 
 Build Log (for compile errors):
 [...truncated 19239 lines...]
 
 
 
 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional
 commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-3506) tests for verifying that assertions are enabled do nothing since they ignore AssertionError

2011-10-25 Thread Uwe Schindler (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13134997#comment-13134997
 ] 

Uwe Schindler commented on LUCENE-3506:
---

+1 to commit!

 tests for verifying that assertions are enabled do nothing since they ignore 
 AssertionError
 ---

 Key: LUCENE-3506
 URL: https://issues.apache.org/jira/browse/LUCENE-3506
 Project: Lucene - Java
  Issue Type: Bug
  Components: general/test
Reporter: Doron Cohen
Assignee: Doron Cohen
Priority: Minor
 Attachments: LUCENE-3506.patch


 Follow-up from LUCENE-3501

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-3506) tests for verifying that assertions are enabled do nothing since they ignore AssertionError

2011-10-25 Thread Michael McCandless (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135002#comment-13135002
 ] 

Michael McCandless commented on LUCENE-3506:


I'm confused here -- the changes to TestSegmentMerger look like they'll allow 
the test to pass when assertions are disabled?

(Whereas today if you run that test w/o assertions you get a failure, albeit a 
confusing one).

 tests for verifying that assertions are enabled do nothing since they ignore 
 AssertionError
 ---

 Key: LUCENE-3506
 URL: https://issues.apache.org/jira/browse/LUCENE-3506
 Project: Lucene - Java
  Issue Type: Bug
  Components: general/test
Reporter: Doron Cohen
Assignee: Doron Cohen
Priority: Minor
 Attachments: LUCENE-3506.patch


 Follow-up from LUCENE-3501

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-3506) tests for verifying that assertions are enabled do nothing since they ignore AssertionError

2011-10-25 Thread Robert Muir (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135004#comment-13135004
 ] 

Robert Muir commented on LUCENE-3506:
-

maybe make assertionsEnabled() static in LuceneTestCase, and add 
assertTrue(assertionsEnabled()) in LuceneTestCase's beforeClass?

This way all tests will fail if assertions are not enabled.

The other day I committed an accidental change to common-build that disabled 
assertions, and it was a little confusing to track down.

 tests for verifying that assertions are enabled do nothing since they ignore 
 AssertionError
 ---

 Key: LUCENE-3506
 URL: https://issues.apache.org/jira/browse/LUCENE-3506
 Project: Lucene - Java
  Issue Type: Bug
  Components: general/test
Reporter: Doron Cohen
Assignee: Doron Cohen
Priority: Minor
 Attachments: LUCENE-3506.patch


 Follow-up from LUCENE-3501

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[JENKINS] Lucene-Solr-tests-only-trunk - Build # 10992 - Failure

2011-10-25 Thread Apache Jenkins Server
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/10992/

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.update.AutoCommitTest

Error Message:
java.lang.AssertionError: directory of test was not closed, opened from: 
org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:33)

Stack Trace:
java.lang.RuntimeException: java.lang.AssertionError: directory of test was not 
closed, opened from: 
org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:33)
at 
org.apache.lucene.util.LuceneTestCase.afterClassLuceneTestCaseJ4(LuceneTestCase.java:469)
at 
org.apache.lucene.util.LuceneTestCase.checkResourcesAfterClass(LuceneTestCase.java:527)
at 
org.apache.lucene.util.LuceneTestCase.afterClassLuceneTestCaseJ4(LuceneTestCase.java:437)




Build Log (for compile errors):
[...truncated 7895 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-3506) tests for verifying that assertions are enabled do nothing since they ignore AssertionError

2011-10-25 Thread Doron Cohen (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135086#comment-13135086
 ] 

Doron Cohen commented on LUCENE-3506:
-

bq. (Whereas today if you run that test w/o assertions you get a failure, 
albeit a confusing one).

Actually today when you run the tests - with assertions, without assertions, - 
you get no failures at all - which is what I was trying to fix here (unless I 
missed something seriously) - because:
- the original tests, after deciding to fail, invoked fail()
- this threw AssertionError
- but it was ignored as part of their wrong logic. 

bq. I'm confused here – the changes to TestSegmentMerger look like they'll 
allow the test to pass when assertions are disabled?

Right, I fixed it such that *only if* assertions are enabled, they verify that 
the expected assertion errors are not thrown, so they allow you to run tests 
also without enabling assertions. See my comment above only one test I 
take it that this kind of flexibility is not required. So will change it so 
that these tests will fail if assertions are not enabled.

bq. The other day I committed an accidental change to common-build that 
disabled assertions, and it was a little confusing to track down.

I see, so we make the entire test framework to fail if assertions are not 
enabled.
I'll update the patch.

 tests for verifying that assertions are enabled do nothing since they ignore 
 AssertionError
 ---

 Key: LUCENE-3506
 URL: https://issues.apache.org/jira/browse/LUCENE-3506
 Project: Lucene - Java
  Issue Type: Bug
  Components: general/test
Reporter: Doron Cohen
Assignee: Doron Cohen
Priority: Minor
 Attachments: LUCENE-3506.patch


 Follow-up from LUCENE-3501

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-3508) Decompounders based on CompoundWordTokenFilterBase cannot be used with custom attributes

2011-10-25 Thread Spyros Kapnissis (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135103#comment-13135103
 ] 

Spyros Kapnissis commented on LUCENE-3508:
--

Looks great now, thanks a lot guys!

 Decompounders based on CompoundWordTokenFilterBase cannot be used with custom 
 attributes
 

 Key: LUCENE-3508
 URL: https://issues.apache.org/jira/browse/LUCENE-3508
 Project: Lucene - Java
  Issue Type: Bug
  Components: modules/analysis
Affects Versions: 3.4, 4.0
Reporter: Spyros Kapnissis
Assignee: Uwe Schindler
 Fix For: 3.5, 4.0

 Attachments: LUCENE-3508.patch, LUCENE-3508.patch, LUCENE-3508.patch, 
 LUCENE-3508.patch, LUCENE-3508.patch


 The CompoundWordTokenFilterBase.setToken method will call clearAttributes() 
 and then will reset only the default Token attributes (term, position, flags, 
 etc) resulting in any custom attributes losing their value. Commenting out 
 clearAttributes() seems to do the trick, but will fail the 
 TestCompoundWordTokenFilter tests..

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-3534) Backport FilteredQuery/IndexSearcher changes to 3.x: Remove filter logic from IndexSearcher and delegate to FilteredQuery

2011-10-25 Thread Uwe Schindler (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135128#comment-13135128
 ] 

Uwe Schindler commented on LUCENE-3534:
---

My problem is that this backport is almost impossible: Searchable and Searcher 
abstract class prevent this from being done.

We have several choices:
- we can heavily break backwards and maybe nuke Searcher and Searchable earlier
- we keep API intact, only call FilteredQuery.weight() from inside 
IndexSaercher instead of duplicating code
- close issue as won't fix

Any comments?

 Backport FilteredQuery/IndexSearcher changes to 3.x: Remove filter logic from 
 IndexSearcher and delegate to FilteredQuery
 -

 Key: LUCENE-3534
 URL: https://issues.apache.org/jira/browse/LUCENE-3534
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Uwe Schindler
Assignee: Uwe Schindler

 Spinoff from LUCENE-1536: We simplified the code in IndexSearcher to no 
 longer do the filtering there, instead wrap all Query with FilteredQuery, if 
 a non-null filter is given. The conjunction code would then only exist in 
 FilteredQuery which makes it easier to maintain. Currently both 
 implementations differ in 3.x, in trunk we used the more optimized 
 IndexSearcher variant with addition of a simplified in-order conjunction code.
 This issue will backport those changes (without random access bits).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-3534) Backport FilteredQuery/IndexSearcher changes to 3.x: Remove filter logic from IndexSearcher and delegate to FilteredQuery

2011-10-25 Thread Robert Muir (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135131#comment-13135131
 ] 

Robert Muir commented on LUCENE-3534:
-

i dont think we should heavy-break in a minor release, especially since we 
cannot use random access Bits.


 Backport FilteredQuery/IndexSearcher changes to 3.x: Remove filter logic from 
 IndexSearcher and delegate to FilteredQuery
 -

 Key: LUCENE-3534
 URL: https://issues.apache.org/jira/browse/LUCENE-3534
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Uwe Schindler
Assignee: Uwe Schindler

 Spinoff from LUCENE-1536: We simplified the code in IndexSearcher to no 
 longer do the filtering there, instead wrap all Query with FilteredQuery, if 
 a non-null filter is given. The conjunction code would then only exist in 
 FilteredQuery which makes it easier to maintain. Currently both 
 implementations differ in 3.x, in trunk we used the more optimized 
 IndexSearcher variant with addition of a simplified in-order conjunction code.
 This issue will backport those changes (without random access bits).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-3534) Backport FilteredQuery/IndexSearcher changes to 3.x: Remove filter logic from IndexSearcher and delegate to FilteredQuery

2011-10-25 Thread Uwe Schindler (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135132#comment-13135132
 ] 

Uwe Schindler commented on LUCENE-3534:
---

I tend to the second solution, as the whole issue was about removing the code 
duplication between IS and FQ (which also have different implementations of the 
same code).

 Backport FilteredQuery/IndexSearcher changes to 3.x: Remove filter logic from 
 IndexSearcher and delegate to FilteredQuery
 -

 Key: LUCENE-3534
 URL: https://issues.apache.org/jira/browse/LUCENE-3534
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Uwe Schindler
Assignee: Uwe Schindler

 Spinoff from LUCENE-1536: We simplified the code in IndexSearcher to no 
 longer do the filtering there, instead wrap all Query with FilteredQuery, if 
 a non-null filter is given. The conjunction code would then only exist in 
 FilteredQuery which makes it easier to maintain. Currently both 
 implementations differ in 3.x, in trunk we used the more optimized 
 IndexSearcher variant with addition of a simplified in-order conjunction code.
 This issue will backport those changes (without random access bits).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (LUCENE-3506) tests for verifying that assertions are enabled do nothing since they ignore AssertionError

2011-10-25 Thread Doron Cohen (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doron Cohen updated LUCENE-3506:


Attachment: LUCENE-3506.patch

Updated patch as suggested, thanks guys for reviewing and your helpful comments.

 tests for verifying that assertions are enabled do nothing since they ignore 
 AssertionError
 ---

 Key: LUCENE-3506
 URL: https://issues.apache.org/jira/browse/LUCENE-3506
 Project: Lucene - Java
  Issue Type: Bug
  Components: general/test
Reporter: Doron Cohen
Assignee: Doron Cohen
Priority: Minor
 Attachments: LUCENE-3506.patch, LUCENE-3506.patch


 Follow-up from LUCENE-3501

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Created] (LUCENE-3535) move PerFieldCodecWrapper into codecs package

2011-10-25 Thread Robert Muir (Created) (JIRA)
move PerFieldCodecWrapper into codecs package
-

 Key: LUCENE-3535
 URL: https://issues.apache.org/jira/browse/LUCENE-3535
 Project: Lucene - Java
  Issue Type: Task
Affects Versions: 4.0
Reporter: Robert Muir


PerFieldCodecWrapper is a codec, but its 'hardwired' as lucene's only codec 
currently (except for PreFlex/3.x case)

it lets you choose a format for the postings lists per-field.

I think we should move this to the codecs package as a start... just a rote 
refactor.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (LUCENE-3535) move PerFieldCodecWrapper into codecs package

2011-10-25 Thread Robert Muir (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Muir updated LUCENE-3535:


Attachment: LUCENE-3535.patch

 move PerFieldCodecWrapper into codecs package
 -

 Key: LUCENE-3535
 URL: https://issues.apache.org/jira/browse/LUCENE-3535
 Project: Lucene - Java
  Issue Type: Task
Affects Versions: 4.0
Reporter: Robert Muir
 Attachments: LUCENE-3535.patch


 PerFieldCodecWrapper is a codec, but its 'hardwired' as lucene's only codec 
 currently (except for PreFlex/3.x case)
 it lets you choose a format for the postings lists per-field.
 I think we should move this to the codecs package as a start... just a rote 
 refactor.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-3534) Backport FilteredQuery/IndexSearcher changes to 3.x: Remove filter logic from IndexSearcher and delegate to FilteredQuery

2011-10-25 Thread Robert Muir (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135141#comment-13135141
 ] 

Robert Muir commented on LUCENE-3534:
-

for the second solution, we get some benefits of cleanup, without breaks right? 
Sounds good to me.

 Backport FilteredQuery/IndexSearcher changes to 3.x: Remove filter logic from 
 IndexSearcher and delegate to FilteredQuery
 -

 Key: LUCENE-3534
 URL: https://issues.apache.org/jira/browse/LUCENE-3534
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Uwe Schindler
Assignee: Uwe Schindler

 Spinoff from LUCENE-1536: We simplified the code in IndexSearcher to no 
 longer do the filtering there, instead wrap all Query with FilteredQuery, if 
 a non-null filter is given. The conjunction code would then only exist in 
 FilteredQuery which makes it easier to maintain. Currently both 
 implementations differ in 3.x, in trunk we used the more optimized 
 IndexSearcher variant with addition of a simplified in-order conjunction code.
 This issue will backport those changes (without random access bits).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[JENKINS] Lucene-Solr-tests-only-trunk - Build # 10993 - Still Failing

2011-10-25 Thread Apache Jenkins Server
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/10993/

All tests passed

Build Log (for compile errors):
[...truncated 13128 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



Re: SIGSEGV in JCCEnv::setClassPath

2011-10-25 Thread Stein, Ruben
Hi Andi,

thank you very much for pointing me to the right direction. The
thread-attachment call in ReviewBoard was in fact missing. For anyone
interested in the actual solution for ReviewBoard see:
http://groups.google.com/group/reviewboard/browse_thread/thread/60cff51dc87
673f4

Thanks again, Ruben



[JENKINS-MAVEN] Lucene-Solr-Maven-trunk #276: POMs out of sync

2011-10-25 Thread Apache Jenkins Server
Build: https://builds.apache.org/job/Lucene-Solr-Maven-trunk/276/

No tests ran.

Build Log (for compile errors):
[...truncated 7270 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (LUCENE-3534) Backport FilteredQuery/IndexSearcher changes to 3.x: Remove filter logic from IndexSearcher and delegate to FilteredQuery

2011-10-25 Thread Uwe Schindler (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-3534:
--

Attachment: LUCENE-3534.patch

Here the easy implementation, that simply removes code duplication and uses the 
same scorer for both implementations (IS and FQ).

This patch uses a hack, to provide access to the wrapped scorer from inside 
IndexSearcher.

This is all with no public API change at all.

 Backport FilteredQuery/IndexSearcher changes to 3.x: Remove filter logic from 
 IndexSearcher and delegate to FilteredQuery
 -

 Key: LUCENE-3534
 URL: https://issues.apache.org/jira/browse/LUCENE-3534
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Uwe Schindler
Assignee: Uwe Schindler
 Attachments: LUCENE-3534.patch


 Spinoff from LUCENE-1536: We simplified the code in IndexSearcher to no 
 longer do the filtering there, instead wrap all Query with FilteredQuery, if 
 a non-null filter is given. The conjunction code would then only exist in 
 FilteredQuery which makes it easier to maintain. Currently both 
 implementations differ in 3.x, in trunk we used the more optimized 
 IndexSearcher variant with addition of a simplified in-order conjunction code.
 This issue will backport those changes (without random access bits).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-3506) tests for verifying that assertions are enabled do nothing since they ignore AssertionError

2011-10-25 Thread Michael McCandless (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135179#comment-13135179
 ] 

Michael McCandless commented on LUCENE-3506:


{quote}
Actually today when you run the tests - with assertions, without assertions, - 
you get no failures at all - which is what I was trying to fix here (unless I 
missed something seriously) - because:

* the original tests, after deciding to fail, invoked fail()
* this threw AssertionError
* but it was ignored as part of their wrong logic.
{quote}

Ahh, OK, I missed that fail() throws AssertionError which this then caught  
ignored.  OK.  Patch looks good!

 tests for verifying that assertions are enabled do nothing since they ignore 
 AssertionError
 ---

 Key: LUCENE-3506
 URL: https://issues.apache.org/jira/browse/LUCENE-3506
 Project: Lucene - Java
  Issue Type: Bug
  Components: general/test
Reporter: Doron Cohen
Assignee: Doron Cohen
Priority: Minor
 Attachments: LUCENE-3506.patch, LUCENE-3506.patch


 Follow-up from LUCENE-3501

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[JENKINS] Lucene-Solr-tests-only-trunk - Build # 10994 - Still Failing

2011-10-25 Thread Apache Jenkins Server
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/10994/

All tests passed

Build Log (for compile errors):
[...truncated 13134 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (LUCENE-3534) Backport FilteredQuery/IndexSearcher changes to 3.x: Remove filter logic from IndexSearcher and delegate to FilteredQuery

2011-10-25 Thread Uwe Schindler (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-3534:
--

Attachment: LUCENE-3463.patch

Small optimization, as in 3.x we dont need to move the filter to first doc, so 
the scorer has one less if-check.

Also the previous patch was missing the deprecated similarity pass-through to 
Scorer.

I think thats ready to commit, any comments?

 Backport FilteredQuery/IndexSearcher changes to 3.x: Remove filter logic from 
 IndexSearcher and delegate to FilteredQuery
 -

 Key: LUCENE-3534
 URL: https://issues.apache.org/jira/browse/LUCENE-3534
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Uwe Schindler
Assignee: Uwe Schindler
 Attachments: LUCENE-3463.patch, LUCENE-3534.patch


 Spinoff from LUCENE-1536: We simplified the code in IndexSearcher to no 
 longer do the filtering there, instead wrap all Query with FilteredQuery, if 
 a non-null filter is given. The conjunction code would then only exist in 
 FilteredQuery which makes it easier to maintain. Currently both 
 implementations differ in 3.x, in trunk we used the more optimized 
 IndexSearcher variant with addition of a simplified in-order conjunction code.
 This issue will backport those changes (without random access bits).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Issue Comment Edited] (LUCENE-3534) Backport FilteredQuery/IndexSearcher changes to 3.x: Remove filter logic from IndexSearcher and delegate to FilteredQuery

2011-10-25 Thread Robert Muir (Issue Comment Edited) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135224#comment-13135224
 ] 

Robert Muir edited comment on LUCENE-3534 at 10/25/11 4:59 PM:
---

+1, i prefer the hackedy-häck-hack over duplicated code :)

  was (Author: rcmuir):
+1, i prefer the hackedy-hack-hack over duplicated code :)
  
 Backport FilteredQuery/IndexSearcher changes to 3.x: Remove filter logic from 
 IndexSearcher and delegate to FilteredQuery
 -

 Key: LUCENE-3534
 URL: https://issues.apache.org/jira/browse/LUCENE-3534
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Uwe Schindler
Assignee: Uwe Schindler
 Attachments: LUCENE-3463.patch, LUCENE-3534.patch


 Spinoff from LUCENE-1536: We simplified the code in IndexSearcher to no 
 longer do the filtering there, instead wrap all Query with FilteredQuery, if 
 a non-null filter is given. The conjunction code would then only exist in 
 FilteredQuery which makes it easier to maintain. Currently both 
 implementations differ in 3.x, in trunk we used the more optimized 
 IndexSearcher variant with addition of a simplified in-order conjunction code.
 This issue will backport those changes (without random access bits).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-3534) Backport FilteredQuery/IndexSearcher changes to 3.x: Remove filter logic from IndexSearcher and delegate to FilteredQuery

2011-10-25 Thread Uwe Schindler (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135236#comment-13135236
 ] 

Uwe Schindler commented on LUCENE-3534:
---

bq. Robert Muir added a comment - 25/Oct/11 16:51 - *edited*

Much better! I will commit soon and move forward to nuking SpanFilters :-)

 Backport FilteredQuery/IndexSearcher changes to 3.x: Remove filter logic from 
 IndexSearcher and delegate to FilteredQuery
 -

 Key: LUCENE-3534
 URL: https://issues.apache.org/jira/browse/LUCENE-3534
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Uwe Schindler
Assignee: Uwe Schindler
 Attachments: LUCENE-3463.patch, LUCENE-3534.patch


 Spinoff from LUCENE-1536: We simplified the code in IndexSearcher to no 
 longer do the filtering there, instead wrap all Query with FilteredQuery, if 
 a non-null filter is given. The conjunction code would then only exist in 
 FilteredQuery which makes it easier to maintain. Currently both 
 implementations differ in 3.x, in trunk we used the more optimized 
 IndexSearcher variant with addition of a simplified in-order conjunction code.
 This issue will backport those changes (without random access bits).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[JENKINS] Lucene-Solr-tests-only-3.x-java7 - Build # 735 - Failure

2011-10-25 Thread Apache Jenkins Server
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-3.x-java7/735/

1 tests failed.
REGRESSION:  
org.apache.solr.client.solrj.embedded.LargeVolumeJettyTest.testMultiThreaded

Error Message:
java.lang.AssertionError: Some threads threw uncaught exceptions!

Stack Trace:
java.lang.RuntimeException: java.lang.AssertionError: Some threads threw 
uncaught exceptions!
at 
org.apache.lucene.util.LuceneTestCase.tearDown(LuceneTestCase.java:574)
at org.apache.solr.SolrTestCaseJ4.tearDown(SolrTestCaseJ4.java:96)
at 
org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:147)
at 
org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:50)
at 
org.apache.lucene.util.LuceneTestCase.checkUncaughtExceptionsAfter(LuceneTestCase.java:602)
at 
org.apache.lucene.util.LuceneTestCase.tearDown(LuceneTestCase.java:546)




Build Log (for compile errors):
[...truncated 14516 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (SOLR-2849) Solr maven dependencies: logging

2011-10-25 Thread David Smiley (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley updated SOLR-2849:
---

Attachment: SOLR-2849_maven_dependencies.patch

Attached is a patch making a fair amount of changes to dependencies in the 
maven poms. One of the biggest changes to the poms was shifting dependency 
exclusions at the solr war level down to the origins of where those 
dependencies were introduced.  Below is a summary of the changes I made, 
including notes on changes I did *not* make.

* /modules/queryparser
** make lucene-sandbox an optional dependency. The dependency is isolated to 
the XML builder CorePlusExtensionsParser subclass, so it is quite optional.
* /solr/core
** added direct dependency on lucene-core, instead of leaving it to transitive 
resolution (a matter of taste)
** exclude commons-logging from commons-httpclient dependency
** depend on jcl-over-sfl4j
** exclude many unused dependencies from velocity
** remove slf4j-jdk14 (moved to war pom)
** I noticed the ant build depends on guava, but we *still* don't actually use 
it, so I opted to not add this dependency.
* /solr/solrj
** exclude log4j and jline from zookeeper
** depend on log4j-over-slf4j
** exclude commons-logging from commons-httpclient dependency
** depend on jcl-over-sfl4j
** I noticed the ant distribution suggests woodstox is a solrj dependency. I 
opted to not include it here.  The client doesn't actually depend on it, and 
SolrJ is usually going to use the efficient binary format any way.
* /solr/webapp
** removed numerous exclusions from the solr-core dependency, since these were 
effectively moved
** depend on slf4j-jdk scope=runtime optional=true (actually moved it from 
solr-core)

I reviewed the built war for consistency with the ant build, as well as the 
solrj dependencies.

 Solr maven dependencies: logging
 

 Key: SOLR-2849
 URL: https://issues.apache.org/jira/browse/SOLR-2849
 Project: Solr
  Issue Type: Improvement
  Components: Build
Affects Versions: 4.0
Reporter: David Smiley
Priority: Trivial
 Attachments: SOLR-2849_maven_dependencies.patch


 I was looking at my maven based project's Solr-core dependencies (trunk), and 
 observed some issues that I think should be fixed in Solr's maven poms. I ran 
 {{mvn dependency:tree}} -- the output is further below.  There are two 
 changes I see needed, related to logging:
 * slf4j-jdk14 should be runtime scope, and optional.
 * httpclient depends on commons-logging.  Exclude this dependency from the 
 httpclient dependency, and add a dependency on jcl-over-slf4j with compile 
 scope.
 * Zookeeper depends on Log4j, unfortunately. There is an issue to change this 
 to SLF4J: ZOOKEEPER-850. In the mean time we should exclude it and use 
 log4j-over-slf4j with compile scope, at the solrj pom.
 As an aside, it's unfortunate to see all those velocity dependencies.  It 
 even depends on struts -- seriously?!  I hope solritas gets put back into a 
 contrib sometime: SOLR-2588
 Steve, if you'd like to me to create the patch, I will.
 {code}
 [INFO] +- org.apache.solr:solr-core:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.solr:solr-solrj:jar:4.0-SNAPSHOT:compile
 [INFO] |  |  \- org.apache.zookeeper:zookeeper:jar:3.3.3:compile
 [INFO] |  | +- log4j:log4j:jar:1.2.15:compile
 [INFO] |  | |  \- javax.mail:mail:jar:1.4:compile
 [INFO] |  | | \- javax.activation:activation:jar:1.1:compile
 [INFO] |  | \- jline:jline:jar:0.9.94:compile
 [INFO] |  +- org.apache.solr:solr-noggit:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- 
 org.apache.lucene:lucene-analyzers-phonetic:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-highlighter:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-memory:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-misc:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-queryparser:jar:4.0-SNAPSHOT:compile
 [INFO] |  |  \- org.apache.lucene:lucene-sandbox:jar:4.0-SNAPSHOT:compile
 [INFO] |  | \- jakarta-regexp:jakarta-regexp:jar:1.4:compile
 [INFO] |  +- org.apache.lucene:lucene-spatial:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-suggest:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-grouping:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.solr:solr-commons-csv:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- commons-codec:commons-codec:jar:1.4:compile
 [INFO] |  +- commons-fileupload:commons-fileupload:jar:1.2.1:compile
 [INFO] |  +- commons-httpclient:commons-httpclient:jar:3.1:compile
 [INFO] |  |  \- commons-logging:commons-logging:jar:1.0.4:compile
 [INFO] |  +- commons-io:commons-io:jar:1.4:compile
 [INFO] |  +- org.apache.velocity:velocity:jar:1.6.4:compile
 [INFO] |  |  +- 

[jira] [Resolved] (LUCENE-3506) tests for verifying that assertions are enabled do nothing since they ignore AssertionError

2011-10-25 Thread Doron Cohen (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Doron Cohen resolved LUCENE-3506.
-

Resolution: Fixed

Fixed: 
- r1188777 - trunk
- r1188801 - 3x

Mike, Uwe, Robert, thanks for reviewing!


 tests for verifying that assertions are enabled do nothing since they ignore 
 AssertionError
 ---

 Key: LUCENE-3506
 URL: https://issues.apache.org/jira/browse/LUCENE-3506
 Project: Lucene - Java
  Issue Type: Bug
  Components: general/test
Reporter: Doron Cohen
Assignee: Doron Cohen
Priority: Minor
 Attachments: LUCENE-3506.patch, LUCENE-3506.patch


 Follow-up from LUCENE-3501

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Resolved] (LUCENE-3534) Backport FilteredQuery/IndexSearcher changes to 3.x: Remove filter logic from IndexSearcher and delegate to FilteredQuery

2011-10-25 Thread Uwe Schindler (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler resolved LUCENE-3534.
---

   Resolution: Fixed
Fix Version/s: 3.5
Lucene Fields: New,Patch Available  (was: New)

Committed 3.x revision: 1188805

 Backport FilteredQuery/IndexSearcher changes to 3.x: Remove filter logic from 
 IndexSearcher and delegate to FilteredQuery
 -

 Key: LUCENE-3534
 URL: https://issues.apache.org/jira/browse/LUCENE-3534
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Uwe Schindler
Assignee: Uwe Schindler
 Fix For: 3.5

 Attachments: LUCENE-3463.patch, LUCENE-3534.patch


 Spinoff from LUCENE-1536: We simplified the code in IndexSearcher to no 
 longer do the filtering there, instead wrap all Query with FilteredQuery, if 
 a non-null filter is given. The conjunction code would then only exist in 
 FilteredQuery which makes it easier to maintain. Currently both 
 implementations differ in 3.x, in trunk we used the more optimized 
 IndexSearcher variant with addition of a simplified in-order conjunction code.
 This issue will backport those changes (without random access bits).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-3506) tests for verifying that assertions are enabled do nothing since they ignore AssertionError

2011-10-25 Thread Dawid Weiss (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135247#comment-13135247
 ] 

Dawid Weiss commented on LUCENE-3506:
-

Err... how is this different:
{code}
assert Boolean.FALSE.booleanValue();
{code}
from 
{code}
assert false;
{code}

Is there any compile-time code elimination? I ask specifically because I've 
implemented a dedicated validator for this purpose in RandomizedRunner here:

https://github.com/carrotsearch/randomizedtesting/blob/master/runner/src/main/java/com/carrotsearch/randomizedtesting/validators/EnsureAssertionsEnabled.java

and this seems to work just fine (checked with and without -ea).

 tests for verifying that assertions are enabled do nothing since they ignore 
 AssertionError
 ---

 Key: LUCENE-3506
 URL: https://issues.apache.org/jira/browse/LUCENE-3506
 Project: Lucene - Java
  Issue Type: Bug
  Components: general/test
Reporter: Doron Cohen
Assignee: Doron Cohen
Priority: Minor
 Attachments: LUCENE-3506.patch, LUCENE-3506.patch


 Follow-up from LUCENE-3501

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



RE: [jira] [Commented] (LUCENE-3506) tests for verifying that assertions are enabled do nothing since they ignore AssertionError

2011-10-25 Thread Uwe Schindler
That is different, because assert false; will make javac angry - it simply 
refuses to compile this (some error like unreachable statement blabla). When I 
implemented this test for the first time, this was the only way to get this 
running, copied from some code example on the net (I think it was the assertion 
guide shipped with Java 1.4).

-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de


 -Original Message-
 From: Dawid Weiss (Commented) (JIRA) [mailto:j...@apache.org]
 Sent: Tuesday, October 25, 2011 7:21 PM
 To: dev@lucene.apache.org
 Subject: [jira] [Commented] (LUCENE-3506) tests for verifying that assertions
 are enabled do nothing since they ignore AssertionError
 
 
 [ https://issues.apache.org/jira/browse/LUCENE-
 3506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-
 tabpanelfocusedCommentId=13135247#comment-13135247 ]
 
 Dawid Weiss commented on LUCENE-3506:
 -
 
 Err... how is this different:
 {code}
 assert Boolean.FALSE.booleanValue();
 {code}
 from
 {code}
 assert false;
 {code}
 
 Is there any compile-time code elimination? I ask specifically because I've
 implemented a dedicated validator for this purpose in RandomizedRunner
 here:
 
 https://github.com/carrotsearch/randomizedtesting/blob/master/runner/src/m
 ain/java/com/carrotsearch/randomizedtesting/validators/EnsureAssertionsEna
 bled.java
 
 and this seems to work just fine (checked with and without -ea).
 
  tests for verifying that assertions are enabled do nothing since they
  ignore AssertionError
  --
  -
 
  Key: LUCENE-3506
  URL: https://issues.apache.org/jira/browse/LUCENE-3506
  Project: Lucene - Java
   Issue Type: Bug
   Components: general/test
 Reporter: Doron Cohen
 Assignee: Doron Cohen
 Priority: Minor
  Attachments: LUCENE-3506.patch, LUCENE-3506.patch
 
 
  Follow-up from LUCENE-3501
 
 --
 This message is automatically generated by JIRA.
 If you think it was sent incorrectly, please contact your JIRA administrators:
 https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
 For more information on JIRA, see: http://www.atlassian.com/software/jira
 
 
 
 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional
 commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



Re: Request for Feedback for Patch to Allow DIH to Archive Files

2011-10-25 Thread Martijn v Groningen
Thanks Josh!

On 25 October 2011 14:41, Josh Harness josh.harn...@jtv.com wrote:
 Martijn,

  Thanks for the feedback. I have created SOLR-2851 for this feature
 request. Please let me know if there's anything else you'd like me to do.

 Thanks!

 Josh

 On Tue, Oct 25, 2011 at 2:29 AM, Martijn v Groningen
 martijn.v.gronin...@gmail.com wrote:

 Hi Josh,

 I think this functionality is useful. I'd create an Jira issue and
 attach your code as a patch. I think that the functionality should be
 added to the FileListEntityProcessor since it seems to be a more
 natural place for it. Maybe we need something more generic, like a
 post action if a file has been processed.

 Martijn

 On 24 October 2011 21:31, Josh Harness josh.harn...@jtv.com wrote:
  Hi -
 
   We are using SOLR to process XML input files using the Data Import
  Handler. I didn't see a way to move the xml files out of the way after
  processing, so I wrote a small extension to allow this. The How to
  Contribute page says to pitch the request to the developer list in
  order to
  decide whether or not to submit a patch. As such, here goes:
 
   The new code basically extends FileDataSource and wraps the
  underlying
  reader such that when the close method on the input stream is called,
  the
  file is moved to a configurable archive directory. It is unclear to me
  whether this is the correct place to put it (I pondered changing the
  FileListEntityProcessor but this somehow felt safer). I realize that a
  more
  robust implementation would consider the success status of the file
  being
  processed and would also allow for configurable policies rather than a
  concrete implementation. Nonetheless, I didn't want the perfect to be
  the
  enemy of the good.
 
   Please peruse the attached source code file and provide feedback as
  to
  the merit of the idea, whether I ought to submit a JIRA ticket/patch and
  if
  my approach is correct.
 
  Thanks!
 
  Josh Harness
 
 
  -
  To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
  For additional commands, e-mail: dev-h...@lucene.apache.org
 



 --
 Met vriendelijke groet,

 Martijn van Groningen

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org






-- 
Met vriendelijke groet,

Martijn van Groningen

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-3506) tests for verifying that assertions are enabled do nothing since they ignore AssertionError

2011-10-25 Thread Uwe Schindler (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135256#comment-13135256
 ] 

Uwe Schindler commented on LUCENE-3506:
---

It may no longer apply to Java 6 javac, but when I implemented this test for 
the first time (was Java 4 or Java 5) the assert false; made javac angry - it 
simply refused to compile this (some error like unreachable statement blabla). 
This was the only way to get this running, copied from some code example on the 
net (I think it was the assertion guide shipped with Java 1.4).

 tests for verifying that assertions are enabled do nothing since they ignore 
 AssertionError
 ---

 Key: LUCENE-3506
 URL: https://issues.apache.org/jira/browse/LUCENE-3506
 Project: Lucene - Java
  Issue Type: Bug
  Components: general/test
Reporter: Doron Cohen
Assignee: Doron Cohen
Priority: Minor
 Attachments: LUCENE-3506.patch, LUCENE-3506.patch


 Follow-up from LUCENE-3501

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



Re: (LUCENE-3506) tests for verifying that assertions are enabled do nothing since they ignore AssertionError

2011-10-25 Thread Dawid Weiss
Ok, thanks Uwe. Must have been a javac problem then because I compiled
it fine with Maven (javac is the default if I'm not mistaken).

Dawid

On Tue, Oct 25, 2011 at 7:28 PM, Uwe Schindler (Commented) (JIRA)
j...@apache.org wrote:

    [ 
 https://issues.apache.org/jira/browse/LUCENE-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135256#comment-13135256
  ]

 Uwe Schindler commented on LUCENE-3506:
 ---

 It may no longer apply to Java 6 javac, but when I implemented this test for 
 the first time (was Java 4 or Java 5) the assert false; made javac angry - 
 it simply refused to compile this (some error like unreachable statement 
 blabla). This was the only way to get this running, copied from some code 
 example on the net (I think it was the assertion guide shipped with Java 1.4).

 tests for verifying that assertions are enabled do nothing since they ignore 
 AssertionError
 ---

                 Key: LUCENE-3506
                 URL: https://issues.apache.org/jira/browse/LUCENE-3506
             Project: Lucene - Java
          Issue Type: Bug
          Components: general/test
            Reporter: Doron Cohen
            Assignee: Doron Cohen
            Priority: Minor
         Attachments: LUCENE-3506.patch, LUCENE-3506.patch


 Follow-up from LUCENE-3501

 --
 This message is automatically generated by JIRA.
 If you think it was sent incorrectly, please contact your JIRA 
 administrators: 
 https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
 For more information on JIRA, see: http://www.atlassian.com/software/jira



 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[JENKINS] Lucene-Solr-tests-only-trunk - Build # 10995 - Still Failing

2011-10-25 Thread Apache Jenkins Server
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/10995/

All tests passed

Build Log (for compile errors):
[...truncated 13149 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



joins and filter queries effecting scoring

2011-10-25 Thread Jason Toy
I sent this question on the user mailing lists, but didn't hear any replies.


I have 2 types of docs, users and posts.
I want to view all the docs that belong to certain users by joining posts
and users together.  I have to filter the users with a filter query of
is_active_boolean:true so that the score is not effected,but since I do a
join, I have to move the filter query to the query parameter so that I can
get the filter applied. The problem is that since the is_active_boolean is
moved to the query, the score is affected which returns back an order that I
don't want.
  If I leave the is_active_boolean:true in the fq paramater, I get no
results back.

My question is how can I apply a filter query to users so that the score is
not effected?


RE: [JENKINS] Lucene-Solr-tests-only-trunk - Build # 10995 - Still Failing

2011-10-25 Thread Uwe Schindler
Another Javadoc failure, sorry. Should be fine now.

-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de


 -Original Message-
 From: Apache Jenkins Server [mailto:jenk...@builds.apache.org]
 Sent: Tuesday, October 25, 2011 8:17 PM
 To: dev@lucene.apache.org
 Subject: [JENKINS] Lucene-Solr-tests-only-trunk - Build # 10995 - Still 
 Failing
 
 Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/10995/
 
 All tests passed
 
 Build Log (for compile errors):
 [...truncated 13149 lines...]
 
 
 
 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional
 commands, e-mail: dev-h...@lucene.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



Re: (LUCENE-3506) tests for verifying that assertions are enabled do nothing since they ignore AssertionError

2011-10-25 Thread Dawid Weiss
 I don't remember exactly what happened, but it appeared to me as a safe way 
 to not make any optimizer/javac/... get angry.

I think I know what happened: that had to be a bug in javac because
any code around the assertion is still physically reachable (assert is
after all implemented as a compound if/ throw clause). As for Version
-- constants propagation at compile time can be problematic, but it is
documented well in that Lucene snippet, everything's fine, didn't have
problems understanding that particular piece. Thanks,

Dawid

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Resolved] (SOLR-2055) CLONE -woodstox dependency in maven solrj has wrong groupId

2011-10-25 Thread Steven Rowe (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-2055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steven Rowe resolved SOLR-2055.
---

   Resolution: Fixed
Fix Version/s: 3.1
 Assignee: Steven Rowe  (was: Shalin Shekhar Mangar)

 CLONE -woodstox dependency in maven solrj has wrong groupId
 ---

 Key: SOLR-2055
 URL: https://issues.apache.org/jira/browse/SOLR-2055
 Project: Solr
  Issue Type: Bug
  Components: clients - java
Affects Versions: 1.4.1
Reporter: Sebastian Annies
Assignee: Steven Rowe
 Fix For: 3.1


 Woodstox project has long changed their groupId and artifacts under the old 
 groupId are actually not defined correctly in the maven2, causing various 
 silly issues with maven:
 pom file of Solrj 1.4-SNAPSHOT has woodstox Sax dependency defined as 
 dependency
   groupIdwoodstox/groupId
   artifactIdwstx-asl/artifactId
   version3.2.7/version
 /dependency
 where the correct dependency should be 
   dependency
 groupIdorg.codehaus.woodstox/groupId
 artifactIdwstx-asl/artifactId
 version3.2.7/version
   /dependency

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Resolved] (SOLR-1950) SolrJ POM still refers to stax parser

2011-10-25 Thread Steven Rowe (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steven Rowe resolved SOLR-1950.
---

   Resolution: Fixed
Fix Version/s: 3.1
 Assignee: Steven Rowe

 SolrJ POM still refers to stax parser
 -

 Key: SOLR-1950
 URL: https://issues.apache.org/jira/browse/SOLR-1950
 Project: Solr
  Issue Type: Bug
  Components: clients - java
Affects Versions: 1.4
Reporter: Peter Karich
Assignee: Steven Rowe
Priority: Minor
  Labels: maven, pom
 Fix For: 3.1

   Original Estimate: 1h
  Remaining Estimate: 1h

 See the issue at https://issues.apache.org/jira/browse/SOLR-787 which seems 
 to be incorrectly fixed. (I cannot reopen that issue, so I create this one 
 here)
 Using the following deps:
dependency
 groupIdorg.apache.solr/groupId
 artifactIdsolr-solrj/artifactId
 version1.4.0/version
 /dependency
 dependency
 artifactIdsolr-core/artifactId
 groupIdorg.apache.solr/groupId
 version1.4.0/version
 /dependency
 will lead to duplicate jars. (Solr uses woodstox and SolrJ uses the different 
 artifact (but same jar) org.codehaus.woodstox )
 But maybe the artifacts are only incorrectly deployed? Where can I find the 
 original pom files?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (LUCENE-3509) Add settings to IWC to optimize IDV indices for CPU or RAM respectivly

2011-10-25 Thread Martijn van Groningen (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-3509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Martijn van Groningen updated LUCENE-3509:
--

Attachment: LUCENE-3509.patch

Updated the patch. The option with name sortedBytesFasterButMoreRam is now in 
DocValuesWriterBase class and defaults to true.

 Add settings to IWC to optimize IDV indices for CPU or RAM respectivly
 --

 Key: LUCENE-3509
 URL: https://issues.apache.org/jira/browse/LUCENE-3509
 Project: Lucene - Java
  Issue Type: Improvement
  Components: core/index
Affects Versions: 4.0
Reporter: Simon Willnauer
Priority: Minor
 Fix For: 4.0

 Attachments: LUCENE-3509.patch, LUCENE-3509.patch


 spinnoff from LUCENE-3496 - we are seeing much better performance if required 
 bits for PackedInts are rounded up to a 8/16/32/64. We should add this option 
 to IWC and default to round up ie. more RAM  faster lookups.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Updated] (SOLR-2849) Solr maven dependencies: logging

2011-10-25 Thread David Smiley (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley updated SOLR-2849:
---

Attachment: SOLR-2849_maven_dependencies.patch

I updated the patch to account for another problem I noticed.  It turns out 
that a provided scoped dependency is not transitive. solr-core has a provided 
dependency on the servlet api. As a consequence, dependencies on solr-core 
(such as solr-test-framework) didn't pick up this dependency, and so my project 
using solr-test-framework otherwise had to add this explicit dependency.  So I 
made it this dependency compile scope in solr-core, and then in the webapp I 
added the dependency to specify provided, so it doesn't end up in the war.

 Solr maven dependencies: logging
 

 Key: SOLR-2849
 URL: https://issues.apache.org/jira/browse/SOLR-2849
 Project: Solr
  Issue Type: Improvement
  Components: Build
Affects Versions: 4.0
Reporter: David Smiley
Priority: Trivial
 Attachments: SOLR-2849_maven_dependencies.patch, 
 SOLR-2849_maven_dependencies.patch


 I was looking at my maven based project's Solr-core dependencies (trunk), and 
 observed some issues that I think should be fixed in Solr's maven poms. I ran 
 {{mvn dependency:tree}} -- the output is further below.  There are two 
 changes I see needed, related to logging:
 * slf4j-jdk14 should be runtime scope, and optional.
 * httpclient depends on commons-logging.  Exclude this dependency from the 
 httpclient dependency, and add a dependency on jcl-over-slf4j with compile 
 scope.
 * Zookeeper depends on Log4j, unfortunately. There is an issue to change this 
 to SLF4J: ZOOKEEPER-850. In the mean time we should exclude it and use 
 log4j-over-slf4j with compile scope, at the solrj pom.
 As an aside, it's unfortunate to see all those velocity dependencies.  It 
 even depends on struts -- seriously?!  I hope solritas gets put back into a 
 contrib sometime: SOLR-2588
 Steve, if you'd like to me to create the patch, I will.
 {code}
 [INFO] +- org.apache.solr:solr-core:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.solr:solr-solrj:jar:4.0-SNAPSHOT:compile
 [INFO] |  |  \- org.apache.zookeeper:zookeeper:jar:3.3.3:compile
 [INFO] |  | +- log4j:log4j:jar:1.2.15:compile
 [INFO] |  | |  \- javax.mail:mail:jar:1.4:compile
 [INFO] |  | | \- javax.activation:activation:jar:1.1:compile
 [INFO] |  | \- jline:jline:jar:0.9.94:compile
 [INFO] |  +- org.apache.solr:solr-noggit:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- 
 org.apache.lucene:lucene-analyzers-phonetic:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-highlighter:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-memory:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-misc:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-queryparser:jar:4.0-SNAPSHOT:compile
 [INFO] |  |  \- org.apache.lucene:lucene-sandbox:jar:4.0-SNAPSHOT:compile
 [INFO] |  | \- jakarta-regexp:jakarta-regexp:jar:1.4:compile
 [INFO] |  +- org.apache.lucene:lucene-spatial:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-suggest:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-grouping:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.solr:solr-commons-csv:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- commons-codec:commons-codec:jar:1.4:compile
 [INFO] |  +- commons-fileupload:commons-fileupload:jar:1.2.1:compile
 [INFO] |  +- commons-httpclient:commons-httpclient:jar:3.1:compile
 [INFO] |  |  \- commons-logging:commons-logging:jar:1.0.4:compile
 [INFO] |  +- commons-io:commons-io:jar:1.4:compile
 [INFO] |  +- org.apache.velocity:velocity:jar:1.6.4:compile
 [INFO] |  |  +- commons-collections:commons-collections:jar:3.2.1:compile
 [INFO] |  |  \- oro:oro:jar:2.0.8:compile
 [INFO] |  +- org.apache.velocity:velocity-tools:jar:2.0:compile
 [INFO] |  |  +- commons-beanutils:commons-beanutils:jar:1.7.0:compile
 [INFO] |  |  +- commons-digester:commons-digester:jar:1.8:compile
 [INFO] |  |  +- commons-chain:commons-chain:jar:1.1:compile
 [INFO] |  |  +- commons-validator:commons-validator:jar:1.3.1:compile
 [INFO] |  |  +- dom4j:dom4j:jar:1.1:compile
 [INFO] |  |  +- sslext:sslext:jar:1.2-0:compile
 [INFO] |  |  +- org.apache.struts:struts-core:jar:1.3.8:compile
 [INFO] |  |  |  \- antlr:antlr:jar:2.7.2:compile
 [INFO] |  |  +- org.apache.struts:struts-taglib:jar:1.3.8:compile
 [INFO] |  |  \- org.apache.struts:struts-tiles:jar:1.3.8:compile
 [INFO] |  +- org.slf4j:slf4j-jdk14:jar:1.6.1:compile
 [INFO] |  \- org.codehaus.woodstox:wstx-asl:jar:3.2.7:runtime
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 

[jira] [Commented] (LUCENE-3509) Add settings to IWC to optimize IDV indices for CPU or RAM respectivly

2011-10-25 Thread Simon Willnauer (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135365#comment-13135365
 ] 

Simon Willnauer commented on LUCENE-3509:
-

bq. Updated the patch. The option with name sortedBytesFasterButMoreRam is 
now in DocValuesWriterBase class and defaults to true.

it might make sense to give it a more general name since we use packed ints in 
various places? The question is if we want to use this option elsewhere... not 
sure though

 Add settings to IWC to optimize IDV indices for CPU or RAM respectivly
 --

 Key: LUCENE-3509
 URL: https://issues.apache.org/jira/browse/LUCENE-3509
 Project: Lucene - Java
  Issue Type: Improvement
  Components: core/index
Affects Versions: 4.0
Reporter: Simon Willnauer
Priority: Minor
 Fix For: 4.0

 Attachments: LUCENE-3509.patch, LUCENE-3509.patch


 spinnoff from LUCENE-3496 - we are seeing much better performance if required 
 bits for PackedInts are rounded up to a 8/16/32/64. We should add this option 
 to IWC and default to round up ie. more RAM  faster lookups.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-2849) Solr maven dependencies: logging

2011-10-25 Thread Steven Rowe (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135372#comment-13135372
 ] 

Steven Rowe commented on SOLR-2849:
---

bq. I noticed the ant distribution suggests woodstox is a solrj dependency. I 
opted to not include it here. The client doesn't actually depend on it, and 
SolrJ is usually going to use the efficient binary format any way.

While it's true that Solr's dist target doesn't insist that woodstox is a solrj 
dependency (hence your suggests), there is no mechanism for insistence; all 
Solr modules' dependencies under the Ant build are suggestions.

David, as you may recall, we've previously had this discussion [on 
SOLR-2756|https://issues.apache.org/jira/browse/SOLR-2756#comment-13103074] and 
[on #lucene-dev 
IRC|http://colabti.org/irclogger/irclogger_log/lucene-dev?date=2011-09-12#l143].
 
[This February 2010 Apache CXF email message from Daniel 
Kulp|http://www.mail-archive.com/users@cxf.apache.org/msg12750.html] that 
[Yonik linked to on 
SOLR-2042|https://issues.apache.org/jira/browse/SOLR-2042#comment-12899967] 
provides decent rationale for keeping woodstox as a runtime dependency: it's a 
lot faster than the JVM implementation.  The CXF project's Common Utilities 
module [still has this 
dependency|http://svn.apache.org/viewvc/cxf/trunk/common/common/pom.xml?view=markup#l117]
 (see [the parent 
POM|http://svn.apache.org/viewvc/cxf/trunk/parent/pom.xml?view=markup#l92] for 
the definition of the stax implementation groupId and artifactId).

As I said in the previous version of this argument, getting rid of the woodstox 
dependency should be done project wide, if it is to happen, not just in the 
Maven build.  My position is essentially that the Maven build should track the 
Ant build as closely as possible.

As I'm sure you're aware, all you have to do to not use the woodstox dependency 
when you depend on solrj is add an exclusion.  This is no great hardship.


 Solr maven dependencies: logging
 

 Key: SOLR-2849
 URL: https://issues.apache.org/jira/browse/SOLR-2849
 Project: Solr
  Issue Type: Improvement
  Components: Build
Affects Versions: 4.0
Reporter: David Smiley
Priority: Trivial
 Attachments: SOLR-2849_maven_dependencies.patch, 
 SOLR-2849_maven_dependencies.patch


 I was looking at my maven based project's Solr-core dependencies (trunk), and 
 observed some issues that I think should be fixed in Solr's maven poms. I ran 
 {{mvn dependency:tree}} -- the output is further below.  There are two 
 changes I see needed, related to logging:
 * slf4j-jdk14 should be runtime scope, and optional.
 * httpclient depends on commons-logging.  Exclude this dependency from the 
 httpclient dependency, and add a dependency on jcl-over-slf4j with compile 
 scope.
 * Zookeeper depends on Log4j, unfortunately. There is an issue to change this 
 to SLF4J: ZOOKEEPER-850. In the mean time we should exclude it and use 
 log4j-over-slf4j with compile scope, at the solrj pom.
 As an aside, it's unfortunate to see all those velocity dependencies.  It 
 even depends on struts -- seriously?!  I hope solritas gets put back into a 
 contrib sometime: SOLR-2588
 Steve, if you'd like to me to create the patch, I will.
 {code}
 [INFO] +- org.apache.solr:solr-core:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.solr:solr-solrj:jar:4.0-SNAPSHOT:compile
 [INFO] |  |  \- org.apache.zookeeper:zookeeper:jar:3.3.3:compile
 [INFO] |  | +- log4j:log4j:jar:1.2.15:compile
 [INFO] |  | |  \- javax.mail:mail:jar:1.4:compile
 [INFO] |  | | \- javax.activation:activation:jar:1.1:compile
 [INFO] |  | \- jline:jline:jar:0.9.94:compile
 [INFO] |  +- org.apache.solr:solr-noggit:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- 
 org.apache.lucene:lucene-analyzers-phonetic:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-highlighter:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-memory:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-misc:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-queryparser:jar:4.0-SNAPSHOT:compile
 [INFO] |  |  \- org.apache.lucene:lucene-sandbox:jar:4.0-SNAPSHOT:compile
 [INFO] |  | \- jakarta-regexp:jakarta-regexp:jar:1.4:compile
 [INFO] |  +- org.apache.lucene:lucene-spatial:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-suggest:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-grouping:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.solr:solr-commons-csv:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- commons-codec:commons-codec:jar:1.4:compile
 [INFO] |  +- commons-fileupload:commons-fileupload:jar:1.2.1:compile
 [INFO] |  +- commons-httpclient:commons-httpclient:jar:3.1:compile
 [INFO] |  |  \- 

[jira] [Commented] (SOLR-2849) Solr maven dependencies: logging

2011-10-25 Thread David Smiley (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135375#comment-13135375
 ] 

David Smiley commented on SOLR-2849:


Ok.  FWIW, I didn't introduce the woodstox or guava discrepancies in the build 
between maven and ant; I just opted to let them be.  I completely understand 
that it is better to be consistent.  I will update the patch to fix these two 
discrepancies. 

 Solr maven dependencies: logging
 

 Key: SOLR-2849
 URL: https://issues.apache.org/jira/browse/SOLR-2849
 Project: Solr
  Issue Type: Improvement
  Components: Build
Affects Versions: 4.0
Reporter: David Smiley
Priority: Trivial
 Attachments: SOLR-2849_maven_dependencies.patch, 
 SOLR-2849_maven_dependencies.patch


 I was looking at my maven based project's Solr-core dependencies (trunk), and 
 observed some issues that I think should be fixed in Solr's maven poms. I ran 
 {{mvn dependency:tree}} -- the output is further below.  There are two 
 changes I see needed, related to logging:
 * slf4j-jdk14 should be runtime scope, and optional.
 * httpclient depends on commons-logging.  Exclude this dependency from the 
 httpclient dependency, and add a dependency on jcl-over-slf4j with compile 
 scope.
 * Zookeeper depends on Log4j, unfortunately. There is an issue to change this 
 to SLF4J: ZOOKEEPER-850. In the mean time we should exclude it and use 
 log4j-over-slf4j with compile scope, at the solrj pom.
 As an aside, it's unfortunate to see all those velocity dependencies.  It 
 even depends on struts -- seriously?!  I hope solritas gets put back into a 
 contrib sometime: SOLR-2588
 Steve, if you'd like to me to create the patch, I will.
 {code}
 [INFO] +- org.apache.solr:solr-core:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.solr:solr-solrj:jar:4.0-SNAPSHOT:compile
 [INFO] |  |  \- org.apache.zookeeper:zookeeper:jar:3.3.3:compile
 [INFO] |  | +- log4j:log4j:jar:1.2.15:compile
 [INFO] |  | |  \- javax.mail:mail:jar:1.4:compile
 [INFO] |  | | \- javax.activation:activation:jar:1.1:compile
 [INFO] |  | \- jline:jline:jar:0.9.94:compile
 [INFO] |  +- org.apache.solr:solr-noggit:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- 
 org.apache.lucene:lucene-analyzers-phonetic:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-highlighter:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-memory:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-misc:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-queryparser:jar:4.0-SNAPSHOT:compile
 [INFO] |  |  \- org.apache.lucene:lucene-sandbox:jar:4.0-SNAPSHOT:compile
 [INFO] |  | \- jakarta-regexp:jakarta-regexp:jar:1.4:compile
 [INFO] |  +- org.apache.lucene:lucene-spatial:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-suggest:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.lucene:lucene-grouping:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- org.apache.solr:solr-commons-csv:jar:4.0-SNAPSHOT:compile
 [INFO] |  +- commons-codec:commons-codec:jar:1.4:compile
 [INFO] |  +- commons-fileupload:commons-fileupload:jar:1.2.1:compile
 [INFO] |  +- commons-httpclient:commons-httpclient:jar:3.1:compile
 [INFO] |  |  \- commons-logging:commons-logging:jar:1.0.4:compile
 [INFO] |  +- commons-io:commons-io:jar:1.4:compile
 [INFO] |  +- org.apache.velocity:velocity:jar:1.6.4:compile
 [INFO] |  |  +- commons-collections:commons-collections:jar:3.2.1:compile
 [INFO] |  |  \- oro:oro:jar:2.0.8:compile
 [INFO] |  +- org.apache.velocity:velocity-tools:jar:2.0:compile
 [INFO] |  |  +- commons-beanutils:commons-beanutils:jar:1.7.0:compile
 [INFO] |  |  +- commons-digester:commons-digester:jar:1.8:compile
 [INFO] |  |  +- commons-chain:commons-chain:jar:1.1:compile
 [INFO] |  |  +- commons-validator:commons-validator:jar:1.3.1:compile
 [INFO] |  |  +- dom4j:dom4j:jar:1.1:compile
 [INFO] |  |  +- sslext:sslext:jar:1.2-0:compile
 [INFO] |  |  +- org.apache.struts:struts-core:jar:1.3.8:compile
 [INFO] |  |  |  \- antlr:antlr:jar:2.7.2:compile
 [INFO] |  |  +- org.apache.struts:struts-taglib:jar:1.3.8:compile
 [INFO] |  |  \- org.apache.struts:struts-tiles:jar:1.3.8:compile
 [INFO] |  +- org.slf4j:slf4j-jdk14:jar:1.6.1:compile
 [INFO] |  \- org.codehaus.woodstox:wstx-asl:jar:3.2.7:runtime
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Created] (SOLR-2852) SolrJ doesn't need woodstox jar

2011-10-25 Thread David Smiley (Created) (JIRA)
SolrJ doesn't need woodstox jar
---

 Key: SOLR-2852
 URL: https://issues.apache.org/jira/browse/SOLR-2852
 Project: Solr
  Issue Type: Improvement
  Components: clients - java
Reporter: David Smiley
Priority: Trivial


The /dist/solrj-lib/ directory contains wstx-asl-3.2.7.jar (Woodstox StAX API). 
 SolrJ doesn't actually have any type of dependency on this library. The maven 
build doesn't have it as a dependency and the tests pass.  Perhaps Woodstox is 
faster than the JDK's StAX, I don't know, but I find that point quite moot 
since SolrJ can use the efficient binary format.  Woodstox is not a small 
library either, weighting in at 524KB, and of course if someone actually wants 
to use it, they can.

I propose woodstox be removed as a SolrJ dependency.  I am *not* proposing it 
be removed as a Solr WAR dependency since it is actually required there due to 
an obscure XSLT issue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[JENKINS] Lucene-Solr-tests-only-trunk - Build # 10997 - Failure

2011-10-25 Thread Apache Jenkins Server
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/10997/

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.update.AutoCommitTest

Error Message:
java.lang.AssertionError: directory of test was not closed, opened from: 
org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:33)

Stack Trace:
java.lang.RuntimeException: java.lang.AssertionError: directory of test was not 
closed, opened from: 
org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:33)
at 
org.apache.lucene.util.LuceneTestCase.afterClassLuceneTestCaseJ4(LuceneTestCase.java:472)
at 
org.apache.lucene.util.LuceneTestCase.checkResourcesAfterClass(LuceneTestCase.java:530)
at 
org.apache.lucene.util.LuceneTestCase.afterClassLuceneTestCaseJ4(LuceneTestCase.java:440)




Build Log (for compile errors):
[...truncated 7819 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-2588) Make Velocity an optional dependency in SolrCore

2011-10-25 Thread Erik Hatcher (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-2588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135394#comment-13135394
 ] 

Erik Hatcher commented on SOLR-2588:


I'm going to start making some commits on this issue.  I've re-done the work 
locally on a fresh trunk checkout just to make sure things are clean, and I'm 
not seeing the test failures I saw before.  Maybe some dependencies on the 
example environment were removed?   Or...?

Anyway, I'll get this rolling in the next day or so and hopefully have this 
resolved then.

 Make Velocity an optional dependency in SolrCore
 

 Key: SOLR-2588
 URL: https://issues.apache.org/jira/browse/SOLR-2588
 Project: Solr
  Issue Type: Wish
Affects Versions: 3.2
Reporter: Gunnar Wagenknecht
Assignee: Erik Hatcher
Priority: Minor
 Fix For: 3.5, 4.0

 Attachments: SOLR-2588.patch, SOLR-2588.patch, SOLR-2588.patch, 
 SOLR-2588.patch, SOLR-2588.patch, 
 SOLR-2588_Don_t_fail_if_velocity_libs_not_present_.patch


 In 1.4. it was fine to run Solr without Velocity on the classpath. However, 
 in 3.2. SolrCore won't load because of a hard reference to the Velocity 
 response writer in a static initializer.
 {noformat}
 ... ERROR org.apache.solr.core.CoreContainer - 
 java.lang.NoClassDefFoundError: org/apache/velocity/context/Context
   at org.apache.solr.core.SolrCore.clinit(SolrCore.java:1447)
   at org.apache.solr.core.CoreContainer.create(CoreContainer.java:463)
   at org.apache.solr.core.CoreContainer.load(CoreContainer.java:316)
   at org.apache.solr.core.CoreContainer.load(CoreContainer.java:207)
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Created] (LUCENE-3536) lucene-join and lucene-facet artifacts are missing from 'ant generate-maven-artifacts' output

2011-10-25 Thread Steven Rowe (Created) (JIRA)
lucene-join and lucene-facet artifacts are missing from 'ant 
generate-maven-artifacts' output
-

 Key: LUCENE-3536
 URL: https://issues.apache.org/jira/browse/LUCENE-3536
 Project: Lucene - Java
  Issue Type: Bug
  Components: general/build
Affects Versions: 4.0
Reporter: Steven Rowe
Priority: Minor


David Smiley reported [on #lucene-dev 
IRC|http://colabti.org/irclogger/irclogger_log/lucene-dev?date=2011-10-25#l243] 
that lucene-facet module was missing from the maven artifacts produced by the 
nightly Jenkins Maven trunk build 
[https://builds.apache.org/job/Lucene-Solr-Maven-trunk/lastSuccessfulBuild/artifact/maven_artifacts/org/apache/lucene/].
  It turns out that the lucene-join module is also missing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-3509) Add settings to IWC to optimize IDV indices for CPU or RAM respectivly

2011-10-25 Thread Martijn van Groningen (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135423#comment-13135423
 ] 

Martijn van Groningen commented on LUCENE-3509:
---

We currently only use it for types BYTES_VAR_SORTED and BYTES_FIXED_SORTED 
hence the name. I think we can use it for the other types as well. This option 
basically decides whether DirectInt or PacketInt is used (in 
PacketInt#getReader()). The DirectInt classes are faster for when using 
BYTES_VAR_SORTED and BYTES_FIXED_SORTED types. I'll try and test if this is 
also the case for for the other types (I think it will be faster for the other 
types as well).

 Add settings to IWC to optimize IDV indices for CPU or RAM respectivly
 --

 Key: LUCENE-3509
 URL: https://issues.apache.org/jira/browse/LUCENE-3509
 Project: Lucene - Java
  Issue Type: Improvement
  Components: core/index
Affects Versions: 4.0
Reporter: Simon Willnauer
Priority: Minor
 Fix For: 4.0

 Attachments: LUCENE-3509.patch, LUCENE-3509.patch


 spinnoff from LUCENE-3496 - we are seeing much better performance if required 
 bits for PackedInts are rounded up to a 8/16/32/64. We should add this option 
 to IWC and default to round up ie. more RAM  faster lookups.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Resolved] (LUCENE-3536) lucene-join and lucene-facet artifacts are missing from 'ant generate-maven-artifacts' output

2011-10-25 Thread Steven Rowe (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-3536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steven Rowe resolved LUCENE-3536.
-

   Resolution: Fixed
Fix Version/s: 4.0
 Assignee: Steven Rowe
Lucene Fields:   (was: New)

 lucene-join and lucene-facet artifacts are missing from 'ant 
 generate-maven-artifacts' output
 -

 Key: LUCENE-3536
 URL: https://issues.apache.org/jira/browse/LUCENE-3536
 Project: Lucene - Java
  Issue Type: Bug
  Components: general/build
Affects Versions: 4.0
Reporter: Steven Rowe
Assignee: Steven Rowe
Priority: Minor
 Fix For: 4.0

 Attachments: LUCENE-3536.patch


 David Smiley reported [on #lucene-dev 
 IRC|http://colabti.org/irclogger/irclogger_log/lucene-dev?date=2011-10-25#l243]
  that lucene-facet module was missing from the maven artifacts produced by 
 the nightly Jenkins Maven trunk build 
 [https://builds.apache.org/job/Lucene-Solr-Maven-trunk/lastSuccessfulBuild/artifact/maven_artifacts/org/apache/lucene/].
   It turns out that the lucene-join module is also missing.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-2853) SpellCheckCollator.collate method creates the a PossibilityIterator with maxTries instead of maxCollations

2011-10-25 Thread James Dyer (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-2853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135468#comment-13135468
 ] 

James Dyer commented on SOLR-2853:
--

Matt,

Thanks for taking such a deep dive into this code.  Its great to see people 
checking for things like this.  I think the current code is correct, however.  
What PossibilityIterator is returning is a set of word combinations that 
SpellCheckCollator then needs to test against the index.  So 
PossibilityIterator will return up to maxTries word combinations.  But some 
of these possibilities could be nonsense and will return 0 hits when queried 
for against the index. SpellCheckCollator will throw these 0-hit possibilities 
out, trying each possibility until it has as many good ones as requested by 
maxCollations, or until it has exhausted the list.  (If the user sets 
maxCollationTries to zero, SpellCheckCollator won't test any and in this case 
will just return the first maxCollations possibilities back to the user.)  
Make sense? 

 SpellCheckCollator.collate method creates the a PossibilityIterator with 
 maxTries instead of maxCollations
 --

 Key: SOLR-2853
 URL: https://issues.apache.org/jira/browse/SOLR-2853
 Project: Solr
  Issue Type: Bug
  Components: spellchecker
Affects Versions: 3.3, 4.0
Reporter: Matt Traynham
Priority: Minor

 Class SpellCheckCollator creates a new possibility iterator based on the 
 spellcheck results.  The iterator is created with: 
 PossibilityIterator possibilityIter = new 
 PossibilityIterator(result.getSuggestions(), maxTries, maxEvaluations);
 The issue is maxTries, should be maxCollations.  Correct me if I'm wrong.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-2403) contrib/HighFreqTerms should use ByteRefs but provide human-readable output

2011-10-25 Thread Steven Rowe (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135522#comment-13135522
 ] 

Steven Rowe commented on LUCENE-2403:
-

The nabble link in the description is dead.  Here is the Lucid version: 
http://www.lucidimagination.com/search/document/4bcb8f5f5ba4dd8/bug_in_contrib_misc_highfreqterms_java

 contrib/HighFreqTerms should use ByteRefs but provide human-readable output
 ---

 Key: LUCENE-2403
 URL: https://issues.apache.org/jira/browse/LUCENE-2403
 Project: Lucene - Java
  Issue Type: Bug
  Components: modules/other
Affects Versions: 3.1
Reporter: Tom Burton-West
Priority: Trivial
 Fix For: 4.0


 contrib HighFreqTerms was upgraded to use the flex APIs but currently 
 displays hex code if you do not give a field argument and strings if you do.
 The conversion to a string should be consistent and should occur just before 
 output rather than when loading the priority queue
 See: 
 http://n3.nabble.com/Bug-in-contrib-misc-HighFreqTerms-java-tc719202.html#a719202

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-3000) Lucene release artifacts should be named apache-lucene-*

2011-10-25 Thread Steven Rowe (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135535#comment-13135535
 ] 

Steven Rowe commented on LUCENE-3000:
-

Grant, can we resolve this as WONTFIX?

 Lucene release artifacts should be named apache-lucene-*
 

 Key: LUCENE-3000
 URL: https://issues.apache.org/jira/browse/LUCENE-3000
 Project: Lucene - Java
  Issue Type: Bug
Affects Versions: 4.0
Reporter: Grant Ingersoll
Priority: Minor
 Fix For: 4.0


 Our artifact names should be prefixed with apache-, as in 
 apache-lucene-4.0-src.tar.gz (or whatever)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-2853) SpellCheckCollator.collate method creates the a PossibilityIterator with maxTries instead of maxCollations

2011-10-25 Thread Matt Traynham (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-2853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135542#comment-13135542
 ] 

Matt Traynham commented on SOLR-2853:
-

Ahh yes, that makes sense.  I was under the assumption that one might want up 
to a maxCollations of possibilities returned, instead of just the first one, 
even if maxTries is zero.  But thanks that clears up my issue.

:) No problem looking into code, it's very interesting work.

 SpellCheckCollator.collate method creates the a PossibilityIterator with 
 maxTries instead of maxCollations
 --

 Key: SOLR-2853
 URL: https://issues.apache.org/jira/browse/SOLR-2853
 Project: Solr
  Issue Type: Bug
  Components: spellchecker
Affects Versions: 3.3, 4.0
Reporter: Matt Traynham
Priority: Minor

 Class SpellCheckCollator creates a new possibility iterator based on the 
 spellcheck results.  The iterator is created with: 
 PossibilityIterator possibilityIter = new 
 PossibilityIterator(result.getSuggestions(), maxTries, maxEvaluations);
 The issue is maxTries, should be maxCollations.  Correct me if I'm wrong.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-3000) Lucene release artifacts should be named apache-lucene-*

2011-10-25 Thread Robert Muir (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135543#comment-13135543
 ] 

Robert Muir commented on LUCENE-3000:
-

apache httpd's artifacts arent even prefixed with apache (just httpd-xxx.tar.gz)

So, I don't think we need to change any naming here.

 Lucene release artifacts should be named apache-lucene-*
 

 Key: LUCENE-3000
 URL: https://issues.apache.org/jira/browse/LUCENE-3000
 Project: Lucene - Java
  Issue Type: Bug
Affects Versions: 4.0
Reporter: Grant Ingersoll
Priority: Minor
 Fix For: 4.0


 Our artifact names should be prefixed with apache-, as in 
 apache-lucene-4.0-src.tar.gz (or whatever)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Closed] (SOLR-2853) SpellCheckCollator.collate method creates the a PossibilityIterator with maxTries instead of maxCollations

2011-10-25 Thread Matt Traynham (Closed) (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-2853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Traynham closed SOLR-2853.
---

Resolution: Invalid

Marking as invalid, based upon Mr. Dyer's comment.

 SpellCheckCollator.collate method creates the a PossibilityIterator with 
 maxTries instead of maxCollations
 --

 Key: SOLR-2853
 URL: https://issues.apache.org/jira/browse/SOLR-2853
 Project: Solr
  Issue Type: Bug
  Components: spellchecker
Affects Versions: 3.3, 4.0
Reporter: Matt Traynham
Priority: Minor

 Class SpellCheckCollator creates a new possibility iterator based on the 
 spellcheck results.  The iterator is created with: 
 PossibilityIterator possibilityIter = new 
 PossibilityIterator(result.getSuggestions(), maxTries, maxEvaluations);
 The issue is maxTries, should be maxCollations.  Correct me if I'm wrong.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-2974) the hudson nightly for lucene should check out lucene by itself

2011-10-25 Thread Steven Rowe (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135552#comment-13135552
 ] 

Steven Rowe commented on LUCENE-2974:
-

This issue is about excluding the top-level {{dev-tools/}} dir from the 
Lucene-only nightly build checkouts, right?

 the hudson nightly for lucene should check out lucene by itself
 ---

 Key: LUCENE-2974
 URL: https://issues.apache.org/jira/browse/LUCENE-2974
 Project: Lucene - Java
  Issue Type: Bug
  Components: general/build
Reporter: Robert Muir
 Fix For: 3.5, 4.0


 Currently its too easy to break the lucene-only packaging and build.
 the hudson job for lucene should check out lucene by itself, this will
 prevent it from being broken.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-2974) the hudson nightly for lucene should check out lucene by itself

2011-10-25 Thread Robert Muir (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1313#comment-1313
 ] 

Robert Muir commented on LUCENE-2974:
-

and solr and modules, too?

Currently, when issuing releases, people want lucene to compile/javadocs/etc by 
itself, and the release manager has to test this (and fix any issues before 
creating an RC, or fix and respin). This is because the lucene src distribution 
is rooted at the lucene/ folder.

So it would be better if hudson tested this continuously rather than it being 
on the RM?


 the hudson nightly for lucene should check out lucene by itself
 ---

 Key: LUCENE-2974
 URL: https://issues.apache.org/jira/browse/LUCENE-2974
 Project: Lucene - Java
  Issue Type: Bug
  Components: general/build
Reporter: Robert Muir
 Fix For: 3.5, 4.0


 Currently its too easy to break the lucene-only packaging and build.
 the hudson job for lucene should check out lucene by itself, this will
 prevent it from being broken.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-2974) the hudson nightly for lucene should check out lucene by itself

2011-10-25 Thread Steven Rowe (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135557#comment-13135557
 ] 

Steven Rowe commented on LUCENE-2974:
-

Currently trunk {{modules/}} is also tested by the Jenkins nightly build.  You 
think this should change?  (If so, I guess a new build for {{modules/}} is in 
order.)

 the hudson nightly for lucene should check out lucene by itself
 ---

 Key: LUCENE-2974
 URL: https://issues.apache.org/jira/browse/LUCENE-2974
 Project: Lucene - Java
  Issue Type: Bug
  Components: general/build
Reporter: Robert Muir
 Fix For: 3.5, 4.0


 Currently its too easy to break the lucene-only packaging and build.
 the hudson job for lucene should check out lucene by itself, this will
 prevent it from being broken.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Resolved] (LUCENE-2866) Unexpected search results

2011-10-25 Thread Steven Rowe (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-2866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steven Rowe resolved LUCENE-2866.
-

   Resolution: Incomplete
 Assignee: Steven Rowe
Lucene Fields:   (was: New)

Alejandro,

It's not clear from your description whether Lucene is the source of the 
problems.

In the future, please post questions like this to the 
java-u...@lucene.apache.org mailing list.  When you do, you should include 
enough information that people can reproduce the problem.  (This usually means 
providing a simple example that triggers the problem.)


 Unexpected search results
 -

 Key: LUCENE-2866
 URL: https://issues.apache.org/jira/browse/LUCENE-2866
 Project: Lucene - Java
  Issue Type: Bug
  Components: core/search
 Environment: *Operating System:* Windows Server 2003 and Windows 
 Server 2008 R2
 *System type:* 32 bits (Win Server 2003) and 64 bits (Win Server 2008)
 *Platform:* Alfresco Community 3.3.g
 *Processor:* Intel Celeron 1.80GHz 1.80GHz
 *RAM Memory*: 2GB
Reporter: Alejandro
Assignee: Steven Rowe

 Hello...
 I'm using Lucene search with Alfresco 3.3.g (I'm not sure what version of 
 Lucene is used), and I'm havin problems when the search get me results... 
 sometimes one search can bring me just 1 result, but when I instantly do the 
 same search at a second time it can bring me a lot of results...
 Sometimes the search takes too much time to bring results...
 and sometimes the search stops at 1000 results.
 I'm using simple and boolean searches and both types have the same mistakes.
 Thanks for reading and for your support.
 Alejandro Villa Betancur

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-2974) the hudson nightly for lucene should check out lucene by itself

2011-10-25 Thread Robert Muir (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135562#comment-13135562
 ] 

Robert Muir commented on LUCENE-2974:
-

I think it should all be tested!

But I think we should try to test what we actually release. Maybe the nightly 
build should even build release artifacts, untar them, and run javadocs/tests 
this way?

And this is also related to the packaging issue (LUCENE-2999) !

 the hudson nightly for lucene should check out lucene by itself
 ---

 Key: LUCENE-2974
 URL: https://issues.apache.org/jira/browse/LUCENE-2974
 Project: Lucene - Java
  Issue Type: Bug
  Components: general/build
Reporter: Robert Muir
 Fix For: 3.5, 4.0


 Currently its too easy to break the lucene-only packaging and build.
 the hudson job for lucene should check out lucene by itself, this will
 prevent it from being broken.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (LUCENE-3452) The native FS lock used in test-framework's o.a.l.util.LuceneJUnitResultFormatter prohibits testing on a multi-user system

2011-10-25 Thread Steven Rowe (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135563#comment-13135563
 ] 

Steven Rowe commented on LUCENE-3452:
-

I just successfully ran all trunk Lucene/Solr tests with this patch, and 
everything passed.

+1 to commit.

 The native FS lock used in test-framework's 
 o.a.l.util.LuceneJUnitResultFormatter prohibits testing on a multi-user system
 --

 Key: LUCENE-3452
 URL: https://issues.apache.org/jira/browse/LUCENE-3452
 Project: Lucene - Java
  Issue Type: Bug
  Components: general/test
Affects Versions: 3.4, 4.0
Reporter: Steven Rowe
Priority: Minor
 Attachments: LUCENE-3452.patch


 {{LuceneJUnitResultFormatter}} uses a lock to buffer test suites' output, so 
 that when run in parallel, they don't interrupt each other when they are 
 displayed on the console.
 The current implementation uses a fixed directory ({{lucene_junit_lock/}} in 
 {{java.io.tmpdir}} (by default {{/tmp/}} on Unix/Linux systems) as the 
 location of this lock.  This functionality was introduced on SOLR-1835.
 As Shawn Heisey reported on SOLR-2739, some tests fail when run as root, but 
 succeed when run as a non-root user.  
 On #lucene IRC today, Shawn wrote:
 {quote}
 (2:06:07 PM) elyograg: Now that I know I can't run the tests as root, I have 
 discovered /tmp/lucene_junit_lock.  Once you run the tests as user A, you 
 cannot run them again as user B until that directory is deleted, and only 
 root or the original user can do so.
 {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Resolved] (LUCENE-3452) The native FS lock used in test-framework's o.a.l.util.LuceneJUnitResultFormatter prohibits testing on a multi-user system

2011-10-25 Thread Robert Muir (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENE-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Muir resolved LUCENE-3452.
-

   Resolution: Fixed
Fix Version/s: 4.0
   3.5

 The native FS lock used in test-framework's 
 o.a.l.util.LuceneJUnitResultFormatter prohibits testing on a multi-user system
 --

 Key: LUCENE-3452
 URL: https://issues.apache.org/jira/browse/LUCENE-3452
 Project: Lucene - Java
  Issue Type: Bug
  Components: general/test
Affects Versions: 3.4, 4.0
Reporter: Steven Rowe
Priority: Minor
 Fix For: 3.5, 4.0

 Attachments: LUCENE-3452.patch


 {{LuceneJUnitResultFormatter}} uses a lock to buffer test suites' output, so 
 that when run in parallel, they don't interrupt each other when they are 
 displayed on the console.
 The current implementation uses a fixed directory ({{lucene_junit_lock/}} in 
 {{java.io.tmpdir}} (by default {{/tmp/}} on Unix/Linux systems) as the 
 location of this lock.  This functionality was introduced on SOLR-1835.
 As Shawn Heisey reported on SOLR-2739, some tests fail when run as root, but 
 succeed when run as a non-root user.  
 On #lucene IRC today, Shawn wrote:
 {quote}
 (2:06:07 PM) elyograg: Now that I know I can't run the tests as root, I have 
 discovered /tmp/lucene_junit_lock.  Once you run the tests as user A, you 
 cannot run them again as user B until that directory is deleted, and only 
 root or the original user can do so.
 {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-1786) Solr (trunk rev. 912116) suffers from PDFBOX-537 [Endless loop in org.apache.pdfbox.pdfparser.BaseParser.parseCOSDictionary()] fixed in PDFbox 1.0?

2011-10-25 Thread Steven Rowe (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135582#comment-13135582
 ] 

Steven Rowe commented on SOLR-1786:
---

Solr Cell upgraded to Tika 0.8, which included PDFbox 1.1.0, in the Solr 3.1 
release.

The Solr 3.5 release will include Tika 0.10, which includes PDFbox 1.6.0.

Likely this problem has been addressed.

Jan, can you test Solr 3.1+ to confirm?

 Solr (trunk rev. 912116) suffers from PDFBOX-537 [Endless loop in 
 org.apache.pdfbox.pdfparser.BaseParser.parseCOSDictionary()]  fixed in PDFbox 
 1.0?
 

 Key: SOLR-1786
 URL: https://issues.apache.org/jira/browse/SOLR-1786
 Project: Solr
  Issue Type: Bug
  Components: contrib - Solr Cell (Tika extraction)
Affects Versions: 1.5
 Environment: Ubuntu 9.10, 32bit
Reporter: Jan Iwaszkiewicz
Priority: Critical
  Labels: PDFbox
 Fix For: 3.5, 4.0


 I tried indexing several thousand PDF documents but could not finish as Solr 
 was falling into an endless loop for some of them, for instance: 
 http://cdsweb.cern.ch/record/702585/files/sl-note-2000-019.pdf (the PDF seems 
 OK).
 Can Solr start using PDFbox 1.0?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Resolved] (SOLR-2550) Apache Solr needs an updated TIKA version in its extraction libraries

2011-10-25 Thread Steven Rowe (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-2550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steven Rowe resolved SOLR-2550.
---

   Resolution: Fixed
Fix Version/s: 3.1
 Assignee: Steven Rowe

Solr Cell upgraded to Tika 0.8, which included PDFbox 1.1.0, in the Solr 3.1 
release.


 Apache Solr needs an updated TIKA version in its extraction libraries
 -

 Key: SOLR-2550
 URL: https://issues.apache.org/jira/browse/SOLR-2550
 Project: Solr
  Issue Type: Bug
  Components: contrib - Solr Cell (Tika extraction)
Affects Versions: 1.4.1
Reporter: Surendranadh Puranam
Assignee: Steven Rowe
Priority: Critical
  Labels: extraction, indexing, pdf, secure
 Fix For: 3.1, 1.4.2


 There are issues with some PDF documents when it gets indexed (extracted?). 
 There is an issue being fixed by PDFBOX in the version PDFBox 1.1.0. But 
 Apache solr 1.4.1 doesn't have the latest version of these jars which is 
 causing these failures. We have tika-pareser0.4 in this solr 1.4.1 
 distribution which has to be updated to 0.9 version.
 Reference for the issue and the solution : 
 https://issues.apache.org/jira/browse/PDFBOX-617

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-2760) Cannot ant dist or ant example

2011-10-25 Thread Steven Rowe (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-2760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135592#comment-13135592
 ] 

Steven Rowe commented on SOLR-2760:
---

Bill, is this still a problem for you?

 Cannot ant dist or ant example
 

 Key: SOLR-2760
 URL: https://issues.apache.org/jira/browse/SOLR-2760
 Project: Solr
  Issue Type: Bug
Reporter: Bill Bell

 Path: .
 URL: http://svn.apache.org/repos/asf/lucene/dev/trunk/solr
 Repository Root: http://svn.apache.org/repos/asf
 Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68
 Revision: 1170435
 Node Kind: directory
 Schedule: normal
 Last Changed Author: chrism
 Last Changed Rev: 1170425
 Last Changed Date: 2011-09-13 21:36:56 -0600 (Tue, 13 Sep 2011)
 Then
  ant dist or ant example
 compile-core:
 [javac] Compiling 23 source files to 
 /Users/bill/solr/trunk/modules/queries/build/classes/java
 [javac] 
 /Users/bill/solr/trunk/modules/queries/src/java/org/apache/lucene/queries/function/valuesource/NormValueSource.java:48:
  warning: [unchecked] unchecked call to put(K,V) as a member of the raw type 
 java.util.Map
 [javac] context.put(searcher,searcher);
 [javac]^
 [javac] 
 /Users/bill/solr/trunk/modules/queries/src/java/org/apache/lucene/queries/function/valuesource/NormValueSource.java:61:
  cannot find symbol
 [javac] symbol  : class ConstDoubleDocValues
 [javac] location: class 
 org.apache.lucene.queries.function.valuesource.NormValueSource
 [javac]   return new ConstDoubleDocValues(0.0, this);
 [javac]  ^
 [javac] 
 /Users/bill/solr/trunk/modules/queries/src/java/org/apache/lucene/queries/function/valuesource/NumDocsValueSource.java:40:
  cannot find symbol
 [javac] symbol  : class ConstIntDocValues
 [javac] location: class 
 org.apache.lucene.queries.function.valuesource.NumDocsValueSource
 [javac] return new 
 ConstIntDocValues(ReaderUtil.getTopLevelContext(readerContext).reader.numDocs(),
  this);
 [javac]^
 [javac] 
 /Users/bill/solr/trunk/modules/queries/src/java/org/apache/lucene/queries/function/valuesource/QueryValueSource.java:73:
  warning: [unchecked] unchecked call to put(K,V) as a member of the raw type 
 java.util.Map
 [javac] context.put(this, w);
 [javac]^
 [javac] 
 /Users/bill/solr/trunk/modules/queries/src/java/org/apache/lucene/queries/function/valuesource/ScaleFloatFunction.java:96:
  warning: [unchecked] unchecked call to put(K,V) as a member of the raw type 
 java.util.Map
 [javac] context.put(this.source, scaleInfo);
 [javac]^
 [javac] 
 /Users/bill/solr/trunk/modules/queries/src/java/org/apache/lucene/queries/function/valuesource/SumTotalTermFreqValueSource.java:68:
  warning: [unchecked] unchecked call to put(K,V) as a member of the raw type 
 java.util.Map
 [javac] context.put(this, new LongDocValues(this) {
 [javac]^
 [javac] 
 /Users/bill/solr/trunk/modules/queries/src/java/org/apache/lucene/queries/function/valuesource/TotalTermFreqValueSource.java:68:
  warning: [unchecked] unchecked call to put(K,V) as a member of the raw type 
 java.util.Map
 [javac] context.put(this, new LongDocValues(this) {
 [javac]^
 [javac] 2 errors
 [javac] 5 warnings

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Resolved] (SOLR-2460) Some European characters cannot be parsed correctly for some PDFs

2011-10-25 Thread Steven Rowe (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-2460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steven Rowe resolved SOLR-2460.
---

   Resolution: Fixed
Fix Version/s: 3.5
 Assignee: Steven Rowe

In Solr 3.5, Tika will be upgraded to v0.10, which includes PDFbox 1.6.0.  (See 
SOLR-2372)

 Some European characters cannot be parsed correctly for some PDFs
 -

 Key: SOLR-2460
 URL: https://issues.apache.org/jira/browse/SOLR-2460
 Project: Solr
  Issue Type: Bug
  Components: contrib - Solr Cell (Tika extraction)
Affects Versions: 1.4.1, 3.1
 Environment: Tika, PDFBox
Reporter: Erlend Garåsen
Assignee: Steven Rowe
Priority: Minor
 Fix For: 3.5, 3.1.1


 The Norwegian characters (æ, ø and å) in the following PDF document will not 
 display correctly after Solr has indexed it, using Solr Cell:
 http://ridder.uio.no/dokument.pdf
 If I manually change the version of PDFBox (one of Tika's dependencies) to 
 1.4.0, the document will parse correctly.
 I suggest that the next release of Solr ships with version 0.9 of Tika which 
 also has updated its PDFBox dependencies to 1.4.0

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Resolved] (SOLR-2835) Compilation problem with maven - package org.apache.solr.core does not exist

2011-10-25 Thread Steven Rowe (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/SOLR-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steven Rowe resolved SOLR-2835.
---

Resolution: Incomplete
  Assignee: Steven Rowe

 Compilation problem with maven - package org.apache.solr.core does not exist
 

 Key: SOLR-2835
 URL: https://issues.apache.org/jira/browse/SOLR-2835
 Project: Solr
  Issue Type: Bug
  Components: Build
Affects Versions: 4.0
Reporter: Ukyo Virgden
Assignee: Steven Rowe
  Labels: maven, solrj

 maven compilation fails.
 Steps to reproduce are;
 * svn update
 * ant get-maven-poms
 * mvn -N -Pbootstrap install
 * mvn -DskipTests install
 Build works until solrj test compilation Here's the start of the errors.
 {code}
 [INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ 
 solr-solrj ---
 [INFO] Compiling 157 source files to 
 /users/ukyo/lucene-solr/solr/build/solr-solrj/classes/test
 [INFO] -
 [ERROR] COMPILATION ERROR : 
 [INFO] -
 [ERROR] 
 /users/ukyo/lucene-solr/solr/solrj/src/test/org/apache/solr/client/solrj/embedded/MergeIndexesEmbeddedTest.java:[24,27]
  package org.apache.solr.core does not exist
 [ERROR] 
 /users/ukyo/lucene-solr/solr/solrj/src/test/org/apache/solr/client/solrj/MergeIndexesExampleTestBase.java:[28,27]
  package org.apache.solr.core does not exist
 [ERROR] 
 /users/ukyo/lucene-solr/solr/solrj/src/test/org/apache/solr/client/solrj/MergeIndexesExampleTestBase.java:[29,27]
  package org.apache.solr.core does not exist
 [ERROR] 
 /users/ukyo/lucene-solr/solr/solrj/src/test/org/apache/solr/client/solrj/MergeIndexesExampleTestBase.java:[30,27]
  package org.apache.solr.util does not exist
 [ERROR] 
 /users/ukyo/lucene-solr/solr/solrj/src/test/org/apache/solr/client/solrj/SolrExampleTestBase.java:[21,27]
  package org.apache.solr.util does not exist
 [ERROR] 
 /users/ukyo/lucene-solr/solr/solrj/src/test/org/apache/solr/client/solrj/SolrExampleTestBase.java:[31,50]
  cannot find symbol
 {code}
 All following errors are the same package not found.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Commented] (SOLR-2846) SolrTestCaseJ4 bug get udate/json Json update handler

2011-10-25 Thread Steven Rowe (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/SOLR-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13135601#comment-13135601
 ] 

Steven Rowe commented on SOLR-2846:
---

Seems to be a typo, introduced by Yonik in SOLR-2236; it was backported to 
branch_3x, so the same problem is there.

 SolrTestCaseJ4 bug get udate/json Json update handler
 ---

 Key: SOLR-2846
 URL: https://issues.apache.org/jira/browse/SOLR-2846
 Project: Solr
  Issue Type: Bug
Affects Versions: 3.4
Reporter: Linbin Chen
Priority: Trivial
 Fix For: 3.5, 4.0

 Attachments: SOLR-2846.patch


 org.apache.solr.SolrTestCaseJ4.updateJ(String, SolrParams)
 mistake  udate/json for update/json

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[jira] [Created] (SOLR-2854) Limit remote streaming to update handlers

2011-10-25 Thread David Smiley (Created) (JIRA)
Limit remote streaming to update handlers
-

 Key: SOLR-2854
 URL: https://issues.apache.org/jira/browse/SOLR-2854
 Project: Solr
  Issue Type: Improvement
Reporter: David Smiley


I think the remote streaming feature should be limited to update request 
processors. I'm not sure if there is even any use of using it on a /select, but 
even if there is, it's an unintended security risk.  Observe this URL that is 
roughly the equivalent of an SQL injection attack:

http://localhost:8983/solr/select?q=*:*indent=onwt=rubyrows=2stream.url=http%3A%2F%2Flocalhost%3A8983%2Fsolr%2Fupdate%3Fcommit%3Dtruetream.body%3D%3Cdelete%3E%3Cquery%3E*%3A*%3C%2Fquery%3E%3C%2Fdelete%3E

Yep; that's right -- this *search* deletes all the data in your Solr instance! 
If you blocked off access to /update* based on IP then that isn't good enough.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[JENKINS] Lucene-Solr-tests-only-trunk-java7 - Build # 736 - Failure

2011-10-25 Thread Apache Jenkins Server
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk-java7/736/

1 tests failed.
REGRESSION:  org.apache.solr.core.TestJmxIntegration.testJmxOnCoreReload

Error Message:
Number of registered MBeans is not the same as info registry size expected:56 
but was:54

Stack Trace:
junit.framework.AssertionFailedError: Number of registered MBeans is not the 
same as info registry size expected:56 but was:54
at 
org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:149)
at 
org.apache.lucene.util.LuceneTestCaseRunner.runChild(LuceneTestCaseRunner.java:51)
at 
org.apache.solr.core.TestJmxIntegration.testJmxOnCoreReload(TestJmxIntegration.java:134)
at 
org.apache.lucene.util.LuceneTestCase$2$1.evaluate(LuceneTestCase.java:613)




Build Log (for compile errors):
[...truncated 11106 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



  1   2   >