date:20140717


[ 
https://issues.apache.org/jira/browse/LUCENE-5819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14064820#comment-14064820
 ] 

Robert Muir commented on LUCENE-5819:
-

Why is FSTOrdsOutput just treating the rest of the output as a byte array? Wont 
this be ineffective here (e.g. we have pointer to block in terms dict, what 
else is in there?) Do we really need a generic solution or should this just 
have its own output geared at what it does?

 Add block tree postings format that supports term ords
 --

 Key: LUCENE-5819
 URL: https://issues.apache.org/jira/browse/LUCENE-5819
 Project: Lucene - Core
  Issue Type: Improvement
  Components: modules/other
Reporter: Michael McCandless
Assignee: Michael McCandless
 Fix For: 5.0, 4.10

 Attachments: LUCENE-5819.patch, LUCENE-5819.patch


 BlockTree is our default terms dictionary today, but it doesn't
 support term ords, which is an optional API in the postings format to
 retrieve the ordinal for the currently seek'd term, and also later
 seek by that ordinal e.g. to lookup the term.
 This can possibly be useful for e.g. faceting, and maybe at some point
 we can share the postings terms dict with the one used by sorted/set
 DV for cases when app wants to invert and facet on a given field.
 The older (3.x) block terms dict can easily support ords, and we have
 a Lucene41OrdsPF in test-framework, but it's not as fast / compact as
 block-tree, and doesn't (can't easily) implement an optimized
 intersect, but it could be for fields we'd want to facet on, these
 tradeoffs don't matter.  It's nice to have options...



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-6253) Plugin init failure for custom analysis filter

2014-07-17 Thread Sharmilee S (JIRA)

Sharmilee S created SOLR-6253:
-

 Summary: Plugin init failure for custom analysis filter
 Key: SOLR-6253
 URL: https://issues.apache.org/jira/browse/SOLR-6253
 Project: Solr
  Issue Type: Bug
  Components: Schema and Analysis
Affects Versions: 4.6
 Environment: Linux
Reporter: Sharmilee S


Getting this error below on adding new custom filter to schema.xml:
org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: 
Plugin init failure for [schema.xml] fieldType textCustom: Plugin init 
failure for [schema.xml] analyzer/filter: Error instantiating class: 
'org.apache.solr.analysis.Custom.CustomFilterFactory'. Schema file is 
/opt/solr5/collection3/schema.xml

I tried adding jar of my custom filter to /solr/collection/lib path as 
mentioned in this link: 
http://stackoverflow.com/questions/7828619/basetokenfilterfactory-not-found-by-solr-with-custom-filter
 but same error.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-6253) Plugin init failure for custom analysis filter

2014-07-17 Thread Sharmilee S (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-6253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sharmilee S updated SOLR-6253:
--

Remaining Estimate: (was: 840h)
 Original Estimate: (was: 840h)

 Plugin init failure for custom analysis filter
 --

 Key: SOLR-6253
 URL: https://issues.apache.org/jira/browse/SOLR-6253
 Project: Solr
  Issue Type: Bug
  Components: Schema and Analysis
Affects Versions: 4.6
 Environment: Linux
Reporter: Sharmilee S
Priority: Critical
  Labels: analysis, customPlugin, filter, filterfactory, solr

 Getting this error below on adding new custom filter to schema.xml:
 org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: 
 Plugin init failure for [schema.xml] fieldType textCustom: Plugin init 
 failure for [schema.xml] analyzer/filter: Error instantiating class: 
 'org.apache.solr.analysis.Custom.CustomFilterFactory'. Schema file is 
 /opt/solr5/collection3/schema.xml
 I tried adding jar of my custom filter to /solr/collection/lib path as 
 mentioned in this link: 
 http://stackoverflow.com/questions/7828619/basetokenfilterfactory-not-found-by-solr-with-custom-filter
  but same error.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-6253) Plugin init failure for custom analysis filter

2014-07-17 Thread Sharmilee S (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-6253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sharmilee S updated SOLR-6253:
--

Priority: Critical  (was: Major)

 Plugin init failure for custom analysis filter
 --

 Key: SOLR-6253
 URL: https://issues.apache.org/jira/browse/SOLR-6253
 Project: Solr
  Issue Type: Bug
  Components: Schema and Analysis
Affects Versions: 4.6
 Environment: Linux
Reporter: Sharmilee S
Priority: Critical
  Labels: analysis, customPlugin, filter, filterfactory, solr
   Original Estimate: 840h
  Remaining Estimate: 840h

 Getting this error below on adding new custom filter to schema.xml:
 org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: 
 Plugin init failure for [schema.xml] fieldType textCustom: Plugin init 
 failure for [schema.xml] analyzer/filter: Error instantiating class: 
 'org.apache.solr.analysis.Custom.CustomFilterFactory'. Schema file is 
 /opt/solr5/collection3/schema.xml
 I tried adding jar of my custom filter to /solr/collection/lib path as 
 mentioned in this link: 
 http://stackoverflow.com/questions/7828619/basetokenfilterfactory-not-found-by-solr-with-custom-filter
  but same error.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2875) Incorrect url of tika-data-config.xml in example-DIH

2014-07-17 Thread Frank Ren (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-2875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14064868#comment-14064868
 ] 

Frank Ren commented on SOLR-2875:
-

It would be helpful to inform people of this somewhere, say,
example/example-DIH/README.txt. Some instruction for tika would also be
appreciated. Thanks :D


On Thu, Jul 17, 2014 at 4:08 PM, Shinichiro Abe (JIRA) j...@apache.org



 Incorrect url of tika-data-config.xml in example-DIH
 

 Key: SOLR-2875
 URL: https://issues.apache.org/jira/browse/SOLR-2875
 Project: Solr
  Issue Type: Bug
  Components: contrib - DataImportHandler
Affects Versions: 4.0-ALPHA
 Environment: solr boot:java 
 -Dsolr.solr.home=~/trunk/solr/example/example-DIH/solr/tika -jar start.jar 
Reporter: Shinichiro Abe
Assignee: Koji Sekiguchi
Priority: Trivial
 Fix For: 3.5, 4.0-ALPHA

 Attachments: SOLR-2875.patch


 The specified url in tika-data-config.xml is not correct path. So when 
 running full-import, exception is thrown.
 {quote}
 2011/11/04 16:48:26 org.apache.solr.common.SolrException log
 ?v???I: Full Import failed:java.lang.RuntimeException: 
 org.apache.solr.handler.dataimport.DataImportHandlerException: 
 java.lang.RuntimeException: java.io.FileNotFoundException: Could not find 
 file: ../contrib/extraction/src/test/resources/solr-word.pdf
   at 
 org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:261)
   at 
 org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:372)
  :
  :
 Caused by: java.io.FileNotFoundException: Could not find file: 
 ../contrib/extraction/src/test/resources/solr-word.pdf
   at 
 org.apache.solr.handler.dataimport.FileDataSource.getFile(FileDataSource.java:110)
 {quote}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-5830) Fix some missing/assert codec checks to throw CorruptIndexException

Robert Muir created LUCENE-5830:
---

 Summary: Fix some missing/assert codec checks to throw 
CorruptIndexException
 Key: LUCENE-5830
 URL: https://issues.apache.org/jira/browse/LUCENE-5830
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Robert Muir
 Attachments: LUCENE-5830.patch

I thought i had tackled all these before, but I missed a few good ones.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5830) Fix some missing/assert codec checks to throw CorruptIndexException


 [ 
https://issues.apache.org/jira/browse/LUCENE-5830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Muir updated LUCENE-5830:


Attachment: LUCENE-5830.patch

 Fix some missing/assert codec checks to throw CorruptIndexException
 ---

 Key: LUCENE-5830
 URL: https://issues.apache.org/jira/browse/LUCENE-5830
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Robert Muir
 Attachments: LUCENE-5830.patch


 I thought i had tackled all these before, but I missed a few good ones.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5830) Fix some missing/assert codec checks to throw CorruptIndexException

2014-07-17 Thread Michael McCandless (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14064872#comment-14064872
 ] 

Michael McCandless commented on LUCENE-5830:


+1

 Fix some missing/assert codec checks to throw CorruptIndexException
 ---

 Key: LUCENE-5830
 URL: https://issues.apache.org/jira/browse/LUCENE-5830
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Robert Muir
 Attachments: LUCENE-5830.patch


 I thought i had tackled all these before, but I missed a few good ones.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5819) Add block tree postings format that supports term ords

2014-07-17 Thread Michael McCandless (JIRA)

[
https://issues.apache.org/jira/browse/LUCENE-5819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14064886#comment-14064886
]

Michael McCandless commented on LUCENE-5819:

bq. Do we really need a generic solution or should this just have its own
output geared at what it does?

Well, really I just preserved that from the current block tree impl. That
byte[] has the pointer to the block, plus a couple flags
(isLeafBlock/isFloorBlock) and then any floor block index data (file pointer
offsets to get to the floor blocks). I agree at Outputs impl that broke these
things out would be nicer ... but I think this should probably be done
separately.

Add block tree postings format that supports term ords
--

Key: LUCENE-5819
URL: https://issues.apache.org/jira/browse/LUCENE-5819
Project: Lucene - Core
Issue Type: Improvement
Components: modules/other
Reporter: Michael McCandless
Assignee: Michael McCandless
Fix For: 5.0, 4.10

Attachments: LUCENE-5819.patch, LUCENE-5819.patch

BlockTree is our default terms dictionary today, but it doesn't
support term ords, which is an optional API in the postings format to
retrieve the ordinal for the currently seek'd term, and also later
seek by that ordinal e.g. to lookup the term.
This can possibly be useful for e.g. faceting, and maybe at some point
we can share the postings terms dict with the one used by sorted/set
DV for cases when app wants to invert and facet on a given field.
The older (3.x) block terms dict can easily support ords, and we have
a Lucene41OrdsPF in test-framework, but it's not as fast / compact as
block-tree, and doesn't (can't easily) implement an optimized
intersect, but it could be for fields we'd want to facet on, these
tradeoffs don't matter. It's nice to have options...

--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3619) Rename 'example' dir to 'server' and pull examples into an 'examples' directory

2014-07-17 Thread Timothy Potter (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14064938#comment-14064938
 ] 

Timothy Potter commented on SOLR-3619:
--

Resurrecting interest in this as I'd like to get this done in the near term. 
Main question is whether this will only be for Solr 5 or should be backported 
to 4.x?

 Rename 'example' dir to 'server' and pull examples into an 'examples' 
 directory
 ---

 Key: SOLR-3619
 URL: https://issues.apache.org/jira/browse/SOLR-3619
 Project: Solr
  Issue Type: Improvement
Reporter: Mark Miller
 Fix For: 4.9, 5.0

 Attachments: SOLR-3619.patch, server-name-layout.png






--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5830) Fix some missing/assert codec checks to throw CorruptIndexException

2014-07-17 Thread Uwe Schindler (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14064943#comment-14064943
 ] 

Uwe Schindler commented on LUCENE-5830:
---

Strong +1. We need more hard checks, hard checks, hard checks, hard checks. 
Asserts should not be used when disk IO is involved :-)

 Fix some missing/assert codec checks to throw CorruptIndexException
 ---

 Key: LUCENE-5830
 URL: https://issues.apache.org/jira/browse/LUCENE-5830
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Robert Muir
 Attachments: LUCENE-5830.patch


 I thought i had tackled all these before, but I missed a few good ones.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3619) Rename 'example' dir to 'server' and pull examples into an 'examples' directory

2014-07-17 Thread Yonik Seeley (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14064953#comment-14064953
 ] 

Yonik Seeley commented on SOLR-3619:


Good luck getting consensus ;-)  I sort of lost interest when it looked like 
some of the proposals would have made things more complex (or at least as 
complex) as what we have now.

 Rename 'example' dir to 'server' and pull examples into an 'examples' 
 directory
 ---

 Key: SOLR-3619
 URL: https://issues.apache.org/jira/browse/SOLR-3619
 Project: Solr
  Issue Type: Improvement
Reporter: Mark Miller
 Fix For: 4.9, 5.0

 Attachments: SOLR-3619.patch, server-name-layout.png






--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

CMS diff:

2014-07-17 Thread Mathieu

Clone URL (Committers only):
https://cms.apache.org/redirect?new=anonymous;action=diff;uri=http://lucene.apache.org/solr%2Fbooks.mdtext

Mathieu

Index: trunk/content/solr/books.mdtext
===
--- trunk/content/solr/books.mdtext (revision 1608973)
+++ trunk/content/solr/books.mdtext (working copy)
@@ -1,6 +1,19 @@
 # Solrspan style=vertical-align: super; font-size: xx-smallTM/span Books
 
 [TOC]
+
+## Mastering Apache Solr
+
+Mathieu Nayrolles and Inkstall Publications are proud to announce their latest 
book —a 
href=http://www.amazon.com/Mastering-Apache-Solr-practical-guide/dp/8192784509;Mastering
 Apache Solr/a.  This book will empower you to provide a world-class search 
experience to your end users through the discovery of the powerful mechanisms 
presented in it.
+
+a 
href=http://www.amazon.com/Mastering-Apache-Solr-practical-guide/dp/8192784509;
 title=Mastering Apache Solrimg alt=Solr in Action cover 
class=float-right src=http://ecx.images-amazon.com/images/I/41MDmx45l9L.jpg; 
height=236px width=192px//a
+
+Mastering Apache Solr is a short, focused, practical, hands-on guide 
containing crisp, relevant, systematically arranged, progressive chapters. 
These chapters contain a wealth of information presented in a direct and 
easy-to-understand manner. Highlighting Solr's supremacy over classical 
databases in full-text search, this book covers key technical concepts which 
will help you accelerate your progress in the Solr world.
+ 
+Mastering Apache Solr starts with an introduction to Apache Solr, its 
underlying technologies, the main differences between the classical database 
engines, and gradually moves to more advance topics such as boosting 
performance. In this book, we will look under the hood of a large number of 
topics and discuss answers to pertinent questions such as why denormalize data, 
how to import classical databases' data inside Apache Solr, how to serve Solr 
through five different web servers, how to optimize them to serve Solr even 
faster. An important and major topic covered in this book is Solr's querying 
mechanism, which will prove to be a strong ally in our journey through this 
book. We then look at boosting performance and deploying Solr using several 
servlet servers. Finally, we cover how to communicate with Solr using different 
programming languages, before deploying it in a cloud-based environment.
+ 
+Mastering Apache Solr is written lucidly and has a clear simple approach. From 
the first page to the last, the book remains practical and focuses on the most 
important topics used in the world of Apache Solr without neglecting important 
theoretical fundamentals that help you build a strong foundation. 
+
 ## Solr in Action
 
 Trey Grainger, Timothy Potter, and Manning Publications are proud to announce 
a href=http://solrinaction.com/Solr in Action/a, a comprehensive (638 
pg.), example-driven guide covering through Solr 4.7.


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting

[
https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065060#comment-14065060
]

Hoss Man commented on SOLR-2894:

bq. When those two things combine ... *Fixed*

Awesome .. i love it when tests uncover bugs you never even suspected.

bq. Are you still seeing the infinite recursion problem? The seeds you provided
earlier pass locally for me.

Changing the encoding mechanism stoped the reliably reproducing infinite loop
-- my concern though is that -- even if it's hard to test for -- we should make
sure the overall algorithm for refinement isn't susceptible to infinite looping
in the event of aberrant shard behavior.

At the moment, from what i can tell, the general behavior of the refinement
logic is along the lines of...

{code}
while ( ! values_needing_refined.isEmpty() ) {
foreach (value : values_needing_refined) {
foreach (shard_not_responded : value) {
shardrequstor.enque_refinement_request(shard_not_responded, value)
}
}
shardrequestor.send_refinement_requests_to_shards_that_need_them()
}
{code}

Now imagine a situation where, for whatever reason, some shard will *never*
respond back as expected from a refinement request -- ie: maybe we just started
a rolling config upgrade and the new solrconfig.xml permanently disables
faceting via a {{facet=false}} invariant? that's going to be an infinite loop.

We need a safety valve in the refinement logic.

Instead of simply sending refinement requests to a shard if there is a value
whose shard count we don't know, potentially asking for hte same refinement
over and over; we should instead keep track of which shards we've already
_asked_ for a refinement, and if we've already asked a shard once, and still
don't have a response then we should just give up and return an error.

does that make sense?

Implement distributed pivot faceting

Key: SOLR-2894
URL: https://issues.apache.org/jira/browse/SOLR-2894
Project: Solr
Issue Type: Improvement
Reporter: Erik Hatcher
Assignee: Hoss Man
Fix For: 4.9, 5.0

Attachments: SOLR-2894-mincount-minification.patch,
SOLR-2894-reworked.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch,
SOLR-2894_cloud_test.patch, dateToObject.patch, pivot_mincount_problem.sh

Following up on SOLR-792, pivot faceting currently only supports
undistributed mode. Distributed pivot faceting needs to be implemented.

--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5831) ant precommit should depend on clean-jars


[ 
https://issues.apache.org/jira/browse/LUCENE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065164#comment-14065164
 ] 

Steve Rowe commented on LUCENE-5831:


Simple patch:

{code:xml}
Index: build.xml
===
--- build.xml   (revision 1611400)
+++ build.xml   (working copy)
@@ -39,7 +39,7 @@
   property name=tests.heap-dump-dir location=heapdumps/
   
   target name=precommit description=Run basic checks before committing
-  depends=check-svn-working-copy,validate,documentation-lint/
+  
depends=clean-jars,check-svn-working-copy,validate,documentation-lint/
 
   target name=test description=Test both Lucene and Solr
 subant buildpath=. antfile=extra-targets.xml target=-run-test 
inheritall=false failonerror=true /
{code}

 ant precommit should depend on clean-jars
 -

 Key: LUCENE-5831
 URL: https://issues.apache.org/jira/browse/LUCENE-5831
 Project: Lucene - Core
  Issue Type: Bug
  Components: general/build
Reporter: Steve Rowe
Priority: Minor

 Ivy's bug that fails to remove differently versioned dependencies in 
 (test-)lib/ dirs even though we set {{sync=true}} on {{ivy:retrieve}} (I 
 couldn't find a JIRA for this) continues to cause trouble/confusion (see 
 related LUCENE-5467).
 We should make the ant {{precommit}} target depend on {{clean-jars}}, so that 
 people won't think they need to run {{ant jar-checksums}} because of stale 
 jars Ivy leaves in {{lib/}} or {{test-lib/}} directories, which currently 
 causes {{ant precommit}} to bitch that there are missing checksums.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-5831) ant precommit should depend on clean-jars

Steve Rowe created LUCENE-5831:
--

 Summary: ant precommit should depend on clean-jars
 Key: LUCENE-5831
 URL: https://issues.apache.org/jira/browse/LUCENE-5831
 Project: Lucene - Core
  Issue Type: Bug
  Components: general/build
Reporter: Steve Rowe
Priority: Minor


Ivy's bug that fails to remove differently versioned dependencies in 
(test-)lib/ dirs even though we set {{sync=true}} on {{ivy:retrieve}} (I 
couldn't find a JIRA for this) continues to cause trouble/confusion (see 
related LUCENE-5467).

We should make the ant {{precommit}} target depend on {{clean-jars}}, so that 
people won't think they need to run {{ant jar-checksums}} because of stale jars 
Ivy leaves in {{lib/}} or {{test-lib/}} directories, which currently causes 
{{ant precommit}} to bitch that there are missing checksums.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5831) ant precommit should depend on clean-jars


[ 
https://issues.apache.org/jira/browse/LUCENE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065176#comment-14065176
 ] 

Robert Muir commented on LUCENE-5831:
-

This destroys and destabilizes my IDE environment (ant clean-jars) every time 
its run!

 ant precommit should depend on clean-jars
 -

 Key: LUCENE-5831
 URL: https://issues.apache.org/jira/browse/LUCENE-5831
 Project: Lucene - Core
  Issue Type: Bug
  Components: general/build
Reporter: Steve Rowe
Priority: Minor

 Ivy's bug that fails to remove differently versioned dependencies in 
 (test-)lib/ dirs even though we set {{sync=true}} on {{ivy:retrieve}} (I 
 couldn't find a JIRA for this) continues to cause trouble/confusion (see 
 related LUCENE-5467).
 We should make the ant {{precommit}} target depend on {{clean-jars}}, so that 
 people won't think they need to run {{ant jar-checksums}} because of stale 
 jars Ivy leaves in {{lib/}} or {{test-lib/}} directories, which currently 
 causes {{ant precommit}} to bitch that there are missing checksums.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3619) Rename 'example' dir to 'server' and pull examples into an 'examples' directory

2014-07-17 Thread Timothy Potter (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065174#comment-14065174
 ] 

Timothy Potter commented on SOLR-3619:
--

A fair point Yonik ;-) We need to come together on this quickly as I hear often 
how unintuitive the Solr directory layout is compared to other open source 
search engines based on Lucene. 

What about leaving example untouched and introducing a new server directory? 
That would make backporting easier and satisfy the bulk of the concerns of new 
users looking at Solr's directory structure.

 Rename 'example' dir to 'server' and pull examples into an 'examples' 
 directory
 ---

 Key: SOLR-3619
 URL: https://issues.apache.org/jira/browse/SOLR-3619
 Project: Solr
  Issue Type: Improvement
Reporter: Mark Miller
 Fix For: 4.9, 5.0

 Attachments: SOLR-3619.patch, server-name-layout.png






--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5831) ant precommit should depend on clean-jars


[ 
https://issues.apache.org/jira/browse/LUCENE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065178#comment-14065178
 ] 

Robert Muir commented on LUCENE-5831:
-

I don't think we need to run this every time we precommit, adding third party 
dependencies is something that shouldnt happen 10x a day, and should be done 
carefully (taking into account licensing, and other aspects).

 ant precommit should depend on clean-jars
 -

 Key: LUCENE-5831
 URL: https://issues.apache.org/jira/browse/LUCENE-5831
 Project: Lucene - Core
  Issue Type: Bug
  Components: general/build
Reporter: Steve Rowe
Priority: Minor

 Ivy's bug that fails to remove differently versioned dependencies in 
 (test-)lib/ dirs even though we set {{sync=true}} on {{ivy:retrieve}} (I 
 couldn't find a JIRA for this) continues to cause trouble/confusion (see 
 related LUCENE-5467).
 We should make the ant {{precommit}} target depend on {{clean-jars}}, so that 
 people won't think they need to run {{ant jar-checksums}} because of stale 
 jars Ivy leaves in {{lib/}} or {{test-lib/}} directories, which currently 
 causes {{ant precommit}} to bitch that there are missing checksums.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-6233) Provide basic command line tools for checking Solr status and health.

2014-07-17 Thread Timothy Potter (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-6233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Timothy Potter updated SOLR-6233:
-

Summary: Provide basic command line tools for checking Solr status and 
health.  (was: Need an option on the admin system info handler to get a smaller 
set of essential information about a server.)

 Provide basic command line tools for checking Solr status and health.
 -

 Key: SOLR-6233
 URL: https://issues.apache.org/jira/browse/SOLR-6233
 Project: Solr
  Issue Type: Improvement
Reporter: Timothy Potter
Priority: Minor

 As part of the start script development work SOLR-3617, example restructuring 
 SOLR-3619, and the overall curb appeal work SOLR-4430, I'd like to have an 
 option on the SystemInfoHandler that gives a shorter, well formatted JSON 
 synopsis of essential information. I know essential is vague ;-) but right 
 now using curl to http://host:port/solr/admin/info/system?wt=json gives too 
 much information when I just want a synopsis of a Solr server. 
 Maybe something like overview=true?
 Result would be:
 {noformat}
 {
   address: http://localhost:8983/solr;,
   mode: solrcloud,
   zookeeper: localhost:2181/foo,
   uptime: 2 days, 3 hours, 4 minutes, 5 seconds,
   version: 5.0-SNAPSHOT,
   status: healthy,
   memory: 4.2g of 6g
 }
 {noformat}
 Now of course, one may argue all this information can be easily parsed from 
 the JSON but consider cross-platform command-line tools that don't have 
 immediate access to a JSON parser, such as the bin/solr start script.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5831) ant precommit should depend on clean-jars


[ 
https://issues.apache.org/jira/browse/LUCENE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065217#comment-14065217
 ] 

Hoss Man commented on LUCENE-5831:
--

maybe instead of modifying the ant deps, we should instead update the error 
message produced by precommit when it isn't happy with the jar checksums to 
help the user sanity check themselves

something like...

{noformat}
Missing checksum for foo.jar

If you recently modified any ivy.xml or ivy.properties files to add new jars, 
make sure you run ant clean-jars jar-checksums before running precommit.
{noformat}

 ant precommit should depend on clean-jars
 -

 Key: LUCENE-5831
 URL: https://issues.apache.org/jira/browse/LUCENE-5831
 Project: Lucene - Core
  Issue Type: Bug
  Components: general/build
Reporter: Steve Rowe
Priority: Minor

 Ivy's bug that fails to remove differently versioned dependencies in 
 (test-)lib/ dirs even though we set {{sync=true}} on {{ivy:retrieve}} (I 
 couldn't find a JIRA for this) continues to cause trouble/confusion (see 
 related LUCENE-5467).
 We should make the ant {{precommit}} target depend on {{clean-jars}}, so that 
 people won't think they need to run {{ant jar-checksums}} because of stale 
 jars Ivy leaves in {{lib/}} or {{test-lib/}} directories, which currently 
 causes {{ant precommit}} to bitch that there are missing checksums.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5831) ant precommit should depend on clean-jars


[ 
https://issues.apache.org/jira/browse/LUCENE-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065221#comment-14065221
 ] 

Steve Rowe commented on LUCENE-5831:


+1 sounds good Hoss

 ant precommit should depend on clean-jars
 -

 Key: LUCENE-5831
 URL: https://issues.apache.org/jira/browse/LUCENE-5831
 Project: Lucene - Core
  Issue Type: Bug
  Components: general/build
Reporter: Steve Rowe
Priority: Minor

 Ivy's bug that fails to remove differently versioned dependencies in 
 (test-)lib/ dirs even though we set {{sync=true}} on {{ivy:retrieve}} (I 
 couldn't find a JIRA for this) continues to cause trouble/confusion (see 
 related LUCENE-5467).
 We should make the ant {{precommit}} target depend on {{clean-jars}}, so that 
 people won't think they need to run {{ant jar-checksums}} because of stale 
 jars Ivy leaves in {{lib/}} or {{test-lib/}} directories, which currently 
 causes {{ant precommit}} to bitch that there are missing checksums.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3619) Rename 'example' dir to 'server' and pull examples into an 'examples' directory


[ 
https://issues.apache.org/jira/browse/SOLR-3619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065222#comment-14065222
 ] 

Mark Miller commented on SOLR-3619:
---

I think we should make this change in 4x and 5x. No reason we should have to 
wait for 5x to get a sensible modern directory layout.

bq. What about leaving example untouched and introducing a new server 
directory? 

As long as we pull jetty and what not out into the server directory. We can 
improve on the sprawling example dir later. I would hate to see another set of 
configs introduced though.





 Rename 'example' dir to 'server' and pull examples into an 'examples' 
 directory
 ---

 Key: SOLR-3619
 URL: https://issues.apache.org/jira/browse/SOLR-3619
 Project: Solr
  Issue Type: Improvement
Reporter: Mark Miller
 Fix For: 4.9, 5.0

 Attachments: SOLR-3619.patch, server-name-layout.png






--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting

2014-07-17 Thread Andrew Muldowney (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065242#comment-14065242
 ] 

Andrew Muldowney commented on SOLR-2894:


bq. We need a safety valve in the refinement logic.

I'm with you. Brett and I have talked through some options and I think we have 
the requisite accounting already in place to be able to check if a shard did 
not respond with all the refinement values we asked for. We should be able to 
check the refinements and see that they have had data contributed from the 
shard in question, if not well throw an error.

If a shard never responds, does the searcher handle that by eventually timing 
out? The pivot facet code is predicated on waiting for all shards to respond 
before moving forward with the next level of refinement so if a shard never 
responds at all then it'll just wait forever. We're assuming that other 
processes are watching for searches that take much too long and kill them.

 Implement distributed pivot faceting
 

 Key: SOLR-2894
 URL: https://issues.apache.org/jira/browse/SOLR-2894
 Project: Solr
  Issue Type: Improvement
Reporter: Erik Hatcher
Assignee: Hoss Man
 Fix For: 4.9, 5.0

 Attachments: SOLR-2894-mincount-minification.patch, 
 SOLR-2894-reworked.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894_cloud_test.patch, dateToObject.patch, pivot_mincount_problem.sh


 Following up on SOLR-792, pivot faceting currently only supports 
 undistributed mode.  Distributed pivot faceting needs to be implemented.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting

[
https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065279#comment-14065279
]

Hoss Man commented on SOLR-2894:

bq. If a shard never responds, does the searcher handle that by eventually
timing out?

Off the top of my head i'm not sure -- i'm pretty sure that will generate an
error at a much lower level then the individual search components, so it's not
something the pivot code needs to worry about.

I think the main thing is that when the pivot codes is looking at a
ShardResponse, it should be able to say this response doesn't contain a count
for the refinementX we asked for in the corrisponding ShardRequest, fail! (as
opposed to know where i believe it reQueues it) ... we can leave worrying about
whether or not a ShardResponse was ever returned at all to the SearchHandler.

bq. ... I think we have the requisite accounting already in place to be able to
check if a shard did not respond with all the refinement values we asked for.
We should be able to check the refinements and see that they have had data
contributed from the shard in question, if not well throw an error.

that should be all we need.

Implement distributed pivot faceting

Key: SOLR-2894
URL: https://issues.apache.org/jira/browse/SOLR-2894
Project: Solr
Issue Type: Improvement
Reporter: Erik Hatcher
Assignee: Hoss Man
Fix For: 4.9, 5.0

Following up on SOLR-792, pivot faceting currently only supports
undistributed mode. Distributed pivot faceting needs to be implemented.

--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-6254) Failing tests due to timeouts caused by SSL depleting random entropy on Jenkins

Steve Rowe created SOLR-6254:


 Summary: Failing tests due to timeouts caused by SSL depleting 
random entropy on Jenkins
 Key: SOLR-6254
 URL: https://issues.apache.org/jira/browse/SOLR-6254
 Project: Solr
  Issue Type: Task
  Components: Tests
Reporter: Steve Rowe


Tests using SSL can block on Jenkins when random entropy is depleted, causing 
timeouts that trigger test failures.

I found some info about /dev/random problems on FreeBSD here: 
[https://wiki.freebsd.org/201308DevSummit/Security/DevRandom], which lead me to 
/etc/rc.d/iinitrandom, which gets around the limited entropy by cat'ing a bunch 
of shit to /dev/random:

{code}
( ps -fauxww; sysctl -a; date; df -ib; dmesg; ps -fauxww ) \
  | dd of=/dev/random bs=8k 2/dev/null
cat /bin/ls | dd of=/dev/random bs=8k 2/dev/null
{code}

I think we should try the same strategy in a crontab every X minutes, to see if 
that addresses the test failures.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting

2014-07-17 Thread Brett Lucey (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065352#comment-14065352
 ] 

Brett Lucey commented on SOLR-2894:
---

bq. I think the main thing is that when the pivot codes is looking at a 
ShardResponse, it should be able to say this response doesn't contain a count 
for the refinementX we asked for in the corrisponding ShardRequest, fail! (as 
opposed to know where i believe it reQueues it) ... we can leave worrying about 
whether or not a ShardResponse was ever returned at all to the SearchHandler.

We mark a shard as having contributed a value already, and we have a list of 
refinements we requested from the shard.  All we'll need to do is iterate over 
that list after merging the shard contribution and making sure that the shard 
bit is set on that value.  If it isn't, then we know we got a response from 
that shard but that it didn't tell us what we asked it for, and therefore some 
sort of error has occurred.  We're working on implementing this now.

 Implement distributed pivot faceting
 

 Key: SOLR-2894
 URL: https://issues.apache.org/jira/browse/SOLR-2894
 Project: Solr
  Issue Type: Improvement
Reporter: Erik Hatcher
Assignee: Hoss Man
 Fix For: 4.9, 5.0

 Attachments: SOLR-2894-mincount-minification.patch, 
 SOLR-2894-reworked.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894_cloud_test.patch, dateToObject.patch, pivot_mincount_problem.sh


 Following up on SOLR-792, pivot faceting currently only supports 
 undistributed mode.  Distributed pivot faceting needs to be implemented.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6254) Failing tests due to timeouts caused by SSL depleting random entropy on Jenkins


[ 
https://issues.apache.org/jira/browse/SOLR-6254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065370#comment-14065370
 ] 

Mark Miller commented on SOLR-6254:
---

bq. I think we should try the same strategy in a crontab every X minutes, to 
see if that addresses the test failures.

I think it's fine as a short term workaround, but not a great solution. We 
probably should just disable SSL unless we can address it in a portable way.

 Failing tests due to timeouts caused by SSL depleting random entropy on 
 Jenkins
 ---

 Key: SOLR-6254
 URL: https://issues.apache.org/jira/browse/SOLR-6254
 Project: Solr
  Issue Type: Task
  Components: Tests
Reporter: Steve Rowe

 Tests using SSL can block on Jenkins when random entropy is depleted, causing 
 timeouts that trigger test failures.
 I found some info about /dev/random problems on FreeBSD here: 
 [https://wiki.freebsd.org/201308DevSummit/Security/DevRandom], which lead me 
 to /etc/rc.d/iinitrandom, which gets around the limited entropy by cat'ing a 
 bunch of shit to /dev/random:
 {code}
 ( ps -fauxww; sysctl -a; date; df -ib; dmesg; ps -fauxww ) \
   | dd of=/dev/random bs=8k 2/dev/null
 cat /bin/ls | dd of=/dev/random bs=8k 2/dev/null
 {code}
 I think we should try the same strategy in a crontab every X minutes, to see 
 if that addresses the test failures.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6254) Failing tests due to timeouts caused by SSL depleting random entropy on Jenkins


[ 
https://issues.apache.org/jira/browse/SOLR-6254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065369#comment-14065369
 ] 

Mark Miller commented on SOLR-6254:
---

This is essentially a dupe of https://issues.apache.org/jira/browse/SOLR-5776, 
though that doesn't have an updated title.

 Failing tests due to timeouts caused by SSL depleting random entropy on 
 Jenkins
 ---

 Key: SOLR-6254
 URL: https://issues.apache.org/jira/browse/SOLR-6254
 Project: Solr
  Issue Type: Task
  Components: Tests
Reporter: Steve Rowe

 Tests using SSL can block on Jenkins when random entropy is depleted, causing 
 timeouts that trigger test failures.
 I found some info about /dev/random problems on FreeBSD here: 
 [https://wiki.freebsd.org/201308DevSummit/Security/DevRandom], which lead me 
 to /etc/rc.d/iinitrandom, which gets around the limited entropy by cat'ing a 
 bunch of shit to /dev/random:
 {code}
 ( ps -fauxww; sysctl -a; date; df -ib; dmesg; ps -fauxww ) \
   | dd of=/dev/random bs=8k 2/dev/null
 cat /bin/ls | dd of=/dev/random bs=8k 2/dev/null
 {code}
 I think we should try the same strategy in a crontab every X minutes, to see 
 if that addresses the test failures.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5813) Directory should implement Accountable

2014-07-17 Thread Adrien Grand (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-5813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand updated LUCENE-5813:
-

Attachment: LUCENE-5813.patch

Here is a patch. Directory implements Accountable and BaseDirectoryTestCase has 
a new test to check ram usage estimations

 Directory should implement Accountable
 --

 Key: LUCENE-5813
 URL: https://issues.apache.org/jira/browse/LUCENE-5813
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor
 Fix For: 4.10

 Attachments: LUCENE-5813.patch


 Follow-up of LUCENE-5812.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5813) Directory should implement Accountable

2014-07-17 Thread Uwe Schindler (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065389#comment-14065389
 ] 

Uwe Schindler commented on LUCENE-5813:
---

I don't think Directory itsself should implement Accountable. In my opinion, 
just because *some* directories consume RAM, I don't think we should make all 
directories implement that.

For the issue mentioned in LUCENE-5812 I think the ramBytesUsed there should 
simply check with instanceof if the inner directory implements accountable and 
then add its size to itsself. Something thats not intended to be Accountable 
like a purely disk based dir, should not implement it. Because otherwise a 
NIOFSDirectory would need to return the size of its inner ByteBuffer caches or 
whatever.

-1 to make Directory itsself Accountable.

 Directory should implement Accountable
 --

 Key: LUCENE-5813
 URL: https://issues.apache.org/jira/browse/LUCENE-5813
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor
 Fix For: 4.10

 Attachments: LUCENE-5813.patch


 Follow-up of LUCENE-5812.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-6244) Error 500 when want reindex or update solr

2014-07-17 Thread Erick Erickson (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-6244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson resolved SOLR-6244.
--

Resolution: Invalid

Please raise this issue on the appropriate user's list (RSolr?). If and when 
it's a confirmed _Solr_ bug we can reopen this issue.

 Error 500 when want reindex or update solr
 --

 Key: SOLR-6244
 URL: https://issues.apache.org/jira/browse/SOLR-6244
 Project: Solr
  Issue Type: Bug
Affects Versions: 4.9
 Environment: mac mavericks rails sunspot
Reporter: igor
Priority: Critical

 i try reindex and when start output are:
 Error - RSolr::Error::Http - 500 Internal Server Error - ignoring...
 [#] [  
 700/48865] [  1.43%] [00:01] [01:25] [  560.27/s]Error - RSolr::Error::Http - 
 500 Internal Server Error - retrying...
 Error - RSolr::Error::Http - 500 Internal Server Error - ignoring...
 Error - RSolr::Error::Http - 500 Internal Server Error - retrying...
 Error - RSolr::Error::Http - 500 Internal Server Error - ignoring...
 Error - RSolr::Error::Http - 500 Internal Server Error - retrying...
 i try delete and restore dir, kill process restart reindex but not work!!!
 and when try use my app when update write me = java.lang.NullPointerException 
 in log
 help



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-6254) Failing tests due to timeouts caused by SSL depleting random entropy on Jenkins


 [ 
https://issues.apache.org/jira/browse/SOLR-6254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe resolved SOLR-6254.
--

Resolution: Duplicate

bq. This is essentially a dupe of 
https://issues.apache.org/jira/browse/SOLR-5776, though that doesn't have an 
updated title.

Thanks Mark, I knew I'd seen you discuss this somewhere, but my search-fu 
failed me.

 Failing tests due to timeouts caused by SSL depleting random entropy on 
 Jenkins
 ---

 Key: SOLR-6254
 URL: https://issues.apache.org/jira/browse/SOLR-6254
 Project: Solr
  Issue Type: Task
  Components: Tests
Reporter: Steve Rowe

 Tests using SSL can block on Jenkins when random entropy is depleted, causing 
 timeouts that trigger test failures.
 I found some info about /dev/random problems on FreeBSD here: 
 [https://wiki.freebsd.org/201308DevSummit/Security/DevRandom], which lead me 
 to /etc/rc.d/iinitrandom, which gets around the limited entropy by cat'ing a 
 bunch of shit to /dev/random:
 {code}
 ( ps -fauxww; sysctl -a; date; df -ib; dmesg; ps -fauxww ) \
   | dd of=/dev/random bs=8k 2/dev/null
 cat /bin/ls | dd of=/dev/random bs=8k 2/dev/null
 {code}
 I think we should try the same strategy in a crontab every X minutes, to see 
 if that addresses the test failures.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-6253) Plugin init failure for custom analysis filter

2014-07-17 Thread Erick Erickson (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-6253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson resolved SOLR-6253.
--

Resolution: Invalid

Please raise this issue on the Solr user's list before raising a JIRA, JIRAs 
are for confirmed bugs.

When you do, please provide the full stack trace. My bet is that you've 
incorrectly specified the path in solrconfig.xml to your jar file (personally I 
prefer that to adding it to the Solr lib directory.

If this is confirmed to be a Solr bug, we an re-open this JIRA.

 Plugin init failure for custom analysis filter
 --

 Key: SOLR-6253
 URL: https://issues.apache.org/jira/browse/SOLR-6253
 Project: Solr
  Issue Type: Bug
  Components: Schema and Analysis
Affects Versions: 4.6
 Environment: Linux
Reporter: Sharmilee S
Priority: Critical
  Labels: analysis, customPlugin, filter, filterfactory, solr

 Getting this error below on adding new custom filter to schema.xml:
 org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: 
 Plugin init failure for [schema.xml] fieldType textCustom: Plugin init 
 failure for [schema.xml] analyzer/filter: Error instantiating class: 
 'org.apache.solr.analysis.Custom.CustomFilterFactory'. Schema file is 
 /opt/solr5/collection3/schema.xml
 I tried adding jar of my custom filter to /solr/collection/lib path as 
 mentioned in this link: 
 http://stackoverflow.com/questions/7828619/basetokenfilterfactory-not-found-by-solr-with-custom-filter
  but same error.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting

2014-07-17 Thread Andrew Muldowney (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065429#comment-14065429
 ] 

Andrew Muldowney commented on SOLR-2894:


Hey Hoss. How would we test this?

I've verfied it works by commenting out the mergeResponse lines and seeing it 
error since we expected shard contributions but failed. But how do I write a 
test where a shard responds in a canned way that is bad?

 Implement distributed pivot faceting
 

 Key: SOLR-2894
 URL: https://issues.apache.org/jira/browse/SOLR-2894
 Project: Solr
  Issue Type: Improvement
Reporter: Erik Hatcher
Assignee: Hoss Man
 Fix For: 4.9, 5.0

 Attachments: SOLR-2894-mincount-minification.patch, 
 SOLR-2894-reworked.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894_cloud_test.patch, dateToObject.patch, pivot_mincount_problem.sh


 Following up on SOLR-792, pivot faceting currently only supports 
 undistributed mode.  Distributed pivot faceting needs to be implemented.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6254) Failing tests due to timeouts caused by SSL depleting random entropy on Jenkins


[ 
https://issues.apache.org/jira/browse/SOLR-6254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065437#comment-14065437
 ] 

Steve Rowe commented on SOLR-6254:
--

bq. We probably should just disable SSL unless we can address it in a portable 
way.

I did this for the test I recently committed on SOLR-6137 
(TestCloudSchemaless), but I suppose you mean disabling SSL everywhere.

 Failing tests due to timeouts caused by SSL depleting random entropy on 
 Jenkins
 ---

 Key: SOLR-6254
 URL: https://issues.apache.org/jira/browse/SOLR-6254
 Project: Solr
  Issue Type: Task
  Components: Tests
Reporter: Steve Rowe

 Tests using SSL can block on Jenkins when random entropy is depleted, causing 
 timeouts that trigger test failures.
 I found some info about /dev/random problems on FreeBSD here: 
 [https://wiki.freebsd.org/201308DevSummit/Security/DevRandom], which lead me 
 to /etc/rc.d/iinitrandom, which gets around the limited entropy by cat'ing a 
 bunch of shit to /dev/random:
 {code}
 ( ps -fauxww; sysctl -a; date; df -ib; dmesg; ps -fauxww ) \
   | dd of=/dev/random bs=8k 2/dev/null
 cat /bin/ls | dd of=/dev/random bs=8k 2/dev/null
 {code}
 I think we should try the same strategy in a crontab every X minutes, to see 
 if that addresses the test failures.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5776) Look at speeding up using SSL with tests.


[ 
https://issues.apache.org/jira/browse/SOLR-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065480#comment-14065480
 ] 

Steve Rowe commented on SOLR-5776:
--

[Haveged|http://www.issihosts.com/haveged/] is designed to solve the problem of 
low-entropy hosts (e.g. headless servers), though it appears  to be Linux-only.

 Look at speeding up using SSL with tests.
 -

 Key: SOLR-5776
 URL: https://issues.apache.org/jira/browse/SOLR-5776
 Project: Solr
  Issue Type: Test
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: 4.9, 5.0

 Attachments: SOLR-5776.patch, SOLR-5776.patch


 We have to disable SSL on a bunch of tests now because it appears to sometime 
 be ridiculously slow - especially in slow envs (I never see timeouts on my 
 machine).
 I was talking to Robert about this, and he mentioned that there might be some 
 settings we could change to speed it up.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5813) Directory should implement Accountable

2014-07-17 Thread Adrien Grand (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065487#comment-14065487
 ] 

Adrien Grand commented on LUCENE-5813:
--

I don' t think using {{instanceof Accountable}} works in general to measure 
memory usage. That's what I initially wanted to do with doc id sets, but this 
doesn't work as soon as you start using wrappers since they hide what interface 
the wrapped class implements. So for example if your NRT dir wraps a 
RateLimitedDirectoryWrapper that wraps another directory that implements 
Accountable, you would miss it.

If we don't want of Accountable on Directory, I would rather revert LUCENE-5812 
to make clear that it only reports memory usage for the cache?

 Directory should implement Accountable
 --

 Key: LUCENE-5813
 URL: https://issues.apache.org/jira/browse/LUCENE-5813
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Assignee: Adrien Grand
Priority: Minor
 Fix For: 4.10

 Attachments: LUCENE-5813.patch


 Follow-up of LUCENE-5812.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6060) Unable to deploy SOLR Webapps on Wildfly 8+ due to guava 14.0.1.jar old version

2014-07-17 Thread Glaucio Scheibel (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-6060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065492#comment-14065492
 ] 

Glaucio Scheibel commented on SOLR-6060:


In my case, it's a blocker issue. I am using JBoss 7.2, and if I update guava, 
jboss fails some deployments, and if I don't, solr fails.


 Unable to deploy SOLR Webapps on Wildfly 8+ due to guava 14.0.1.jar old 
 version
 ---

 Key: SOLR-6060
 URL: https://issues.apache.org/jira/browse/SOLR-6060
 Project: Solr
  Issue Type: Improvement
Affects Versions: 4.8
 Environment: Linux CentOS release 6.4 (Final)
 Wildfly 8.0 8.1 RC1
Reporter: Guilhem RAMBAL

 Error 
 ESC[0mESC[33m2014-05-12 14:18:35,283 WARN  [org.jboss.modules] 
 (weld-worker-2) Failed to define class 
 org.apache.hadoop.hdfs.web.resources.UserProvider in Module 
 deployment.solr.war:main from Service Module Loader: 
 java.lang.LinkageError: Failed to link 
 org/apache/hadoop/hdfs/web/resources/UserProvider (Module 
 deployment.solr.war:main from Service Module Loader)
 at 
 org.jboss.modules.ModuleClassLoader.defineClass(ModuleClassLoader.java:487) 
 [jboss-modules.jar:1.3.3.Final]
 at 
 org.jboss.modules.ModuleClassLoader.loadClassLocal(ModuleClassLoader.java:277)
  [jboss-modules.jar:1.3.3.Final]
 at 
 org.jboss.modules.ModuleClassLoader$1.loadClassLocal(ModuleClassLoader.java:92)
  [jboss-modules.jar:1.3.3.Final]
 at org.jboss.modules.Module.loadModuleClass(Module.java:568) 
 [jboss-modules.jar:1.3.3.Final]
 at 
 org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:205) 
 [jboss-modules.jar:1.3.3.Final]
 at 
 org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:459)
  [jboss-modules.jar:1.3.3.Final]
 at 
 org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:408)
  [jboss-modules.jar:1.3.3.Final]
 at 
 org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:389)
  [jboss-modules.jar:1.3.3.Final]
 at 
 org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:134)
  [jboss-modules.jar:1.3.3.Final]
 at 
 org.jboss.as.weld.WeldModuleResourceLoader.classForName(WeldModuleResourceLoader.java:68)
  [wildfly-weld-8.1.0.CR1.jar:8.1.0.CR1]
 at 
 org.jboss.weld.bootstrap.BeanDeployer.loadClass(BeanDeployer.java:106) 
 [weld-core-impl-2.1.2.Final.jar:2014-01-09 09:23]
 at 
 org.jboss.weld.bootstrap.BeanDeployer.addClass(BeanDeployer.java:94) 
 [weld-core-impl-2.1.2.Final.jar:2014-01-09 09:23]
 at 
 org.jboss.weld.bootstrap.ConcurrentBeanDeployer$1.doWork(ConcurrentBeanDeployer.java:62)
  [weld-core-impl-2.1.2.Final.jar:2014-01-09 09:23]
 at 
 org.jboss.weld.bootstrap.ConcurrentBeanDeployer$1.doWork(ConcurrentBeanDeployer.java:60)
  [weld-core-impl-2.1.2.Final.jar:2014-01-09 09:23]
 at 
 org.jboss.weld.executor.IterativeWorkerTaskFactory$1.call(IterativeWorkerTaskFactory.java:60)
  [weld-core-impl-2.1.2.Final.jar:2014-01-09 09:23]
 at 
 org.jboss.weld.executor.IterativeWorkerTaskFactory$1.call(IterativeWorkerTaskFactory.java:53)
  [weld-core-impl-2.1.2.Final.jar:2014-01-09 09:23]
 at java.util.concurrent.FutureTask.run(Unknown Source) 
 [rt.jar:1.7.0_51]
 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) 
 [rt.jar:1.7.0_51]
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) 
 [rt.jar:1.7.0_51]
 at java.lang.Thread.run(Unknown Source) [rt.jar:1.7.0_51]
 Caused by: java.lang.NoClassDefFoundError: 
 com/sun/jersey/spi/inject/InjectableProvider
 at java.lang.ClassLoader.defineClass1(Native Method) [rt.jar:1.7.0_51]
 at java.lang.ClassLoader.defineClass(Unknown Source) [rt.jar:1.7.0_51]
 at 
 org.jboss.modules.ModuleClassLoader.doDefineOrLoadClass(ModuleClassLoader.java:361)
  [jboss-modules.jar:1.3.3.Final]
 at 
 org.jboss.modules.ModuleClassLoader.defineClass(ModuleClassLoader.java:482) 
 [jboss-modules.jar:1.3.3.Final]
 ... 19 more
 Caused by: java.lang.ClassNotFoundException: 
 com.sun.jersey.spi.inject.InjectableProvider from [Module 
 deployment.solr.war:main from Service Module Loader]
 at 
 org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:213) 
 [jboss-modules.jar:1.3.3.Final]
 at 
 org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:459)
  [jboss-modules.jar:1.3.3.Final]
 at 
 org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:408)
  [jboss-modules.jar:1.3.3.Final]
 at

[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting


[ 
https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065511#comment-14065511
 ] 

Hoss Man commented on SOLR-2894:


bq. Hey Hoss. How would we test this?

I don't know.  I don't think we can.  Like i mentioned before...

bq. ...That's the sort of edge case that may be really hard to test for, but 
even if we can't explicit test it, we should at least have some sanity check in 
the distrib coordination code...

It's an extreme enough edge case that i don't think we need to jump through a 
crazy amount of hoops to have a test case for it, i just didn't want to leave 
such a dangerous trap in the code

 Implement distributed pivot faceting
 

 Key: SOLR-2894
 URL: https://issues.apache.org/jira/browse/SOLR-2894
 Project: Solr
  Issue Type: Improvement
Reporter: Erik Hatcher
Assignee: Hoss Man
 Fix For: 4.9, 5.0

 Attachments: SOLR-2894-mincount-minification.patch, 
 SOLR-2894-reworked.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894_cloud_test.patch, dateToObject.patch, pivot_mincount_problem.sh


 Following up on SOLR-792, pivot faceting currently only supports 
 undistributed mode.  Distributed pivot faceting needs to be implemented.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6254) Failing tests due to timeouts caused by SSL depleting random entropy on Jenkins


[ 
https://issues.apache.org/jira/browse/SOLR-6254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065530#comment-14065530
 ] 

Mark Miller commented on SOLR-6254:
---

Yeah, I have been doing it one by one, but it's pretty unfriendly to people 
developing new tests and are not familiar with the issue, so perhaps it's just 
better to universally disable until we can address it.

 Failing tests due to timeouts caused by SSL depleting random entropy on 
 Jenkins
 ---

 Key: SOLR-6254
 URL: https://issues.apache.org/jira/browse/SOLR-6254
 Project: Solr
  Issue Type: Task
  Components: Tests
Reporter: Steve Rowe

 Tests using SSL can block on Jenkins when random entropy is depleted, causing 
 timeouts that trigger test failures.
 I found some info about /dev/random problems on FreeBSD here: 
 [https://wiki.freebsd.org/201308DevSummit/Security/DevRandom], which lead me 
 to /etc/rc.d/iinitrandom, which gets around the limited entropy by cat'ing a 
 bunch of shit to /dev/random:
 {code}
 ( ps -fauxww; sysctl -a; date; df -ib; dmesg; ps -fauxww ) \
   | dd of=/dev/random bs=8k 2/dev/null
 cat /bin/ls | dd of=/dev/random bs=8k 2/dev/null
 {code}
 I think we should try the same strategy in a crontab every X minutes, to see 
 if that addresses the test failures.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-5830) Fix some missing/assert codec checks to throw CorruptIndexException


 [ 
https://issues.apache.org/jira/browse/LUCENE-5830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Muir resolved LUCENE-5830.
-

   Resolution: Fixed
Fix Version/s: 4.10
   5.0

 Fix some missing/assert codec checks to throw CorruptIndexException
 ---

 Key: LUCENE-5830
 URL: https://issues.apache.org/jira/browse/LUCENE-5830
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Robert Muir
 Fix For: 5.0, 4.10

 Attachments: LUCENE-5830.patch


 I thought i had tackled all these before, but I missed a few good ones.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6254) Failing tests due to timeouts caused by SSL depleting random entropy on Jenkins

2014-07-17 Thread Dawid Weiss (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-6254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065561#comment-14065561
 ] 

Dawid Weiss commented on SOLR-6254:
---

Is it really that, Steve? It would seem very odd to have ssl connections just 
stall like that -- think of any web server or something... how does it work 
there?

Anyway, my impression from debugging hangs related to sockets was that 
freebsd's port of openjdk is not thoroughly tested (that accept issue would 
have been caught by regular openjdk tests I think). Now I'm left in doubt when 
I see an issue pop up on jenkins -- is this our bug or openjdk's?

 Failing tests due to timeouts caused by SSL depleting random entropy on 
 Jenkins
 ---

 Key: SOLR-6254
 URL: https://issues.apache.org/jira/browse/SOLR-6254
 Project: Solr
  Issue Type: Task
  Components: Tests
Reporter: Steve Rowe

 Tests using SSL can block on Jenkins when random entropy is depleted, causing 
 timeouts that trigger test failures.
 I found some info about /dev/random problems on FreeBSD here: 
 [https://wiki.freebsd.org/201308DevSummit/Security/DevRandom], which lead me 
 to /etc/rc.d/iinitrandom, which gets around the limited entropy by cat'ing a 
 bunch of shit to /dev/random:
 {code}
 ( ps -fauxww; sysctl -a; date; df -ib; dmesg; ps -fauxww ) \
   | dd of=/dev/random bs=8k 2/dev/null
 cat /bin/ls | dd of=/dev/random bs=8k 2/dev/null
 {code}
 I think we should try the same strategy in a crontab every X minutes, to see 
 if that addresses the test failures.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting

2014-07-17 Thread Andrew Muldowney (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-2894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065598#comment-14065598
 ] 

Andrew Muldowney commented on SOLR-2894:


So I may have jumped the gun on the fix. Previously refinement requests would 
return counts of zero for things that it was asked about but had no values for, 
this is now no longer true. (It shrinks the required work to merge in 
refinements and limits how much data we send across the wire). This means that 
we cannot ask if a refinement has been fulfilled because if a shard doesn't 
know about a valuePath it will not include it in its response.
If the shard is responding properly it *should* still return a {{facet_pivot}} 
with the top level requested pivots, just with everything below that merely an 
empty list.
So really the only thing we can check is that the {{facet_pivot}} isn't null on 
a response, since that would indicate that something really awful happened.

 Implement distributed pivot faceting
 

 Key: SOLR-2894
 URL: https://issues.apache.org/jira/browse/SOLR-2894
 Project: Solr
  Issue Type: Improvement
Reporter: Erik Hatcher
Assignee: Hoss Man
 Fix For: 4.9, 5.0

 Attachments: SOLR-2894-mincount-minification.patch, 
 SOLR-2894-reworked.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894.patch, SOLR-2894.patch, SOLR-2894.patch, 
 SOLR-2894_cloud_test.patch, dateToObject.patch, pivot_mincount_problem.sh


 Following up on SOLR-792, pivot faceting currently only supports 
 undistributed mode.  Distributed pivot faceting needs to be implemented.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6254) Failing tests due to timeouts caused by SSL depleting random entropy on Jenkins


[ 
https://issues.apache.org/jira/browse/SOLR-6254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14065631#comment-14065631
 ] 

Steve Rowe commented on SOLR-6254:
--

bq. Is it really that, Steve? It would seem very odd to have ssl connections 
just stall like that – think of any web server or something... how does it work 
there?

I don't know - see SOLR-5776 for more info about SSL-related test failures on 
Jenkins.

All of the recent TestCloudSchemaless failures on Jenkins were stuck reading 
SSL sockets, so if it's not entropy depletion, it's something related to SSL 
sockets.  (We'll see in a day or two if TestCloudSchemaless stops failing 
regularly; I added the {{@SuppressSSL}} annotation to it.)

More FreeBSD-specific entropy info:

# FreeBSD's /dev/urandom is symlinked to /dev/random:
\\
{noformat}
sarowe@lucene[867]$ ls -ld /dev/random /dev/urandom
crw-rw-rw-  1 root  wheel0,  13 Jul 17 18:38 /dev/random
lrwxr-xr-x  1 root  wheel 6 Jun 18 23:22 /dev/urandom - random
{noformat}
# FreeBSD's /dev/random blocks when there is insufficient entropy; from 
[http://www.freebsd.org/cgi/man.cgi?format=htmlquery=random(4)]:
\\
{quote}
The kern.random.sys.seeded  variable indicates whether or not the random
device is in an acceptably  secure state as a result of reseeding.  If set
to  0, the device will block (on read) until the next reseed (which can be
from an explicit write, or  as a result of entropy harvesting).  A reseed
will set the value  to 1 (non-blocking).
{quote}
\\
and from [https://wiki.freebsd.org/201308DevSummit/Security/DevRandom]:
{quote}
How /dev/random works today: The primary source of randomness is the [Yarrow 
PRNG|http://www.schneier.com/yarrow.html].
\[...]
Yarrow \[...] need\[s] to accumulate a certain amount of entropy before \[it] 
can start generating random numbers. Until that happens, reads from /dev/random 
will block.
{quote}
\\
More info about randomness sources at 
[https://wiki.freebsd.org/201308DevSummit/Security/DevRandom].

 Failing tests due to timeouts caused by SSL depleting random entropy on 
 Jenkins
 ---

 Key: SOLR-6254
 URL: https://issues.apache.org/jira/browse/SOLR-6254
 Project: Solr
  Issue Type: Task
  Components: Tests
Reporter: Steve Rowe

 Tests using SSL can block on Jenkins when random entropy is depleted, causing 
 timeouts that trigger test failures.
 I found some info about /dev/random problems on FreeBSD here: 
 [https://wiki.freebsd.org/201308DevSummit/Security/DevRandom], which lead me 
 to /etc/rc.d/iinitrandom, which gets around the limited entropy by cat'ing a 
 bunch of shit to /dev/random:
 {code}
 ( ps -fauxww; sysctl -a; date; df -ib; dmesg; ps -fauxww ) \
   | dd of=/dev/random bs=8k 2/dev/null
 cat /bin/ls | dd of=/dev/random bs=8k 2/dev/null
 {code}
 I think we should try the same strategy in a crontab every X minutes, to see 
 if that addresses the test failures.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2894) Implement distributed pivot faceting