[jira] [Updated] (SOLR-3597) seems like a lot of wasted whitespace at the top of the admin screens

[Stefan Matheis (steffkes) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Matheis (steffkes) updated SOLR-3597:


Attachment: SOLR-3597.patch

Right, didn't notice that .. updated Patch attached

> seems like a lot of wasted whitespace at the top of the admin screens
> -
>
> Key: SOLR-3597
> URL: https://issues.apache.org/jira/browse/SOLR-3597
> Project: Solr
>  Issue Type: Improvement
>  Components: web gui
>Reporter: Hoss Man
>Assignee: Stefan Matheis (steffkes)
> Fix For: 4.0
>
> Attachments: error-divs-overlap.png, SOLR-3597-hack.patch, 
> SOLR-3597.patch, SOLR-3597.patch
>
>
> because of how the UI is laid out, the solr logo is all by itself above the 
> main table, leaving a lot of whitespace to the right of the solr logo.
> this seems like a waste that should be easily curable?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4352) only the test runner should be able to System.exit

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447515#comment-13447515
 ] 

Uwe Schindler commented on LUCENE-4352:
---

For the test-framework an example for using AccessControler.doPrivileged() 
would be:
- Policy denies e.g. file access outside the test runner JAR (which may be a 
good idea) -> "untrusted code"
- But Policy allows writing files outside the J0 working directory for the 
test-framework.jar file itself -> "trusted code"

The problem here would be: 
- "untrusted" code calls "trusted" code, e.g. the test ("untrusted") calls a 
method from test-framework that writes outside base directory. The calling 
"untrusted" code is not allowed to do this, but the test-framework itsself 
("trusted") is allowed to do this.
- SecurityManager would disallow this, but that may not what we want.

The naming of AccessController.doPrivileged is a little bit misleading. It does 
not mean something is "privileged", it just tells that the closure should not 
be handled by access checks of the code calling you, but with the context of 
your own code. This is always needed when untrusted code calls trusted code 
which would normally have higher rights.

The above example might be a good solution (limit write access to test-runner's 
local private "J0" directory only).

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/test
>Affects Versions: 4.0-BETA
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Fix For: 5.0, 4.0
>
> Attachments: LUCENE-4352.patch, LUCENE-4352.patch, LUCENE-4352.patch, 
> LUCENE-4352.patch, LUCENE-4352.patch
>
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4352) only the test runner should be able to System.exit

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447513#comment-13447513
 ] 

Dawid Weiss commented on LUCENE-4352:
-

Ok, I get it.

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/test
>Affects Versions: 4.0-BETA
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Fix For: 5.0, 4.0
>
> Attachments: LUCENE-4352.patch, LUCENE-4352.patch, LUCENE-4352.patch, 
> LUCENE-4352.patch, LUCENE-4352.patch
>
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4352) only the test runner should be able to System.exit

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447511#comment-13447511
 ] 

Uwe Schindler commented on LUCENE-4352:
---

bq. Stack traces are retrieved from all over the place in the test framework 
too (on test timeouts, on thread leaks). Will it work?

If you install a SecurityManager that does not allow this, it will of course 
not work. The thing here is more special and has nothing to do with 
Test-Framework: If the Security Manager itsself has to do additional checks 
that may be denied, it must du this using his own security context (thats what 
AccessController is doing here) and not with the one of the caller.

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/test
>Affects Versions: 4.0-BETA
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Fix For: 5.0, 4.0
>
> Attachments: LUCENE-4352.patch, LUCENE-4352.patch, LUCENE-4352.patch, 
> LUCENE-4352.patch, LUCENE-4352.patch
>
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4352) only the test runner should be able to System.exit

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447509#comment-13447509
 ] 

Dawid Weiss commented on LUCENE-4352:
-

bq. The access checks must be done with the SecurityManager's access context, 
otherwise things like getting stack trace could be denied

Stack traces are retrieved from all over the place in the test framework too 
(on test timeouts, on thread leaks). Will it work?

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/test
>Affects Versions: 4.0-BETA
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Fix For: 5.0, 4.0
>
> Attachments: LUCENE-4352.patch, LUCENE-4352.patch, LUCENE-4352.patch, 
> LUCENE-4352.patch, LUCENE-4352.patch
>
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4345) Create a Classification module

[Lance Norskog (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447429#comment-13447429
 ] 

Lance Norskog commented on LUCENE-4345:
---

bq. would make training slower but it could be useful to improve accuracy
If you use index data which is already analyzed with the same analyzer as your 
test (unseen) documents, you can use a lot more documents as input. More is 
better. As the training data increases, signal drives out noise. Once you add 
the ability to store & load models, training speed becomes less important. 

Look at the Mahout project for ideas about text classifiers. The 
ConfusionMatrix class and the html page it prints are really handy for 
summarizing and probing the classifier's performance.


> Create a Classification module
> --
>
> Key: LUCENE-4345
> URL: https://issues.apache.org/jira/browse/LUCENE-4345
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
>Priority: Minor
> Attachments: LUCENE-4345_2.patch, LUCENE-4345.patch, 
> SOLR-3700_2.patch, SOLR-3700.patch
>
>
> Lucene/Solr can host huge sets of documents containing lots of information in 
> fields so that these can be used as training examples (w/ features) in order 
> to very quickly create classifiers algorithms to use on new documents and / 
> or to provide an additional service.
> So the idea is to create a contrib module (called 'classification') to host a 
> ClassificationComponent that will use already seen data (the indexed 
> documents / fields) to classify new documents / text fragments.
> The first version will contain a (simplistic) Lucene based Naive Bayes 
> classifier but more implementations should be added in the future.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-4352) only the test runner should be able to System.exit

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler resolved LUCENE-4352.
---

Resolution: Fixed

Comitted trunk revision: 1380391
Committed 4.x revision: 1380392

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/test
>Affects Versions: 4.0-BETA
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Fix For: 5.0, 4.0
>
> Attachments: LUCENE-4352.patch, LUCENE-4352.patch, LUCENE-4352.patch, 
> LUCENE-4352.patch, LUCENE-4352.patch
>
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4352) only the test runner should be able to System.exit

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-4352:
--

Attachment: LUCENE-4352.patch

My final version:
- The access checks must be done with the SecurityManager's access context, 
otherwise things like getting stack trace could be denied. This is ensured by 
AccessController.
- The SecurityException also gives exit code and method called
- Also Runtime.halt() is handled.

I will commit this now.

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/test
>Affects Versions: 4.0-BETA
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Fix For: 5.0, 4.0
>
> Attachments: LUCENE-4352.patch, LUCENE-4352.patch, LUCENE-4352.patch, 
> LUCENE-4352.patch, LUCENE-4352.patch
>
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-3076) Solr should support block joins

[Mikhail Khludnev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447419#comment-13447419
 ] 

Mikhail Khludnev edited comment on SOLR-3076 at 9/4/12 9:19 AM:


Hi

I can't check it right now, can't remember how deep I covered it, but I 
remember that there was a test or even level of certainty that nested docs are 
passed through DisttributedUpdateProcessor by using a top level doc primary 
key. Therefore whole block should land into particular shard.  If this 
necessary condition is met query side should work out of the box. 
Also you can find about performance benefits

http://blog.griddynamics.com/2012/08/block-join-query-performs.html

  was (Author: mkhludnev):
Hi

I can't check it right now, can't remember how deep I covered it, but I 
remember that there was a test or even level of certainty that nested docs are 
passed through DisttributedUpdateProcessor by using a top level doc primary 
key. Therefore whole block should land into particular shard.  If this 
necessary condition is met query side should work out of the box. 
Also you can find about performance benefits
blog.griddynamics.com/2012/08/block-join-query-performs.htmli
  
> Solr should support block joins
> ---
>
> Key: SOLR-3076
> URL: https://issues.apache.org/jira/browse/SOLR-3076
> Project: Solr
>  Issue Type: New Feature
>Reporter: Grant Ingersoll
> Attachments: 27M-singlesegment-histogram.png, 27M-singlesegment.png, 
> bjq-vs-filters-backward-disi.patch, bjq-vs-filters-illegal-state.patch, 
> child-bjqparser.patch, parent-bjq-qparser.patch, parent-bjq-qparser.patch, 
> Screen Shot 2012-07-17 at 1.12.11 AM.png, SOLR-3076.patch, SOLR-3076.patch, 
> SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, 
> solrconf-bjq-erschema-snippet.xml, solrconfig.xml.patch, 
> tochild-bjq-filtered-search-fix.patch
>
>
> Lucene has the ability to do block joins, we should add it to Solr.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3076) Solr should support block joins

[Mikhail Khludnev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447419#comment-13447419
 ] 

Mikhail Khludnev commented on SOLR-3076:


Hi

I can't check it right now, can't remember how deep I covered it, but I 
remember that there was a test or even level of certainty that nested docs are 
passed through DisttributedUpdateProcessor by using a top level doc primary 
key. Therefore whole block should land into particular shard.  If this 
necessary condition is met query side should work out of the box. 
Also you can find about performance benefits
blog.griddynamics.com/2012/08/block-join-query-performs.htmli

> Solr should support block joins
> ---
>
> Key: SOLR-3076
> URL: https://issues.apache.org/jira/browse/SOLR-3076
> Project: Solr
>  Issue Type: New Feature
>Reporter: Grant Ingersoll
> Attachments: 27M-singlesegment-histogram.png, 27M-singlesegment.png, 
> bjq-vs-filters-backward-disi.patch, bjq-vs-filters-illegal-state.patch, 
> child-bjqparser.patch, parent-bjq-qparser.patch, parent-bjq-qparser.patch, 
> Screen Shot 2012-07-17 at 1.12.11 AM.png, SOLR-3076.patch, SOLR-3076.patch, 
> SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, 
> solrconf-bjq-erschema-snippet.xml, solrconfig.xml.patch, 
> tochild-bjq-filtered-search-fix.patch
>
>
> Lucene has the ability to do block joins, we should add it to Solr.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4352) only the test runner should be able to System.exit

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447409#comment-13447409
 ] 

Dawid Weiss commented on LUCENE-4352:
-

I think it's fine as far as my limited knowledge of security architecture 
allows me to tell.

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/test
>Affects Versions: 4.0-BETA
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Fix For: 5.0, 4.0
>
> Attachments: LUCENE-4352.patch, LUCENE-4352.patch, LUCENE-4352.patch, 
> LUCENE-4352.patch
>
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4352) only the test runner should be able to System.exit

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-4352:
--

  Component/s: general/test
Affects Version/s: 4.0-BETA
Fix Version/s: 4.0
   5.0

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/test
>Affects Versions: 4.0-BETA
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Fix For: 5.0, 4.0
>
> Attachments: LUCENE-4352.patch, LUCENE-4352.patch, LUCENE-4352.patch, 
> LUCENE-4352.patch
>
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (LUCENE-4352) only the test runner should be able to System.exit

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler reassigned LUCENE-4352:
-

Assignee: Uwe Schindler

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/test
>Affects Versions: 4.0-BETA
>Reporter: Robert Muir
>Assignee: Uwe Schindler
> Fix For: 5.0, 4.0
>
> Attachments: LUCENE-4352.patch, LUCENE-4352.patch, LUCENE-4352.patch, 
> LUCENE-4352.patch
>
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4352) only the test runner should be able to System.exit

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-4352:
--

Attachment: LUCENE-4352.patch

Attached with additional javadocs and the delegation to super readded (so the 
default AccessController has a chance to recognize the system exit and do 
something).

The loop is indeed better, while watching TV I had the same in mind but you 
were faster.

Do you think we should commit this hack? It's tiny and we already have a 
SecurityManager, so no overhead.

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/test
>Affects Versions: 4.0-BETA
>Reporter: Robert Muir
> Fix For: 5.0, 4.0
>
> Attachments: LUCENE-4352.patch, LUCENE-4352.patch, LUCENE-4352.patch, 
> LUCENE-4352.patch
>
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4352) only the test runner should be able to System.exit

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447402#comment-13447402
 ] 

Dawid Weiss commented on LUCENE-4352:
-

bq. If you override this method, then you should make a call to super.checkExit 
at the point the overridden method would normally throw an exception.

It isn't mandatory -- they say "you should" not "must". I don't understand the 
purpose (and wording) of it either, other than just delegate to check the 
default policy. I think in our case this won't matter much.

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: LUCENE-4352.patch, LUCENE-4352.patch, LUCENE-4352.patch
>
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3755) shard splitting

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447400#comment-13447400
 ] 

Yonik Seeley commented on SOLR-3755:


It seems like we need logical shard parameters (i.e. Slice class), but we don't 
currently have a place for them.
These parameters would include:
 - collection (this is somewhat redundant, but belongs more on a slice than on 
a replica)
 - replication factor (i.e. in time based sharding, one may want more replicas 
of recent shards to handle greater query throughput)
 - hash range(s) covered by the slice
 - maybe a pointer to the leader, rather than having to search through the 
nodes?

You can see the previous structure of cloudstate from my previous message.

One fix is to introduce a "nodes" or "replicas" level to contain the nodes and 
leave the other properties as top-level:

{code}
  "shard1": {
"replication_factor" : 3,
"range" : "-3fff",
"nodes" : {
  "Rogue:8983_solr_collection1":{
"state" : "active"
  }
}
  }
{code}

Another way is to introduce a "props" to store properties:

{code}
  "shard1": {
"props" : {
  "replication_factor" : 3,
  "range" : "-3fff"
},
"Rogue:8983_solr_collection1":{
  "state" : "active"
}
  }
{code}

The first option feels more natural to me - properties are directly under the 
shard, and the nodes of a shard are simply another property.

> shard splitting
> ---
>
> Key: SOLR-3755
> URL: https://issues.apache.org/jira/browse/SOLR-3755
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Reporter: Yonik Seeley
> Attachments: SOLR-3755.patch, SOLR-3755.patch
>
>
> We can currently easily add replicas to handle increases in query volume, but 
> we should also add a way to add additional shards dynamically by splitting 
> existing shards.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3076) Solr should support block joins

[Pavel Goncharik (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447399#comment-13447399
 ] 

Pavel Goncharik commented on SOLR-3076:
---

Hi,

I'm wondering if patches in this issue implement "block joins" for a single 
Solr shard only, or implementation works correctly out-of-the-box in 
distributed mode (SolrCloud)?

> Solr should support block joins
> ---
>
> Key: SOLR-3076
> URL: https://issues.apache.org/jira/browse/SOLR-3076
> Project: Solr
>  Issue Type: New Feature
>Reporter: Grant Ingersoll
> Attachments: 27M-singlesegment-histogram.png, 27M-singlesegment.png, 
> bjq-vs-filters-backward-disi.patch, bjq-vs-filters-illegal-state.patch, 
> child-bjqparser.patch, parent-bjq-qparser.patch, parent-bjq-qparser.patch, 
> Screen Shot 2012-07-17 at 1.12.11 AM.png, SOLR-3076.patch, SOLR-3076.patch, 
> SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, SOLR-3076.patch, 
> solrconf-bjq-erschema-snippet.xml, solrconfig.xml.patch, 
> tochild-bjq-filtered-search-fix.patch
>
>
> Lucene has the ability to do block joins, we should add it to Solr.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4352) only the test runner should be able to System.exit

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447397#comment-13447397
 ] 

Uwe Schindler commented on LUCENE-4352:
---

The upwards delegation is mandatory, the javadocs require this (although 
doesn't matter here).

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: LUCENE-4352.patch, LUCENE-4352.patch, LUCENE-4352.patch
>
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4352) only the test runner should be able to System.exit

[Dawid Weiss (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dawid Weiss updated LUCENE-4352:


Attachment: LUCENE-4352.patch

Isn't this equivalent with a slightly easier to read logic, Uwe? Also I didn't 
delegate up to the system security manager but just return -- why did you 
delegate upwards, was it intentional?

Anyway, works for me!

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: LUCENE-4352.patch, LUCENE-4352.patch, LUCENE-4352.patch
>
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4352) only the test runner should be able to System.exit

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-4352:
--

Attachment: LUCENE-4352.patch

Of course we can! Attached is new patch - using 
"(e.getClassName().startsWith("com.carrotsearch.ant.tasks.junit4."))". The 
important thing is that there is nothing between the last System.exit() or 
Runtime.exit() in the stack trace before the test runner package (as of above 
if statement), so its garanteed that the test runner itsself is calling exit() 
and not something deeper in the stack trace.

The loop in this code looks ugly but allows both patterns:
- test-runner calling System.exit() which itsself calls 
Runtime.getRuntime().exit()
- or test-runner clalling Runtime.exit() directly

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: LUCENE-4352.patch, LUCENE-4352.patch
>
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4352) only the test runner should be able to System.exit

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447370#comment-13447370
 ] 

Dawid Weiss commented on LUCENE-4352:
-

bq. This patch works for me, but I need a list of all "valid" exit points with 
stack traces that can occur in Dawid's code.

Uwe, can we restrict to a package (and subpackages) instead? 
com.carrotsearch.randomizedtesting.* or something will do. I can provide all 
the exit points but it'd be nicer if we just do the package -- I can then 
refactor things without looking at this stuff.

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: LUCENE-4352.patch
>
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4352) only the test runner should be able to System.exit

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-4352:
--

Attachment: LUCENE-4352.patch

Here is a quick patch thet prevents System.exit() with a dirty hack:

- This installs a subclass of Java's original java.lang.SecurityManager, so the 
tests.policy file is still taken into account (see common-build.xml)
- The method checkExit(int) was overridden to:

# create a SecurityException, but don't throw (just to get the stack trace).
# Inspect stack trace and saerch for System.exit (to walk over JVM-internal 
things)
# The *next* element in the stack trace (important, no gaps inbetween) after 
System.exit() must be SlaveMain.main() from Junit4's ANT runner. If this stack 
element is found, checkExit() delegates to superclass (which will always pass 
without SecurityException)
# If no valid stack trace element is found, checkExit() will finally throw the 
already instantiated SecurityException

This patch works for me, but I need a list of all "valid" exit points with 
stack traces that can occur in Dawid's code.

THIS IS A HACK, BUT WORKS!

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
> Attachments: LUCENE-4352.patch
>
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4340) Move all codecs but Lucene40 to a module

[Steven Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447366#comment-13447366
 ] 

Steven Rowe commented on LUCENE-4340:
-

bq. does anyone know how to produce something both readable and applyable 
through svn?

I post the svn (move/copy/etc.) script to the issue and generate the patch 
using {{svn diff --no-diff-deleted}}.  Somebody wanting to apply the patch just 
runs the svn script first.

> Move all codecs but Lucene40 to a module
> 
>
> Key: LUCENE-4340
> URL: https://issues.apache.org/jira/browse/LUCENE-4340
> Project: Lucene - Core
>  Issue Type: Task
>  Components: core/codecs, general/build
>Reporter: Adrien Grand
> Fix For: 5.0, 4.0
>
>
> We should move all concrete postings formats and codecs but Lucene40 to a 
> module.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4340) Move all codecs but Lucene40 to a module

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447360#comment-13447360
 ] 

Adrien Grand commented on LUCENE-4340:
--

I tried the proposed approach on my local copy (unfortunately, the diff is 
pretty meaningless because of the svn mvoves, does anyone know how to produce 
something both readable and applyable through svn?):

 - moved  all sub packages of oal.codecs except perfield and lucene40 to a new 
{{codecs}} module,
 - also moved BlockTerms{Reader,Writer}, FixedGapTermsIndex{Reader, Writer}, 
TermsIndex{Reader,Writer}Base, VariableGapTermsIndex{Reader,Writer},
 - lucene-codecs has been added to the classpath of test-framework,
 - lucene-codecs has been added to the test classpath of lucene-core.

Compilation and tests seem to work fine...

Does it look good to you? I tried to take care of javadocs, eclipse, idea and 
Maven, is there anything else I should worry about?

> Move all codecs but Lucene40 to a module
> 
>
> Key: LUCENE-4340
> URL: https://issues.apache.org/jira/browse/LUCENE-4340
> Project: Lucene - Core
>  Issue Type: Task
>  Components: core/codecs, general/build
>Reporter: Adrien Grand
> Fix For: 5.0, 4.0
>
>
> We should move all concrete postings formats and codecs but Lucene40 to a 
> module.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3755) shard splitting

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447347#comment-13447347
 ] 

Yonik Seeley commented on SOLR-3755:


Just committed some more progress.

http://svn.apache.org/viewvc?rev=1380287&view=rev

I started up a ZK cluster with one shard, one node.
curl "http://localhost:8983/solr/admin/cores?core=collection1&action=SPLIT";

The cloud state after looks like
{code}
{"collection1":{
"shard1":{"Rogue:8983_solr_collection1":{
"shard":"shard1",
"roles":null,
"leader":"true",
"state":"active",
"core":"collection1",
"collection":"collection1",
"node_name":"Rogue:8983_solr",
"base_url":"http://Rogue:8983/solr"}},
"shard1_0":{"Rogue:8983_solr_collection1_0":{
"shard":"shard1_0",
"leader":"true",
"roles":null,
"state":"active",
"core":"collection1_0",
"collection":"collection1",
"node_name":"Rogue:8983_solr",
"base_url":"http://Rogue:8983/solr"}},
"shard1_1":{"Rogue:8983_solr_collection1_1":{
"shard":"shard1_1",
"roles":null,
"leader":"true",
"state":"active",
"core":"collection1_1",
"collection":"collection1",
"node_name":"Rogue:8983_solr",
"base_url":"http://Rogue:8983/solr"
{code}

The original core had 32 docs.  After I did a manual commit on both of the new 
cores, the first showed 14 docs and the second 18.


> shard splitting
> ---
>
> Key: SOLR-3755
> URL: https://issues.apache.org/jira/browse/SOLR-3755
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Reporter: Yonik Seeley
> Attachments: SOLR-3755.patch, SOLR-3755.patch
>
>
> We can currently easily add replicas to handle increases in query volume, but 
> we should also add a way to add additional shards dynamically by splitting 
> existing shards.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4352) only the test runner should be able to System.exit

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447343#comment-13447343
 ] 

Uwe Schindler commented on LUCENE-4352:
---

I think, simply passing another SecurityManager with -D should handle all this 
(and checking the stack trace from it like Preflex-Codec). The reason why 
exitVM permission is granted automatically to all classes from within 
-classpath is tricky: Otherwise, the code could never exit itsself (it would 
not even allow to exit after its own main() method ends!).

I'll play around with it. The Zookeeper issue is well known, so they are 
waiting for their patch to apply again (ZOOKEEPER-1544 was closed as duplicate, 
ZOOKEEPER-575 is pending since long time - I don't expect a solution soon).

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4352) only the test runner should be able to System.exit

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447334#comment-13447334
 ] 

Dawid Weiss commented on LUCENE-4352:
-

Thanks for checking, Uwe. I could integrate a sub-class loader for test 
classes, but it'll take me some time, I'm busy at the moment. Don't know about 
overriding the default system class loader trick -- perhaps it'd work (it's 
what Maven and ANT are using if I remember correctly).

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4352) only the test runner should be able to System.exit

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447323#comment-13447323
 ] 

Uwe Schindler commented on LUCENE-4352:
---

Running with:

{noformat}
ant test -Dargs="-Djava.security.debug=all" ...
{noformat}

verifies this, it grants exitVM to every package on classpath - just read the 
debug output (very interesting...!)

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4352) only the test runner should be able to System.exit

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447320#comment-13447320
 ] 

Uwe Schindler commented on LUCENE-4352:
---

See also: 
http://stackoverflow.com/questions/8270340/prohibit-the-call-to-system-exit and 
http://stackoverflow.com/questions/309396/java-how-to-test-methods-that-call-system-exit/309427#309427

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4352) only the test runner should be able to System.exit

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447314#comment-13447314
 ] 

Uwe Schindler commented on LUCENE-4352:
---

I played a little bit around and the above javadoc behaves exactly as proposed: 
Code from the default classloader can always call System.exit/System.halt. I 
played around and removed the RuntimePermission at all to verify this, Lucene 
tests still passed with:

{noformat}
  permission java.lang.RuntimePermission "createClassLoader";
  permission java.lang.RuntimePermission "getClassLoader";
  permission java.lang.RuntimePermission "setIO";
  permission java.lang.RuntimePermission "modifyThread";
  permission java.lang.RuntimePermission "modifyThreadGroup";
  permission java.lang.RuntimePermission "accessDeclaredMembers";
  permission java.lang.RuntimePermission "getStackTrace";
  permission java.lang.RuntimePermission "setDefaultUncaughtExceptionHandler";
  permission java.lang.RuntimePermission "shutdownHooks";
{noformat}

So theoretically we can restrict a lot of stuff (like calling Thread.stop(), 
which is deprecated) or setting another SecurityManager that circumvents our 
HighSecurity(tm) environment (LOL), but not System.exit().

There is only one possibility that I will check now: We currently pass:

{code:xml}

{code}

If the test VM calls System.exit(), the SecurityManager's method checkExit() is 
in all cases invoked and can jump in... So we can provide a custom 
SecurityManger from the test framework that passes everything to the default 
manager (so enforcing our Policy), but on checkExit(int), it will inspect the 
stack trace like the preflex-RW codec does in Mike's hack and look for the 
test-runners "exit" signature to allow exit only for this case; in all other 
cases throw SecurityException.

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-3782) A leader going down while updates are coming in can cause shard inconsistency.

[Mark Miller (JIRA)]

Mark Miller created SOLR-3782:
-

 Summary: A leader going down while updates are coming in can cause 
shard inconsistency.
 Key: SOLR-3782
 URL: https://issues.apache.org/jira/browse/SOLR-3782
 Project: Solr
  Issue Type: Bug
  Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: 4.0, 5.0


Harpoon into the head of the great whale I have been chasing for a couple weeks 
now.

ChaosMonkey test was exposing this.

Turns out the problem was the solr cmd distrib executor - when closing the 
leader CoreContainer, we would close the zkController while updates can still 
flow through the distrib executor. The result was that we would send updates 
from the leader briefly even though there was a new leader.

I had suspected something similar to this at one point in the hunt and started 
adding some defensive state checks that we wanted to add anyway. I don't think 
they caught all of this issue due to the limited tightness one of the state 
checks can get to (checking the cloudstate leader from a replica against the 
leader indicated by the request).

So the answer is to finally work out how to stop the solr cmd distrib executor 
- because we need to stop it before closing zkController and giving up our role 
as leader.

I've worked that all out and the issue no longer seems to be a problem.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4345) Create a Classification module

[Tommaso Teofili (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447293#comment-13447293
 ] 

Tommaso Teofili commented on LUCENE-4345:
-

bq. Nice! I've found that filtering for nouns & verbs makes another NLP task 
(latent semantic indexing) work much better. This will benefit from 
parts-of-speech filtering.

my former comment is partially correct as the Analyzer is currently used only 
on the unseen text rather than on the whole set of docs too, using it (or other 
Analyzers) with the existing docs' text would make training slower but it could 
be useful to improve accuracy. Maybe a subclass of the current one which is 
capable of doing that would be a nice addition.

> Create a Classification module
> --
>
> Key: LUCENE-4345
> URL: https://issues.apache.org/jira/browse/LUCENE-4345
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
>Priority: Minor
> Attachments: LUCENE-4345_2.patch, LUCENE-4345.patch, 
> SOLR-3700_2.patch, SOLR-3700.patch
>
>
> Lucene/Solr can host huge sets of documents containing lots of information in 
> fields so that these can be used as training examples (w/ features) in order 
> to very quickly create classifiers algorithms to use on new documents and / 
> or to provide an additional service.
> So the idea is to create a contrib module (called 'classification') to host a 
> ClassificationComponent that will use already seen data (the indexed 
> documents / fields) to classify new documents / text fragments.
> The first version will contain a (simplistic) Lucene based Naive Bayes 
> classifier but more implementations should be added in the future.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4345) Create a Classification module

[Tommaso Teofili (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tommaso Teofili updated LUCENE-4345:


Attachment: LUCENE-4345_2.patch

> Create a Classification module
> --
>
> Key: LUCENE-4345
> URL: https://issues.apache.org/jira/browse/LUCENE-4345
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
>Priority: Minor
> Attachments: LUCENE-4345_2.patch, LUCENE-4345.patch, 
> SOLR-3700_2.patch, SOLR-3700.patch
>
>
> Lucene/Solr can host huge sets of documents containing lots of information in 
> fields so that these can be used as training examples (w/ features) in order 
> to very quickly create classifiers algorithms to use on new documents and / 
> or to provide an additional service.
> So the idea is to create a contrib module (called 'classification') to host a 
> ClassificationComponent that will use already seen data (the indexed 
> documents / fields) to classify new documents / text fragments.
> The first version will contain a (simplistic) Lucene based Naive Bayes 
> classifier but more implementations should be added in the future.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4345) Create a Classification module

[Tommaso Teofili (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tommaso Teofili updated LUCENE-4345:


Attachment: (was: LUCENE-4345_2.patch)

> Create a Classification module
> --
>
> Key: LUCENE-4345
> URL: https://issues.apache.org/jira/browse/LUCENE-4345
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
>Priority: Minor
> Attachments: LUCENE-4345.patch, SOLR-3700_2.patch, SOLR-3700.patch
>
>
> Lucene/Solr can host huge sets of documents containing lots of information in 
> fields so that these can be used as training examples (w/ features) in order 
> to very quickly create classifiers algorithms to use on new documents and / 
> or to provide an additional service.
> So the idea is to create a contrib module (called 'classification') to host a 
> ClassificationComponent that will use already seen data (the indexed 
> documents / fields) to classify new documents / text fragments.
> The first version will contain a (simplistic) Lucene based Naive Bayes 
> classifier but more implementations should be added in the future.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: svn commit: r1379362 - in /lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud: BasicDistributedZk2Test.java BasicDistributedZkTest.java

[Dawid Weiss]

I filed this:
https://issues.apache.org/jira/browse/ZOOKEEPER-1544

Dawid

On Mon, Sep 3, 2012 at 1:51 PM, Robert Muir  wrote:
> we should open a bug at zookeeper.
>
> On Mon, Sep 3, 2012 at 7:03 AM, Dawid Weiss
>  wrote:
>> Oh man... now I know why the hell the JVM terminates with error 11...
>>
>> SyncThread's interrupt causes this:
>>
>> } catch (Throwable t) {
>> LOG.fatal("Severe unrecoverable error, exiting", t);
>> System.exit(11);
>> }
>>
>> D.
>>
>> On Fri, Aug 31, 2012 at 5:21 PM, Dawid Weiss
>>  wrote:
>>> Oh yeah -- you mentioned it to me once, but I forgot about it. Thanks,
>>> I'll see if this was the cause.
>>>
>>> Dawid
>>>
>>> On Fri, Aug 31, 2012 at 3:12 PM, Robert Muir  wrote:
 FYI: I'm not sure if this is related, but there is a thread from
 zookeeper "SyncThread" that you should never interrupt.

 Back in the day interrupting this thread in freebsd jenkins (but also
 locally on windows or linux) would cause JVM crashes.

 so the old huge hairy LuceneTestCase had this code:

 if (!t.getName().startsWith("SyncThread")) // avoid zookeeper jre crash
 t.interrupt()

 Not sure if its still applicable.

 On Fri, Aug 31, 2012 at 4:39 AM,   wrote:
> Author: dweiss
> Date: Fri Aug 31 08:39:29 2012
> New Revision: 1379362
>
> URL: http://svn.apache.org/viewvc?rev=1379362&view=rev
> Log:
> Don't interrupt threads in this test, just report thread leak errors. 
> Should help get past the broken (?) freebsd jvm.
>
> Modified:
> 
> lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZk2Test.java
> 
> lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZkTest.java
>
> Modified: 
> lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZk2Test.java
> URL: 
> http://svn.apache.org/viewvc/lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZk2Test.java?rev=1379362&r1=1379361&r2=1379362&view=diff
> ==
> --- 
> lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZk2Test.java
>  (original)
> +++ 
> lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZk2Test.java
>  Fri Aug 31 08:39:29 2012
> @@ -34,16 +34,16 @@ import org.apache.solr.common.params.Com
>  import org.apache.solr.common.params.ModifiableSolrParams;
>  import org.apache.solr.servlet.SolrDispatchFilter;
>
> +import com.carrotsearch.randomizedtesting.annotations.ThreadLeakAction;
> +import 
> com.carrotsearch.randomizedtesting.annotations.ThreadLeakAction.Action;
> +
>  /**
>   * This test simply does a bunch of basic things in solrcloud mode and 
> asserts things
>   * work as expected.
>   */
> +@ThreadLeakAction({Action.WARN})
>  public class BasicDistributedZk2Test extends 
> AbstractFullDistribZkTestBase {
> -
> -  public BasicDistributedZk2Test() {
> -super();
> -  }
> -
> +
>/*
> * (non-Javadoc)
> *
>
> Modified: 
> lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZkTest.java
> URL: 
> http://svn.apache.org/viewvc/lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZkTest.java?rev=1379362&r1=1379361&r2=1379362&view=diff
> ==
> --- 
> lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZkTest.java
>  (original)
> +++ 
> lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZkTest.java
>  Fri Aug 31 08:39:29 2012
> @@ -70,11 +70,15 @@ import org.apache.solr.common.util.Named
>  import org.apache.solr.update.SolrCmdDistributor.Request;
>  import org.apache.solr.util.DefaultSolrThreadFactory;
>
> +import com.carrotsearch.randomizedtesting.annotations.ThreadLeakAction;
> +import 
> com.carrotsearch.randomizedtesting.annotations.ThreadLeakAction.Action;
> +
>  /**
>   * This test simply does a bunch of basic things in solrcloud mode and 
> asserts things
>   * work as expected.
>   */
>  @Slow
> +@ThreadLeakAction({Action.WARN})
>  public class BasicDistributedZkTest extends AbstractDistribZkTestBase {
>
>private static final String DEFAULT_COLLECTION = "collection1";
>
>



 --
 lucidworks.com

 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h

[jira] [Commented] (LUCENE-4352) only the test runner should be able to System.exit

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447280#comment-13447280
 ] 

Dawid Weiss commented on LUCENE-4352:
-

bq. Does the test runner call System.exit() at all or does it just let the 
main() method finish?

It calls system.exit() because otherwise it wouldn't be able to terminate the 
forked JVM if background non-daemon threads are started from tests.

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3865) MemoryIndex does not allow user to add multiple values for a single field name

[Jasper van Veghel (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-3865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447263#comment-13447263
 ] 

Jasper van Veghel commented on LUCENE-3865:
---

Is MemoryIndex still a relevant way of doing Prospective Search? I see a couple 
of other issues that talk about using a regular IndexWriter with a MemoryCodec.

I myself am also looking for a way to do Prospective Search — I'd like to push 
documents into both Apache Solr (general search) and something like MemoryIndex 
(for alerting a large number of users instantly.) I've implemented it to the 
point where I have a Solr IndexSchema to provide me with the same Analyzers as 
the ones the Apache Solr instance has, but now I'm running into the issue that 
MemoryIndex does not support multiple fields and only supports String-valued 
elements (less of an issue than the lack of multiple-field support) .. Should I 
make an effort to try and add multiple-field support to MemoryIndex or should I 
look towards another way of achieving the same? :-)

> MemoryIndex does not allow user to add multiple values for a single field name
> --
>
> Key: LUCENE-3865
> URL: https://issues.apache.org/jira/browse/LUCENE-3865
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/other
>Affects Versions: 3.5
> Environment: All
>Reporter: Dave Seltzer
>Priority: Minor
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> When using MemoryIndex.addField the following operation throws an 
> IllegalArgumentException:
> index.addField("foobar", "value1", LuceneAnalyzer); 
> index.addField("foobar", "value2", LuceneAnalyzer); 
> This throws:
> java.lang.IllegalArgumentException: field must not be added more than once
> According to Uwe Schindler on the java-user mailing list this violates the 
> IndexWriter/IndexReader contract.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4345) Create a Classification module

[Tommaso Teofili (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tommaso Teofili updated LUCENE-4345:


Attachment: LUCENE-4345_2.patch

attaching the updated patch (without Maps)

> Create a Classification module
> --
>
> Key: LUCENE-4345
> URL: https://issues.apache.org/jira/browse/LUCENE-4345
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
>Priority: Minor
> Attachments: LUCENE-4345_2.patch, LUCENE-4345.patch, 
> SOLR-3700_2.patch, SOLR-3700.patch
>
>
> Lucene/Solr can host huge sets of documents containing lots of information in 
> fields so that these can be used as training examples (w/ features) in order 
> to very quickly create classifiers algorithms to use on new documents and / 
> or to provide an additional service.
> So the idea is to create a contrib module (called 'classification') to host a 
> ClassificationComponent that will use already seen data (the indexed 
> documents / fields) to classify new documents / text fragments.
> The first version will contain a (simplistic) Lucene based Naive Bayes 
> classifier but more implementations should be added in the future.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4352) only the test runner should be able to System.exit

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447246#comment-13447246
 ] 

Uwe Schindler commented on LUCENE-4352:
---

The thing we can do is to explicitely all actions for RuntimePermission and 
leave out exit(): 
http://docs.oracle.com/javase/6/docs/api/java/lang/RuntimePermission.html

Unfortunately this confuses me: 'This allows an attacker to mount a 
denial-of-service attack by automatically forcing the virtual machine to halt. 
Note: The "exitVM.*" permission is automatically granted to all code loaded 
from the application class path, thus enabling applications to terminate 
themselves. Also, the "exitVM" permission is equivalent to "exitVM.*".'
So this appears to me that exiting is always possible from classes that are 
inside the application ClassLoader (passed via -classpath to JVM). We could 
only forbid it, if Solr would load ZooKeeper in a separate ClassLoader. I have 
to try this out.

The second problem is that policy files only allow "grants" not "deny", so if 
you grant RuntimePermission.* (as it is currently) you cannot explicitely 
forbid it for one code part. You have to do it vice versa: Disallow it for all 
(by having the long list of RuntimePermissions instead of "*") and only allow 
it for one classloader.

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4345) Create a Classification module

[Tommaso Teofili (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447243#comment-13447243
 ] 

Tommaso Teofili commented on LUCENE-4345:
-

bq. So we could consider performance-driven heuristics/approximations like 
MoreLikeThis does based on things like local term frequency within the 
document/term length, whatever to save on docFreq() calls, if it makes sense (i 
have to look at the formula in more detail here).

The generic formula is _C = argmax( P(doc|class) * P(class) )_ , I agree it 
makes sense to incrementally see if we can find good heuristics / 
approximations which low the computational cost of this calculation.

bq. the current code, given a word that appears many times in the document, 
will do many computations when instead we could really just work across the 
unique terms within the document.

another good point where we can improve, thanks :)

I managed to remove all the Maps from the code, I'll attach the patch shortly. 
I'll then work on removing the tokenizeDoc() loop.

> Create a Classification module
> --
>
> Key: LUCENE-4345
> URL: https://issues.apache.org/jira/browse/LUCENE-4345
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
>Priority: Minor
> Attachments: LUCENE-4345.patch, SOLR-3700_2.patch, SOLR-3700.patch
>
>
> Lucene/Solr can host huge sets of documents containing lots of information in 
> fields so that these can be used as training examples (w/ features) in order 
> to very quickly create classifiers algorithms to use on new documents and / 
> or to provide an additional service.
> So the idea is to create a contrib module (called 'classification') to host a 
> ClassificationComponent that will use already seen data (the indexed 
> documents / fields) to classify new documents / text fragments.
> The first version will contain a (simplistic) Lucene based Naive Bayes 
> classifier but more implementations should be added in the future.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4345) Create a Classification module

[Tommaso Teofili (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447240#comment-13447240
 ] 

Tommaso Teofili commented on LUCENE-4345:
-

bq. This will benefit from parts-of-speech filtering.

sure, and that can be done by passing the correct Analyzer to the 
Classifier#train() method.

> Create a Classification module
> --
>
> Key: LUCENE-4345
> URL: https://issues.apache.org/jira/browse/LUCENE-4345
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Tommaso Teofili
>Assignee: Tommaso Teofili
>Priority: Minor
> Attachments: LUCENE-4345.patch, SOLR-3700_2.patch, SOLR-3700.patch
>
>
> Lucene/Solr can host huge sets of documents containing lots of information in 
> fields so that these can be used as training examples (w/ features) in order 
> to very quickly create classifiers algorithms to use on new documents and / 
> or to provide an additional service.
> So the idea is to create a contrib module (called 'classification') to host a 
> ClassificationComponent that will use already seen data (the indexed 
> documents / fields) to classify new documents / text fragments.
> The first version will contain a (simplistic) Lucene based Naive Bayes 
> classifier but more implementations should be added in the future.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4352) only the test runner should be able to System.exit

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-4352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447236#comment-13447236
 ] 

Uwe Schindler commented on LUCENE-4352:
---

The problem is, thats not easy possible with a policy file, as we can only 
forbid it globally or allow it. If we would have signed code (means the JAR 
file of the test-runner is signed by Verisign or like that, you can allow it 
selectively for specific signers), or by directory (must be absolute URL - 
e.g., with file:/fullpath/...). This would need tpo be preprocessed by the 
build.xml to make absolute and convert to URL/URI (new 
File(...).asURI().toAsciiString()).

The question: Does the test runner call System.exit() at all or does it just 
let the main() method finish? If the latter is true, then its easy, just 
disallow System.exit().

> only the test runner should be able to System.exit
> --
>
> Key: LUCENE-4352
> URL: https://issues.apache.org/jira/browse/LUCENE-4352
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Robert Muir
>
> All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-4352) only the test runner should be able to System.exit

[Robert Muir (JIRA)]

Robert Muir created LUCENE-4352:
---

 Summary: only the test runner should be able to System.exit
 Key: LUCENE-4352
 URL: https://issues.apache.org/jira/browse/LUCENE-4352
 Project: Lucene - Core
  Issue Type: Bug
Reporter: Robert Muir


All others should get SecurityException

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: svn commit: r1379362 - in /lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud: BasicDistributedZk2Test.java BasicDistributedZkTest.java

[Robert Muir]

we should open a bug at zookeeper.

On Mon, Sep 3, 2012 at 7:03 AM, Dawid Weiss
 wrote:
> Oh man... now I know why the hell the JVM terminates with error 11...
>
> SyncThread's interrupt causes this:
>
> } catch (Throwable t) {
> LOG.fatal("Severe unrecoverable error, exiting", t);
> System.exit(11);
> }
>
> D.
>
> On Fri, Aug 31, 2012 at 5:21 PM, Dawid Weiss
>  wrote:
>> Oh yeah -- you mentioned it to me once, but I forgot about it. Thanks,
>> I'll see if this was the cause.
>>
>> Dawid
>>
>> On Fri, Aug 31, 2012 at 3:12 PM, Robert Muir  wrote:
>>> FYI: I'm not sure if this is related, but there is a thread from
>>> zookeeper "SyncThread" that you should never interrupt.
>>>
>>> Back in the day interrupting this thread in freebsd jenkins (but also
>>> locally on windows or linux) would cause JVM crashes.
>>>
>>> so the old huge hairy LuceneTestCase had this code:
>>>
>>> if (!t.getName().startsWith("SyncThread")) // avoid zookeeper jre crash
>>> t.interrupt()
>>>
>>> Not sure if its still applicable.
>>>
>>> On Fri, Aug 31, 2012 at 4:39 AM,   wrote:
 Author: dweiss
 Date: Fri Aug 31 08:39:29 2012
 New Revision: 1379362

 URL: http://svn.apache.org/viewvc?rev=1379362&view=rev
 Log:
 Don't interrupt threads in this test, just report thread leak errors. 
 Should help get past the broken (?) freebsd jvm.

 Modified:
 
 lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZk2Test.java
 
 lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZkTest.java

 Modified: 
 lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZk2Test.java
 URL: 
 http://svn.apache.org/viewvc/lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZk2Test.java?rev=1379362&r1=1379361&r2=1379362&view=diff
 ==
 --- 
 lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZk2Test.java
  (original)
 +++ 
 lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZk2Test.java
  Fri Aug 31 08:39:29 2012
 @@ -34,16 +34,16 @@ import org.apache.solr.common.params.Com
  import org.apache.solr.common.params.ModifiableSolrParams;
  import org.apache.solr.servlet.SolrDispatchFilter;

 +import com.carrotsearch.randomizedtesting.annotations.ThreadLeakAction;
 +import 
 com.carrotsearch.randomizedtesting.annotations.ThreadLeakAction.Action;
 +
  /**
   * This test simply does a bunch of basic things in solrcloud mode and 
 asserts things
   * work as expected.
   */
 +@ThreadLeakAction({Action.WARN})
  public class BasicDistributedZk2Test extends 
 AbstractFullDistribZkTestBase {
 -
 -  public BasicDistributedZk2Test() {
 -super();
 -  }
 -
 +
/*
 * (non-Javadoc)
 *

 Modified: 
 lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZkTest.java
 URL: 
 http://svn.apache.org/viewvc/lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZkTest.java?rev=1379362&r1=1379361&r2=1379362&view=diff
 ==
 --- 
 lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZkTest.java
  (original)
 +++ 
 lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZkTest.java
  Fri Aug 31 08:39:29 2012
 @@ -70,11 +70,15 @@ import org.apache.solr.common.util.Named
  import org.apache.solr.update.SolrCmdDistributor.Request;
  import org.apache.solr.util.DefaultSolrThreadFactory;

 +import com.carrotsearch.randomizedtesting.annotations.ThreadLeakAction;
 +import 
 com.carrotsearch.randomizedtesting.annotations.ThreadLeakAction.Action;
 +
  /**
   * This test simply does a bunch of basic things in solrcloud mode and 
 asserts things
   * work as expected.
   */
  @Slow
 +@ThreadLeakAction({Action.WARN})
  public class BasicDistributedZkTest extends AbstractDistribZkTestBase {

private static final String DEFAULT_COLLECTION = "collection1";


>>>
>>>
>>>
>>> --
>>> lucidworks.com
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>



-- 
lucidworks.com

Re: svn commit: r1379362 - in /lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud: BasicDistributedZk2Test.java BasicDistributedZkTest.java

[Dawid Weiss]

Oh man... now I know why the hell the JVM terminates with error 11...

SyncThread's interrupt causes this:

} catch (Throwable t) {
LOG.fatal("Severe unrecoverable error, exiting", t);
System.exit(11);
}

D.

On Fri, Aug 31, 2012 at 5:21 PM, Dawid Weiss
 wrote:
> Oh yeah -- you mentioned it to me once, but I forgot about it. Thanks,
> I'll see if this was the cause.
>
> Dawid
>
> On Fri, Aug 31, 2012 at 3:12 PM, Robert Muir  wrote:
>> FYI: I'm not sure if this is related, but there is a thread from
>> zookeeper "SyncThread" that you should never interrupt.
>>
>> Back in the day interrupting this thread in freebsd jenkins (but also
>> locally on windows or linux) would cause JVM crashes.
>>
>> so the old huge hairy LuceneTestCase had this code:
>>
>> if (!t.getName().startsWith("SyncThread")) // avoid zookeeper jre crash
>> t.interrupt()
>>
>> Not sure if its still applicable.
>>
>> On Fri, Aug 31, 2012 at 4:39 AM,   wrote:
>>> Author: dweiss
>>> Date: Fri Aug 31 08:39:29 2012
>>> New Revision: 1379362
>>>
>>> URL: http://svn.apache.org/viewvc?rev=1379362&view=rev
>>> Log:
>>> Don't interrupt threads in this test, just report thread leak errors. 
>>> Should help get past the broken (?) freebsd jvm.
>>>
>>> Modified:
>>> 
>>> lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZk2Test.java
>>> 
>>> lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZkTest.java
>>>
>>> Modified: 
>>> lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZk2Test.java
>>> URL: 
>>> http://svn.apache.org/viewvc/lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZk2Test.java?rev=1379362&r1=1379361&r2=1379362&view=diff
>>> ==
>>> --- 
>>> lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZk2Test.java
>>>  (original)
>>> +++ 
>>> lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZk2Test.java
>>>  Fri Aug 31 08:39:29 2012
>>> @@ -34,16 +34,16 @@ import org.apache.solr.common.params.Com
>>>  import org.apache.solr.common.params.ModifiableSolrParams;
>>>  import org.apache.solr.servlet.SolrDispatchFilter;
>>>
>>> +import com.carrotsearch.randomizedtesting.annotations.ThreadLeakAction;
>>> +import 
>>> com.carrotsearch.randomizedtesting.annotations.ThreadLeakAction.Action;
>>> +
>>>  /**
>>>   * This test simply does a bunch of basic things in solrcloud mode and 
>>> asserts things
>>>   * work as expected.
>>>   */
>>> +@ThreadLeakAction({Action.WARN})
>>>  public class BasicDistributedZk2Test extends AbstractFullDistribZkTestBase 
>>> {
>>> -
>>> -  public BasicDistributedZk2Test() {
>>> -super();
>>> -  }
>>> -
>>> +
>>>/*
>>> * (non-Javadoc)
>>> *
>>>
>>> Modified: 
>>> lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZkTest.java
>>> URL: 
>>> http://svn.apache.org/viewvc/lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZkTest.java?rev=1379362&r1=1379361&r2=1379362&view=diff
>>> ==
>>> --- 
>>> lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZkTest.java
>>>  (original)
>>> +++ 
>>> lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/cloud/BasicDistributedZkTest.java
>>>  Fri Aug 31 08:39:29 2012
>>> @@ -70,11 +70,15 @@ import org.apache.solr.common.util.Named
>>>  import org.apache.solr.update.SolrCmdDistributor.Request;
>>>  import org.apache.solr.util.DefaultSolrThreadFactory;
>>>
>>> +import com.carrotsearch.randomizedtesting.annotations.ThreadLeakAction;
>>> +import 
>>> com.carrotsearch.randomizedtesting.annotations.ThreadLeakAction.Action;
>>> +
>>>  /**
>>>   * This test simply does a bunch of basic things in solrcloud mode and 
>>> asserts things
>>>   * work as expected.
>>>   */
>>>  @Slow
>>> +@ThreadLeakAction({Action.WARN})
>>>  public class BasicDistributedZkTest extends AbstractDistribZkTestBase {
>>>
>>>private static final String DEFAULT_COLLECTION = "collection1";
>>>
>>>
>>
>>
>>
>> --
>> lucidworks.com
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3776) Parse NamedList for JaspellLookup factory, so fuzzy and edit distance can be enabled

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447189#comment-13447189
 ] 

Uwe Schindler commented on SOLR-3776:
-

I did some testing on the weekend and found out that Jaspell's fuzzy algorithm 
is not really useful for suggester, because it can do only prefix matching or a 
fuzzy match, but not combined. This means, if you enter a prefix term, the 
jaspell suggest algorithm only returns results in the ternary search tree that 
are exactly the distance away from the entered text.  It would be better to 
match the "prefix" in a fuzzy way, but the current approach does not do this 
and I see no solution how to do this.

>From my code analysis, the remaining code to do prefix matching is identical 
>to TSTLookup, so I wonder why we have both at all? Let's remove the Jaspell 
>one completely, its just the same concept with more ugly code than the 
>straight-forward TSTLookup!

I will close this issue as "won't fix".

> Parse NamedList for JaspellLookup factory, so fuzzy and edit distance can be 
> enabled
> 
>
> Key: SOLR-3776
> URL: https://issues.apache.org/jira/browse/SOLR-3776
> Project: Solr
>  Issue Type: Bug
>  Components: spellchecker
>Affects Versions: 4.0-BETA
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
> Fix For: 4.0, 5.0
>
> Attachments: SOLR-3776.patch
>
>
> Curretly JaspellLookup in Lucene already supports enabling fuzzy matches in 
> its ternary tree, but the factory in Solr ignores the NamedList completely 
> and just uses "new JaspellLookup()". This should be extended, so fuzzy 
> suggestions would be possible (if dictionary is not too big). This is better 
> than no alternative until a FST-based fuzzy work -> Mike, hey! :-)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Closed] (SOLR-3776) Parse NamedList for JaspellLookup factory, so fuzzy and edit distance can be enabled

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-3776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler closed SOLR-3776.
---

Resolution: Won't Fix

> Parse NamedList for JaspellLookup factory, so fuzzy and edit distance can be 
> enabled
> 
>
> Key: SOLR-3776
> URL: https://issues.apache.org/jira/browse/SOLR-3776
> Project: Solr
>  Issue Type: Bug
>  Components: spellchecker
>Affects Versions: 4.0-BETA
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
> Fix For: 4.0, 5.0
>
> Attachments: SOLR-3776.patch
>
>
> Curretly JaspellLookup in Lucene already supports enabling fuzzy matches in 
> its ternary tree, but the factory in Solr ignores the NamedList completely 
> and just uses "new JaspellLookup()". This should be extended, so fuzzy 
> suggestions would be possible (if dictionary is not too big). This is better 
> than no alternative until a FST-based fuzzy work -> Mike, hey! :-)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Build fail for tests

[Sivan Yogev]

Thanks David and Uwe, a clean ant installation did the work.

Sivan

On Mon, Sep 3, 2012 at 11:41 AM, Dawid Weiss
wrote:

> Yep, this may also be the reason, thanks Uwe.
>
> Dawid
>
> On Mon, Sep 3, 2012 at 10:33 AM, Uwe Schindler  wrote:
> > Hi,
> >
> >
> >
> > The problem is that you have an older jUnit version in your
> $ANT_HOME/lib or
> > ~/.ant/lib classpath. Please use a freshly and clean installed ANT
> version
> > (not one from your Linux distribution, those are often cluttered with
> > outdated stuff). Lucene needs no such libraries like JUnit in classpath,
> as
> > it downloads then via IVY automatically and uses it locally. But if there
> > are old versions already in the classpath, it cannot do anything.
> >
> >
> >
> > Uwe
> >
> >
> >
> > -
> >
> > Uwe Schindler
> >
> > H.-H.-Meier-Allee 63, D-28213 Bremen
> >
> > http://www.thetaphi.de
> >
> > eMail: u...@thetaphi.de
> >
> >
> >
> > From: Sivan Yogev [mailto:sivan.yo...@gmail.com]
> > Sent: Monday, September 03, 2012 10:26 AM
> > To: dev@lucene.apache.org
> > Subject: Build fail for tests
> >
> >
> >
> > Hi,
> >
> > I am trying to run the tests for lucene with some modifications, and
> getting
> > this output:
> >
> > compile-core:
> > [mkdir] Created dir:
> > D:\workspace\lucene-trunk\lucene\build\test-framework\classes\java
> > [javac] Compiling 87 source files to
> > D:\workspace\lucene-trunk\lucene\build\test-framework\classes\java
> > [javac] Note: Some input files use or override a deprecated API.
> > [javac] Note: Recompile with -Xlint:deprecation for details.
> >  [copy] Copying 3 files to
> > D:\workspace\lucene-trunk\lucene\build\test-framework\classes\java
> >
> > compile-test:
> > [mkdir] Created dir:
> > D:\workspace\lucene-trunk\lucene\build\core\classes\test
> > [javac] Compiling 325 source files to
> > D:\workspace\lucene-trunk\lucene\build\core\classes\test
> > [javac] Note: Some input files use or override a deprecated API.
> > [javac] Note: Recompile with -Xlint:deprecation for details.
> >  [copy] Copying 29 files to
> > D:\workspace\lucene-trunk\lucene\build\core\classes\test
> >
> > install-junit4-taskdef:
> >
> > validate:
> >
> > test:
> > [mkdir] Created dir: D:\workspace\lucene-trunk\lucene\build\core\test
> >
> > BUILD FAILED
> > D:\workspace\lucene-trunk\lucene\build.xml:49: The following error
> occurred
> > while executing this line:
> > D:\workspace\lucene-trunk\lucene\common-build.xml:1056: The following
> error
> > occurred while executing this line:
> > D:\workspace\lucene-trunk\lucene\common-build.xml:741: At least JUnit
> > version 4.10 is required on junit4's taskdef classpath.
> >
> > Does anyone know what can cause this?
> >
> > Thanks,
> > Sivan
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

Re: Build fail for tests

[Dawid Weiss]

Yep, this may also be the reason, thanks Uwe.

Dawid

On Mon, Sep 3, 2012 at 10:33 AM, Uwe Schindler  wrote:
> Hi,
>
>
>
> The problem is that you have an older jUnit version in your $ANT_HOME/lib or
> ~/.ant/lib classpath. Please use a freshly and clean installed ANT version
> (not one from your Linux distribution, those are often cluttered with
> outdated stuff). Lucene needs no such libraries like JUnit in classpath, as
> it downloads then via IVY automatically and uses it locally. But if there
> are old versions already in the classpath, it cannot do anything.
>
>
>
> Uwe
>
>
>
> -
>
> Uwe Schindler
>
> H.-H.-Meier-Allee 63, D-28213 Bremen
>
> http://www.thetaphi.de
>
> eMail: u...@thetaphi.de
>
>
>
> From: Sivan Yogev [mailto:sivan.yo...@gmail.com]
> Sent: Monday, September 03, 2012 10:26 AM
> To: dev@lucene.apache.org
> Subject: Build fail for tests
>
>
>
> Hi,
>
> I am trying to run the tests for lucene with some modifications, and getting
> this output:
>
> compile-core:
> [mkdir] Created dir:
> D:\workspace\lucene-trunk\lucene\build\test-framework\classes\java
> [javac] Compiling 87 source files to
> D:\workspace\lucene-trunk\lucene\build\test-framework\classes\java
> [javac] Note: Some input files use or override a deprecated API.
> [javac] Note: Recompile with -Xlint:deprecation for details.
>  [copy] Copying 3 files to
> D:\workspace\lucene-trunk\lucene\build\test-framework\classes\java
>
> compile-test:
> [mkdir] Created dir:
> D:\workspace\lucene-trunk\lucene\build\core\classes\test
> [javac] Compiling 325 source files to
> D:\workspace\lucene-trunk\lucene\build\core\classes\test
> [javac] Note: Some input files use or override a deprecated API.
> [javac] Note: Recompile with -Xlint:deprecation for details.
>  [copy] Copying 29 files to
> D:\workspace\lucene-trunk\lucene\build\core\classes\test
>
> install-junit4-taskdef:
>
> validate:
>
> test:
> [mkdir] Created dir: D:\workspace\lucene-trunk\lucene\build\core\test
>
> BUILD FAILED
> D:\workspace\lucene-trunk\lucene\build.xml:49: The following error occurred
> while executing this line:
> D:\workspace\lucene-trunk\lucene\common-build.xml:1056: The following error
> occurred while executing this line:
> D:\workspace\lucene-trunk\lucene\common-build.xml:741: At least JUnit
> version 4.10 is required on junit4's taskdef classpath.
>
> Does anyone know what can cause this?
>
> Thanks,
> Sivan

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

RE: Build fail for tests

[Uwe Schindler]

Hi,

 

The problem is that you have an older jUnit version in your $ANT_HOME/lib or 
~/.ant/lib classpath. Please use a freshly and clean installed ANT version (not 
one from your Linux distribution, those are often cluttered with outdated 
stuff). Lucene needs no such libraries like JUnit in classpath, as it downloads 
then via IVY automatically and uses it locally. But if there are old versions 
already in the classpath, it cannot do anything.

 

Uwe

 

-

Uwe Schindler

H.-H.-Meier-Allee 63, D-28213 Bremen

http://www.thetaphi.de  

eMail: u...@thetaphi.de

 

From: Sivan Yogev [mailto:sivan.yo...@gmail.com] 
Sent: Monday, September 03, 2012 10:26 AM
To: dev@lucene.apache.org
Subject: Build fail for tests

 

Hi,

I am trying to run the tests for lucene with some modifications, and getting 
this output:

compile-core:
[mkdir] Created dir: 
D:\workspace\lucene-trunk\lucene\build\test-framework\classes\java
[javac] Compiling 87 source files to 
D:\workspace\lucene-trunk\lucene\build\test-framework\classes\java
[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
 [copy] Copying 3 files to 
D:\workspace\lucene-trunk\lucene\build\test-framework\classes\java

compile-test:
[mkdir] Created dir: 
D:\workspace\lucene-trunk\lucene\build\core\classes\test
[javac] Compiling 325 source files to 
D:\workspace\lucene-trunk\lucene\build\core\classes\test
[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
 [copy] Copying 29 files to 
D:\workspace\lucene-trunk\lucene\build\core\classes\test

install-junit4-taskdef:

validate:

test:
[mkdir] Created dir: D:\workspace\lucene-trunk\lucene\build\core\test

BUILD FAILED
D:\workspace\lucene-trunk\lucene\build.xml:49: The following error occurred 
while executing this line:
D:\workspace\lucene-trunk\lucene\common-build.xml:1056: The following error 
occurred while executing this line:
D:\workspace\lucene-trunk\lucene\common-build.xml:741: At least JUnit version 
4.10 is required on junit4's taskdef classpath.

Does anyone know what can cause this?

Thanks,
Sivan

Re: Build fail for tests

[Dawid Weiss]

> D:\workspace\lucene-trunk\lucene\common-build.xml:741: At least JUnit
> version 4.10 is required on junit4's taskdef classpath.
>
> Does anyone know what can cause this?

Yes. You need JUnit 4.10 or later on classpath where junit4 is
defined. Exactly what it says above. If you share your modifications
it'll be easier to help you. Look at junit4 taskdef though --

  





  

  

the ivy file declares dependencies:

  
  
  

Dawid

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org