[jira] [Updated] (SOLR-6892) Make update processors toplevel components
[
https://issues.apache.org/jira/browse/SOLR-6892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noble Paul updated SOLR-6892:
-
Description:
The current update processor chain is rather cumbersome and we should be able
to use the updateprocessors without a chain.
The scope of this ticket is
* A new tag updateProcessor becomes a toplevel tag and it will be equivalent
to the {{processor}} tag inside {{updateRequestProcessorChain}} . The only
difference is that it should require a {{name}} attribute. The
{{updateProcessorChain}} tag will continue to exist and it should be possible
to define processor inside as well . It should also be possible to reference
a named URP in a chain.
* Any update request will be able to pass a param {{processor=a,b,c}} , where
a,b,c are names of update processors. A just in time chain will be created with
those URPs
* Some in built update processors (wherever possible) will be predefined with
standard names and can be directly used in requests
* What happens when I say processor=a,b,c in a request? It will execute the
default chain after the just-in-time chain {{a-b-c}} .
* How to execute a different chain other than the default chain? the same old
mechanism of update.chain=x means that the chain {{x}} will be applied after
{{a,b,c}}
* How to avoid the default processor chain from being executed ? There will be
an implicit URP called {{STOP}} . send your request as processor=a,b,c,STOP.
was:
The current update processor chain is rather cumbersome and we should be able
to use the updateprocessors without a chain.
The scope of this ticket is
* updateProcessor tag becomes a toplevel tag and it will be equivalent to the
processor tag inside updateRequestProcessorChain . The only difference is
that it should require a {{name}} attribute
* Any update request will be able to pass a param {{processor=a,b,c}} , where
a,b,c are names of update processors. A just in time chain will be created with
those update processors
* Some in built update processors (wherever possible) will be predefined with
standard names and can be directly used in requests
Make update processors toplevel components
---
Key: SOLR-6892
URL: https://issues.apache.org/jira/browse/SOLR-6892
Project: Solr
Issue Type: Bug
Reporter: Noble Paul
Assignee: Noble Paul
The current update processor chain is rather cumbersome and we should be able
to use the updateprocessors without a chain.
The scope of this ticket is
* A new tag updateProcessor becomes a toplevel tag and it will be
equivalent to the {{processor}} tag inside
{{updateRequestProcessorChain}} . The only difference is that it should
require a {{name}} attribute. The {{updateProcessorChain}} tag will
continue to exist and it should be possible to define processor inside as
well . It should also be possible to reference a named URP in a chain.
* Any update request will be able to pass a param {{processor=a,b,c}} ,
where a,b,c are names of update processors. A just in time chain will be
created with those URPs
* Some in built update processors (wherever possible) will be predefined with
standard names and can be directly used in requests
* What happens when I say processor=a,b,c in a request? It will execute the
default chain after the just-in-time chain {{a-b-c}} .
* How to execute a different chain other than the default chain? the same old
mechanism of update.chain=x means that the chain {{x}} will be applied after
{{a,b,c}}
* How to avoid the default processor chain from being executed ? There will
be an implicit URP called {{STOP}} . send your request as
processor=a,b,c,STOP.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6892) Make it possible to define update request processors as toplevel components
[
https://issues.apache.org/jira/browse/SOLR-6892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noble Paul updated SOLR-6892:
-
Summary: Make it possible to define update request processors as toplevel
components (was: Make update processors toplevel components )
Make it possible to define update request processors as toplevel components
Key: SOLR-6892
URL: https://issues.apache.org/jira/browse/SOLR-6892
Project: Solr
Issue Type: Bug
Reporter: Noble Paul
Assignee: Noble Paul
The current update processor chain is rather cumbersome and we should be able
to use the updateprocessors without a chain.
The scope of this ticket is
* A new tag updateProcessor becomes a toplevel tag and it will be
equivalent to the {{processor}} tag inside
{{updateRequestProcessorChain}} . The only difference is that it should
require a {{name}} attribute. The {{updateProcessorChain}} tag will
continue to exist and it should be possible to define processor inside as
well . It should also be possible to reference a named URP in a chain.
* Any update request will be able to pass a param {{processor=a,b,c}} ,
where a,b,c are names of update processors. A just in time chain will be
created with those URPs
* Some in built update processors (wherever possible) will be predefined with
standard names and can be directly used in requests
* What happens when I say processor=a,b,c in a request? It will execute the
default chain after the just-in-time chain {{a-b-c}} .
* How to execute a different chain other than the default chain? the same old
mechanism of update.chain=x means that the chain {{x}} will be applied after
{{a,b,c}}
* How to avoid the default processor chain from being executed ? There will
be an implicit URP called {{STOP}} . send your request as
processor=a,b,c,STOP.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6892) Make update processors toplevel components
[
https://issues.apache.org/jira/browse/SOLR-6892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259600#comment-14259600
]
Noble Paul commented on SOLR-6892:
--
I've updated the description with more meat this time. Please comment
Make update processors toplevel components
---
Key: SOLR-6892
URL: https://issues.apache.org/jira/browse/SOLR-6892
Project: Solr
Issue Type: Bug
Reporter: Noble Paul
Assignee: Noble Paul
The current update processor chain is rather cumbersome and we should be able
to use the updateprocessors without a chain.
The scope of this ticket is
* A new tag updateProcessor becomes a toplevel tag and it will be
equivalent to the {{processor}} tag inside
{{updateRequestProcessorChain}} . The only difference is that it should
require a {{name}} attribute. The {{updateProcessorChain}} tag will
continue to exist and it should be possible to define processor inside as
well . It should also be possible to reference a named URP in a chain.
* Any update request will be able to pass a param {{processor=a,b,c}} ,
where a,b,c are names of update processors. A just in time chain will be
created with those URPs
* Some in built update processors (wherever possible) will be predefined with
standard names and can be directly used in requests
* What happens when I say processor=a,b,c in a request? It will execute the
default chain after the just-in-time chain {{a-b-c}} .
* How to execute a different chain other than the default chain? the same old
mechanism of update.chain=x means that the chain {{x}} will be applied after
{{a,b,c}}
* How to avoid the default processor chain from being executed ? There will
be an implicit URP called {{STOP}} . send your request as
processor=a,b,c,STOP.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6879) Have an option to disable autoAddReplicas temporarily for all collections
[
https://issues.apache.org/jira/browse/SOLR-6879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259603#comment-14259603
]
Varun Thacker commented on SOLR-6879:
-
Thanks Steve.
Couple of ref guide updates that we would need to make -
1.
https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api11
Instead of As of Solr 4.7, only the property urlScheme is supported.
we could have something like - The two supported properties are - 'urlScheme'
and 'autoAddReplicas'
2. Under AutoAddReplica Settings on
https://cwiki.apache.org/confluence/display/solr/Running+Solr+on+HDFS we could
add the following section-
Temporarily disable autoAddReplicas for the entire cluster - When doing
offline maintenance on the cluster and for various other use cases where an
admin would like to disable autoAddReplicas temporarily, one could use the
following APIs to disable and enable autoAddReplicas cluster wide.
API to disable autoAddReplicas:
{code}
admin/collections?action=CLUSTERPROPname=autoAddReplicasval=false
{code}
API to enable autoAddReplicas:
{code}
admin/collections?action=CLUSTERPROPname=autoAddReplicas
{code}
Have an option to disable autoAddReplicas temporarily for all collections
-
Key: SOLR-6879
URL: https://issues.apache.org/jira/browse/SOLR-6879
Project: Solr
Issue Type: Improvement
Reporter: Varun Thacker
Assignee: Steve Rowe
Fix For: 5.0, Trunk
Attachments: SOLR-6879.patch, SOLR-6879.patch
When doing offline maintenance on my cluster I would like to disable
autoAddReplicas since I don't want replicas being created on nodes during
this time.
It would be useful to have an option to enable/disable autoAddReplicas via an
API.
This API would disable autoAddReplicas:
{code}
admin/collections?action=CLUSTERPROPname=autoAddReplicasval=false
{code}
Now when I want to enable it again the API call would look like this:
{code}
admin/collections?action=CLUSTERPROPname=autoAddReplicas
{code}
This uses the CLUSTERPROP semantics of unsetting a property when the {{val}}
param is not provided.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6135) re-number fields randomly in tests
[
https://issues.apache.org/jira/browse/LUCENE-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259611#comment-14259611
]
ASF subversion and git services commented on LUCENE-6135:
-
Commit 1648188 from [~rcmuir] in branch 'dev/trunk'
[ https://svn.apache.org/r1648188 ]
LUCENE-6135: renumber fields randomly in tests
re-number fields randomly in tests
--
Key: LUCENE-6135
URL: https://issues.apache.org/jira/browse/LUCENE-6135
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-6135.patch
Currently some code (such as stored fields bulk merge) depends upon
consistent field number ordering.
In the case field numbers do not align, then optimizations are disabled,
because the would cause crazy corruption where values are mixed up across
different fields.
But this is hardly tested today. If i introduce an intentional bug into this
logic, then only one lone test fails: TestAddIndexes.testFieldNamesChanged,
and only about 10% of the time at best. In general tests pass.
{code}
---
lucene/core/src/java/org/apache/lucene/codecs/compressing/MatchingReaders.java
(revision 1647793)
+++
lucene/core/src/java/org/apache/lucene/codecs/compressing/MatchingReaders.java
(working copy)
@@ -52,6 +52,10 @@
for (int i = 0; i numReaders; i++) {
for (FieldInfo fi : mergeState.fieldInfos[i]) {
FieldInfo other = mergeState.mergeFieldInfos.fieldInfo(fi.number);
+// nocommit:
+if (true) {
+ break;
+}
if (other == null || !other.name.equals(fi.name)) {
continue nextReader;
}
{code}
Don't get me wrong, its a great simple test, but it should not be the only
one doing this. And it would be great if it failed more often, i havent
looked as to why it only fails rarely if there is a bug in this stuff.
But in general, we should simulate this more. My current idea is to shuffle
up field numbers in MockRandomMergePolicy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6135) re-number fields randomly in tests
[
https://issues.apache.org/jira/browse/LUCENE-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259621#comment-14259621
]
ASF subversion and git services commented on LUCENE-6135:
-
Commit 1648189 from [~rcmuir] in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1648189 ]
LUCENE-6135: renumber fields randomly in tests
re-number fields randomly in tests
--
Key: LUCENE-6135
URL: https://issues.apache.org/jira/browse/LUCENE-6135
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-6135.patch
Currently some code (such as stored fields bulk merge) depends upon
consistent field number ordering.
In the case field numbers do not align, then optimizations are disabled,
because the would cause crazy corruption where values are mixed up across
different fields.
But this is hardly tested today. If i introduce an intentional bug into this
logic, then only one lone test fails: TestAddIndexes.testFieldNamesChanged,
and only about 10% of the time at best. In general tests pass.
{code}
---
lucene/core/src/java/org/apache/lucene/codecs/compressing/MatchingReaders.java
(revision 1647793)
+++
lucene/core/src/java/org/apache/lucene/codecs/compressing/MatchingReaders.java
(working copy)
@@ -52,6 +52,10 @@
for (int i = 0; i numReaders; i++) {
for (FieldInfo fi : mergeState.fieldInfos[i]) {
FieldInfo other = mergeState.mergeFieldInfos.fieldInfo(fi.number);
+// nocommit:
+if (true) {
+ break;
+}
if (other == null || !other.name.equals(fi.name)) {
continue nextReader;
}
{code}
Don't get me wrong, its a great simple test, but it should not be the only
one doing this. And it would be great if it failed more often, i havent
looked as to why it only fails rarely if there is a bug in this stuff.
But in general, we should simulate this more. My current idea is to shuffle
up field numbers in MockRandomMergePolicy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-6135) re-number fields randomly in tests
[
https://issues.apache.org/jira/browse/LUCENE-6135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Muir resolved LUCENE-6135.
-
Resolution: Fixed
Fix Version/s: Trunk
5.0
re-number fields randomly in tests
--
Key: LUCENE-6135
URL: https://issues.apache.org/jira/browse/LUCENE-6135
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Fix For: 5.0, Trunk
Attachments: LUCENE-6135.patch
Currently some code (such as stored fields bulk merge) depends upon
consistent field number ordering.
In the case field numbers do not align, then optimizations are disabled,
because the would cause crazy corruption where values are mixed up across
different fields.
But this is hardly tested today. If i introduce an intentional bug into this
logic, then only one lone test fails: TestAddIndexes.testFieldNamesChanged,
and only about 10% of the time at best. In general tests pass.
{code}
---
lucene/core/src/java/org/apache/lucene/codecs/compressing/MatchingReaders.java
(revision 1647793)
+++
lucene/core/src/java/org/apache/lucene/codecs/compressing/MatchingReaders.java
(working copy)
@@ -52,6 +52,10 @@
for (int i = 0; i numReaders; i++) {
for (FieldInfo fi : mergeState.fieldInfos[i]) {
FieldInfo other = mergeState.mergeFieldInfos.fieldInfo(fi.number);
+// nocommit:
+if (true) {
+ break;
+}
if (other == null || !other.name.equals(fi.name)) {
continue nextReader;
}
{code}
Don't get me wrong, its a great simple test, but it should not be the only
one doing this. And it would be great if it failed more often, i havent
looked as to why it only fails rarely if there is a bug in this stuff.
But in general, we should simulate this more. My current idea is to shuffle
up field numbers in MockRandomMergePolicy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6494) Query filters applied in a wrong order
[
https://issues.apache.org/jira/browse/SOLR-6494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259627#comment-14259627
]
Alexander S. commented on SOLR-6494:
As I was told already, Solr does not apply filters incrementally, instead each
filter runs through the entire data set, then Solr caches the results. In the
case with filters that contain ranges cache is not effective, especially when
we need NRT search and commits being triggered multiple times per minute. Then
big caches make no sense and big autowarming numbers causing Solr to fail. My
point is that cache is not always efficient and for such cases Solr need to use
another strategy and apply filters incrementally (read as post filters).
So this:
{quote}
By design, fq clauses like this are calculated for the entire document set and
the results cached, there is no ordering for that part. Otherwise, how could
they be re-used for a different query?
{quote}
does not work in all cases.
Something like this:
{code}
fq={!cache=false cost=101}field:value # to run as a post filter
{code}
would definitely solve the problem, but this is not supported.
The frange parser has support for this, but it is not always suitable and fails
with different errors, like can not use FieldCache on multivalued field:
type, etc.
Does that look like a missing feature? I mean for me it definitely does, but
could this be considered as a wish and implemented some day? How can Solr
community help with missing features?
Query filters applied in a wrong order
--
Key: SOLR-6494
URL: https://issues.apache.org/jira/browse/SOLR-6494
Project: Solr
Issue Type: Bug
Affects Versions: 4.8.1
Reporter: Alexander S.
This query:
{code}
{
fq: [type:Award::Nomination],
sort: score desc,
start: 0,
rows: 20,
q: *:*
}
{code}
takes just a few milliseconds, but this one:
{code}
{
fq: [
type:Award::Nomination,
created_at_d:[* TO 2014-09-08T23:59:59Z]
],
sort: score desc,
start: 0,
rows: 20,
q: *:*
}
{code}
takes almost 15 seconds.
I have just ≈12k of documents with type Award::Nomination, but around half
a billion with created_at_d field set. And it seems Solr applies the
created_at_d filter first going through all documents where this field is
set, which is not very smart.
I think if it can't do anything better than applying filters in the alphabet
order it should apply them in the order they were received.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-6494) Query filters applied in a wrong order
[
https://issues.apache.org/jira/browse/SOLR-6494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259627#comment-14259627
]
Alexander S. edited comment on SOLR-6494 at 12/28/14 12:50 PM:
---
As I was told already, Solr does not apply filters incrementally, instead each
filter runs through the entire data set, then Solr caches the results. In the
case with filters that contain ranges cache is not effective, especially when
we need NRT search and commits being triggered multiple times per minute. Then
big caches make no sense and big autowarming numbers causing Solr to fail. My
point is that cache is not always efficient and for such cases Solr need to use
another strategy and apply filters incrementally (read as post filters).
So this:
{quote}
By design, fq clauses like this are calculated for the entire document set and
the results cached, there is no ordering for that part. Otherwise, how could
they be re-used for a different query?
{quote}
does not work in all cases.
Something like this:
{code}
# cost 100 to run as a post filter, but something like post=true would be
better I think
fq={!cache=false cost=101}field:value
{code}
would definitely solve the problem, but this is not supported.
The frange parser has support for this, but it is not always suitable and fails
with different errors, like can not use FieldCache on multivalued field:
type, etc.
Does that look like a missing feature? I mean for me it definitely does, but
could this be considered as a wish and implemented some day? How can Solr
community help with missing features?
was (Author: aheaven):
As I was told already, Solr does not apply filters incrementally, instead each
filter runs through the entire data set, then Solr caches the results. In the
case with filters that contain ranges cache is not effective, especially when
we need NRT search and commits being triggered multiple times per minute. Then
big caches make no sense and big autowarming numbers causing Solr to fail. My
point is that cache is not always efficient and for such cases Solr need to use
another strategy and apply filters incrementally (read as post filters).
So this:
{quote}
By design, fq clauses like this are calculated for the entire document set and
the results cached, there is no ordering for that part. Otherwise, how could
they be re-used for a different query?
{quote}
does not work in all cases.
Something like this:
{code}
fq={!cache=false cost=101}field:value # to run as a post filter
{code}
would definitely solve the problem, but this is not supported.
The frange parser has support for this, but it is not always suitable and fails
with different errors, like can not use FieldCache on multivalued field:
type, etc.
Does that look like a missing feature? I mean for me it definitely does, but
could this be considered as a wish and implemented some day? How can Solr
community help with missing features?
Query filters applied in a wrong order
--
Key: SOLR-6494
URL: https://issues.apache.org/jira/browse/SOLR-6494
Project: Solr
Issue Type: Bug
Affects Versions: 4.8.1
Reporter: Alexander S.
This query:
{code}
{
fq: [type:Award::Nomination],
sort: score desc,
start: 0,
rows: 20,
q: *:*
}
{code}
takes just a few milliseconds, but this one:
{code}
{
fq: [
type:Award::Nomination,
created_at_d:[* TO 2014-09-08T23:59:59Z]
],
sort: score desc,
start: 0,
rows: 20,
q: *:*
}
{code}
takes almost 15 seconds.
I have just ≈12k of documents with type Award::Nomination, but around half
a billion with created_at_d field set. And it seems Solr applies the
created_at_d filter first going through all documents where this field is
set, which is not very smart.
I think if it can't do anything better than applying filters in the alphabet
order it should apply them in the order they were received.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5507) Admin UI - Refactoring using AngularJS
[ https://issues.apache.org/jira/browse/SOLR-5507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259635#comment-14259635 ] Upayavira commented on SOLR-5507: - [~erickerickson] please feel free to make a dependent issue for your SolrCloud enhancement, it is a good idea that shouldn't be lost. Admin UI - Refactoring using AngularJS -- Key: SOLR-5507 URL: https://issues.apache.org/jira/browse/SOLR-5507 Project: Solr Issue Type: Improvement Components: web gui Reporter: Stefan Matheis (steffkes) Assignee: Stefan Matheis (steffkes) Priority: Minor Attachments: SOLR-5507.patch On the LSR in Dublin, i've talked again to [~upayavira] and this time we talked about Refactoring the existing UI - using AngularJS: providing (more, internal) structure and what not ; He already started working on the Refactoring, so this is more a 'tracking' issue about the progress he/we do there. Will extend this issue with a bit more context additional information, w/ thoughts about the possible integration in the existing UI and more (: -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6138) ItalianLightStemmer doesn't apply on words shorter then 6 chars in length
[ https://issues.apache.org/jira/browse/LUCENE-6138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259655#comment-14259655 ] Massimo Pasquini commented on LUCENE-6138: -- The issue you pointed out is related to a different stemmer for Russian language. I see no connection to the Italian light stemmer. According to the rules of the Italian grammar, I think the bug I found can be fixed (it possibly cannot be done in the Russian stemmer according to what I've read on the other post). So I suppose the ItalianLightStemmer can evolve to a better implementation: it is possible to find some simple rules on words suffixes in order to make a decision about applying the stemming on short words (shorter then 6 characters). Notice my thoughts are not based on a deep and accurate study of the problem though. But I think it could be worth to investigate about it. I may suggest to submit this issue to the author of the code and see if he got a better solution (I saw he's in the field of computational linguistics). According to the notes in the source, the algorithm was written in 2005 as the result of some experiments. We don't know if they've put further efforts in investigating the problem and they possibly wrote a best algorithm they agree to publish according to Lucene's license. I don't expect the stemmer to be 100% successful, but the issue I pointed out affects an important range on words. ItalianLightStemmer doesn't apply on words shorter then 6 chars in length - Key: LUCENE-6138 URL: https://issues.apache.org/jira/browse/LUCENE-6138 Project: Lucene - Core Issue Type: Bug Components: modules/analysis Affects Versions: 4.10.2 Reporter: Massimo Pasquini Priority: Minor I expect a stemmer to transform nouns in their singular and plural forms into a shorter common form. The implementation of the ItalianLightStemmer doesn't apply any stemming to words shorter then 6 characters in length. This leads to some annoying results: singular form | plural form 4|5 chars in length (no stemming) alga - alga | alghe - alghe fuga - fuga | fughe - fughe lega - lega | leghe - leghe 5|6 chars in length (stemming only on plural form) vanga - vanga | vanghe - vang verga - verga | verghe - verg I suppose that such limitation on words length is to avoid other side effects on shorter words not in the set above, but I think something must be reviewed in the code for better results. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5507) Admin UI - Refactoring using AngularJS
[ https://issues.apache.org/jira/browse/SOLR-5507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259657#comment-14259657 ] Jack Krupansky commented on SOLR-5507: -- bq. All I ask, though, is that you forgive the occasional burst of ebullient enthusiasm! No need for it to be forgiven... all ebullient enthusiasm is always welcome and encouraged. Admin UI - Refactoring using AngularJS -- Key: SOLR-5507 URL: https://issues.apache.org/jira/browse/SOLR-5507 Project: Solr Issue Type: Improvement Components: web gui Reporter: Stefan Matheis (steffkes) Assignee: Stefan Matheis (steffkes) Priority: Minor Attachments: SOLR-5507.patch On the LSR in Dublin, i've talked again to [~upayavira] and this time we talked about Refactoring the existing UI - using AngularJS: providing (more, internal) structure and what not ; He already started working on the Refactoring, so this is more a 'tracking' issue about the progress he/we do there. Will extend this issue with a bit more context additional information, w/ thoughts about the possible integration in the existing UI and more (: -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6141) simplify stored fields bulk merge logic
[ https://issues.apache.org/jira/browse/LUCENE-6141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259660#comment-14259660 ] Robert Muir commented on LUCENE-6141: - We already have a test: testMixedCompressions. I will improve it with the commit though. simplify stored fields bulk merge logic --- Key: LUCENE-6141 URL: https://issues.apache.org/jira/browse/LUCENE-6141 Project: Lucene - Core Issue Type: Task Reporter: Robert Muir Attachments: LUCENE-6141.patch The current logic checks that the same chunksize and compression algorithm were used, but this is obselete as it no longer iterates chunks. We only need to check that the format version is the same. This also allows for bulk merging across compression algorithms. A good use case is BEST_SPEED - BEST_COMPRESSION for archiving. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6141) simplify stored fields bulk merge logic
[ https://issues.apache.org/jira/browse/LUCENE-6141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259664#comment-14259664 ] ASF subversion and git services commented on LUCENE-6141: - Commit 1648221 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1648221 ] LUCENE-6141: simplify stored fields bulk merge logic simplify stored fields bulk merge logic --- Key: LUCENE-6141 URL: https://issues.apache.org/jira/browse/LUCENE-6141 Project: Lucene - Core Issue Type: Task Reporter: Robert Muir Attachments: LUCENE-6141.patch The current logic checks that the same chunksize and compression algorithm were used, but this is obselete as it no longer iterates chunks. We only need to check that the format version is the same. This also allows for bulk merging across compression algorithms. A good use case is BEST_SPEED - BEST_COMPRESSION for archiving. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6892) Make it possible to define update request processors as toplevel components
[
https://issues.apache.org/jira/browse/SOLR-6892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259665#comment-14259665
]
Jack Krupansky commented on SOLR-6892:
--
Thanks for the description updates. Comments...
1. We need to be explicit about how and when the hard-wired processors are
invoked. In particular the run update processor. The log update processor
is somewhat special in that it is not mandatory, but a lot of people are not
explicitly aware of it, so if they leave it out, they will be wondering why
they don't get logging of updates.
2. I suggest three parameters: pre.processors to specify processors before
the default chain, post.processors to specify processors after the default
chain (before or after run update and log update??), and processors to
specify a processor list to completely replace the default chain.
3. Make log update be automatically added at the end unless a nolog
processor is specified.
4. Make run update be automatically added at the end unless a norun
processor is specified.
5. Discuss processor vs. processors - I prefer the latter since it is
explicit, but maybe allow both since the singular/plural can be confusing.
6. Consider supporting both a single parameter with a csv list as well as
multiple parameters each with a single value. I prefer having the choice.
Having a separate parameter for each processor can be more explicit sometimes.
7. Consider a single-processor parameter with the option to specify the
parameters for that processor. That would make it possible to invoke the
various field mutating update processors, which would be especially cool and
convenient.
Make it possible to define update request processors as toplevel components
Key: SOLR-6892
URL: https://issues.apache.org/jira/browse/SOLR-6892
Project: Solr
Issue Type: Bug
Reporter: Noble Paul
Assignee: Noble Paul
The current update processor chain is rather cumbersome and we should be able
to use the updateprocessors without a chain.
The scope of this ticket is
* A new tag updateProcessor becomes a toplevel tag and it will be
equivalent to the {{processor}} tag inside
{{updateRequestProcessorChain}} . The only difference is that it should
require a {{name}} attribute. The {{updateProcessorChain}} tag will
continue to exist and it should be possible to define processor inside as
well . It should also be possible to reference a named URP in a chain.
* Any update request will be able to pass a param {{processor=a,b,c}} ,
where a,b,c are names of update processors. A just in time chain will be
created with those URPs
* Some in built update processors (wherever possible) will be predefined with
standard names and can be directly used in requests
* What happens when I say processor=a,b,c in a request? It will execute the
default chain after the just-in-time chain {{a-b-c}} .
* How to execute a different chain other than the default chain? the same old
mechanism of update.chain=x means that the chain {{x}} will be applied after
{{a,b,c}}
* How to avoid the default processor chain from being executed ? There will
be an implicit URP called {{STOP}} . send your request as
processor=a,b,c,STOP.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6141) simplify stored fields bulk merge logic
[ https://issues.apache.org/jira/browse/LUCENE-6141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259668#comment-14259668 ] ASF subversion and git services commented on LUCENE-6141: - Commit 1648222 from [~rcmuir] in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1648222 ] LUCENE-6141: simplify stored fields bulk merge logic simplify stored fields bulk merge logic --- Key: LUCENE-6141 URL: https://issues.apache.org/jira/browse/LUCENE-6141 Project: Lucene - Core Issue Type: Task Reporter: Robert Muir Fix For: 5.0, Trunk Attachments: LUCENE-6141.patch The current logic checks that the same chunksize and compression algorithm were used, but this is obselete as it no longer iterates chunks. We only need to check that the format version is the same. This also allows for bulk merging across compression algorithms. A good use case is BEST_SPEED - BEST_COMPRESSION for archiving. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-6141) simplify stored fields bulk merge logic
[ https://issues.apache.org/jira/browse/LUCENE-6141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir resolved LUCENE-6141. - Resolution: Fixed Fix Version/s: Trunk 5.0 simplify stored fields bulk merge logic --- Key: LUCENE-6141 URL: https://issues.apache.org/jira/browse/LUCENE-6141 Project: Lucene - Core Issue Type: Task Reporter: Robert Muir Fix For: 5.0, Trunk Attachments: LUCENE-6141.patch The current logic checks that the same chunksize and compression algorithm were used, but this is obselete as it no longer iterates chunks. We only need to check that the format version is the same. This also allows for bulk merging across compression algorithms. A good use case is BEST_SPEED - BEST_COMPRESSION for archiving. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6892) Make it possible to define update request processors as toplevel components
[
https://issues.apache.org/jira/browse/SOLR-6892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259670#comment-14259670
]
Noble Paul commented on SOLR-6892:
--
[~jkrupan]
A lot of your suggestions are not realy in the scope of this ticket and should
be addressed separately
bq. Consider supporting both a single parameter with a csv list as well as
multiple parameters each with a single value. I prefer having the choice.
I think having choices is not really good if it doesn't offer anything better.
bq.Consider a single-processor parameter with the option to specify the
parameters for that processor.
This is just too complex to handle with the flat http parameter structure.
Again beyond the scope
URPs are really powerful but they are a bit hard to configure and use. I wish
to make it easier and more widely used. That is the objective of this ticket
Make it possible to define update request processors as toplevel components
Key: SOLR-6892
URL: https://issues.apache.org/jira/browse/SOLR-6892
Project: Solr
Issue Type: Bug
Reporter: Noble Paul
Assignee: Noble Paul
The current update processor chain is rather cumbersome and we should be able
to use the updateprocessors without a chain.
The scope of this ticket is
* A new tag updateProcessor becomes a toplevel tag and it will be
equivalent to the {{processor}} tag inside
{{updateRequestProcessorChain}} . The only difference is that it should
require a {{name}} attribute. The {{updateProcessorChain}} tag will
continue to exist and it should be possible to define processor inside as
well . It should also be possible to reference a named URP in a chain.
* Any update request will be able to pass a param {{processor=a,b,c}} ,
where a,b,c are names of update processors. A just in time chain will be
created with those URPs
* Some in built update processors (wherever possible) will be predefined with
standard names and can be directly used in requests
* What happens when I say processor=a,b,c in a request? It will execute the
default chain after the just-in-time chain {{a-b-c}} .
* How to execute a different chain other than the default chain? the same old
mechanism of update.chain=x means that the chain {{x}} will be applied after
{{a,b,c}}
* How to avoid the default processor chain from being executed ? There will
be an implicit URP called {{STOP}} . send your request as
processor=a,b,c,STOP.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6140) simplify inflater usage in deflate CompressionMode
[ https://issues.apache.org/jira/browse/LUCENE-6140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259671#comment-14259671 ] ASF subversion and git services commented on LUCENE-6140: - Commit 1648224 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1648224 ] LUCENE-6140: simplify inflater usage in CompressionMode simplify inflater usage in deflate CompressionMode -- Key: LUCENE-6140 URL: https://issues.apache.org/jira/browse/LUCENE-6140 Project: Lucene - Core Issue Type: Task Reporter: Robert Muir Attachments: LUCENE-6140.patch This currently loops-n-grows the output byte[]. But we always decompress the whole block (we dont emit flushes or anything to allow otherwise) and ignore offset/length to the end, and we know how big the uncompressed size is up-front... we can just call inflate one time. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-6140) simplify inflater usage in deflate CompressionMode
[ https://issues.apache.org/jira/browse/LUCENE-6140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir resolved LUCENE-6140. - Resolution: Fixed Fix Version/s: Trunk 5.0 simplify inflater usage in deflate CompressionMode -- Key: LUCENE-6140 URL: https://issues.apache.org/jira/browse/LUCENE-6140 Project: Lucene - Core Issue Type: Task Reporter: Robert Muir Fix For: 5.0, Trunk Attachments: LUCENE-6140.patch This currently loops-n-grows the output byte[]. But we always decompress the whole block (we dont emit flushes or anything to allow otherwise) and ignore offset/length to the end, and we know how big the uncompressed size is up-front... we can just call inflate one time. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6140) simplify inflater usage in deflate CompressionMode
[ https://issues.apache.org/jira/browse/LUCENE-6140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259673#comment-14259673 ] ASF subversion and git services commented on LUCENE-6140: - Commit 1648226 from [~rcmuir] in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1648226 ] LUCENE-6140: simplify inflater usage in CompressionMode simplify inflater usage in deflate CompressionMode -- Key: LUCENE-6140 URL: https://issues.apache.org/jira/browse/LUCENE-6140 Project: Lucene - Core Issue Type: Task Reporter: Robert Muir Fix For: 5.0, Trunk Attachments: LUCENE-6140.patch This currently loops-n-grows the output byte[]. But we always decompress the whole block (we dont emit flushes or anything to allow otherwise) and ignore offset/length to the end, and we know how big the uncompressed size is up-front... we can just call inflate one time. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6892) Make it possible to define update request processors as toplevel components
[
https://issues.apache.org/jira/browse/SOLR-6892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noble Paul updated SOLR-6892:
-
Issue Type: Improvement (was: Bug)
Make it possible to define update request processors as toplevel components
Key: SOLR-6892
URL: https://issues.apache.org/jira/browse/SOLR-6892
Project: Solr
Issue Type: Improvement
Reporter: Noble Paul
Assignee: Noble Paul
The current update processor chain is rather cumbersome and we should be able
to use the updateprocessors without a chain.
The scope of this ticket is
* A new tag updateProcessor becomes a toplevel tag and it will be
equivalent to the {{processor}} tag inside
{{updateRequestProcessorChain}} . The only difference is that it should
require a {{name}} attribute. The {{updateProcessorChain}} tag will
continue to exist and it should be possible to define processor inside as
well . It should also be possible to reference a named URP in a chain.
* Any update request will be able to pass a param {{processor=a,b,c}} ,
where a,b,c are names of update processors. A just in time chain will be
created with those URPs
* Some in built update processors (wherever possible) will be predefined with
standard names and can be directly used in requests
* What happens when I say processor=a,b,c in a request? It will execute the
default chain after the just-in-time chain {{a-b-c}} .
* How to execute a different chain other than the default chain? the same old
mechanism of update.chain=x means that the chain {{x}} will be applied after
{{a,b,c}}
* How to avoid the default processor chain from being executed ? There will
be an implicit URP called {{STOP}} . send your request as
processor=a,b,c,STOP.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-6893) DIH doen't work using URL and wget
Dani created SOLR-6893: -- Summary: DIH doen't work using URL and wget Key: SOLR-6893 URL: https://issues.apache.org/jira/browse/SOLR-6893 Project: Solr Issue Type: Bug Affects Versions: 4.10.2, 4.8.1, 4.6.1 Environment: Linux (Ubuntu, CentOS) Reporter: Dani Priority: Minor [ related to #SOLR-2305 ] I put this URL on browser and import correctly starts: http://localhost:8983/solr/#/sintesicontratti/dataimport//importazione?command=full-importentity=fileImportclean=truecommit=true But if I use wget it doens't work: wget http://localhost:8983/solr/#/sintesicontratti/dataimport//importazione?command=full-importentity=fileImportclean=truecommit=true; nor also using escape: wget http://localhost:8983/solr/#/sintesicontratti/dataimport//importazione?command=full-import\entity=fileImport\clean=true\commit=true; -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6138) ItalianLightStemmer doesn't apply on words shorter then 6 chars in length
[ https://issues.apache.org/jira/browse/LUCENE-6138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259684#comment-14259684 ] Erick Erickson commented on LUCENE-6138: Right, the important part of the discussion (I should have pointed it out) was that the stemmers are not part of the Solr code base, they're another project and that project would be the place to raise possible bugs or submit patches, bq: Can you propose your changes to http://members.unine.ch/jacques.savoy/clef/index.html? Sorry for the confusion ItalianLightStemmer doesn't apply on words shorter then 6 chars in length - Key: LUCENE-6138 URL: https://issues.apache.org/jira/browse/LUCENE-6138 Project: Lucene - Core Issue Type: Bug Components: modules/analysis Affects Versions: 4.10.2 Reporter: Massimo Pasquini Priority: Minor I expect a stemmer to transform nouns in their singular and plural forms into a shorter common form. The implementation of the ItalianLightStemmer doesn't apply any stemming to words shorter then 6 characters in length. This leads to some annoying results: singular form | plural form 4|5 chars in length (no stemming) alga - alga | alghe - alghe fuga - fuga | fughe - fughe lega - lega | leghe - leghe 5|6 chars in length (stemming only on plural form) vanga - vanga | vanghe - vang verga - verga | verghe - verg I suppose that such limitation on words length is to avoid other side effects on shorter words not in the set above, but I think something must be reviewed in the code for better results. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6893) DIH doen't work using URL and wget
[ https://issues.apache.org/jira/browse/SOLR-6893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dani updated SOLR-6893: --- Description: I put this URL on browser and import correctly starts: http://localhost:8983/solr/#/sintesicontratti/dataimport//importazione?command=full-importentity=fileImportclean=truecommit=true But if I use wget it doens't work: wget http://localhost:8983/solr/#/sintesicontratti/dataimport//importazione?command=full-importentity=fileImportclean=truecommit=true; nor also using escape: wget http://localhost:8983/solr/#/sintesicontratti/dataimport//importazione?command=full-import\entity=fileImport\clean=true\commit=true; was: [ related to #SOLR-2305 ] I put this URL on browser and import correctly starts: http://localhost:8983/solr/#/sintesicontratti/dataimport//importazione?command=full-importentity=fileImportclean=truecommit=true But if I use wget it doens't work: wget http://localhost:8983/solr/#/sintesicontratti/dataimport//importazione?command=full-importentity=fileImportclean=truecommit=true; nor also using escape: wget http://localhost:8983/solr/#/sintesicontratti/dataimport//importazione?command=full-import\entity=fileImport\clean=true\commit=true; DIH doen't work using URL and wget -- Key: SOLR-6893 URL: https://issues.apache.org/jira/browse/SOLR-6893 Project: Solr Issue Type: Bug Affects Versions: 4.6.1, 4.8.1, 4.10.2 Environment: Linux (Ubuntu, CentOS) Reporter: Dani Priority: Minor I put this URL on browser and import correctly starts: http://localhost:8983/solr/#/sintesicontratti/dataimport//importazione?command=full-importentity=fileImportclean=truecommit=true But if I use wget it doens't work: wget http://localhost:8983/solr/#/sintesicontratti/dataimport//importazione?command=full-importentity=fileImportclean=truecommit=true; nor also using escape: wget http://localhost:8983/solr/#/sintesicontratti/dataimport//importazione?command=full-import\entity=fileImport\clean=true\commit=true; -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5507) Admin UI - Refactoring using AngularJS
[ https://issues.apache.org/jira/browse/SOLR-5507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259692#comment-14259692 ] Erick Erickson commented on SOLR-5507: -- My desire here is for the really annoying error messages to go away if the files are misplaced... I have no real feel for whether they are useful enough to try to keep around anyway, up to the people doing the work of course. Admin UI - Refactoring using AngularJS -- Key: SOLR-5507 URL: https://issues.apache.org/jira/browse/SOLR-5507 Project: Solr Issue Type: Improvement Components: web gui Reporter: Stefan Matheis (steffkes) Assignee: Stefan Matheis (steffkes) Priority: Minor Attachments: SOLR-5507.patch On the LSR in Dublin, i've talked again to [~upayavira] and this time we talked about Refactoring the existing UI - using AngularJS: providing (more, internal) structure and what not ; He already started working on the Refactoring, so this is more a 'tracking' issue about the progress he/we do there. Will extend this issue with a bit more context additional information, w/ thoughts about the possible integration in the existing UI and more (: -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6082) Umbrella JIRA for Admin UI and SolrCloud.
[ https://issues.apache.org/jira/browse/SOLR-6082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259693#comment-14259693 ] Erick Erickson commented on SOLR-6082: -- [~shalinmangar] Don't know whether this is getting much of your attention, but see SOLR-5507. Upayavira is working on moving the admin UI to AngularJS, coordinating the two efforts seems like A Good Thing An open question is how to prioritize this and the work on SOLR-5507. How/when should priority be given to reproducing the current admin UI which looks at things largely on the basis of individual nodes with SolrCloud bolted on as opposed to adding SolrCloud-centric features to the admin UI? Recording here for posterity, as always the people doing the work are the ones who get to choose. Umbrella JIRA for Admin UI and SolrCloud. - Key: SOLR-6082 URL: https://issues.apache.org/jira/browse/SOLR-6082 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 4.9, Trunk Reporter: Erick Erickson Assignee: Shalin Shekhar Mangar It would be very helpful if the admin UI were more cloud friendly. This is an umbrella JIRA so we can collect sub-tasks as necessary. I think there might be scattered JIRAs about this, let's link them in as we find them. [~steffkes] - I've taken the liberty of assigning it to you since you expressed some interest. Feel free to assign it back if you want... Let's imagine that a user has a cluster with _no_ collections assigned and start from there. Here's a simple way to set this up. Basically you follow the reference guide tutorial but _don't_ define a collection. 1 completely delete the collection1 directory from example 2 cp -r example example2 3 in example, execute java -DzkRun -jar start.jar 4 in example2, execute java -Djetty.port=7574 -DzkHost=localhost:9983 -jar start.jar Now the cloud link appears. If you expand the tree view, you see the two live nodes. But, there's nothing in the graph view, no cores are selectable, etc. First problem (need to solve before any sub-jiras, so including it here): You have to push a configuration directory to ZK. [~thetapi] The _last_ time Stefan and I started allowing files to be written to Solr from the UI it was...unfortunate. I'm assuming that there's something similar here. That is, we shouldn't allow pushing the Solr config _to_ ZooKeeper through the Admin UI, where they'd be distributed to all the solr nodes. Is that true? If this is a security issue, we can keep pushing the config dirs to ZK a manual step for now... Once we determine how to get configurations up, we can work on the various sub-jiras. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6876) Remove unused legacy scripts.conf
[ https://issues.apache.org/jira/browse/SOLR-6876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson updated SOLR-6876: - Attachment: SOLR-6876.patch Patch with entry in CHANGES.txt Remove unused legacy scripts.conf - Key: SOLR-6876 URL: https://issues.apache.org/jira/browse/SOLR-6876 Project: Solr Issue Type: Bug Affects Versions: 4.10.2, 5.0, Trunk Reporter: Alexandre Rafalovitch Assignee: Erick Erickson Priority: Minor Attachments: SOLR-6876.patch Some of the example collections include *scripts.conf* in the *conf* directory. It is not used by anything in the distribution and is somehow left over from the Solr 1.x legacy days. It should be possible to safe delete it to avoid confusing users trying to understand what different files actually do. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6876) Remove unused legacy scripts.conf
[ https://issues.apache.org/jira/browse/SOLR-6876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259709#comment-14259709 ] ASF subversion and git services commented on SOLR-6876: --- Commit 1648246 from [~erickoerickson] in branch 'dev/trunk' [ https://svn.apache.org/r1648246 ] SOLR-6876: Remove unused legacy scripts.conf Remove unused legacy scripts.conf - Key: SOLR-6876 URL: https://issues.apache.org/jira/browse/SOLR-6876 Project: Solr Issue Type: Bug Affects Versions: 4.10.2, 5.0, Trunk Reporter: Alexandre Rafalovitch Assignee: Erick Erickson Priority: Minor Attachments: SOLR-6876.patch Some of the example collections include *scripts.conf* in the *conf* directory. It is not used by anything in the distribution and is somehow left over from the Solr 1.x legacy days. It should be possible to safe delete it to avoid confusing users trying to understand what different files actually do. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6893) DIH doen't work using URL and wget
[
https://issues.apache.org/jira/browse/SOLR-6893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259713#comment-14259713
]
Tomoko Uchida commented on SOLR-6893:
-
Hi,
At my environment (Mac OS), DIH works well with wget command.
Maybe you should remove '/#/' from request url.
I have tested two patterns with /#/ and without /#/
It works.
{code}
$ wget -O response
http://localhost:8983/solr/wikipedia/dataimport?command=full-importindent=true;
$ wget -O response
http://localhost:8983/solr/wikipedia/dataimport?command=statusindent=true;
$ cat response
?xml version=1.0 encoding=UTF-8?
response
lst name=responseHeader
int name=status0/int
int name=QTime0/int
/lst
lst name=initArgs
lst name=defaults
str name=configdb-data-config.xml/str
/lst
/lst
str name=commandstatus/str
str name=statusbusy/str
str name=importResponseA command is still running.../str
lst name=statusMessages
str name=Time Elapsed0:6:59.792/str
str name=Total Requests made to DataSource1/str
str name=Total Rows Fetched121772/str
str name=Total Documents Processed60885/str
str name=Total Documents Skipped0/str
str name=Full Dump Started2014-12-28 19:38:24/str
/lst
/response
{code}
It does not work.
{code}
$ wget -O response
http://localhost:8983/solr/#/wikipedia/dataimport?command=full-importindent=true;
{code}
Please try again after removing '#' character (this character has special
meaning in URI specification.)
DIH doen't work using URL and wget
--
Key: SOLR-6893
URL: https://issues.apache.org/jira/browse/SOLR-6893
Project: Solr
Issue Type: Bug
Affects Versions: 4.6.1, 4.8.1, 4.10.2
Environment: Linux (Ubuntu, CentOS)
Reporter: Dani
Priority: Minor
I put this URL on browser and import correctly starts:
http://localhost:8983/solr/#/sintesicontratti/dataimport//importazione?command=full-importentity=fileImportclean=truecommit=true
But if I use wget it doens't work:
wget
http://localhost:8983/solr/#/sintesicontratti/dataimport//importazione?command=full-importentity=fileImportclean=truecommit=true;
nor also using escape:
wget
http://localhost:8983/solr/#/sintesicontratti/dataimport//importazione?command=full-import\entity=fileImport\clean=true\commit=true;
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6876) Remove unused legacy scripts.conf
[ https://issues.apache.org/jira/browse/SOLR-6876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259726#comment-14259726 ] ASF subversion and git services commented on SOLR-6876: --- Commit 1648252 from [~erickoerickson] in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1648252 ] SOLR-6876: Remove unused legacy scripts.conf Remove unused legacy scripts.conf - Key: SOLR-6876 URL: https://issues.apache.org/jira/browse/SOLR-6876 Project: Solr Issue Type: Bug Affects Versions: 4.10.2, 5.0, Trunk Reporter: Alexandre Rafalovitch Assignee: Erick Erickson Priority: Minor Fix For: 5.0, Trunk Attachments: SOLR-6876.patch Some of the example collections include *scripts.conf* in the *conf* directory. It is not used by anything in the distribution and is somehow left over from the Solr 1.x legacy days. It should be possible to safe delete it to avoid confusing users trying to understand what different files actually do. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-6876) Remove unused legacy scripts.conf
[ https://issues.apache.org/jira/browse/SOLR-6876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson resolved SOLR-6876. -- Resolution: Fixed Fix Version/s: Trunk 5.0 Thanks Alexandre! Remove unused legacy scripts.conf - Key: SOLR-6876 URL: https://issues.apache.org/jira/browse/SOLR-6876 Project: Solr Issue Type: Bug Affects Versions: 4.10.2, 5.0, Trunk Reporter: Alexandre Rafalovitch Assignee: Erick Erickson Priority: Minor Fix For: 5.0, Trunk Attachments: SOLR-6876.patch Some of the example collections include *scripts.conf* in the *conf* directory. It is not used by anything in the distribution and is somehow left over from the Solr 1.x legacy days. It should be possible to safe delete it to avoid confusing users trying to understand what different files actually do. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5507) Admin UI - Refactoring using AngularJS
[ https://issues.apache.org/jira/browse/SOLR-5507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259729#comment-14259729 ] Erick Erickson commented on SOLR-5507: -- [~upayavira] Added a comment to SOLR-6082 about prioritization, I think that's enough. Admin UI - Refactoring using AngularJS -- Key: SOLR-5507 URL: https://issues.apache.org/jira/browse/SOLR-5507 Project: Solr Issue Type: Improvement Components: web gui Reporter: Stefan Matheis (steffkes) Assignee: Stefan Matheis (steffkes) Priority: Minor Attachments: SOLR-5507.patch On the LSR in Dublin, i've talked again to [~upayavira] and this time we talked about Refactoring the existing UI - using AngularJS: providing (more, internal) structure and what not ; He already started working on the Refactoring, so this is more a 'tracking' issue about the progress he/we do there. Will extend this issue with a bit more context additional information, w/ thoughts about the possible integration in the existing UI and more (: -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-5.x-Windows (32bit/jdk1.8.0_40-ea-b09) - Build # 4412 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/4412/ Java: 32bit/jdk1.8.0_40-ea-b09 -client -XX:+UseParallelGC (asserts: false) No tests ran. Build Log: [...truncated 10932 lines...] FATAL: java.io.IOException: Unexpected termination of the channel hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel at hudson.remoting.Request.abort(Request.java:295) at hudson.remoting.Channel.terminate(Channel.java:814) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:69) at ..remote call to Windows VBOX(Native Method) at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1356) at hudson.remoting.Request.call(Request.java:171) at hudson.remoting.Channel.call(Channel.java:751) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:179) at com.sun.proxy.$Proxy75.join(Unknown Source) at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:979) at hudson.Launcher$ProcStarter.join(Launcher.java:388) at hudson.tasks.Ant.perform(Ant.java:217) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:770) at hudson.model.Build$BuildExecution.build(Build.java:199) at hudson.model.Build$BuildExecution.doRun(Build.java:160) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:533) at hudson.model.Run.execute(Run.java:1759) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:89) at hudson.model.Executor.run(Executor.java:240) Caused by: java.io.IOException: Unexpected termination of the channel at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50) Caused by: java.io.EOFException at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2325) at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2794) at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:801) at java.io.ObjectInputStream.init(ObjectInputStream.java:299) at hudson.remoting.ObjectInputStreamEx.init(ObjectInputStreamEx.java:40) at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS-MAVEN] Lucene-Solr-Maven-5.x #801: POMs out of sync
Build: https://builds.apache.org/job/Lucene-Solr-Maven-5.x/801/ 3 tests failed. FAILED: org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest.org.apache.solr.hadoop.MorphlineGoLiveMiniMRTest Error Message: null Stack Trace: java.lang.AssertionError: null at __randomizedtesting.SeedInfo.seed([42DA430FA16CA04]:0) at org.apache.lucene.util.TestRuleTemporaryFilesCleanup.before(TestRuleTemporaryFilesCleanup.java:105) at com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.before(TestRuleAdapter.java:26) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:35) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at java.lang.Thread.run(Thread.java:745) FAILED: org.apache.solr.hadoop.MorphlineBasicMiniMRTest.testPathParts Error Message: Test abandoned because suite timeout was reached. Stack Trace: java.lang.Exception: Test abandoned because suite timeout was reached. at __randomizedtesting.SeedInfo.seed([B98A46E3988AA203]:0) FAILED: org.apache.solr.hadoop.MorphlineBasicMiniMRTest.org.apache.solr.hadoop.MorphlineBasicMiniMRTest Error Message: Suite timeout exceeded (= 720 msec). Stack Trace: java.lang.Exception: Suite timeout exceeded (= 720 msec). at __randomizedtesting.SeedInfo.seed([B98A46E3988AA203]:0) Build Log: [...truncated 53961 lines...] BUILD FAILED /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Maven-5.x/build.xml:552: The following error occurred while executing this line: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Maven-5.x/build.xml:204: The following error occurred while executing this line: : Java returned: 1 Total time: 380 minutes 16 seconds Build step 'Invoke Ant' marked build as failure Recording test results Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.9.0-ea-b34) - Build # 11824 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11824/ Java: 64bit/jdk1.9.0-ea-b34 -XX:-UseCompressedOops -XX:+UseG1GC (asserts: true) 2 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest Error Message: Suite timeout exceeded (= 720 msec). Stack Trace: java.lang.Exception: Suite timeout exceeded (= 720 msec). at __randomizedtesting.SeedInfo.seed([78D62EC33B461E33]:0) FAILED: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch Error Message: Test abandoned because suite timeout was reached. Stack Trace: java.lang.Exception: Test abandoned because suite timeout was reached. at __randomizedtesting.SeedInfo.seed([78D62EC33B461E33]:0) Build Log: [...truncated 10414 lines...] [junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest [junit4] 2 Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest-78D62EC33B461E33-001/init-core-data-001 [junit4] 2 1691221 T5088 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false) [junit4] 2 1691221 T5088 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: / [junit4] 2 1691223 T5088 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch [junit4] 2 1691224 T5088 oasc.ZkTestServer.run STARTING ZK TEST SERVER [junit4] 1 client port:0.0.0.0/0.0.0.0:0 [junit4] 2 1691224 T5089 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server [junit4] 2 1691324 T5088 oasc.ZkTestServer.run start zk server on port:45567 [junit4] 2 1691325 T5088 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2 1691325 T5088 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2 1691328 T5096 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@733c650b name:ZooKeeperConnection Watcher:127.0.0.1:45567 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2 1691328 T5088 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2 1691329 T5088 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2 1691329 T5088 oascc.SolrZkClient.makePath makePath: /solr [junit4] 2 1691332 T5088 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2 1691333 T5088 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2 1691333 T5099 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7099daea name:ZooKeeperConnection Watcher:127.0.0.1:45567/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2 1691334 T5088 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2 1691334 T5088 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2 1691334 T5088 oascc.SolrZkClient.makePath makePath: /collections/collection1 [junit4] 2 1691336 T5088 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards [junit4] 2 1691338 T5088 oascc.SolrZkClient.makePath makePath: /collections/control_collection [junit4] 2 1691339 T5088 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards [junit4] 2 1691340 T5088 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml [junit4] 2 1691340 T5088 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml [junit4] 2 1691342 T5088 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml [junit4] 2 1691342 T5088 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml [junit4] 2 1691343 T5088 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2 1691344 T5088 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2 1691345 T5088 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt [junit4] 2 1691345 T5088 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt [junit4] 2 1691346 T5088 oasc.AbstractZkTestCase.putConfig put
[jira] [Commented] (SOLR-6888) Decompressing documents on first-pass distributed queries to get docId is inefficient, use indexed values instead?
[ https://issues.apache.org/jira/browse/SOLR-6888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14259787#comment-14259787 ] David Smiley commented on SOLR-6888: Erick, I enjoyed reading what you have here. I think this issue duplicates SOLR-5478, for which I have a patch in fact. I encourage you to review it and kick the tires on it! Decompressing documents on first-pass distributed queries to get docId is inefficient, use indexed values instead? -- Key: SOLR-6888 URL: https://issues.apache.org/jira/browse/SOLR-6888 Project: Solr Issue Type: Improvement Affects Versions: 5.0, Trunk Reporter: Erick Erickson Assignee: Erick Erickson Attachments: SOLR-6888-hacktiming.patch Assigning this to myself to just not lose track of it, but I won't be working on this in the near term; anyone feeling ambitious should feel free to grab it. Note, docId used here is whatever is defined for uniqueKey... Since Solr 4.1, the compression/decompression process is based on 16K blocks and is automatic, and not configurable. So, to get a single stored value one must decompress an entire 16K block. At least. For SolrCloud (and distributed processing in general), we make two trips, one to get the doc id and score (or other sort criteria) and one to return the actual data. The first pass here requires that we return the top N docIDs and sort criteria, which means that each and every sub-request has to unpack at least one 16K block (and sometimes more) to get just the doc ID. So if we have 20 shards and only want 20 rows, 95% of the decompression cycles will be wasted. Not to mention all the disk reads. It seems like we should be able to do better than that. Can we argue that doc ids are 'special' and should be cached somehow? Let's discuss what this would look like. I can think of a couple of approaches: 1 Since doc IDs are special, can we say that for this purpose returning the indexed version is OK? We'd need to return the actual stored value when the full doc was requested, but for the sub-request only what about returning the indexed value instead of the stored one? On the surface I don't see a problem here, but what do I know? Storing these as DocValues seems useful in this case. 1a A variant is treating numeric docIds specially since the indexed value and the stored value should be identical. And DocValues here would be useful it seems. But this seems an unnecessary specialization if 1 is implemented well. 2 We could cache individual doc IDs, although I'm not sure what use that really is. Would maintaining the cache overwhelm the savings of not decompressing? I really don't like this idea, but am throwing it out there. Doing this from stored data up front would essentially mean decompressing every doc so that seems untenable to try up-front. 3 We could maintain an array[maxDoc] that held document IDs, perhaps lazily initializing it. I'm not particularly a fan of this either, doesn't seem like a Good Thing. I can see lazy loading being almost, but not quite totally, useless, i.e. a hit ratio near 0, especially since it'd be thrown out on every openSearcher. Really, the only one of these that seems viable is 1/1a. The others would all involve decompressing the docs anyway to get the ID, and I suspect that caching would be of very limited usefulness. I guess 1's viability hinges on whether, for internal use, the indexed form of DocId is interchangeable with the stored value. Or are there other ways to approach this? Or isn't it something to really worry about? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 717 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/717/ 1 tests failed. REGRESSION: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch Error Message: Error from server at http://127.0.0.1:54363: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: Could not get shard id for core: halfcollection_shard1_replica1 Stack Trace: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Error from server at http://127.0.0.1:54363: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: Could not get shard id for core: halfcollection_shard1_replica1 at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:566) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:213) at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:209) at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:452) at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:201) at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at
[JENKINS] Lucene-Solr-Tests-5.x-Java7 - Build # 2394 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-Tests-5.x-Java7/2394/ 2 tests failed. REGRESSION: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch Error Message: Test abandoned because suite timeout was reached. Stack Trace: java.lang.Exception: Test abandoned because suite timeout was reached. at __randomizedtesting.SeedInfo.seed([2FE41655C33758DD]:0) FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest Error Message: Suite timeout exceeded (= 720 msec). Stack Trace: java.lang.Exception: Suite timeout exceeded (= 720 msec). at __randomizedtesting.SeedInfo.seed([2FE41655C33758DD]:0) Build Log: [...truncated 10475 lines...] [junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest [junit4] 2 Creating dataDir: /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeySafeLeaderTest-2FE41655C33758DD-001/init-core-data-001 [junit4] 2 1667952 T3808 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false) [junit4] 2 1667952 T3808 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /dt_/lw [junit4] 2 1667959 T3808 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch [junit4] 2 1667960 T3808 oasc.ZkTestServer.run STARTING ZK TEST SERVER [junit4] 1 client port:0.0.0.0/0.0.0.0:0 [junit4] 2 1667961 T3809 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server [junit4] 2 1668060 T3808 oasc.ZkTestServer.run start zk server on port:11132 [junit4] 2 1668061 T3808 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2 1668062 T3808 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2 1668065 T3816 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@418cfca8 name:ZooKeeperConnection Watcher:127.0.0.1:11132 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2 1668066 T3808 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2 1668066 T3808 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2 1668066 T3808 oascc.SolrZkClient.makePath makePath: /solr [junit4] 2 1668069 T3808 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider [junit4] 2 1668070 T3808 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper [junit4] 2 1668072 T3819 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4b266679 name:ZooKeeperConnection Watcher:127.0.0.1:11132/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None [junit4] 2 1668072 T3808 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper [junit4] 2 1668072 T3808 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider [junit4] 2 1668073 T3808 oascc.SolrZkClient.makePath makePath: /collections/collection1 [junit4] 2 1668075 T3808 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards [junit4] 2 1668076 T3808 oascc.SolrZkClient.makePath makePath: /collections/control_collection [junit4] 2 1668078 T3808 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards [junit4] 2 1668080 T3808 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml [junit4] 2 1668080 T3808 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml [junit4] 2 1668083 T3808 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml [junit4] 2 1668083 T3808 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml [junit4] 2 1668185 T3808 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2 1668186 T3808 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml [junit4] 2 1668188 T3808 oasc.AbstractZkTestCase.putConfig put /usr/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-5.x-Java7/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt [junit4] 2 1668189 T3808 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt [junit4] 2 1668190 T3808 oasc.AbstractZkTestCase.putConfig put
Multiple CFS files are generated in lucene 4.10.2
Hi
I am trying index around 612 record using lucene 4.10.2. It is creating lot
of CF files in index directory.
around 626 CFS file are created. It is taking more time to index.
ENV : java 8, window 7
http://lucene.472066.n3.nabble.com/file/n4176336/index_files.jpg
Directory dir = FSDirectory.open(file);
IndexWriterConfig config = new
IndexWriterConfig(Version.LUCENE_4_10_2, new ClassicAnalyzer());
if(bufferSizeMB != 0 bufferSizeMB != -1){
config.setRAMBufferSizeMB(bufferSizeMB);
} else {
config.setRAMBufferSizeMB(DEFAULT_RAM_BUFFER_SIZE_MB);
}
config.setMaxBufferedDocs(1000);
config.setMaxBufferedDeleteTerms(1000);
config.setMergePolicy(new LogDocMergePolicy());
IndexWriter iwriter = new IndexWriter(dir, config);
iwriter.getConfig().setMaxBufferedDeleteTerms(1000);
iwriter.getConfig().setMaxBufferedDocs(1000);
iwriter.getConfig().setRAMBufferSizeMB(bufferSizeMB)
Thanks and Regards
jaga india
--
View this message in context:
http://lucene.472066.n3.nabble.com/Multiple-CFS-files-are-generated-in-lucene-4-10-2-tp4176336.html
Sent from the Lucene - Java Developer mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
