[jira] [Updated] (LUCENE-7700) Move throughput control and merge aborting out of IndexWriter's core?
[ https://issues.apache.org/jira/browse/LUCENE-7700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dawid Weiss updated LUCENE-7700: Description: Here is a bit of a background: - I wanted to implement a custom merging strategy that would have a custom i/o flow control (global), - currently, the CMS is tightly coupled with a few classes -- MergeRateLimiter, OneMerge, IndexWriter. Looking at the code it seems to me that everything with respect to I/O control could be nicely pulled out into classes that explicitly control the merging process, that is only MergePolicy and MergeScheduler. By default, one could even run without any additional I/O accounting overhead (which is currently in there, even if one doesn't use the CMS's throughput control). Such refactoring would also give a chance to nicely move things where they belong -- job aborting into OneMerge (currently in RateLimiter), rate limiter lifecycle bound to OneMerge (MergeScheduler could then use per-merge or global accounting, as it pleases). Just a thought and some initial refactorings for discussion. was: Here is a bit of a background: - I wanted to implement a custom merging strategy that would have a custom i/o flow control (global), - currently, the CMS is tightly bound with a few classes -- MergeRateLimiter, OneMerge, IndexWriter. Looking at the code it seems to me that everything with respect to I/O control could be nicely pulled out into classes that explicitly control the merging process, that is only MergePolicy and MergeScheduler. By default, one could even run without any additional I/O accounting overhead (which is currently in there, even if one doesn't use the CMS's throughput control). Such refactoring would also give a chance to nicely move things where they belong -- job aborting into OneMerge (currently in RateLimiter), rate limiter lifecycle bound to OneMerge (MergeScheduler could then use per-merge or global accounting, as it pleases). Just a thought and some initial refactorings for discussion. > Move throughput control and merge aborting out of IndexWriter's core? > - > > Key: LUCENE-7700 > URL: https://issues.apache.org/jira/browse/LUCENE-7700 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Minor > Fix For: 6.x, master (7.0), 6.5 > > Attachments: LUCENE-7700.patch, LUCENE-7700.patch, LUCENE-7700.patch > > > Here is a bit of a background: > - I wanted to implement a custom merging strategy that would have a custom > i/o flow control (global), > - currently, the CMS is tightly coupled with a few classes -- > MergeRateLimiter, OneMerge, IndexWriter. > Looking at the code it seems to me that everything with respect to I/O > control could be nicely pulled out into classes that explicitly control the > merging process, that is only MergePolicy and MergeScheduler. By default, one > could even run without any additional I/O accounting overhead (which is > currently in there, even if one doesn't use the CMS's throughput control). > Such refactoring would also give a chance to nicely move things where they > belong -- job aborting into OneMerge (currently in RateLimiter), rate limiter > lifecycle bound to OneMerge (MergeScheduler could then use per-merge or > global accounting, as it pleases). > Just a thought and some initial refactorings for discussion. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-7700) Move throughput control and merge aborting out of IndexWriter's core?
[ https://issues.apache.org/jira/browse/LUCENE-7700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dawid Weiss updated LUCENE-7700: Attachment: LUCENE-7700.patch Final patch. > Move throughput control and merge aborting out of IndexWriter's core? > - > > Key: LUCENE-7700 > URL: https://issues.apache.org/jira/browse/LUCENE-7700 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Minor > Fix For: 6.x, master (7.0), 6.5 > > Attachments: LUCENE-7700.patch, LUCENE-7700.patch, LUCENE-7700.patch, > LUCENE-7700.patch > > > Here is a bit of a background: > - I wanted to implement a custom merging strategy that would have a custom > i/o flow control (global), > - currently, the CMS is tightly coupled with a few classes -- > MergeRateLimiter, OneMerge, IndexWriter. > Looking at the code it seems to me that everything with respect to I/O > control could be nicely pulled out into classes that explicitly control the > merging process, that is only MergePolicy and MergeScheduler. By default, one > could even run without any additional I/O accounting overhead (which is > currently in there, even if one doesn't use the CMS's throughput control). > Such refactoring would also give a chance to nicely move things where they > belong -- job aborting into OneMerge (currently in RateLimiter), rate limiter > lifecycle bound to OneMerge (MergeScheduler could then use per-merge or > global accounting, as it pleases). > Just a thought and some initial refactorings for discussion. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10218) The Schema API "replace-field-type" does not generate the SolrParameters for a SimilarityFactory correctly
[
https://issues.apache.org/jira/browse/SOLR-10218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Troy Mohl updated SOLR-10218:
-
Attachment: SOLR-10218.patch
This patch updates the FieldTypeXmlAdapter to parse the similarity options into
XML elements.
> The Schema API "replace-field-type" does not generate the SolrParameters for
> a SimilarityFactory correctly
> --
>
> Key: SOLR-10218
> URL: https://issues.apache.org/jira/browse/SOLR-10218
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Schema and Analysis
>Affects Versions: 6.4.1
>Reporter: Benjamin Deininger
>Priority: Minor
> Attachments: SOLR-10218.patch
>
>
> When sending a JSON POST to the Schema API to replace a field type, the
> following JSON does not pass the SolrParameters properly to the
> BM25SimilarityFactory.
> {code:javascript}
> {"replace-field-type":{"name":"tint","class":"solr.TrieIntField","positionIncrementGap":"0","precisionStep":"8","similarity":{"class":"solr.BM25SimilarityFactory","k1":1.25,"b":0.75}}}
> {code}
> The `appendAttrs` function in the FieldTypeXmlAdapter parses k1 and b into
> attributes instead of children.
> https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/rest/schema/FieldTypeXmlAdapter.java#L155
> {code:xml}
> class="org.apache.lucene.search.similarities.BM25Similarity" k1="1.25"/>
> {code}
> Based on the XML examples for similarity, this should actually be the
> following :
> {code:xml}
>
> 0.1
> 0.1
>
> {code}
> The similarities block in JSON should be handled differently so that the XML
> is generated appropriately.
> {code:java}
> protected static Element appendSimilarityAttrs(Document doc, Element elm,
> Map json) {
> String clazz = (String) json.get("class");
> elm.setAttribute("class", clazz);
> json.remove("class");
> for (Map.Entry entry : json.entrySet()) {
> Object val = entry.getValue();
> if (val != null && !(val instanceof Map)) {
> Element element =
> doc.createElement(val.getClass().getSimpleName().toLowerCase());
> element.setAttribute("name", entry.getKey());
> element.setTextContent(entry.getValue().toString());
> elm.appendChild(element);
> }
> }
> return elm;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (SOLR-9838) atomic "inc" when adding doc doesn't respect field "default" from schema
[
https://issues.apache.org/jira/browse/SOLR-9838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ishan Chattopadhyaya reassigned SOLR-9838:
--
Assignee: Ishan Chattopadhyaya
> atomic "inc" when adding doc doesn't respect field "default" from schema
>
>
> Key: SOLR-9838
> URL: https://issues.apache.org/jira/browse/SOLR-9838
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Hoss Man
>Assignee: Ishan Chattopadhyaya
> Attachments: SOLR-9838.patch
>
>
> If you do an "atomic update" when adding a document for the first time, then
> the "inc" operator acts as if the field has a default of 0.
> But if the {{}} has an *actual* default in the schema.xml (example:
> {{default="42"}}) then that default is ignored by the atomic update code path.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1105) Use a different stored field for highlighting
[
https://issues.apache.org/jira/browse/SOLR-1105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903258#comment-15903258
]
Matthew Caruana Galizia commented on SOLR-1105:
---
It also doesn't seem to difficult to implement on the UnifiedHighlighter, at
least for someone who's familiar with the code.
> Use a different stored field for highlighting
> -
>
> Key: SOLR-1105
> URL: https://issues.apache.org/jira/browse/SOLR-1105
> Project: Solr
> Issue Type: Improvement
> Components: highlighter
>Reporter: Dmitry Lihachev
> Attachments: SOLR-1105-1_4_1.patch,
> SOLR-1105_shared_content_field_1.3.0.patch
>
>
> DefaultSolrHighlighter uses stored field content to highlight. It has some
> disadvantages, because index grows up fast when using multilingual indexing
> due to several fields has to be stored with same content. This patch allows
> DefaultSolrHighlighter to use "contentField" attribute to loockup content in
> external field.
> Excerpt from old schema:
> {code:xml}
>
>
>
>
> {code}
> The same after patching, highlighter will now get content stored in "title"
> field
> {code:xml}
>
> contentField="title"/>
> contentField="title"/>
> contentField="title"/>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10117) Big docs and the DocumentCache; umbrella issue
[ https://issues.apache.org/jira/browse/SOLR-10117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903226#comment-15903226 ] Alexandre Rafalovitch commented on SOLR-10117: -- No help with code I am afraid, but a question: Would this deduplicate large fields replicated in multiple records? We advise people to denormalize records, so if big stored fields were actually stored once (e.g. repeated description pushed into child records), it would probably be a marketing argument, not just a technical one. > Big docs and the DocumentCache; umbrella issue > -- > > Key: SOLR-10117 > URL: https://issues.apache.org/jira/browse/SOLR-10117 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: David Smiley >Assignee: David Smiley > Attachments: SOLR_10117_large_fields.patch > > > This is an umbrella issue for improved handling of large documents (large > stored fields), generally related to the DocumentCache or SolrIndexSearcher's > doc() methods. Highlighting is affected as it's the primary consumer of this > data. "Large" here is multi-megabyte, especially tens even hundreds of > megabytes. We'd like to support such users without forcing them to choose > between no DocumentCache (bad performance), or having one but hitting OOM due > to massive Strings winding up in there. I've contemplated this for longer > than I'd like to admit and it's a complicated issue with difference concerns > to balance. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Closed] (SOLR-5276) highlighter working using stemmed tokens from another field and text from another
[ https://issues.apache.org/jira/browse/SOLR-5276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Smiley closed SOLR-5276. -- Resolution: Duplicate > highlighter working using stemmed tokens from another field and text from > another > - > > Key: SOLR-5276 > URL: https://issues.apache.org/jira/browse/SOLR-5276 > Project: Solr > Issue Type: Improvement > Components: highlighter >Reporter: Maciej Lizewski >Priority: Minor > > The case is like this: > I have 'content' field with content extracted with tika and several fields > for language versions (like content_pl, content_en, content_es, content_ru, > etc). > I also use custom langid component which can copy 'content' to serveral > content_* fields if it detects more than one language (so those parts are > properly stemmed in every language present in text). > Now to use highlighter in such scenario I need to store all those language > fields even if their contents is always same as the one in 'content' field. > Would be nice if I could configure language specific fields to be not stored, > and configure highlighter to take tokens positions from those fields and > apply them to text in 'content' field... > In other words - to say: take tokens from 'content_pl', and prepare highlight > based on text in 'content' field. > It could be administrator responsibility to guarantee that mapped fields have > same content. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9959) SolrInfoMBean-s category and hierarchy cleanup
[
https://issues.apache.org/jira/browse/SOLR-9959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrzej Bialecki updated SOLR-9959:
Component/s: metrics
> SolrInfoMBean-s category and hierarchy cleanup
> --
>
> Key: SOLR-9959
> URL: https://issues.apache.org/jira/browse/SOLR-9959
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: metrics
>Affects Versions: master (7.0)
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Blocker
> Fix For: master (7.0)
>
>
> SOLR-9947 changed categories of some of {{SolrInfoMBean-s}}, and it also
> added an alternative view in JMX, similar to the one produced by
> {{SolrJmxReporter}}.
> Some changes were left out from that issue because they would break the
> back-compatibility in 6.x, but they should be done before 7.0:
> * remove the old JMX view of {{SolrInfoMBean}}-s and improve the new one so
> that it's more readable and useful.
> * in many cases {{SolrInfoMBean.getName()}} just returns a FQCN, but it could
> be more informative - eg. for highlighter or query plugins this could be the
> symbolic name of a plugin that users know and use in configuration.
> * top-level categories need more thought. On one hand it's best to minimize
> their number, on the other hand they need to meaningfully represent the
> functionality of components that use them. SOLR-9947 made some cosmetic
> changes, but more discussion is necessary (eg. QUERY vs. SEARCHHANDLER)
> * we should consider removing some of the methods in {{SolrInfoMBean}} that
> usually don't return any useful information, eg. {{getDocs}}, {{getSource()}}
> and {{getVersion()}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10247) Support non-numeric metrics
[
https://issues.apache.org/jira/browse/SOLR-10247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrzej Bialecki updated SOLR-10247:
-
Attachment: SOLR-10247.patch
Updated patch.
> Support non-numeric metrics
> ---
>
> Key: SOLR-10247
> URL: https://issues.apache.org/jira/browse/SOLR-10247
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: metrics
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
> Fix For: 6.5, master (7.0)
>
> Attachments: compactFormat.png, currentFormat.png, SOLR-10247.patch,
> SOLR-10247.patch
>
>
> Metrics API currently supports only numeric values. However, it's useful also
> to report non-numeric values such as eg. version, disk type, component state,
> some system properties, etc.
> Codahale {{Gauge}} metric type can be used for this purpose, and
> convenience methods can be added to {{SolrMetricManager}} to make it easier
> to use.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10117) Big docs and the DocumentCache; umbrella issue
[ https://issues.apache.org/jira/browse/SOLR-10117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903287#comment-15903287 ] David Smiley commented on SOLR-10117: - bq. Would this deduplicate large fields replicated in multiple records? No. If I were tasked to do that, I might implement a customized DocValuesFormat that deduplicated per segment (could not dedup at higher tiers) by using the length as a crude hash and then verifying the dedup by re-reading the original. Query time would be no overhead; it'd simply share the internal offset/length pointer. > Big docs and the DocumentCache; umbrella issue > -- > > Key: SOLR-10117 > URL: https://issues.apache.org/jira/browse/SOLR-10117 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: David Smiley >Assignee: David Smiley > Attachments: SOLR_10117_large_fields.patch > > > This is an umbrella issue for improved handling of large documents (large > stored fields), generally related to the DocumentCache or SolrIndexSearcher's > doc() methods. Highlighting is affected as it's the primary consumer of this > data. "Large" here is multi-megabyte, especially tens even hundreds of > megabytes. We'd like to support such users without forcing them to choose > between no DocumentCache (bad performance), or having one but hitting OOM due > to massive Strings winding up in there. I've contemplated this for longer > than I'd like to admit and it's a complicated issue with difference concerns > to balance. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5872) Eliminate overseer queue
[ https://issues.apache.org/jira/browse/SOLR-5872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903303#comment-15903303 ] Erick Erickson commented on SOLR-5872: -- bq: "Also, I do not see why collection A should be aware of collection B state." What is your evidence of this? Because this was changed quite a while ago. Originally there was a single clusterstate.json that held all of the state information for all the collections, but that changed to having a state.json in each collections' Znode. So either you're misinterpreting something or somehow using an older style Zookeeper state. Or something really strange is happening. See the collections API MIGRATESTATEFORMAT. > Eliminate overseer queue > - > > Key: SOLR-5872 > URL: https://issues.apache.org/jira/browse/SOLR-5872 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Reporter: Noble Paul >Assignee: Noble Paul > > The overseer queue is one of the busiest points in the entire system. The > raison d'être of the queue is > * Provide batching of operations for the main clusterstate,json so that > state updates are minimized > * Avoid race conditions and ensure order > Now , as we move the individual collection states out of the main > clusterstate.json, the batching is not useful anymore. > Race conditions can easily be solved by using a compare and set in Zookeeper. > The proposed solution is , whenever an operation is required to be performed > on the clusterstate, the same thread (and of course the same JVM) > # read the fresh state and version of zk node > # construct the new state > # perform a compare and set > # if compare and set fails go to step 1 > This should be limited to all operations performed on external collections > because batching would be required for others -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9959) SolrInfoMBean-s category and hierarchy cleanup
[
https://issues.apache.org/jira/browse/SOLR-9959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903385#comment-15903385
]
Andrzej Bialecki commented on SOLR-9959:
-
There's also considerable duplication of code responsible for reporting metrics
in components that implement both {{SolrInfoMBean}} and {{SolrMetricProducer}}.
We should investigate whether {{SolrInfoMBean.getStatistics()}} is needed at
all - statistics reported by this method theoretically can be equally well
reported using the metrics API.
> SolrInfoMBean-s category and hierarchy cleanup
> --
>
> Key: SOLR-9959
> URL: https://issues.apache.org/jira/browse/SOLR-9959
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: metrics
>Affects Versions: master (7.0)
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Blocker
> Fix For: master (7.0)
>
>
> SOLR-9947 changed categories of some of {{SolrInfoMBean-s}}, and it also
> added an alternative view in JMX, similar to the one produced by
> {{SolrJmxReporter}}.
> Some changes were left out from that issue because they would break the
> back-compatibility in 6.x, but they should be done before 7.0:
> * remove the old JMX view of {{SolrInfoMBean}}-s and improve the new one so
> that it's more readable and useful.
> * in many cases {{SolrInfoMBean.getName()}} just returns a FQCN, but it could
> be more informative - eg. for highlighter or query plugins this could be the
> symbolic name of a plugin that users know and use in configuration.
> * top-level categories need more thought. On one hand it's best to minimize
> their number, on the other hand they need to meaningfully represent the
> functionality of components that use them. SOLR-9947 made some cosmetic
> changes, but more discussion is necessary (eg. QUERY vs. SEARCHHANDLER)
> * we should consider removing some of the methods in {{SolrInfoMBean}} that
> usually don't return any useful information, eg. {{getDocs}}, {{getSource()}}
> and {{getVersion()}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-9959) SolrInfoMBean-s category and hierarchy cleanup
[
https://issues.apache.org/jira/browse/SOLR-9959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903382#comment-15903382
]
Andrzej Bialecki commented on SOLR-9959:
-
There's also considerable duplication of code responsible for reporting metrics
in components that implement both {{SolrInfoMBean}} and {{SolrMetricProducer}}.
We should investigate whether {{SolrInfoMBean.getStatistics()}} is needed at
all - statistics reported by this method theoretically can be equally well
reported using the metrics API.
> SolrInfoMBean-s category and hierarchy cleanup
> --
>
> Key: SOLR-9959
> URL: https://issues.apache.org/jira/browse/SOLR-9959
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: metrics
>Affects Versions: master (7.0)
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Blocker
> Fix For: master (7.0)
>
>
> SOLR-9947 changed categories of some of {{SolrInfoMBean-s}}, and it also
> added an alternative view in JMX, similar to the one produced by
> {{SolrJmxReporter}}.
> Some changes were left out from that issue because they would break the
> back-compatibility in 6.x, but they should be done before 7.0:
> * remove the old JMX view of {{SolrInfoMBean}}-s and improve the new one so
> that it's more readable and useful.
> * in many cases {{SolrInfoMBean.getName()}} just returns a FQCN, but it could
> be more informative - eg. for highlighter or query plugins this could be the
> symbolic name of a plugin that users know and use in configuration.
> * top-level categories need more thought. On one hand it's best to minimize
> their number, on the other hand they need to meaningfully represent the
> functionality of components that use them. SOLR-9947 made some cosmetic
> changes, but more discussion is necessary (eg. QUERY vs. SEARCHHANDLER)
> * we should consider removing some of the methods in {{SolrInfoMBean}} that
> usually don't return any useful information, eg. {{getDocs}}, {{getSource()}}
> and {{getVersion()}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10130) Serious performance degradation in Solr 6.4.1 due to the new metrics collection
[ https://issues.apache.org/jira/browse/SOLR-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Heisey updated SOLR-10130: Affects Version/s: 6.4.0 > Serious performance degradation in Solr 6.4.1 due to the new metrics > collection > --- > > Key: SOLR-10130 > URL: https://issues.apache.org/jira/browse/SOLR-10130 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: metrics >Affects Versions: 6.4.1, 6.4.0 > Environment: Centos 7, OpenJDK 1.8.0 update 111 >Reporter: Ere Maijala >Assignee: Andrzej Bialecki >Priority: Blocker > Labels: perfomance > Fix For: master (7.0), 6.4.2 > > Attachments: SOLR-10130.patch, SOLR-10130.patch, > solr-8983-console-f1.log > > > We've stumbled on serious performance issues after upgrading to Solr 6.4.1. > Looks like the new metrics collection system in MetricsDirectoryFactory is > causing a major slowdown. This happens with an index configuration that, as > far as I can see, has no metrics specific configuration and uses > luceneMatchVersion 5.5.0. In practice a moderate load will completely bog > down the server with Solr threads constantly using up all CPU (600% on 6 core > machine) capacity with a load that normally where we normally see an average > load of < 50%. > I took stack traces (I'll attach them) and noticed that the threads are > spending time in com.codahale.metrics.Meter.mark. I tested building Solr > 6.4.1 with the metrics collection disabled in MetricsDirectoryFactory getByte > and getBytes methods and was unable to reproduce the issue. > As far as I can see there are several issues: > 1. Collecting metrics on every single byte read is slow. > 2. Having it enabled by default is not a good idea. > 3. The comment "enable coarse-grained metrics by default" at > https://github.com/apache/lucene-solr/blob/branch_6x/solr/core/src/java/org/apache/solr/update/SolrIndexConfig.java#L104 > implies that only coarse-grained metrics should be enabled by default, and > this contradicts with collecting metrics on every single byte read. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-1105) Use a different stored field for highlighting
[
https://issues.apache.org/jira/browse/SOLR-1105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903258#comment-15903258
]
Matthew Caruana Galizia edited comment on SOLR-1105 at 3/9/17 3:57 PM:
---
It also doesn't seem too difficult to implement on the UnifiedHighlighter, at
least for someone who's familiar with the code.
was (Author: mcaruanagalizia):
It also doesn't seem to difficult to implement on the UnifiedHighlighter, at
least for someone who's familiar with the code.
> Use a different stored field for highlighting
> -
>
> Key: SOLR-1105
> URL: https://issues.apache.org/jira/browse/SOLR-1105
> Project: Solr
> Issue Type: Improvement
> Components: highlighter
>Reporter: Dmitry Lihachev
> Attachments: SOLR-1105-1_4_1.patch,
> SOLR-1105_shared_content_field_1.3.0.patch
>
>
> DefaultSolrHighlighter uses stored field content to highlight. It has some
> disadvantages, because index grows up fast when using multilingual indexing
> due to several fields has to be stored with same content. This patch allows
> DefaultSolrHighlighter to use "contentField" attribute to loockup content in
> external field.
> Excerpt from old schema:
> {code:xml}
>
>
>
>
> {code}
> The same after patching, highlighter will now get content stored in "title"
> field
> {code:xml}
>
> contentField="title"/>
> contentField="title"/>
> contentField="title"/>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10247) Support non-numeric metrics
[
https://issues.apache.org/jira/browse/SOLR-10247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903323#comment-15903323
]
Andrzej Bialecki commented on SOLR-10247:
--
1. It doesn't, they are now registered as inline lambdas.
3. Yes, I'll fix this.
4. Good idea, I'll add this.
> Support non-numeric metrics
> ---
>
> Key: SOLR-10247
> URL: https://issues.apache.org/jira/browse/SOLR-10247
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: metrics
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
> Fix For: 6.5, master (7.0)
>
> Attachments: compactFormat.png, currentFormat.png, SOLR-10247.patch
>
>
> Metrics API currently supports only numeric values. However, it's useful also
> to report non-numeric values such as eg. version, disk type, component state,
> some system properties, etc.
> Codahale {{Gauge}} metric type can be used for this purpose, and
> convenience methods can be added to {{SolrMetricManager}} to make it easier
> to use.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5872) Eliminate overseer queue
[ https://issues.apache.org/jira/browse/SOLR-5872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903394#comment-15903394 ] albert vico oton commented on SOLR-5872: Already did that, but nodes still notify its state change, apparently collections need to know about other collections status in order to reroute queries to them, this amount of state change msg was killing our io in the ZK cluster, causing a lot of cpu wait time and effectively rendering the system unusable. But honestly that's as far as we went, we moved away from solrcloud and now we are using standalone solr inside each container for each of our collections and doing the balancing between replicas through nginx. > Eliminate overseer queue > - > > Key: SOLR-5872 > URL: https://issues.apache.org/jira/browse/SOLR-5872 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Reporter: Noble Paul >Assignee: Noble Paul > > The overseer queue is one of the busiest points in the entire system. The > raison d'être of the queue is > * Provide batching of operations for the main clusterstate,json so that > state updates are minimized > * Avoid race conditions and ensure order > Now , as we move the individual collection states out of the main > clusterstate.json, the batching is not useful anymore. > Race conditions can easily be solved by using a compare and set in Zookeeper. > The proposed solution is , whenever an operation is required to be performed > on the clusterstate, the same thread (and of course the same JVM) > # read the fresh state and version of zk node > # construct the new state > # perform a compare and set > # if compare and set fails go to step 1 > This should be limited to all operations performed on external collections > because batching would be required for others -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-1105) Use a different stored field for highlighting
[
https://issues.apache.org/jira/browse/SOLR-1105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Smiley updated SOLR-1105:
---
Fix Version/s: (was: 6.0)
(was: 4.9)
Summary: Use a different stored field for highlighting (was: Using
external field content for highlighting)
This would be very useful indeed.
> Use a different stored field for highlighting
> -
>
> Key: SOLR-1105
> URL: https://issues.apache.org/jira/browse/SOLR-1105
> Project: Solr
> Issue Type: Improvement
> Components: highlighter
>Reporter: Dmitry Lihachev
> Attachments: SOLR-1105-1_4_1.patch,
> SOLR-1105_shared_content_field_1.3.0.patch
>
>
> DefaultSolrHighlighter uses stored field content to highlight. It has some
> disadvantages, because index grows up fast when using multilingual indexing
> due to several fields has to be stored with same content. This patch allows
> DefaultSolrHighlighter to use "contentField" attribute to loockup content in
> external field.
> Excerpt from old schema:
> {code:xml}
>
>
>
>
> {code}
> The same after patching, highlighter will now get content stored in "title"
> field
> {code:xml}
>
> contentField="title"/>
> contentField="title"/>
> contentField="title"/>
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10255) Large psuedo-stored fields via BinaryDocValuesField
[
https://issues.apache.org/jira/browse/SOLR-10255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903271#comment-15903271
]
Michael Braun commented on SOLR-10255:
--
We have a use case that would be improved by this.
> Large psuedo-stored fields via BinaryDocValuesField
> ---
>
> Key: SOLR-10255
> URL: https://issues.apache.org/jira/browse/SOLR-10255
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: David Smiley
>Assignee: David Smiley
> Attachments: SOLR-10255.patch
>
>
> (sub-issue of SOLR-10117) This is a proposal for a better way for Solr to
> handle "large" text fields. Large docs that are in Lucene StoredFields slow
> requests that don't involve access to such fields. This is fundamental to
> the fact that StoredFields are row-stored. Worse, the Solr documentCache
> will wind up holding onto massive Strings. While the latter could be tackled
> on it's own somehow as it's the most serious issue, nevertheless it seems
> wrong that such large fields are in row-stored storage to begin with. After
> all, relational DBs seemed to have figured this out and put CLOBs/BLOBs in a
> separate place. Here, we do similarly by using, Lucene
> {{BinaryDocValuesField}}. BDVF isn't well known in the DocValues family as
> it's not for typical DocValues purposes like sorting/faceting etc. The
> default DocValuesFormat doesn't compress these but we could write one that
> does.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-7700) Move throughput control and merge aborting out of IndexWriter's core?
[ https://issues.apache.org/jira/browse/LUCENE-7700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dawid Weiss updated LUCENE-7700: Fix Version/s: master (7.0) 6.x > Move throughput control and merge aborting out of IndexWriter's core? > - > > Key: LUCENE-7700 > URL: https://issues.apache.org/jira/browse/LUCENE-7700 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Minor > Fix For: 6.x, master (7.0), 6.5 > > Attachments: LUCENE-7700.patch, LUCENE-7700.patch, LUCENE-7700.patch > > > Here is a bit of a background: > - I wanted to implement a custom merging strategy that would have a custom > i/o flow control (global), > - currently, the CMS is tightly bound with a few classes -- MergeRateLimiter, > OneMerge, IndexWriter. > Looking at the code it seems to me that everything with respect to I/O > control could be nicely pulled out into classes that explicitly control the > merging process, that is only MergePolicy and MergeScheduler. By default, one > could even run without any additional I/O accounting overhead (which is > currently in there, even if one doesn't use the CMS's throughput control). > Such refactoring would also give a chance to nicely move things where they > belong -- job aborting into OneMerge (currently in RateLimiter), rate limiter > lifecycle bound to OneMerge (MergeScheduler could then use per-merge or > global accounting, as it pleases). > Just a thought and some initial refactorings for discussion. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-7700) Move throughput control and merge aborting out of IndexWriter's core?
[ https://issues.apache.org/jira/browse/LUCENE-7700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dawid Weiss updated LUCENE-7700: Fix Version/s: 6.5 > Move throughput control and merge aborting out of IndexWriter's core? > - > > Key: LUCENE-7700 > URL: https://issues.apache.org/jira/browse/LUCENE-7700 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Minor > Fix For: 6.x, master (7.0), 6.5 > > Attachments: LUCENE-7700.patch, LUCENE-7700.patch, LUCENE-7700.patch > > > Here is a bit of a background: > - I wanted to implement a custom merging strategy that would have a custom > i/o flow control (global), > - currently, the CMS is tightly bound with a few classes -- MergeRateLimiter, > OneMerge, IndexWriter. > Looking at the code it seems to me that everything with respect to I/O > control could be nicely pulled out into classes that explicitly control the > merging process, that is only MergePolicy and MergeScheduler. By default, one > could even run without any additional I/O accounting overhead (which is > currently in there, even if one doesn't use the CMS's throughput control). > Such refactoring would also give a chance to nicely move things where they > belong -- job aborting into OneMerge (currently in RateLimiter), rate limiter > lifecycle bound to OneMerge (MergeScheduler could then use per-merge or > global accounting, as it pleases). > Just a thought and some initial refactorings for discussion. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Issue Comment Deleted] (SOLR-9959) SolrInfoMBean-s category and hierarchy cleanup
[
https://issues.apache.org/jira/browse/SOLR-9959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrzej Bialecki updated SOLR-9959:
Comment: was deleted
(was: There's also considerable duplication of code responsible for reporting
metrics in components that implement both {{SolrInfoMBean}} and
{{SolrMetricProducer}}. We should investigate whether
{{SolrInfoMBean.getStatistics()}} is needed at all - statistics reported by
this method theoretically can be equally well reported using the metrics API.)
> SolrInfoMBean-s category and hierarchy cleanup
> --
>
> Key: SOLR-9959
> URL: https://issues.apache.org/jira/browse/SOLR-9959
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: metrics
>Affects Versions: master (7.0)
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Blocker
> Fix For: master (7.0)
>
>
> SOLR-9947 changed categories of some of {{SolrInfoMBean-s}}, and it also
> added an alternative view in JMX, similar to the one produced by
> {{SolrJmxReporter}}.
> Some changes were left out from that issue because they would break the
> back-compatibility in 6.x, but they should be done before 7.0:
> * remove the old JMX view of {{SolrInfoMBean}}-s and improve the new one so
> that it's more readable and useful.
> * in many cases {{SolrInfoMBean.getName()}} just returns a FQCN, but it could
> be more informative - eg. for highlighter or query plugins this could be the
> symbolic name of a plugin that users know and use in configuration.
> * top-level categories need more thought. On one hand it's best to minimize
> their number, on the other hand they need to meaningfully represent the
> functionality of components that use them. SOLR-9947 made some cosmetic
> changes, but more discussion is necessary (eg. QUERY vs. SEARCHHANDLER)
> * we should consider removing some of the methods in {{SolrInfoMBean}} that
> usually don't return any useful information, eg. {{getDocs}}, {{getSource()}}
> and {{getVersion()}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7734) FieldType copy constructor should accept IndexableFieldType
[
https://issues.apache.org/jira/browse/LUCENE-7734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904404#comment-15904404
]
ASF subversion and git services commented on LUCENE-7734:
-
Commit d2bf30d58fbfc9279bed663500400153b4d4df44 in lucene-solr's branch
refs/heads/master from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d2bf30d ]
LUCENE-7734: FieldType copy constructor widened to IndexableFieldType
> FieldType copy constructor should accept IndexableFieldType
> ---
>
> Key: LUCENE-7734
> URL: https://issues.apache.org/jira/browse/LUCENE-7734
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Minor
> Attachments: LUCENE_7734.patch
>
>
> {{FieldType}} is a concrete implementation of {{IndexableFieldType}}. It has
> a copy-constructor but it demands a {{FieldType}}. It should accept
> {{IndexableFieldType}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7734) FieldType copy constructor should accept IndexableFieldType
[
https://issues.apache.org/jira/browse/LUCENE-7734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904415#comment-15904415
]
David Smiley commented on LUCENE-7734:
--
So I'm in the middle of a cherry pick from master to branch_6x when I see that,
surprise surprise, {{IndexableFieldType}} in 6x does not define
{{numericType()}} or {{numericPrecisionStep()}} (both deprecated things
associated with legacy numerics). I could add an {{instanceof FieldType}} and
cast to this copy constructor on 6x. Or, just let this be a 7x thing. I could
go either way but I'm inclined to just let it be 7x. If I don't hear any
opinion further then I'll simply move the changes.txt entry to 7.0 and not
touch 6x.
> FieldType copy constructor should accept IndexableFieldType
> ---
>
> Key: LUCENE-7734
> URL: https://issues.apache.org/jira/browse/LUCENE-7734
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Minor
> Attachments: LUCENE_7734.patch
>
>
> {{FieldType}} is a concrete implementation of {{IndexableFieldType}}. It has
> a copy-constructor but it demands a {{FieldType}}. It should accept
> {{IndexableFieldType}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-9-ea+159) - Build # 19135 - Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/19135/ Java: 64bit/jdk-9-ea+159 -XX:+UseCompressedOops -XX:+UseSerialGC 3 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.util.TestSolrCLIRunExample Error Message: ObjectTracker found 6 object(s) that were not released!!! [TransactionLog, SolrIndexSearcher, MockDirectoryWrapper, MockDirectoryWrapper, MDCAwareThreadPoolExecutor, MockDirectoryWrapper] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.update.TransactionLog at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.update.TransactionLog.(TransactionLog.java:188) at org.apache.solr.update.UpdateLog.newTransactionLog(UpdateLog.java:443) at org.apache.solr.update.UpdateLog.ensureLog(UpdateLog.java:1102) at org.apache.solr.update.UpdateLog.add(UpdateLog.java:529) at org.apache.solr.update.UpdateLog.add(UpdateLog.java:514) at org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:328) at org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:247) at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:202) at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:67) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55) at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:987) at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1200) at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:749) at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55) at org.apache.solr.update.processor.AddSchemaFieldsUpdateProcessorFactory$AddSchemaFieldsUpdateProcessor.processAdd(AddSchemaFieldsUpdateProcessorFactory.java:336) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55) at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55) at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55) at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55) at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55) at org.apache.solr.update.processor.FieldNameMutatingUpdateProcessorFactory$1.processAdd(FieldNameMutatingUpdateProcessorFactory.java:74) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55) at org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55) at org.apache.solr.update.processor.AbstractDefaultValueUpdateProcessorFactory$DefaultValueUpdateProcessor.processAdd(AbstractDefaultValueUpdateProcessorFactory.java:91) at org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:97) at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:186) at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readIterator(JavaBinUpdateRequestCodec.java:142) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:313) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:258) at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readNamedList(JavaBinUpdateRequestCodec.java:128) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:278) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:258) at org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:180) at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:193) at org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:107) at org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:54) at
[jira] [Updated] (SOLR-10217) Add a query for the background set to the significantTerms streaming expression
[ https://issues.apache.org/jira/browse/SOLR-10217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gethin James updated SOLR-10217: Attachment: SOLR-20217.patch Adding an initial patch for a significant terms background query. I will work next on improving the tests. > Add a query for the background set to the significantTerms streaming > expression > --- > > Key: SOLR-10217 > URL: https://issues.apache.org/jira/browse/SOLR-10217 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Gethin James > Attachments: SOLR-20217.patch > > > Following the work on SOLR-10156 we now have a significantTerms expression. > Currently, the background set is always the full index. It would be great if > we could use a query to define the background set. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10076) Hiding keystore and truststore passwords from /admin/info/* outputs
[
https://issues.apache.org/jira/browse/SOLR-10076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mano Kovacs updated SOLR-10076:
---
Attachment: SOLR-10076.patch
[~markrmil...@gmail.com], thank you for the review and comments!
- I added test for case-sensitive property name, it in fact was not properly
working.
- I changed the redaction text to the one that Greg added. Actually this patch
is the generalization of his original intent.
- I made the system property redaction configurable with default true. 6.x
backport only need to vary by the default value of that configuration to have
it turned off.
> Hiding keystore and truststore passwords from /admin/info/* outputs
> ---
>
> Key: SOLR-10076
> URL: https://issues.apache.org/jira/browse/SOLR-10076
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Mano Kovacs
>Assignee: Mark Miller
> Attachments: SOLR-10076.patch, SOLR-10076.patch
>
>
> Passing keystore and truststore password is done by system properties, via
> cmd line parameter.
> As result, {{/admin/info/properties}} and {{/admin/info/system}} will print
> out the received password.
> Proposing solution to automatically redact value of any system property
> before output, containing the word {{password}}, and replacing its value with
> {{**}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-10258) Facet functions emit date fields as ticks
Chris Eldredge created SOLR-10258:
-
Summary: Facet functions emit date fields as ticks
Key: SOLR-10258
URL: https://issues.apache.org/jira/browse/SOLR-10258
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: faceting
Affects Versions: 5.5.2
Reporter: Chris Eldredge
Priority: Minor
When invoking a facet function in the JSON Facet API, TrieDateField is coerced
into a numeric value and the result of the function is emitted as numeric
instead of being converted back into a formatted date.
Example:
curl
http://localhost:8983/solr/query?q=*:*={most_recent:'max(modified)'}
Produces (in part):
"facets":{
"count":38304,
"most_recent":1.489012400831E12}}
The "most_recent" attribute would be more useful if it was converted back to an
iso8601 formatted date.
There was a thread discussing this issue in 2016:
http://lucene.472066.n3.nabble.com/min-max-on-date-fields-using-JSON-facets-td4288736.html#a4288781
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-10257) Add logarithm StreamEvaluator
Joel Bernstein created SOLR-10257: - Summary: Add logarithm StreamEvaluator Key: SOLR-10257 URL: https://issues.apache.org/jira/browse/SOLR-10257 Project: Solr Issue Type: New Feature Security Level: Public (Default Security Level. Issues are Public) Reporter: Joel Bernstein To support custom scoring formulas with StreamEvaluators it would be useful to have a *log* evaluator that computes the natural log of a numeric value. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10247) Support non-numeric metrics
[
https://issues.apache.org/jira/browse/SOLR-10247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903144#comment-15903144
]
Shalin Shekhar Mangar commented on SOLR-10247:
--
# The patch removes the following gauges from UpdateLog: replayLogsCountGauge,
replayBytesGauge, stateGauge -- I think this is a mistake?
# nit - DirectUpdateHandlerTest has an unused import which will cause precommit
to fail
# In IndexWriter's constructor, while registering the gauge {{() ->
runningMajorMerges}} -- shouldn't that be {{() -> runningMajorMerges.get()}}?
and so on for all others
# I like the compact output -- we can add a version parameter to the
MetricsHandler, say version=2, that outputs the compact format -- this can
become the default in 7.0
> Support non-numeric metrics
> ---
>
> Key: SOLR-10247
> URL: https://issues.apache.org/jira/browse/SOLR-10247
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: metrics
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
> Fix For: 6.5, master (7.0)
>
> Attachments: compactFormat.png, currentFormat.png, SOLR-10247.patch
>
>
> Metrics API currently supports only numeric values. However, it's useful also
> to report non-numeric values such as eg. version, disk type, component state,
> some system properties, etc.
> Codahale {{Gauge}} metric type can be used for this purpose, and
> convenience methods can be added to {{SolrMetricManager}} to make it easier
> to use.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10217) Add a query for the background set to the significantTerms streaming expression
[ https://issues.apache.org/jira/browse/SOLR-10217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gethin James updated SOLR-10217: Attachment: (was: SOLR-20217.patch) > Add a query for the background set to the significantTerms streaming > expression > --- > > Key: SOLR-10217 > URL: https://issues.apache.org/jira/browse/SOLR-10217 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Gethin James > Attachments: SOLR-20217.patch > > > Following the work on SOLR-10156 we now have a significantTerms expression. > Currently, the background set is always the full index. It would be great if > we could use a query to define the background set. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10217) Add a query for the background set to the significantTerms streaming expression
[ https://issues.apache.org/jira/browse/SOLR-10217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gethin James updated SOLR-10217: Attachment: SOLR-20217.patch I added a new patch file. If you prefer a pull request let me know, I have the changes on a branch from master (https://github.com/covolution/lucene-solr/tree/SOLR-10217). > Add a query for the background set to the significantTerms streaming > expression > --- > > Key: SOLR-10217 > URL: https://issues.apache.org/jira/browse/SOLR-10217 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Gethin James > Attachments: SOLR-20217.patch > > > Following the work on SOLR-10156 we now have a significantTerms expression. > Currently, the background set is always the full index. It would be great if > we could use a query to define the background set. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10258) Facet functions emit date fields as ticks
[
https://issues.apache.org/jira/browse/SOLR-10258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yonik Seeley updated SOLR-10258:
Component/s: (was: faceting)
Facet Module
> Facet functions emit date fields as ticks
> -
>
> Key: SOLR-10258
> URL: https://issues.apache.org/jira/browse/SOLR-10258
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module
>Affects Versions: 5.5.2
>Reporter: Chris Eldredge
>Priority: Minor
>
> When invoking a facet function in the JSON Facet API, TrieDateField is
> coerced into a numeric value and the result of the function is emitted as
> numeric instead of being converted back into a formatted date.
> Example:
> curl
> http://localhost:8983/solr/query?q=*:*={most_recent:'max(modified)'}
> Produces (in part):
> "facets":{
> "count":38304,
> "most_recent":1.489012400831E12}}
> The "most_recent" attribute would be more useful if it was converted back to
> an iso8601 formatted date.
> There was a thread discussing this issue in 2016:
> http://lucene.472066.n3.nabble.com/min-max-on-date-fields-using-JSON-facets-td4288736.html#a4288781
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10259) admin/cores?action=STATUS returns 500 when a single core has init failures
[
https://issues.apache.org/jira/browse/SOLR-10259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Bates updated SOLR-10259:
Attachment: SOLR-10259.patch-2.txt
SOLR-10259.patch-1.txt
These are two simple patch ideas.
> admin/cores?action=STATUS returns 500 when a single core has init failures
> --
>
> Key: SOLR-10259
> URL: https://issues.apache.org/jira/browse/SOLR-10259
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 5.3
>Reporter: Oliver Bates
>Priority: Trivial
> Attachments: SOLR-10259.patch-1.txt, SOLR-10259.patch-2.txt
>
>
> When I have a healthy core on a node and I call
> solr/admin/cores?action=STATUS, I get the following healthy response:
> {quote}
>
>
> 0
> 1607
>
>
>
>
> whoisbanana_shard1_replica1
>
> /tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/
>
>
> /tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/
>
> solrconfig.xml
> schema.xml
> 2017-03-08T15:59:50.18Z
> 380431
> active
> 0
>
> 0
> 0
> 0
> 0
> 2
> 0
> true
> false
>
> org.apache.lucene.store.NRTCachingDirectory:NRTCachingDirectory(MMapDirectory@/tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/index
> lockFactory=org.apache.lucene.store.NativeFSLockFactory@762404a0;
> maxCacheMB=48.0 maxMergeSizeMB=4.0)
>
>
> 71
> 71 bytes
>
>
>
>
> {quote}
> If I then corrupt the index file and reload, e.g. like this:
> echo "cheese" >>
> /tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/index/segments_1
> And then I call the same endpoint (solr/admin/cores?action=STATUS), I get a
> 500 back:
> {quote}
>
>
> 500
> 1508
>
>
> Error handling 'status' action
>
> org.apache.solr.common.SolrException: Error handling 'status' action at
> org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:755)
> at
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:231)
> at
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:196)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:146)
> at
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:676)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:443) at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:210)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at com.apple.cie.search.auth.TrustFilter.doFilter(TrustFilter.java:44) at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at com.apple.cie.search.id.IdFilter.doFilter(IdFilter.java:38) at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.eclipse.jetty.server.Server.handle(Server.java:499) at
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> at
[jira] [Updated] (SOLR-10259) admin/cores?action=STATUS returns 500 when a single core has init failures
[
https://issues.apache.org/jira/browse/SOLR-10259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Bates updated SOLR-10259:
Description:
When I have a healthy core on a node and I call solr/admin/cores?action=STATUS,
I get the following healthy response:
{quote}
0
1607
whoisbanana_shard1_replica1
/tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/
/tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/
solrconfig.xml
schema.xml
2017-03-08T15:59:50.18Z
380431
active
0
0
0
0
0
2
0
true
false
org.apache.lucene.store.NRTCachingDirectory:NRTCachingDirectory(MMapDirectory@/tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@762404a0;
maxCacheMB=48.0 maxMergeSizeMB=4.0)
71
71 bytes
{quote}
If I then corrupt the index file and reload, e.g. like this:
echo "cheese" >>
/tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/index/segments_1
And then I call the same endpoint (solr/admin/cores?action=STATUS), I get a 500
back:
{quote}
500
1508
Error handling 'status' action
org.apache.solr.common.SolrException: Error handling 'status' action at
org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:755)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:231)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:196)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:146)
at
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:676)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:443) at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:210)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at com.apple.cie.search.auth.TrustFilter.doFilter(TrustFilter.java:44) at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at com.apple.cie.search.id.IdFilter.doFilter(IdFilter.java:38) at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499) at
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) at
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745) Caused by:
org.apache.lucene.index.CorruptIndexException: misplaced codec footer (file
extended?): remaining=23, expected=16
(resource=BufferedChecksumIndexInput(MMapIndexInput(path="/tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/index/segments_1")))
at org.apache.lucene.codecs.CodecUtil.validateFooter(CodecUtil.java:411) at
org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:331) at
org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:442) at
org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:493) at
org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:490) at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:731)
at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:683)
at
[jira] [Updated] (SOLR-10259) admin/cores?action=STATUS returns 500 when a single core has init failures
[ https://issues.apache.org/jira/browse/SOLR-10259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Bates updated SOLR-10259: Description: When I have a healthy core on a node and I call solr/admin/cores?action=STATUS, I get the following healthy response: 0 1607 whoisbanana_shard1_replica1 /tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/ /tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/ solrconfig.xml schema.xml 2017-03-08T15:59:50.18Z 380431 active 0 0 0 0 0 2 0 true false org.apache.lucene.store.NRTCachingDirectory:NRTCachingDirectory(MMapDirectory@/tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@762404a0; maxCacheMB=48.0 maxMergeSizeMB=4.0) 71 71 bytes If I then corrupt the index file and reload, e.g. like this: echo "cheese" >> /tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/index/segments_1 And then I call the same endpoint (solr/admin/cores?action=STATUS), I get a 500 back: 500 1508 Error handling 'status' action org.apache.solr.common.SolrException: Error handling 'status' action at org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:755) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:231) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:196) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:146) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:676) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:443) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:210) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at com.apple.cie.search.auth.TrustFilter.doFilter(TrustFilter.java:44) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at com.apple.cie.search.id.IdFilter.doFilter(IdFilter.java:38) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.lucene.index.CorruptIndexException: misplaced codec footer (file extended?): remaining=23, expected=16 (resource=BufferedChecksumIndexInput(MMapIndexInput(path="/tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/index/segments_1"))) at org.apache.lucene.codecs.CodecUtil.validateFooter(CodecUtil.java:411) at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:331) at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:442) at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:493) at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:490) at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:731) at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:683) at
[jira] [Commented] (SOLR-5872) Eliminate overseer queue
[ https://issues.apache.org/jira/browse/SOLR-5872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903557#comment-15903557 ] Shawn Heisey commented on SOLR-5872: bq. We were doing our tests with solr 6.4.1 That version has other problems that may be clouding the issue (pun intended). See SOLR-10130. I advise an immediate upgrade to 6.4.2, to be sure that any problems you're encountering are actually caused by SolrCloud, not high CPU usage. > Eliminate overseer queue > - > > Key: SOLR-5872 > URL: https://issues.apache.org/jira/browse/SOLR-5872 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Reporter: Noble Paul >Assignee: Noble Paul > > The overseer queue is one of the busiest points in the entire system. The > raison d'être of the queue is > * Provide batching of operations for the main clusterstate,json so that > state updates are minimized > * Avoid race conditions and ensure order > Now , as we move the individual collection states out of the main > clusterstate.json, the batching is not useful anymore. > Race conditions can easily be solved by using a compare and set in Zookeeper. > The proposed solution is , whenever an operation is required to be performed > on the clusterstate, the same thread (and of course the same JVM) > # read the fresh state and version of zk node > # construct the new state > # perform a compare and set > # if compare and set fails go to step 1 > This should be limited to all operations performed on external collections > because batching would be required for others -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-10259) admin/cores?action=STATUS returns 500 when a single core has init failures
Oliver Bates created SOLR-10259: --- Summary: admin/cores?action=STATUS returns 500 when a single core has init failures Key: SOLR-10259 URL: https://issues.apache.org/jira/browse/SOLR-10259 Project: Solr Issue Type: Bug Security Level: Public (Default Security Level. Issues are Public) Affects Versions: 5.3 Reporter: Oliver Bates Priority: Trivial When I have a healthy core on a node and I call solr/admin/cores?action=STATUS, I get the following: # Healthy response 0 1607 whoisbanana_shard1_replica1 /tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/ /tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/ solrconfig.xml schema.xml 2017-03-08T15:59:50.18Z 380431 active 0 0 0 0 0 2 0 true false org.apache.lucene.store.NRTCachingDirectory:NRTCachingDirectory(MMapDirectory@/tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@762404a0; maxCacheMB=48.0 maxMergeSizeMB=4.0) 71 71 bytes ### If I then corrupt the index file and reload, e.g. like this: echo "cheese" >> /tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/index/segments_1 And then I call the same endpoint (solr/admin/cores?action=STATUS), I get a 500 back: Corrupted response 500 1508 Error handling 'status' action org.apache.solr.common.SolrException: Error handling 'status' action at org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:755) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:231) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:196) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:146) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:676) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:443) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:210) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at com.apple.cie.search.auth.TrustFilter.doFilter(TrustFilter.java:44) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at com.apple.cie.search.id.IdFilter.doFilter(IdFilter.java:38) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.lucene.index.CorruptIndexException: misplaced codec footer (file extended?): remaining=23, expected=16 (resource=BufferedChecksumIndexInput(MMapIndexInput(path="/tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/index/segments_1"))) at org.apache.lucene.codecs.CodecUtil.validateFooter(CodecUtil.java:411) at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:331) at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:442) at
[jira] [Commented] (SOLR-10217) Add a query for the background set to the significantTerms streaming expression
[ https://issues.apache.org/jira/browse/SOLR-10217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903656#comment-15903656 ] Dennis Gove commented on SOLR-10217: What's the reasoning behind having SignificantTermsStream accept a query instead of an incoming stream? If it accepted an incoming stream then the source data could be the result of any other stream whereas with a query the source data can only be the result of a Solr query. > Add a query for the background set to the significantTerms streaming > expression > --- > > Key: SOLR-10217 > URL: https://issues.apache.org/jira/browse/SOLR-10217 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Gethin James > Attachments: SOLR-20217.patch > > > Following the work on SOLR-10156 we now have a significantTerms expression. > Currently, the background set is always the full index. It would be great if > we could use a query to define the background set. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10259) admin/cores?action=STATUS returns 500 when a single core has init failures
[
https://issues.apache.org/jira/browse/SOLR-10259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Bates updated SOLR-10259:
Description:
When I have a healthy core on a node and I call solr/admin/cores?action=STATUS,
I get the following healthy response:
{quote}
0
1607
whoisbanana_shard1_replica1
/tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/
/tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/
solrconfig.xml
schema.xml
2017-03-08T15:59:50.18Z
380431
active
0
0
0
0
0
2
0
true
false
org.apache.lucene.store.NRTCachingDirectory:NRTCachingDirectory(MMapDirectory@/tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/index
lockFactory=org.apache.lucene.store.NativeFSLockFactory@762404a0;
maxCacheMB=48.0 maxMergeSizeMB=4.0)
71
71 bytes
{quote}
If I then corrupt the index file and reload, e.g. like this:
echo "cheese" >>
/tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/index/segments_1
And then I call the same endpoint (solr/admin/cores?action=STATUS), I get a 500
back:
{quote}
500
1508
Error handling 'status' action
org.apache.solr.common.SolrException: Error handling 'status' action at
org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:755)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:231)
at
org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:196)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:146)
at
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:676)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:443) at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:210)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at com.apple.cie.search.auth.TrustFilter.doFilter(TrustFilter.java:44) at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at com.apple.cie.search.id.IdFilter.doFilter(IdFilter.java:38) at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499) at
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) at
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745) Caused by:
org.apache.lucene.index.CorruptIndexException: misplaced codec footer (file
extended?): remaining=23, expected=16
(resource=BufferedChecksumIndexInput(MMapIndexInput(path="/tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/index/segments_1")))
at org.apache.lucene.codecs.CodecUtil.validateFooter(CodecUtil.java:411) at
org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:331) at
org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:442) at
org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:493) at
org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:490) at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:731)
at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:683)
at
[jira] [Updated] (SOLR-10259) admin/cores?action=STATUS returns 500 when a single core has init failures
[ https://issues.apache.org/jira/browse/SOLR-10259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Bates updated SOLR-10259: Description: When I have a healthy core on a node and I call solr/admin/cores?action=STATUS, I get the following healthy response: 0 1607 whoisbanana_shard1_replica1 /tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/ /tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/ solrconfig.xml schema.xml 2017-03-08T15:59:50.18Z 380431 active 0 0 0 0 0 2 0 true false org.apache.lucene.store.NRTCachingDirectory:NRTCachingDirectory(MMapDirectory@/tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@762404a0; maxCacheMB=48.0 maxMergeSizeMB=4.0) 71 71 bytes If I then corrupt the index file and reload, e.g. like this: echo "cheese" >> /tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/index/segments_1 And then I call the same endpoint (solr/admin/cores?action=STATUS), I get a 500 back: 500 1508 Error handling 'status' action org.apache.solr.common.SolrException: Error handling 'status' action at org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:755) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestInternal(CoreAdminHandler.java:231) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:196) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:146) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:676) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:443) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:210) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at com.apple.cie.search.auth.TrustFilter.doFilter(TrustFilter.java:44) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at com.apple.cie.search.id.IdFilter.doFilter(IdFilter.java:38) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:499) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.lucene.index.CorruptIndexException: misplaced codec footer (file extended?): remaining=23, expected=16 (resource=BufferedChecksumIndexInput(MMapIndexInput(path="/tmp/search_integration_test/solr1/whoisbanana_shard1_replica1/data/index/segments_1"))) at org.apache.lucene.codecs.CodecUtil.validateFooter(CodecUtil.java:411) at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:331) at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:442) at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:493) at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:490) at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:731) at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:683) at
[JENKINS] Lucene-Solr-6.4-Windows (64bit/jdk1.8.0_121) - Build # 27 - Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.4-Windows/27/ Java: 64bit/jdk1.8.0_121 -XX:+UseCompressedOops -XX:+UseParallelGC 1 tests failed. FAILED: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI Error Message: No live SolrServers available to handle this request:[http://127.0.0.1:60801/solr/awhollynewcollection_0] Stack Trace: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:60801/solr/awhollynewcollection_0] at __randomizedtesting.SeedInfo.seed([D229BD309379F302:9A5CC984954ADC97]:0) at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:414) at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1347) at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1098) at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1037) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149) at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942) at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:516) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at
[jira] [Created] (SOLR-10260) Collection not found with count(*) and uppercase name
Jordan Diehl created SOLR-10260: --- Summary: Collection not found with count(*) and uppercase name Key: SOLR-10260 URL: https://issues.apache.org/jira/browse/SOLR-10260 Project: Solr Issue Type: Bug Security Level: Public (Default Security Level. Issues are Public) Components: Parallel SQL Affects Versions: 6.4.1 Reporter: Jordan Diehl Priority: Minor I'm on solr 6.4.1 and was testing out using Zeppelin with Solr. I found that Parallel SQL queries using aggregate functions do not seem to work with collections which have capital letters in the name. When searching around for information on this I found [this thread|http://lucene.472066.n3.nabble.com/JDBC-Collection-not-found-with-count-and-uppercase-name-td4287590.html] where someone else encountered the same issue and was told to open a bug on it, but it doesn't look like they ever did. The thread has an example query and stack trace. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10257) Add logarithm StreamEvaluator
[ https://issues.apache.org/jira/browse/SOLR-10257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Bernstein updated SOLR-10257: -- Attachment: SOLR-10257.patch > Add logarithm StreamEvaluator > - > > Key: SOLR-10257 > URL: https://issues.apache.org/jira/browse/SOLR-10257 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Joel Bernstein > Attachments: SOLR-10257.patch > > > To support custom scoring formulas with StreamEvaluators it would be useful > to have a *log* evaluator that computes the natural log of a numeric value. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7734) FieldType copy constructor should accept IndexableFieldType
[
https://issues.apache.org/jira/browse/LUCENE-7734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904063#comment-15904063
]
Adrien Grand commented on LUCENE-7734:
--
True, but this change is API compatible, so if something compiles today, it
will still compile with your patch. I think the only thing it breaks is ABI
compatibility so it would not be possible eg. to replace the Lucene JAR without
recompiling. But this is not something we guarantee anyway, I don't think we
should add an additional constructor on 6.x.
> FieldType copy constructor should accept IndexableFieldType
> ---
>
> Key: LUCENE-7734
> URL: https://issues.apache.org/jira/browse/LUCENE-7734
> Project: Lucene - Core
> Issue Type: Improvement
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Minor
> Attachments: LUCENE_7734.patch
>
>
> {{FieldType}} is a concrete implementation of {{IndexableFieldType}}. It has
> a copy-constructor but it demands a {{FieldType}}. It should accept
> {{IndexableFieldType}}.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5872) Eliminate overseer queue
[ https://issues.apache.org/jira/browse/SOLR-5872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903961#comment-15903961 ] albert vico oton commented on SOLR-5872: thanks for the advice, I'll take a look but the problem we were seeing was in Zookeeper cluster not in solr > Eliminate overseer queue > - > > Key: SOLR-5872 > URL: https://issues.apache.org/jira/browse/SOLR-5872 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Reporter: Noble Paul >Assignee: Noble Paul > > The overseer queue is one of the busiest points in the entire system. The > raison d'être of the queue is > * Provide batching of operations for the main clusterstate,json so that > state updates are minimized > * Avoid race conditions and ensure order > Now , as we move the individual collection states out of the main > clusterstate.json, the batching is not useful anymore. > Race conditions can easily be solved by using a compare and set in Zookeeper. > The proposed solution is , whenever an operation is required to be performed > on the clusterstate, the same thread (and of course the same JVM) > # read the fresh state and version of zk node > # construct the new state > # perform a compare and set > # if compare and set fails go to step 1 > This should be limited to all operations performed on external collections > because batching would be required for others -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10217) Add a query for the background set to the significantTerms streaming expression
[ https://issues.apache.org/jira/browse/SOLR-10217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904049#comment-15904049 ] Joel Bernstein commented on SOLR-10217: --- Gethin and I had discussed using stream for the background query. But it just seemed like the right approach in this scenario to make it a query. The algorithm is written with an AnalyticsQuery and works directly with the index, so collecting a bitset for the background query scales really well. Using a stream we would have to feed docID's back into the AnalyticsQuery to get the background set, just seemed a lot less scalable. > Add a query for the background set to the significantTerms streaming > expression > --- > > Key: SOLR-10217 > URL: https://issues.apache.org/jira/browse/SOLR-10217 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Gethin James > Attachments: SOLR-20217.patch > > > Following the work on SOLR-10156 we now have a significantTerms expression. > Currently, the background set is always the full index. It would be great if > we could use a query to define the background set. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-10217) Add a query for the background set to the significantTerms streaming expression
[ https://issues.apache.org/jira/browse/SOLR-10217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904049#comment-15904049 ] Joel Bernstein edited comment on SOLR-10217 at 3/9/17 11:19 PM: Gethin and I had discussed using a stream for the background query. But it just seemed like the right approach in this scenario was to make it a query. The algorithm is written with an AnalyticsQuery and works directly with the index, so collecting a bitset for the background query scales really well. Using a stream we would have to feed docID's back into the AnalyticsQuery to get the background set, just seemed a lot less scalable. was (Author: joel.bernstein): Gethin and I had discussed using stream for the background query. But it just seemed like the right approach in this scenario to make it a query. The algorithm is written with an AnalyticsQuery and works directly with the index, so collecting a bitset for the background query scales really well. Using a stream we would have to feed docID's back into the AnalyticsQuery to get the background set, just seemed a lot less scalable. > Add a query for the background set to the significantTerms streaming > expression > --- > > Key: SOLR-10217 > URL: https://issues.apache.org/jira/browse/SOLR-10217 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Gethin James > Attachments: SOLR-20217.patch > > > Following the work on SOLR-10156 we now have a significantTerms expression. > Currently, the background set is always the full index. It would be great if > we could use a query to define the background set. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-7735) Suffix compression for points
Adrien Grand created LUCENE-7735: Summary: Suffix compression for points Key: LUCENE-7735 URL: https://issues.apache.org/jira/browse/LUCENE-7735 Project: Lucene - Core Issue Type: Bug Reporter: Adrien Grand Priority: Minor Points already compress fairly well based on shared prefixes. It would be nice if points could also compress suffixes similarly to how doc values perform GCD compression. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10217) Add a query for the background set to the significantTerms streaming expression
[ https://issues.apache.org/jira/browse/SOLR-10217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Bernstein updated SOLR-10217: -- Attachment: SOLR-10257.patch > Add a query for the background set to the significantTerms streaming > expression > --- > > Key: SOLR-10217 > URL: https://issues.apache.org/jira/browse/SOLR-10217 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Gethin James > Attachments: SOLR-10257.patch, SOLR-20217.patch > > > Following the work on SOLR-10156 we now have a significantTerms expression. > Currently, the background set is always the full index. It would be great if > we could use a query to define the background set. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10257) Add logarithm StreamEvaluator
[ https://issues.apache.org/jira/browse/SOLR-10257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Bernstein updated SOLR-10257: -- Attachment: SOLR-10257.patch > Add logarithm StreamEvaluator > - > > Key: SOLR-10257 > URL: https://issues.apache.org/jira/browse/SOLR-10257 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Joel Bernstein > Attachments: SOLR-10257.patch, SOLR-10257.patch > > > To support custom scoring formulas with StreamEvaluators it would be useful > to have a *log* evaluator that computes the natural log of a numeric value. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10217) Add a query for the background set to the significantTerms streaming expression
[ https://issues.apache.org/jira/browse/SOLR-10217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Bernstein updated SOLR-10217: -- Attachment: (was: SOLR-10257.patch) > Add a query for the background set to the significantTerms streaming > expression > --- > > Key: SOLR-10217 > URL: https://issues.apache.org/jira/browse/SOLR-10217 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Gethin James > Attachments: SOLR-20217.patch > > > Following the work on SOLR-10156 we now have a significantTerms expression. > Currently, the background set is always the full index. It would be great if > we could use a query to define the background set. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10257) Add logarithm StreamEvaluator
[ https://issues.apache.org/jira/browse/SOLR-10257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Bernstein updated SOLR-10257: -- Attachment: SOLR-10257.patch > Add logarithm StreamEvaluator > - > > Key: SOLR-10257 > URL: https://issues.apache.org/jira/browse/SOLR-10257 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Joel Bernstein > Attachments: SOLR-10257.patch, SOLR-10257.patch, SOLR-10257.patch > > > To support custom scoring formulas with StreamEvaluators it would be useful > to have a *log* evaluator that computes the natural log of a numeric value. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10184) bin/solr fails to run on java9 due to unrecognized GC options
[
https://issues.apache.org/jira/browse/SOLR-10184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man updated SOLR-10184:
Attachment: SOLR-10184.patch
thanks for the links Uwe, JEP 158 is interesting.
Between the GC logging options that have been removed, and how the unified
logging now deals with multiple tags (gc, all, etc..) & output specifiers
(stdout, stderr, for file) in a single '-Xlog' param; it doesn't look viable to
have an _exact_ java9 equivilent for what the {{bin/solr}} script currently
does with {{SOLR_LOGS_DIR}} & {{GC_LOG_OPTS}}
The attached patch attempts to support the same ideas as best as (i can see)
possible under java9...
* if the user doesn't configure a {{GC_LOG_OPTS}}, then we (still) use a
(verbose) default value for this param via '-Xlog:gc\*'
* if the user configures a blank value for {{GC_LOG_OPTS}}, we (still) leave
the param blank
* if the effective value of {{GC_LOG_OPTS}} is non blank, then foreach param
option specified in {{GC_LOG_OPTS}}:
** if it starts with '-Xlog:gc', and does not include an output specifier, then:
*** add a 'file' output specifier based on {{SOLR_LOGS_DIR}} (to mimic the
'-Xloggc' param we use w/ java8)
*** add {{time,uptime}} decorators (to mimic the -XX params we use w/ java8)
*** add {{filecount=9,filesize=2}} output-options (to mimic the -XX params
we use w/ java8)
(for simplicity, if the user wants to specify multiple JEP158 tags in the
'-Xlog' param , then the tag list has to start with "gc" in order for us to add
a file option -- otherwise we ignore it. likewise if the user wants to specify
their own decorators or output-options, then they must also specify their own
output (file, or stdout, or stderr) specifier as well, since it must come first
in the '-Xlog' format string)
While fixing this, I also did some small cleanup in how the {{java -version}}
output is parsed.
Would appreciate detailed review & manual testing from folks since this type of
script work is hard to write automated tests for.
> bin/solr fails to run on java9 due to unrecognized GC options
> -
>
> Key: SOLR-10184
> URL: https://issues.apache.org/jira/browse/SOLR-10184
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: scripts and tools
>Reporter: Hoss Man
> Labels: java9
> Attachments: SOLR-10184.patch
>
>
> {noformat}
> hossman@tray:~/lucene/dev/solr [master] $ bin/solr start -f
> Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was
> deprecated in version 9.0 and will likely be removed in a future release.
> Java HotSpot(TM) 64-Bit Server VM warning: Option UseParNewGC was deprecated
> in version 9.0 and will likely be removed in a future release.
> Unrecognized VM option 'PrintHeapAtGC'
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> {noformat}
> (untested) workaround is to override GC_LOG_OPTS in {{solr.in.sh}}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10262) Collect request latency metrics for histograms
[
https://issues.apache.org/jira/browse/SOLR-10262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904588#comment-15904588
]
Andrzej Bialecki commented on SOLR-10262:
--
Thanks Otis :) Yes, I'm aware of that implementation and the discussion around
it. It's an easy fix to replace the ExponentiallyDecayingReservoir that metrics
use by default with HdrHistogram, there's even a pull request against Codahale
repo with an implementation.
By "making it possible" did you mean that we make it the new default, but we
also allow people to use a different implementation? This would have to be
configured early on in {{solr.xml}} or even via system properties, which is a
bit ugly.
> Collect request latency metrics for histograms
> --
>
> Key: SOLR-10262
> URL: https://issues.apache.org/jira/browse/SOLR-10262
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: metrics
>Reporter: Otis Gospodnetic
>
> Since [~ab] is on a role with metrics...
> There is no way to accurately compute request latency percentiles from
> metrics exposed by Solr today. We should consider making that possible. c.f.
> https://github.com/HdrHistogram/HdrHistogram
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-10261) TestStressCloudBlindAtomicUpdates.test_dv() fail
Cao Manh Dat created SOLR-10261:
---
Summary: TestStressCloudBlindAtomicUpdates.test_dv() fail
Key: SOLR-10261
URL: https://issues.apache.org/jira/browse/SOLR-10261
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Reporter: Cao Manh Dat
I found a reproducing seed that cause
TestStressCloudBlindAtomicUpdates.test_dv() fail
{code}
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestStressCloudBlindAtomicUpdates -Dtests.method=test_dv
-Dtests.seed=AD8E7B56D53B627F -Dtests.nightly=true -Dtests.slow=true
-Dtests.locale=bg -Dtests.timezone=America/La_Paz -Dtests.asserts=true
-Dtests.file.encoding=UTF-8
[junit4] ERROR 1.21s J2 | TestStressCloudBlindAtomicUpdates.test_dv <<<
[junit4]> Throwable #1: java.util.concurrent.ExecutionException:
java.lang.RuntimeException: Error from server at
http://127.0.0.1:49825/solr/test_col: Async exception during distributed
update: Error from server at
http://127.0.0.1:49824/solr/test_col_shard2_replica2: Server Error
[junit4]> request:
http://127.0.0.1:49824/solr/test_col_shard2_replica2/update?update.distrib=TOLEADER=http%3A%2F%2F127.0.0.1%3A49825%2Fsolr%2Ftest_col_shard5_replica1%2F=javabin=2
[junit4]> Remote error message: Failed synchronous update on shard
StdNode: http://127.0.0.1:49836/solr/test_col_shard2_replica1/ update:
org.apache.solr.client.solrj.request.UpdateRequest@5919dfb3
[junit4]>at
__randomizedtesting.SeedInfo.seed([AD8E7B56D53B627F:9B9A19105F66586E]:0)
[junit4]>at
java.util.concurrent.FutureTask.report(FutureTask.java:122)
[junit4]>at
java.util.concurrent.FutureTask.get(FutureTask.java:192)
[junit4]>at
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:281)
[junit4]>at
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv(TestStressCloudBlindAtomicUpdates.java:193)
[junit4]>at java.lang.Thread.run(Thread.java:745)
[junit4]> Caused by: java.lang.RuntimeException: Error from server at
http://127.0.0.1:49825/solr/test_col: Async exception during distributed
update: Error from server at
http://127.0.0.1:49824/solr/test_col_shard2_replica2: Server Error
[junit4]> request:
http://127.0.0.1:49824/solr/test_col_shard2_replica2/update?update.distrib=TOLEADER=http%3A%2F%2F127.0.0.1%3A49825%2Fsolr%2Ftest_col_shard5_replica1%2F=javabin=2
[junit4]> Remote error message: Failed synchronous update on shard
StdNode: http://127.0.0.1:49836/solr/test_col_shard2_replica1/ update:
org.apache.solr.client.solrj.request.UpdateRequest@5919dfb3
[junit4]>at
org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:409)
[junit4]>at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4]>at
java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4]>at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
[junit4]>at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4]>at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4]>... 1 more
[junit4]> Caused by:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error
from server at http://127.0.0.1:49825/solr/test_col: Async exception during
distributed update: Error from server at
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10257) Add logarithm StreamEvaluator
[ https://issues.apache.org/jira/browse/SOLR-10257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904267#comment-15904267 ] ASF subversion and git services commented on SOLR-10257: Commit b0a973e083ef9750acc69600480212f391448f6a in lucene-solr's branch refs/heads/branch_6x from [~joel.bernstein] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b0a973e ] SOLR-10257: Add logarithm StreamEvaluator > Add logarithm StreamEvaluator > - > > Key: SOLR-10257 > URL: https://issues.apache.org/jira/browse/SOLR-10257 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Joel Bernstein > Attachments: SOLR-10257.patch, SOLR-10257.patch, SOLR-10257.patch > > > To support custom scoring formulas with StreamEvaluators it would be useful > to have a *log* evaluator that computes the natural log of a numeric value. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10257) Add logarithm StreamEvaluator
[ https://issues.apache.org/jira/browse/SOLR-10257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15904255#comment-15904255 ] ASF subversion and git services commented on SOLR-10257: Commit d945a246f6071699790119f07a66fb4c5505cee2 in lucene-solr's branch refs/heads/master from [~joel.bernstein] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d945a24 ] SOLR-10257: Add logarithm StreamEvaluator > Add logarithm StreamEvaluator > - > > Key: SOLR-10257 > URL: https://issues.apache.org/jira/browse/SOLR-10257 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Joel Bernstein > Attachments: SOLR-10257.patch, SOLR-10257.patch, SOLR-10257.patch > > > To support custom scoring formulas with StreamEvaluators it would be useful > to have a *log* evaluator that computes the natural log of a numeric value. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-10262) Collect request latency metrics for histograms
Otis Gospodnetic created SOLR-10262: --- Summary: Collect request latency metrics for histograms Key: SOLR-10262 URL: https://issues.apache.org/jira/browse/SOLR-10262 Project: Solr Issue Type: New Feature Security Level: Public (Default Security Level. Issues are Public) Components: metrics Reporter: Otis Gospodnetic Since [~ab] is on a role with metrics... There is no way to accurately compute request latency percentiles from metrics exposed by Solr today. We should consider making that possible. c.f. https://github.com/HdrHistogram/HdrHistogram -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1183 - Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1183/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC
1 tests failed.
FAILED:
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Error from server at http://127.0.0.1:48825/solr/awhollynewcollection_0:
Expected mime type application/octet-stream but got text/html.
Error 510HTTP ERROR: 510 Problem
accessing /solr/awhollynewcollection_0/select. Reason:
{metadata={error-class=org.apache.solr.common.SolrException,root-error-class=org.apache.solr.common.SolrException},msg={awhollynewcollection_0:6},code=510}
http://eclipse.org/jetty;>Powered by Jetty://
9.3.14.v20161028
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error
from server at http://127.0.0.1:48825/solr/awhollynewcollection_0: Expected
mime type application/octet-stream but got text/html.
Error 510
HTTP ERROR: 510
Problem accessing /solr/awhollynewcollection_0/select. Reason:
{metadata={error-class=org.apache.solr.common.SolrException,root-error-class=org.apache.solr.common.SolrException},msg={awhollynewcollection_0:6},code=510}
http://eclipse.org/jetty;>Powered by Jetty://
9.3.14.v20161028
at
__randomizedtesting.SeedInfo.seed([EF11849929B5888A:A764F02D2F86A71F]:0)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:595)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:439)
at
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:391)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1361)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1112)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1215)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1215)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1215)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1215)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1215)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1042)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:523)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at
[jira] [Commented] (LUCENE-7700) Move throughput control and merge aborting out of IndexWriter's core?
[ https://issues.apache.org/jira/browse/LUCENE-7700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903178#comment-15903178 ] Michael McCandless commented on LUCENE-7700: I think it's fine to backport to 6.x. > Move throughput control and merge aborting out of IndexWriter's core? > - > > Key: LUCENE-7700 > URL: https://issues.apache.org/jira/browse/LUCENE-7700 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Minor > Attachments: LUCENE-7700.patch, LUCENE-7700.patch, LUCENE-7700.patch > > > Here is a bit of a background: > - I wanted to implement a custom merging strategy that would have a custom > i/o flow control (global), > - currently, the CMS is tightly bound with a few classes -- MergeRateLimiter, > OneMerge, IndexWriter. > Looking at the code it seems to me that everything with respect to I/O > control could be nicely pulled out into classes that explicitly control the > merging process, that is only MergePolicy and MergeScheduler. By default, one > could even run without any additional I/O accounting overhead (which is > currently in there, even if one doesn't use the CMS's throughput control). > Such refactoring would also give a chance to nicely move things where they > belong -- job aborting into OneMerge (currently in RateLimiter), rate limiter > lifecycle bound to OneMerge (MergeScheduler could then use per-merge or > global accounting, as it pleases). > Just a thought and some initial refactorings for discussion. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-6.4-Linux (64bit/jdk1.8.0_121) - Build # 166 - Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.4-Linux/166/
Java: 64bit/jdk1.8.0_121 -XX:+UseCompressedOops -XX:+UseSerialGC
2 tests failed.
FAILED: org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test
Error Message:
Expected 2 of 3 replicas to be active but only found 1;
[core_node3:{"core":"c8n_1x3_lf_shard1_replica3","base_url":"http://127.0.0.1:44068","node_name":"127.0.0.1:44068_","state":"active","leader":"true"}];
clusterState:
DocCollection(c8n_1x3_lf//collections/c8n_1x3_lf/state.json/17)={
"replicationFactor":"3", "shards":{"shard1":{
"range":"8000-7fff", "state":"active", "replicas":{
"core_node1":{ "core":"c8n_1x3_lf_shard1_replica2",
"base_url":"http://127.0.0.1:36373;, "node_name":"127.0.0.1:36373_",
"state":"down"}, "core_node2":{ "state":"down",
"base_url":"http://127.0.0.1:38232;,
"core":"c8n_1x3_lf_shard1_replica1", "node_name":"127.0.0.1:38232_"},
"core_node3":{ "core":"c8n_1x3_lf_shard1_replica3",
"base_url":"http://127.0.0.1:44068;, "node_name":"127.0.0.1:44068_",
"state":"active", "leader":"true",
"router":{"name":"compositeId"}, "maxShardsPerNode":"1",
"autoAddReplicas":"false"}
Stack Trace:
java.lang.AssertionError: Expected 2 of 3 replicas to be active but only found
1;
[core_node3:{"core":"c8n_1x3_lf_shard1_replica3","base_url":"http://127.0.0.1:44068","node_name":"127.0.0.1:44068_","state":"active","leader":"true"}];
clusterState: DocCollection(c8n_1x3_lf//collections/c8n_1x3_lf/state.json/17)={
"replicationFactor":"3",
"shards":{"shard1":{
"range":"8000-7fff",
"state":"active",
"replicas":{
"core_node1":{
"core":"c8n_1x3_lf_shard1_replica2",
"base_url":"http://127.0.0.1:36373;,
"node_name":"127.0.0.1:36373_",
"state":"down"},
"core_node2":{
"state":"down",
"base_url":"http://127.0.0.1:38232;,
"core":"c8n_1x3_lf_shard1_replica1",
"node_name":"127.0.0.1:38232_"},
"core_node3":{
"core":"c8n_1x3_lf_shard1_replica3",
"base_url":"http://127.0.0.1:44068;,
"node_name":"127.0.0.1:44068_",
"state":"active",
"leader":"true",
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false"}
at
__randomizedtesting.SeedInfo.seed([B7A4D9D9CA77C0CF:3FF0E603648BAD37]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:168)
at
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1258 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1258/ 2 tests failed. FAILED: org.apache.lucene.search.join.TestBlockJoin.testMultiChildQueriesOfDiffParentLevels Error Message: Java heap space Stack Trace: java.lang.OutOfMemoryError: Java heap space at __randomizedtesting.SeedInfo.seed([49607EEF4FD70C96:98856123AF41F5F0]:0) at java.util.Arrays.copyOf(Arrays.java:3308) at org.apache.lucene.codecs.compressing.CompressingStoredFieldsIndexReader.(CompressingStoredFieldsIndexReader.java:77) at org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.(CompressingStoredFieldsReader.java:134) at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:121) at org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:126) at org.apache.lucene.index.SegmentReader.(SegmentReader.java:77) at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:143) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4486) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3925) at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40) at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2091) at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4971) at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:731) at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5009) at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5000) at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1395) at org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1368) at org.apache.lucene.index.RandomIndexWriter.addDocuments(RandomIndexWriter.java:203) at org.apache.lucene.search.join.TestBlockJoin.testMultiChildQueriesOfDiffParentLevels(TestBlockJoin.java:1433) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) FAILED: org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv Error Message: java.lang.RuntimeException: Error from server at http://127.0.0.1:42500/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:55012/solr/test_col_shard2_replica2: Server Errorrequest: http://127.0.0.1:55012/solr/test_col_shard2_replica2/update?update.distrib=TOLEADER=http%3A%2F%2F127.0.0.1%3A42500%2Fsolr%2Ftest_col_shard3_replica1%2F=javabin=2 Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:50853/solr/test_col_shard2_replica1/ update: org.apache.solr.client.solrj.request.UpdateRequest@524e0599 Stack Trace: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error from server at http://127.0.0.1:42500/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:55012/solr/test_col_shard2_replica2: Server Error request: http://127.0.0.1:55012/solr/test_col_shard2_replica2/update?update.distrib=TOLEADER=http%3A%2F%2F127.0.0.1%3A42500%2Fsolr%2Ftest_col_shard3_replica1%2F=javabin=2 Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:50853/solr/test_col_shard2_replica1/ update: org.apache.solr.client.solrj.request.UpdateRequest@524e0599 at __randomizedtesting.SeedInfo.seed([DCDB68FD95F2034:3BD9D4C953021A25]:0) at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at
[jira] [Commented] (LUCENE-7722) Remove BoostedQuery
[ https://issues.apache.org/jira/browse/LUCENE-7722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902838#comment-15902838 ] Alan Woodward commented on LUCENE-7722: --- Rewriting is a bit academic for now anyway, let's punt that to another issue and just remove the custom scoring queries here. > Remove BoostedQuery > --- > > Key: LUCENE-7722 > URL: https://issues.apache.org/jira/browse/LUCENE-7722 > Project: Lucene - Core > Issue Type: Task >Reporter: Adrien Grand >Priority: Minor > > We already have FunctionScoreQuery, which is more flexible than BoostedQuery > as it can combine scores in arbitrary ways and only requests scores on the > underlying scorer if they are needed. So let's remove BoostedQuery? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7700) Move throughput control and merge aborting out of IndexWriter's core?
[ https://issues.apache.org/jira/browse/LUCENE-7700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902896#comment-15902896 ] Michael McCandless commented on LUCENE-7700: Thanks [~dweiss] the new patch looks great! +1 to push, and thank you for cleaning this up! > Move throughput control and merge aborting out of IndexWriter's core? > - > > Key: LUCENE-7700 > URL: https://issues.apache.org/jira/browse/LUCENE-7700 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Minor > Attachments: LUCENE-7700.patch, LUCENE-7700.patch, LUCENE-7700.patch > > > Here is a bit of a background: > - I wanted to implement a custom merging strategy that would have a custom > i/o flow control (global), > - currently, the CMS is tightly bound with a few classes -- MergeRateLimiter, > OneMerge, IndexWriter. > Looking at the code it seems to me that everything with respect to I/O > control could be nicely pulled out into classes that explicitly control the > merging process, that is only MergePolicy and MergeScheduler. By default, one > could even run without any additional I/O accounting overhead (which is > currently in there, even if one doesn't use the CMS's throughput control). > Such refactoring would also give a chance to nicely move things where they > belong -- job aborting into OneMerge (currently in RateLimiter), rate limiter > lifecycle bound to OneMerge (MergeScheduler could then use per-merge or > global accounting, as it pleases). > Just a thought and some initial refactorings for discussion. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS-EA] Lucene-Solr-6.4-Linux (32bit/jdk-9-ea+159) - Build # 161 - Still Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.4-Linux/161/ Java: 32bit/jdk-9-ea+159 -client -XX:+UseParallelGC 1 tests failed. FAILED: org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit Error Message: expected:<1> but was:<2> Stack Trace: java.lang.AssertionError: expected:<1> but was:<2> at __randomizedtesting.SeedInfo.seed([E7D10676E3E9E46D:1E9C95D9DF9CA9E7]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.failNotEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:128) at org.junit.Assert.assertEquals(Assert.java:472) at org.junit.Assert.assertEquals(Assert.java:456) at org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit(ShardSplitTest.java:280) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:547) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at
[jira] [Commented] (LUCENE-7700) Move throughput control and merge aborting out of IndexWriter's core?
[ https://issues.apache.org/jira/browse/LUCENE-7700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902900#comment-15902900 ] Dawid Weiss commented on LUCENE-7700: - Thanks for reviewing, Mike. I'm guessing this should go to 7.x only, right? Or can we backport it to 6.x as well, risking some minor API incompatibilities (these are internal APIs anyway, but who knows). > Move throughput control and merge aborting out of IndexWriter's core? > - > > Key: LUCENE-7700 > URL: https://issues.apache.org/jira/browse/LUCENE-7700 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Dawid Weiss >Assignee: Dawid Weiss >Priority: Minor > Attachments: LUCENE-7700.patch, LUCENE-7700.patch, LUCENE-7700.patch > > > Here is a bit of a background: > - I wanted to implement a custom merging strategy that would have a custom > i/o flow control (global), > - currently, the CMS is tightly bound with a few classes -- MergeRateLimiter, > OneMerge, IndexWriter. > Looking at the code it seems to me that everything with respect to I/O > control could be nicely pulled out into classes that explicitly control the > merging process, that is only MergePolicy and MergeScheduler. By default, one > could even run without any additional I/O accounting overhead (which is > currently in there, even if one doesn't use the CMS's throughput control). > Such refactoring would also give a chance to nicely move things where they > belong -- job aborting into OneMerge (currently in RateLimiter), rate limiter > lifecycle bound to OneMerge (MergeScheduler could then use per-merge or > global accounting, as it pleases). > Just a thought and some initial refactorings for discussion. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5276) highlighter working using stemmed tokens from another field and text from another
[ https://issues.apache.org/jira/browse/SOLR-5276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902687#comment-15902687 ] Matthew Caruana Galizia commented on SOLR-5276: --- The default highlighter has become the unified highlighter since then, but required feature is the same. > highlighter working using stemmed tokens from another field and text from > another > - > > Key: SOLR-5276 > URL: https://issues.apache.org/jira/browse/SOLR-5276 > Project: Solr > Issue Type: Improvement > Components: highlighter >Reporter: Maciej Lizewski >Priority: Minor > > The case is like this: > I have 'content' field with content extracted with tika and several fields > for language versions (like content_pl, content_en, content_es, content_ru, > etc). > I also use custom langid component which can copy 'content' to serveral > content_* fields if it detects more than one language (so those parts are > properly stemmed in every language present in text). > Now to use highlighter in such scenario I need to store all those language > fields even if their contents is always same as the one in 'content' field. > Would be nice if I could configure language specific fields to be not stored, > and configure highlighter to take tokens positions from those fields and > apply them to text in 'content' field... > In other words - to say: take tokens from 'content_pl', and prepare highlight > based on text in 'content' field. > It could be administrator responsibility to guarantee that mapped fields have > same content. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-5276) highlighter working using stemmed tokens from another field and text from another
[ https://issues.apache.org/jira/browse/SOLR-5276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902687#comment-15902687 ] Matthew Caruana Galizia edited comment on SOLR-5276 at 3/9/17 8:44 AM: --- The default highlighter has become the unified highlighter since then, but required feature is the same as SOLR-1105. was (Author: mcaruanagalizia): The default highlighter has become the unified highlighter since then, but required feature is the same. > highlighter working using stemmed tokens from another field and text from > another > - > > Key: SOLR-5276 > URL: https://issues.apache.org/jira/browse/SOLR-5276 > Project: Solr > Issue Type: Improvement > Components: highlighter >Reporter: Maciej Lizewski >Priority: Minor > > The case is like this: > I have 'content' field with content extracted with tika and several fields > for language versions (like content_pl, content_en, content_es, content_ru, > etc). > I also use custom langid component which can copy 'content' to serveral > content_* fields if it detects more than one language (so those parts are > properly stemmed in every language present in text). > Now to use highlighter in such scenario I need to store all those language > fields even if their contents is always same as the one in 'content' field. > Would be nice if I could configure language specific fields to be not stored, > and configure highlighter to take tokens positions from those fields and > apply them to text in 'content' field... > In other words - to say: take tokens from 'content_pl', and prepare highlight > based on text in 'content' field. > It could be administrator responsibility to guarantee that mapped fields have > same content. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10256) Parentheses in SpellCheckCollator
[
https://issues.apache.org/jira/browse/SOLR-10256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Kumar Singh updated SOLR-10256:
Description:
SpellCheckCollator adds parentheses ( *'('* and *')'* ) around tokens which
have space between them.
This should be configurable, because if *_WordBreakSpellCheckComponent_* is
being used, queries like : *applejuice* will be broken down to *apple juice*.
Such suggestions are being surrounded by braces by current
*SpellCheckCollator*.
And when surrounded by brackets, they represent the same position by
_EdismaxParser_ , which is not required.
https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/spelling/SpellCheckCollator.java#L227
A solution to this will be to have a flag, which can help disable this
parenthesisation of spell check suggestions.
was:
SpellCheckCollator adds parentheses ( *'('* and *')'* ) around tokens which
have space between them.
This should be configurable, because if *_WordBreakSpellCheckComponent_* is
being used, queries like : *applejuice* will be broken down to *apple juice*.
Such suggestions are being surrounded by braces by current
*SpellCheckCollator*.
And when surrounded by brackets, they represent the same position, which is not
required.
https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/spelling/SpellCheckCollator.java#L227
A solution to this will be to have a flag, which can help disable this
parenthesisation of spell check suggestions.
> Parentheses in SpellCheckCollator
> -
>
> Key: SOLR-10256
> URL: https://issues.apache.org/jira/browse/SOLR-10256
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: spellchecker
>Reporter: Abhishek Kumar Singh
>
> SpellCheckCollator adds parentheses ( *'('* and *')'* ) around tokens which
> have space between them.
> This should be configurable, because if *_WordBreakSpellCheckComponent_* is
> being used, queries like : *applejuice* will be broken down to *apple juice*.
> Such suggestions are being surrounded by braces by current
> *SpellCheckCollator*.
> And when surrounded by brackets, they represent the same position by
> _EdismaxParser_ , which is not required.
> https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/spelling/SpellCheckCollator.java#L227
>
> A solution to this will be to have a flag, which can help disable this
> parenthesisation of spell check suggestions.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-5872) Eliminate overseer queue
[ https://issues.apache.org/jira/browse/SOLR-5872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890021#comment-15890021 ] albert vico oton edited comment on SOLR-5872 at 3/9/17 11:51 AM: - Hello, we are currently trying to do a deploy of around 200 collections and solrcloud can't handle it, it just dies due update_status messages propagation everytime we try to add a new collection, each collection has 3 replicas, and sizes are not very large. Also, I do not see why collection A should be aware of collection B state. But moving to the topic, overseer node dies since he can not handle all the stress due the flooding of messages. IMHO we have here a single point of failure in a distributed system, which is not very recommended. since it would be useful for big fat shards, my suggestion would be to make this optional behavior, so people like us, who need to have a more distributed approach, can make use of solrcloud. Since right now it is impossible to. and I'm not talking about "thousands" of collections actually with as few as 100 we are seeing very bad performance. was (Author: alvico): Hello, we are currently trying to do a deploy of around 200 collections and solrcloud can't handle it, it just dies due update_status messages propagation everytime we try to add a new collection, each collection has 3 replicas, and sizes are not very large. Also, I do not see why collection A should be aware of collection B state. But moving to the topic, overseer node dies since he can not handle all the stress due the flooding of messages. IMHO we have here a single point of failure in a distributed system, which is not very recommended. since it would be useful for big fat shards, my suggestion would be to make this optional behavior, so people like use who need to have a more distributed approach can make use of solrcloud. Since right now it is impossible to. and I'm not talking about "thousands" of collections actually with as few as 100 we are seeing very bad performance. > Eliminate overseer queue > - > > Key: SOLR-5872 > URL: https://issues.apache.org/jira/browse/SOLR-5872 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Reporter: Noble Paul >Assignee: Noble Paul > > The overseer queue is one of the busiest points in the entire system. The > raison d'être of the queue is > * Provide batching of operations for the main clusterstate,json so that > state updates are minimized > * Avoid race conditions and ensure order > Now , as we move the individual collection states out of the main > clusterstate.json, the batching is not useful anymore. > Race conditions can easily be solved by using a compare and set in Zookeeper. > The proposed solution is , whenever an operation is required to be performed > on the clusterstate, the same thread (and of course the same JVM) > # read the fresh state and version of zk node > # construct the new state > # perform a compare and set > # if compare and set fails go to step 1 > This should be limited to all operations performed on external collections > because batching would be required for others -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5872) Eliminate overseer queue
[ https://issues.apache.org/jira/browse/SOLR-5872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902928#comment-15902928 ] albert vico oton commented on SOLR-5872: We were doing our tests with solr 6.4.1 > Eliminate overseer queue > - > > Key: SOLR-5872 > URL: https://issues.apache.org/jira/browse/SOLR-5872 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Reporter: Noble Paul >Assignee: Noble Paul > > The overseer queue is one of the busiest points in the entire system. The > raison d'être of the queue is > * Provide batching of operations for the main clusterstate,json so that > state updates are minimized > * Avoid race conditions and ensure order > Now , as we move the individual collection states out of the main > clusterstate.json, the batching is not useful anymore. > Race conditions can easily be solved by using a compare and set in Zookeeper. > The proposed solution is , whenever an operation is required to be performed > on the clusterstate, the same thread (and of course the same JVM) > # read the fresh state and version of zk node > # construct the new state > # perform a compare and set > # if compare and set fails go to step 1 > This should be limited to all operations performed on external collections > because batching would be required for others -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
org.apache.lucene.index.CorruptIndexException: codec header mismatch
Hi, lucene search is throwing back this error what does this mean and how can the indexes be fixed ? Or is rebuilding indexes the only option ? getIndexReader org.apache.lucene.index.CorruptIndexException: codec header mismatch: actual header=2883712 vs expected header=1071082519 (resource: MMapIndexInput(path="\\..\_inwu.si")) at org.apache.lucene.codecs.CodecUtil.checkHeader(CodecUtil.java:128) ~[lucene-core-4.3.0.jar:4.3.0 1477023 - simonw - 2013-04-29 14:55:14] at org.apache.lucene.codecs.lucene40.Lucene40SegmentInfoReader.read(Lucene40SegmentInfoReader.java:53) ~[lucene-core-4.3.0.jar:4.3.0 1477023 - simonw - 2013-04-29 14:55:14] at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:301) ~[lucene-core-4.3.0.jar:4.3.0 1477023 - simonw - 2013-04-29 14:55:14] at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:56) ~[lucene-core-4.3.0.jar:4.3.0 1477023 - simonw - 2013-04-29 14:55:14] at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:783) ~[lucene-core-4.3.0.jar:4.3.0 1477023 - simonw - 2013-04-29 14:55:14] at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:52) ~[lucene-core-4.3.0.jar:4.3.0 1477023 - simonw - 2013-04-29 14:55:14] at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:66) ~[lucene-core-4.3.0.jar:4.3.0 1477023 - simonw - 2013-04-29 14:55:14] at xxx.getIndexReader(FTSearchServiceImpl.java:332) [xx-0.0.1.jar:16.2.2] at xxx.getTextSearch(FTSearchServiceImpl.java:115) [xx-0.0.1.jar:16.2.2] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_25] -- View this message in context: http://lucene.472066.n3.nabble.com/org-apache-lucene-index-CorruptIndexException-codec-header-mismatch-tp4324171.html Sent from the Lucene - Java Developer mailing list archive at Nabble.com. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10247) Support non-numeric metrics
[
https://issues.apache.org/jira/browse/SOLR-10247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrzej Bialecki updated SOLR-10247:
-
Attachment: compactFormat.png
> Support non-numeric metrics
> ---
>
> Key: SOLR-10247
> URL: https://issues.apache.org/jira/browse/SOLR-10247
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: metrics
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
> Fix For: 6.5, master (7.0)
>
> Attachments: compactFormat.png, SOLR-10247.patch
>
>
> Metrics API currently supports only numeric values. However, it's useful also
> to report non-numeric values such as eg. version, disk type, component state,
> some system properties, etc.
> Codahale {{Gauge}} metric type can be used for this purpose, and
> convenience methods can be added to {{SolrMetricManager}} to make it easier
> to use.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10247) Support non-numeric metrics
[
https://issues.apache.org/jira/browse/SOLR-10247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrzej Bialecki updated SOLR-10247:
-
Attachment: currentFormat.png
> Support non-numeric metrics
> ---
>
> Key: SOLR-10247
> URL: https://issues.apache.org/jira/browse/SOLR-10247
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: metrics
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
> Fix For: 6.5, master (7.0)
>
> Attachments: compactFormat.png, currentFormat.png, SOLR-10247.patch
>
>
> Metrics API currently supports only numeric values. However, it's useful also
> to report non-numeric values such as eg. version, disk type, component state,
> some system properties, etc.
> Codahale {{Gauge}} metric type can be used for this purpose, and
> convenience methods can be added to {{SolrMetricManager}} to make it easier
> to use.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-10247) Support non-numeric metrics
[
https://issues.apache.org/jira/browse/SOLR-10247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrzej Bialecki updated SOLR-10247:
-
Attachment: SOLR-10247.patch
Initial patch. Some classes have been converted to use the new style for Gauge
registration eg. {{SolrIndexWriter}}. See {{SolrCore.initializeMetrics}} for an
example of non-numeric registrations.
Also, I added the ability to produce a much more compact NamedList format in
{{MetricUtils}}. We should consider using it by default, although it's not
back-compatible with the format in 6x.
> Support non-numeric metrics
> ---
>
> Key: SOLR-10247
> URL: https://issues.apache.org/jira/browse/SOLR-10247
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: metrics
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
> Fix For: 6.5, master (7.0)
>
> Attachments: SOLR-10247.patch
>
>
> Metrics API currently supports only numeric values. However, it's useful also
> to report non-numeric values such as eg. version, disk type, component state,
> some system properties, etc.
> Codahale {{Gauge}} metric type can be used for this purpose, and
> convenience methods can be added to {{SolrMetricManager}} to make it easier
> to use.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS-EA] Lucene-Solr-6.x-Linux (32bit/jdk-9-ea+159) - Build # 3022 - Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/3022/ Java: 32bit/jdk-9-ea+159 -client -XX:+UseParallelGC 1 tests failed. FAILED: org.apache.solr.handler.TestSQLHandler.doTest Error Message: --> http://127.0.0.1:41686/pk_/sx/collection1:Failed to execute sqlQuery 'select str_s, count(*), sum(field_i), min(field_i), max(field_i), cast(avg(1.0 * field_i) as float) from collection1 where text='' group by str_s order by sum(field_i) asc limit 2' against JDBC connection 'jdbc:calcitesolr:'. Error while executing SQL "select str_s, count(*), sum(field_i), min(field_i), max(field_i), cast(avg(1.0 * field_i) as float) from collection1 where text='' group by str_s order by sum(field_i) asc limit 2": From line 1, column 39 to line 1, column 50: No match found for function signature min() Stack Trace: java.io.IOException: --> http://127.0.0.1:41686/pk_/sx/collection1:Failed to execute sqlQuery 'select str_s, count(*), sum(field_i), min(field_i), max(field_i), cast(avg(1.0 * field_i) as float) from collection1 where text='' group by str_s order by sum(field_i) asc limit 2' against JDBC connection 'jdbc:calcitesolr:'. Error while executing SQL "select str_s, count(*), sum(field_i), min(field_i), max(field_i), cast(avg(1.0 * field_i) as float) from collection1 where text='' group by str_s order by sum(field_i) asc limit 2": From line 1, column 39 to line 1, column 50: No match found for function signature min() at __randomizedtesting.SeedInfo.seed([7D628A7DB31B3E9A:DA2632D9DEA02D23]:0) at org.apache.solr.client.solrj.io.stream.SolrStream.read(SolrStream.java:235) at org.apache.solr.handler.TestSQLHandler.getTuples(TestSQLHandler.java:2337) at org.apache.solr.handler.TestSQLHandler.testBasicGrouping(TestSQLHandler.java:663) at org.apache.solr.handler.TestSQLHandler.doTest(TestSQLHandler.java:81) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:547) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at
[jira] [Comment Edited] (SOLR-10247) Support non-numeric metrics
[
https://issues.apache.org/jira/browse/SOLR-10247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15903064#comment-15903064
]
Andrzej Bialecki edited comment on SOLR-10247 at 3/9/17 1:48 PM:
--
Initial patch. Some classes have been converted to use the new style for Gauge
registration eg. {{SolrIndexWriter}}. See {{SolrCore.initializeMetrics}} for an
example of non-numeric registrations.
Also, I added the ability to produce a much more compact NamedList format in
{{MetricUtils}}, which is used in the output from {{/admin/metrics}}. We should
consider using it by default, although it's not back-compatible with the format
in 6x.
was (Author: ab):
Initial patch. Some classes have been converted to use the new style for Gauge
registration eg. {{SolrIndexWriter}}. See {{SolrCore.initializeMetrics}} for an
example of non-numeric registrations.
Also, I added the ability to produce a much more compact NamedList format in
{{MetricUtils}}. We should consider using it by default, although it's not
back-compatible with the format in 6x.
> Support non-numeric metrics
> ---
>
> Key: SOLR-10247
> URL: https://issues.apache.org/jira/browse/SOLR-10247
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: metrics
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
> Fix For: 6.5, master (7.0)
>
> Attachments: compactFormat.png, SOLR-10247.patch
>
>
> Metrics API currently supports only numeric values. However, it's useful also
> to report non-numeric values such as eg. version, disk type, component state,
> some system properties, etc.
> Codahale {{Gauge}} metric type can be used for this purpose, and
> convenience methods can be added to {{SolrMetricManager}} to make it easier
> to use.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[PATCH] jcc: add python3 support
Hi, I did some work to port jcc to python3, see https://github.com/rudimeier/jcc There are two interesting branches, py2 and py3 py2 should still work for python2 >=2.7 without any behavior change py3 completes experimental python3 support (but still python2 incompatible). regarding py2 branch: - fixes many (but not yet all) python3 incompatibilities - should still work for python2 >=2.7 without any behavior change - succeeds lucene's test-suite completely (pylucene-4.10.1/java-1.7 and (pylucene-6.4.1/java-1.8). - removes compatibility for python < 2.7 (though it would not be hard to keep it) regarding py3 branch: - based on py2 it adds a few more patches for python3 support (>3.1) but still in a way which is python2 incompatible - still fails about 25% of pylucene tests. - Note I just did pylucene's python3 support for testing trivially like '2to3 -w $(find -name "*.py")' Please comment on this. I'd like to get python3 support upstream. Some more notes: My patches were inspired from the old python-3.1 port http://svn.apache.org/repos/asf/lucene/pylucene/branches/python_3/jcc/ I've refactored/rebased it and splitted the huge patches into many smaller ones inclusive keeping python2 support. Almost all commits in my py2 branch are trivially to review and independent of each other. So I would be glad if you would merge as many of them as you like. Since the py3 branch is still not 100% correct (guess still some Bytes/Unicode problems) I would be glad if somebody would help to get it running. Cheers, Rudi
