Hi Boaz,
To install, I ran
bin/plugin --install elasticsearch/marvel/latest
on each node in both clusters, then restarted both clusters.
Since then, I have tried several things, including deleting the indexes
from the monitoring cluster and reinstalling the plugin on the monitoring
cluster. I'll try now to delete all the marvel indexes, uninstall, then
reinstall marvel into both clusters.
I'm a bit stumped otherwise, so I'm all ears for any other suggestions.
Cheers,
Ross
On Tuesday, 28 October 2014 08:30:54 UTC+11, Boaz Leskes wrote:
>
> It looks like something is wrong is indeed wrong with your marvel index
> template which should be there before data is indexed. How did you install
> marvel? Did you perhaps delete the data folder of the monitoring cluster
> after production was already shipping data?
>
> Cheers,
> Boaz
>
> On Monday, October 27, 2014 7:45:34 AM UTC+1, Ross Simpson wrote:
>>
>> To troubleshoot a little more, I rebuilt the monitoring cluster to use
>> ElasticSearch 1.1.1, which matches the ES version used in the production
>> cluster. No luck.
>>
>> On the Overview dashboard, I can see some data (summary, doc count,
>> search and indexing rates are all populated [screenshot attached]), but but
>> both the nodes and indices sections are empty other than the errors
>> mentioned in the previous post. Cluster pulse doesn't show any events at
>> all; node stats and index stats do both show data.
>>
>> Any further suggestions would be greatly appreciated :)
>>
>> Cheers,
>> Ross
>>
>>
>>
>> On Monday, 27 October 2014 11:15:42 UTC+11, Ross Simpson wrote:
>>>
>>> I've got a brand-new Marvel installation, and am having some frustrating
>>> issues with it: on the overview screen, I am constantly getting errors like:
>>> *Oops!* FacetPhaseExecutionException[Facet [timestamp]: failed to find
>>> mapping for node.ip_port.raw]
>>>
>>> *Production cluster:*
>>> * ElasticSearch 1.1.1
>>> * Marvel 1.2.1
>>> * Running in vSphere
>>>
>>> *Monitoring cluster:*
>>> * ElasticSearch 1.3.4
>>> * Marvel 1.2.1
>>> * Running in AWS
>>>
>>> After installing the plugin and bouncing all nodes in both clusters,
>>> Marvel seems to be working -- an index has been created in the monitoring
>>> cluster (.marvel-2014.10.26), and I see thousands of documents in
>>> there. There are documents with the following types: cluster_state,
>>> cluster_stats, index_stats, indices_stats, node_stats. So, it does
>>> seem that data is being shipped from the prod cluster to the monitoring
>>> cluster.
>>>
>>> I've seen in the user group that other people have had similar issues.
>>> Some of those mention problems with the marvel index template. I don't
>>> seem to have any at all templates in my monitoring cluster:
>>>
>>> $ curl -XGET localhost:9200/_template/
>>> {}
>>>
>>> I tried manually adding the default template (as described in
>>> http://www.elasticsearch.org/guide/en/marvel/current/#config-marvel-indices),
>>>
>>> but that didn't seem to have any effect.
>>>
>>> So far, I've seen just two specific errors in Marvel:
>>> * FacetPhaseExecutionException[Facet [timestamp]: failed to find mapping
>>> for node.ip_port.raw]
>>> * FacetPhaseExecutionException[Facet [timestamp]: failed to find mapping
>>> for index.raw]
>>>
>>> I've also looked through the logs on both the production and monitoring
>>> clusters, and the only errors are in the monitoring cluster resulting from
>>> queries from the Marvel UI, like this:
>>>
>>> [2014-10-27 11:08:13,427][DEBUG][action.search.type ] [ip-10-4-1-
>>> 187] [.marvel-2014.10.27][1], node[SR_hriFmTCav-8ofbKU-8g], [R], s[
>>> STARTED]: Failed to execute [org.elasticsearch.action.search.
>>> SearchRequest@661dc47e]
>>> org.elasticsearch.search.SearchParseException: [.marvel-2014.10.27][1]:
>>> query[ConstantScore(BooleanFilter(+*:* +cache(_type:index_stats) +cache(
>>> @timestamp:[1414367880000 TO 1414368540000])))],from[-1],size[10]: Parse
>>> Failure [Failed to parse source [{"size":10,"query":{"filtered":{"query"
>>> :{"match_all":{}},"filter":{"bool":{"must":[{"match_all":{}},{"term":{
>>> "_type":"index_stats"}},{"range":{"@timestamp":{"from":"now-10m/m","to":
>>> "now/m"}}}]}}}},"facets":{"timestamp":{"terms_stats":{"key_field":
>>> "index.raw","value_field":"@timestamp","order":"term","size":2000}},
>>> "primaries.docs.count":{"terms_stats":{"key_field":"index.raw",
>>> "value_field":"primaries.docs.count","order":"term","size":2000}},
>>> "primaries.indexing.index_total":{"terms_stats":{"key_field":"index.raw"
>>> ,"value_field":"primaries.indexing.index_total","order":"term","size":
>>> 2000}},"total.search.query_total":{"terms_stats":{"key_field":
>>> "index.raw","value_field":"total.search.query_total","order":"term",
>>> "size":2000}},"total.merges.total_size_in_bytes":{"terms_stats":{
>>> "key_field":"index.raw","value_field":"total.merges.total_size_in_bytes"
>>> ,"order":"term","size":2000}},"total.fielddata.memory_size_in_bytes":{
>>> "terms_stats":{"key_field":"index.raw","value_field":
>>> "total.fielddata.memory_size_in_bytes","order":"term","size":2000}}}}]]
>>> at org.elasticsearch.search.SearchService.parseSource(
>>> SearchService.java:660)
>>> at org.elasticsearch.search.SearchService.createContext(
>>> SearchService.java:516)
>>> at org.elasticsearch.search.SearchService.createAndPutContext(
>>> SearchService.java:488)
>>> at org.elasticsearch.search.SearchService.executeQueryPhase(
>>> SearchService.java:257)
>>> at org.elasticsearch.search.action.
>>> SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:
>>> 206)
>>> at org.elasticsearch.search.action.
>>> SearchServiceTransportAction$5.call(SearchServiceTransportAction.java:
>>> 203)
>>> at org.elasticsearch.search.action.
>>> SearchServiceTransportAction$23.run(SearchServiceTransportAction.java:
>>> 517)
>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(
>>> ThreadPoolExecutor.java:1145)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
>>> ThreadPoolExecutor.java:615)
>>> at java.lang.Thread.run(Thread.java:745)
>>> Caused by: org.elasticsearch.search.facet.FacetPhaseExecutionException:
>>> Facet [timestamp]: failed to find mapping for index.raw
>>> at org.elasticsearch.search.facet.termsstats.
>>> TermsStatsFacetParser.parse(TermsStatsFacetParser.java:126)
>>> at org.elasticsearch.search.facet.FacetParseElement.parse(
>>> FacetParseElement.java:93)
>>> at org.elasticsearch.search.SearchService.parseSource(
>>> SearchService.java:644</
>>> ...
>>
>>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/7157c765-2576-4ae3-b83f-3a1753f4d2ec%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
