[jira] [Commented] (SOLR-10654) Expose Metrics in Prometheus format

[Keith Laban (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006516#comment-16006516
 ] 

Keith Laban commented on SOLR-10654:


Seems like there a few things here:

1.  
  - a: Pull reporter framework. An interesting idea, but is it over engineering 
for an initial effort? I'm not aware any other mainstream metrics frameworks 
that pull metrics in a very specific format. Any home rolled thing can consume 
the default format we expose. 
  - b: Additionally, we can expose these metrics under {{/metrics/prometheus}} 
like suggested above, to avoid having to change the api if we later decide 
there is a need for more generic framework.

2. Response writers. It might be interesting to expose response writers 
dynamically with a plugin-style interface. Or add a Function to the response 
object that can dictate the writer to be used (optionally). Either way, I think 
this is separate enough from metrics, and useful in other places, that it 
should be pursued in a separate issue. 

3. Dropwizard Exports - yes, there is not feature parity with default solr 
metrics, but is it required for an initial patch? To me it seems like a lot of 
work that isn't required on day one. But I agree it should be added at some 
point. 

I propose that we tackle #2 and #1.b for the initial patch. And circle back to 
#3 and #1.a if we find it necessary.

> Expose Metrics in Prometheus format
> ---
>
> Key: SOLR-10654
> URL: https://issues.apache.org/jira/browse/SOLR-10654
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Reporter: Keith Laban
>
> Expose metrics via a `wt=prometheus` response type.
> Example scape_config in prometheus.yml:
> {code}
> scrape_configs:
>   - job_name: 'solr'
> metrics_path: '/solr/admin/metrics'
> params:
>   wt: ["prometheus"]
> static_configs:
>   - targets: ['localhost:8983']
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10654) Expose Metrics in Prometheus format

[Andrzej Bialecki (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006214#comment-16006214
 ] 

Andrzej Bialecki  commented on SOLR-10654:
--

I see, it's an interesting issue. The metrics API as it is today is focused on 
the push model, but indeed the MetricsHandler seems like the ideal place for 
adding custom formats to support pull models. However, response writers are 
registered globally per SolrCore, and it seems weird and potentially misleading 
to globally register a writer that is a) handler-specific, b) supports only 
numeric values.

OTOH there are precedents for a) above - graphml, schema.xml and filestream 
writers. There are also precedents for b) - handlers that accept requests in 
one format but return response in a different (hardcoded) format, eg. 
{{ExportHandler}}, so maybe it's not such a big deal...

In the current patch the registration is hardcoded - it would have to be made 
more dynamic. Eg. you could consider adding generic support for custom pull 
formats to {{MetricsHandler}}, which would go through every 
{{SolrMetricReporter}} instance that implements a {{SolrMetricPullReporter}} 
(what an ugly name...) or something like that, and register its response 
writers and WT-s that it supports, and then based on either a reporter name, or 
path or WT, call a method from that interface to prepare a report to be 
serialized.

Current patch also ignores all metric / property filtering logic present in 
{{MetricsHandler.handleRequestBody}} - I understand that {{DropwizardExports}} 
doesn't support any filtering, but it should be possible to reimplement this 
using {{MetricUtils.toMaps}} or similar methods from {{MetricUtils}}.

And finally, Solr now uses many non-numeric gauges, most notably {{String}} and 
a dynamic {{MetricsMap}}, so you would have to add a special treatment for 
these.

> Expose Metrics in Prometheus format
> ---
>
> Key: SOLR-10654
> URL: https://issues.apache.org/jira/browse/SOLR-10654
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Reporter: Keith Laban
>
> Expose metrics via a `wt=prometheus` response type.
> Example scape_config in prometheus.yml:
> {code}
> scrape_configs:
>   - job_name: 'solr'
> metrics_path: '/solr/admin/metrics'
> params:
>   wt: ["prometheus"]
> static_configs:
>   - targets: ['localhost:8983']
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10654) Expose Metrics in Prometheus format

[Keith Laban (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16004814#comment-16004814
 ] 

Keith Laban commented on SOLR-10654:


Jan, I think the problem would still be that each handler would most likely 
need their own very specific custom response format. If we were to do this we 
would need to expose them as a raw Filter or Servlet instead of a 
SolrRequestHandler. I'm not aware of anywhere else in solr where this is 
happening. The other option would be to add a custom response writer format for 
each metrics type, kind of like Iike I did here. 

> Expose Metrics in Prometheus format
> ---
>
> Key: SOLR-10654
> URL: https://issues.apache.org/jira/browse/SOLR-10654
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Reporter: Keith Laban
>
> Expose metrics via a `wt=prometheus` response type.
> Example scape_config in prometheus.yml:
> {code}
> scrape_configs:
>   - job_name: 'solr'
> metrics_path: '/solr/admin/metrics'
> params:
>   wt: ["prometheus"]
> static_configs:
>   - targets: ['localhost:8983']
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10654) Expose Metrics in Prometheus format

[JIRA]

[ 
https://issues.apache.org/jira/browse/SOLR-10654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003584#comment-16003584
 ] 

Jan Høydahl commented on SOLR-10654:


Very cool. Perhaps the reporter framework needs to support pull (if it does not 
already)? What if the reporter's defined name in solr.xml would automatically 
be exposed as a sub-path of /admin/metrics, where latest version is served, 
e.g. http://localhost:8983/solr/admin/metrics/prometheus

> Expose Metrics in Prometheus format
> ---
>
> Key: SOLR-10654
> URL: https://issues.apache.org/jira/browse/SOLR-10654
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Reporter: Keith Laban
>
> Expose metrics via a `wt=prometheus` response type.
> Example scape_config in prometheus.yml:
> {code}
> scrape_configs:
>   - job_name: 'solr'
> metrics_path: '/solr/admin/metrics'
> params:
>   wt: ["prometheus"]
> static_configs:
>   - targets: ['localhost:8983']
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10654) Expose Metrics in Prometheus format

[Keith Laban (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003379#comment-16003379
 ] 

Keith Laban commented on SOLR-10654:


Prometheus uses a pull model. I thought of setting up a reporter which also 
setups up a servlet to serve stats in prometheus format but that seemed more 
cumbersome. I also wasn't sure if it was possible to expose an arbitrary 
servlet easily. Using a solr request handler would still require the custom 
response format. 

> Expose Metrics in Prometheus format
> ---
>
> Key: SOLR-10654
> URL: https://issues.apache.org/jira/browse/SOLR-10654
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Reporter: Keith Laban
>
> Expose metrics via a `wt=prometheus` response type.
> Example scape_config in prometheus.yml:
> {code}
> scrape_configs:
>   - job_name: 'solr'
> metrics_path: '/solr/admin/metrics'
> params:
>   wt: ["prometheus"]
> static_configs:
>   - targets: ['localhost:8983']
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10654) Expose Metrics in Prometheus format

[Andrzej Bialecki (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003357#comment-16003357
 ] 

Andrzej Bialecki  commented on SOLR-10654:
--

Does Prometheus use a pull or push model for collecting metrics from monitored 
systems? Metrics API uses {{SolrMetricReporter}} subclasses for pushing metrics 
to external systems using custom formats, so that would be the way to go when 
using push model.

Writer types are a kind of global thing, ie. should not be specific to a 
particular handler.

> Expose Metrics in Prometheus format
> ---
>
> Key: SOLR-10654
> URL: https://issues.apache.org/jira/browse/SOLR-10654
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Reporter: Keith Laban
>
> Expose metrics via a `wt=prometheus` response type.
> Example scape_config in prometheus.yml:
> {code}
> scrape_configs:
>   - job_name: 'solr'
> metrics_path: '/solr/admin/metrics'
> params:
>   wt: ["prometheus"]
> static_configs:
>   - targets: ['localhost:8983']
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10654) Expose Metrics in Prometheus format

[Keith Laban (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003328#comment-16003328
 ] 

Keith Laban commented on SOLR-10654:


Exposing prometheus metrics by creating a new response writer type. I couldn't 
think of a more graceful way to handle this then creating a new response format.

> Expose Metrics in Prometheus format
> ---
>
> Key: SOLR-10654
> URL: https://issues.apache.org/jira/browse/SOLR-10654
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Keith Laban
>
> Expose metrics via a `wt=prometheus` response type.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10654) Expose Metrics in Prometheus format

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003322#comment-16003322
 ] 

ASF GitHub Bot commented on SOLR-10654:
---

GitHub user kelaban opened a pull request:

https://github.com/apache/lucene-solr/pull/200

SOLR-10654 - Expose Prometheus metrics under /admin/metrics with wt=p…

…rometheus

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/kelaban/lucene-solr 
jira/master/SOLR-10654/prometheus-metrics

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/lucene-solr/pull/200.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #200


commit 70d88101b2cb7b893e63f775b63fcd0a91e9d9b7
Author: Keith Laban 
Date:   2017-05-08T20:41:22Z

SOLR-10654 - Expose Prometheus metrics under /admin/metrics with 
wt=prometheus




> Expose Metrics in Prometheus format
> ---
>
> Key: SOLR-10654
> URL: https://issues.apache.org/jira/browse/SOLR-10654
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Keith Laban
>
> Expose metrics via a `wt=prometheus` response type.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org