[jira] [Commented] (CALCITE-642) Avatica server metrics
[ https://issues.apache.org/jira/browse/CALCITE-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15131445#comment-15131445 ] Julian Hyde commented on CALCITE-642: - OK, full regress + javadoc succeeded on JDK 1.7 and 1.8 for change f9ccaa6. > Avatica server metrics > -- > > Key: CALCITE-642 > URL: https://issues.apache.org/jira/browse/CALCITE-642 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Nick Dimiduk >Assignee: Josh Elser > Labels: avatica > Fix For: next > > > Along the same lines as CALCITE-641, avatica server should track some basic > metrics. We can expose them through a "standard" API (such as Hadoop metrics, > or Dropwizard metrics), and include them on some interval in the service log. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CALCITE-642) Avatica server metrics
[ https://issues.apache.org/jira/browse/CALCITE-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15126841#comment-15126841 ] Josh Elser commented on CALCITE-642: Just pushed a big changeset to the pull request. It was a long road around, but I think I'm happy with where we ended up: * Introduced an avatica-metrics API which is all that avatica (client) depends on. This includes a "noop" implementation which removed lots of null/optional checking. * Introduced a dropwizard-metrics v3 implementation of avatica-metrics. * ServiceLoader is used to find one (and only one) implementation. Otherwise the noop implementation is used. (the noop implementation doesn't have a META-INF/services entry for this) * Also included the dropwizard-metrics to hadoop metrics2 as a standalone module in calcite (my glance upstream at dropwizard showed that the next version will be 4.y.z and includes a package renaming -- I'm hoping we can act as the host for this code until we get it pushed upstream and it can flourish there). * Lots of tests One thing I avoided was making a new avatica shaded jar with the metrics implementation included. This really boiled down to a new dependency in the downstream project (phoenix), so I was OK with this for now. More use by consumers other than myself will tell. [~julianhyde], if you'd like to look at the change before I merge, let me know. I believe this approach should be satisfactory to your already-stated concerns. I'm waiting to get a positive from travis (or at least what Travis would run itself). > Avatica server metrics > -- > > Key: CALCITE-642 > URL: https://issues.apache.org/jira/browse/CALCITE-642 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Nick Dimiduk >Assignee: Josh Elser > Labels: avatica > Fix For: next > > > Along the same lines as CALCITE-641, avatica server should track some basic > metrics. We can expose them through a "standard" API (such as Hadoop metrics, > or Dropwizard metrics), and include them on some interval in the service log. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CALCITE-642) Avatica server metrics
[ https://issues.apache.org/jira/browse/CALCITE-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122381#comment-15122381 ] Josh Elser commented on CALCITE-642: So this is what I'm thinking: * Remove all dropwizard API use in Avatica directly, create my own interfaces ala "avatica-metrics" * Implement interfaces in an "avatica-metrics-dropwizard-3" (or something -- need to figure out what/if-at-all is their compat statement) * Have calcite-avatica create two (shaded) artifacts: "calcite-avatica.jar" and "calcite-avatica-without-metrics.jar" (could also do a with-metrics instead of without-metrics but I'm being selfish and hoping the without-metrics jar would be less common) For those running in an AppServer (or some shared environment w/o classpath isolation), they can use the without-metrics jar and provide the version they want. We would also be able to provide bindings for other systems natively (if necessary). Hopefully, the common case would still be the single calcite-avatica.jar, but we wouldn't screw over those whose needs aren't satisfied. > Avatica server metrics > -- > > Key: CALCITE-642 > URL: https://issues.apache.org/jira/browse/CALCITE-642 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Nick Dimiduk >Assignee: Josh Elser > Labels: avatica > Fix For: next > > > Along the same lines as CALCITE-641, avatica server should track some basic > metrics. We can expose them through a "standard" API (such as Hadoop metrics, > or Dropwizard metrics), and include them on some interval in the service log. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CALCITE-642) Avatica server metrics
[
https://issues.apache.org/jira/browse/CALCITE-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122284#comment-15122284
]
Josh Elser commented on CALCITE-642:
Thanks for taking a look, Julian.
bq. Change the commit description to make clear you're adding to Avatica
*server*. We don't need dependency bloat in the client.
So, this isn't entirely true. We have some code which is only intended to run
in the server, but lives in the "client" package, e.g. {{JsonHandler}} handles
the serialization into POJOs. The dropwizard-metrics artifact itself is only
110KB in size, ~100classes and the only provided reporter is an slf4j one
(suddenly the slf4j work becomes more important). Long term, I think having the
ability to use this metrics instrumentation to measure things in the client is
valuable as well (but is presently left as a "todo"). Looks like the
calcite-avatica jar is at ~3.5MB with these changes.
Is including this dependency in the calcite-avatica a deal-breaker for you?
Restructuring stuff for avatica has been on my radar (notably the splitting up
of Service and the POJOs), but I didn't want to try to tackle both of these at
once.
bq. Add a small section to http://calcite.apache.org/docs/avatica_overview.html
and add the package to the "structure" section
Ahh, thanks for the reminder. I meant to write up some docs for users too.
bq. Move the version number in pom.xml into the properties list (consistent
with what I did in CALCITE-999)
Will do.
> Avatica server metrics
> --
>
> Key: CALCITE-642
> URL: https://issues.apache.org/jira/browse/CALCITE-642
> Project: Calcite
> Issue Type: Improvement
> Components: avatica
>Reporter: Nick Dimiduk
>Assignee: Josh Elser
> Labels: avatica
> Fix For: next
>
>
> Along the same lines as CALCITE-641, avatica server should track some basic
> metrics. We can expose them through a "standard" API (such as Hadoop metrics,
> or Dropwizard metrics), and include them on some interval in the service log.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CALCITE-642) Avatica server metrics
[ https://issues.apache.org/jira/browse/CALCITE-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122301#comment-15122301 ] Josh Elser commented on CALCITE-642: bq. I don't want to be a jerk about this Not at all! I'm just trying to make sure I understand the scope of the objection. bq. The client might be used in an environment (say an app server) that has an incompatible version of Dropwizard Let me play with this, we might be able to work around this the same way we did for protobuf (relocating the classes in the shaded jar). This is a bit dirtier since it's not self-contained like protobuf was (we expect users to have some interactions). Let me brainstorm some ideas. > Avatica server metrics > -- > > Key: CALCITE-642 > URL: https://issues.apache.org/jira/browse/CALCITE-642 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Nick Dimiduk >Assignee: Josh Elser > Labels: avatica > Fix For: next > > > Along the same lines as CALCITE-641, avatica server should track some basic > metrics. We can expose them through a "standard" API (such as Hadoop metrics, > or Dropwizard metrics), and include them on some interval in the service log. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CALCITE-642) Avatica server metrics
[ https://issues.apache.org/jira/browse/CALCITE-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15122230#comment-15122230 ] Julian Hyde commented on CALCITE-642: - Very nice work. A few things: * Change the commit description to make clear you're adding to Avatica *server*. We don't need dependency bloat in the client. * Move the version number in pom.xml into the properties list (consistent with what I did in CALCITE-999) * Add a small section to http://calcite.apache.org/docs/avatica_overview.html and add the package to the "structure" section I ran the extended test suite, and everything looks great. So +1 to commit when you've fixed the above. > Avatica server metrics > -- > > Key: CALCITE-642 > URL: https://issues.apache.org/jira/browse/CALCITE-642 > Project: Calcite > Issue Type: Improvement > Components: avatica >Reporter: Nick Dimiduk >Assignee: Josh Elser > Labels: avatica > Fix For: next > > > Along the same lines as CALCITE-641, avatica server should track some basic > metrics. We can expose them through a "standard" API (such as Hadoop metrics, > or Dropwizard metrics), and include them on some interval in the service log. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
