[jira] [Updated] (SOLR-9641) Emit distributed tracing information from Solr
[ https://issues.apache.org/jira/browse/SOLR-9641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated SOLR-9641: Attachment: SOLR-9641.patch Patch v2: * Incorporated feedback from David and Christine as discussed in earlier comments. * Moved core tracing logic out of SolrCore and into HttpSolrCall * Added sample trace configuration section to solr.xml * Added tracing to write response, and more granular tracing in general > Emit distributed tracing information from Solr > -- > > Key: SOLR-9641 > URL: https://issues.apache.org/jira/browse/SOLR-9641 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mike Drob > Fix For: master (7.0) > > Attachments: SOLR-9641.patch, SOLR-9641.patch > > > While Solr already offers a few tools for exposing timing, this information > can be difficult to aggregate and analyze. By integrating distributed tracing > into Solr operations, we can gain new performance and behaviour insights. > One such solution can be accomplished via Apache HTrace (incubating). > (More rationale to follow.) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9641) Emit distributed tracing information from Solr
[ https://issues.apache.org/jira/browse/SOLR-9641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated SOLR-9641: Attachment: (was: SOLR-9641.patch) > Emit distributed tracing information from Solr > -- > > Key: SOLR-9641 > URL: https://issues.apache.org/jira/browse/SOLR-9641 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mike Drob > Fix For: master (7.0) > > Attachments: SOLR-9641.patch > > > While Solr already offers a few tools for exposing timing, this information > can be difficult to aggregate and analyze. By integrating distributed tracing > into Solr operations, we can gain new performance and behaviour insights. > One such solution can be accomplished via Apache HTrace (incubating). > (More rationale to follow.) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9641) Emit distributed tracing information from Solr
[ https://issues.apache.org/jira/browse/SOLR-9641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated SOLR-9641: Attachment: SOLR-9641.patch This is a fairly large patch that covers a lot of ground. A summary of the changes is: * Introduce dependency on new version of HTrace. HTrace-3 (transitively required by Hadoop) cannot communicate with HTrace-4 (used by me here), but they are packaged/namespace separately and can exist in the same application. Hadoop will include HTrace-4 starting in version 2.8 and 3.0, I believe. * Introduce new solr.xml config section, that passes all parameters to the HTrace configuration. If we decide to make the distributed tracing library we use pluggable, then I think we can spend some time to make the section a little more generic, but for now it is fine. * Add trace object to CoreContainer. This seemed like a logical place for a singleton scoped to the entire Solr process, but I ran into issues when trying to add tracing to Overseer operations. This design decision may need to be revisited. * Instrument SolrCore to handle all kinds of core related requests. This gets us most index handlers at once. * Instrument a few of the executors we use to pass along trace ids for tracing over distribution requests. * Instrument CollectionsHandler for create. Same pattern could be applied for other collection operations, but it may be a good idea to make this more generic later. * Added a basic test. There is a start of a test for HDFS, but it's very much a work in progress. > Emit distributed tracing information from Solr > -- > > Key: SOLR-9641 > URL: https://issues.apache.org/jira/browse/SOLR-9641 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mike Drob > Fix For: master (7.0) > > Attachments: SOLR-9641.patch > > > While Solr already offers a few tools for exposing timing, this information > can be difficult to aggregate and analyze. By integrating distributed tracing > into Solr operations, we can gain new performance and behaviour insights. > One such solution can be accomplished via Apache HTrace (incubating). > (More rationale to follow.) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-9641) Emit distributed tracing information from Solr
[ https://issues.apache.org/jira/browse/SOLR-9641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated SOLR-9641: Attachment: SOLR-9641.patch This is a fairly large patch that covers a lot of ground. A summary of the changes is: * Introduce dependency on new version of HTrace. HTrace-3 (transitively required by Hadoop) cannot communicate with HTrace-4 (used by me here), but they are packaged/namespace separately and can exist in the same application. Hadoop will include HTrace-4 starting in version 2.8 and 3.0, I believe. * Introduce new solr.xml config section, that passes all parameters to the HTrace configuration. If we decide to make the distributed tracing library we use pluggable, then I think we can spend some time to make the section a little more generic, but for now it is fine. * Add trace object to CoreContainer. This seemed like a logical place for a singleton scoped to the entire Solr process, but I ran into issues when trying to add tracing to Overseer operations. This design decision may need to be revisited. * Instrument SolrCore to handle all kinds of core related requests. This gets us most index handlers at once. * Instrument a few of the executors we use to pass along trace ids for tracing over distribution requests. * Instrument CollectionsHandler for create. Same pattern could be applied for other collection operations, but it may be a good idea to make this more generic later. * Added a basic test. There is a start of a test for HDFS, but it's very much a work in progress. > Emit distributed tracing information from Solr > -- > > Key: SOLR-9641 > URL: https://issues.apache.org/jira/browse/SOLR-9641 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mike Drob > Fix For: master (7.0) > > Attachments: SOLR-9641.patch > > > While Solr already offers a few tools for exposing timing, this information > can be difficult to aggregate and analyze. By integrating distributed tracing > into Solr operations, we can gain new performance and behaviour insights. > One such solution can be accomplished via Apache HTrace (incubating). > (More rationale to follow.) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
