Hudson commented on PHOENIX-4701:

SUCCESS: Integrated in Jenkins build Phoenix-4.x-HBase-1.3 #136 (See 
PHOENIX-4701 Write client-side metrics asynchronously to (jtaylor: rev 
* (edit) 

> Write client-side metrics asynchronously to SYSTEM.LOG
> ------------------------------------------------------
>                 Key: PHOENIX-4701
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-4701
>             Project: Phoenix
>          Issue Type: Improvement
>            Reporter: James Taylor
>            Assignee: Ankit Singhal
>            Priority: Major
>             Fix For: 4.14.0, 5.0.0
>         Attachments: PHOENIX-4701.patch, PHOENIX-4701_addendum.patch, 
> PHOENIX-4701_addendum2.patch, PHOENIX-4701_master.patch, 
> PHOENIX-4701_v2.patch, PHOENIX-4701_wip1.patch, PHOENIX-4701_wip2.patch, 
> PHOENIX-4701_wip3.patch
> Rather than inventing a new, different set of client-side metrics to persist, 
> we should just persist our [client 
> metrics|http://phoenix.apache.org/metrics.html] in the SYSTEM.LOG. The 
> metrics captures all the same information as your QueryLogInfo (and much 
> more), rolls all the information up to a single set of metrics for each 
> Phoenix statement (aggregating/merging parallel scans, etc), and can emits a 
> single log line (which could be written in a single upsert statement). At 
> SFDC, we emit this information into a file system log in a layer above (and 
> use Splunk to produce nifty dashboard for monitoring), but this could easily 
> be emitted directly in Phoenix and go through your asynchronous write path 
> (and then use Phoenix queries to produce the same kind of dashboards). The 
> only piece would be to add the concept of a log level to each metric to 
> enable statically controlling which metrics are output.
> With this approach, the SYSTEM.LOG table could be declared immutable and use 
> our dense storage format with a single byte for column encoding and get a 
> 3-5x perf gain. This would also open the door for users to potentially add 
> secondary indexes on the table. See schema identified in the wip2 patch.

This message was sent by Atlassian JIRA

Reply via email to