[
https://issues.apache.org/jira/browse/PHOENIX-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16472690#comment-16472690
]
James Taylor commented on PHOENIX-4701:
---------------------------------------
One minor thing, [~an...@apache.org], probably better to use
EnvironmentEdgeManager.currentTimeMillis() instead of
System.currentTimeMillis() here and elsewhere. Will make testing easier.
{code:java}
private QueryLogger(PhoenixConnection connection) {
this.queryId = UUID.randomUUID().toString();
this.queryDisruptor = connection.getQueryServices().getQueryDisruptor();
- this.startTime = System.currentTimeMillis();
logLevel = connection.getLogLevel();
+ log(QueryLogInfo.QUERY_ID_I, queryId);
+ log(QueryLogInfo.START_TIME_I, System.currentTimeMillis());
{code}
> Write client-side metrics asynchronously to SYSTEM.LOG
> ------------------------------------------------------
>
> Key: PHOENIX-4701
> URL: https://issues.apache.org/jira/browse/PHOENIX-4701
> Project: Phoenix
> Issue Type: Bug
> Reporter: James Taylor
> Assignee: James Taylor
> Priority: Major
> Fix For: 4.15.0
>
> Attachments: PHOENIX-4701.patch, PHOENIX-4701_master.patch,
> PHOENIX-4701_wip1.patch, PHOENIX-4701_wip2.patch, PHOENIX-4701_wip3.patch
>
>
> Rather than inventing a new, different set of client-side metrics to persist,
> we should just persist our [client
> metrics|http://phoenix.apache.org/metrics.html] in the SYSTEM.LOG. The
> metrics captures all the same information as your QueryLogInfo (and much
> more), rolls all the information up to a single set of metrics for each
> Phoenix statement (aggregating/merging parallel scans, etc), and can emits a
> single log line (which could be written in a single upsert statement). At
> SFDC, we emit this information into a file system log in a layer above (and
> use Splunk to produce nifty dashboard for monitoring), but this could easily
> be emitted directly in Phoenix and go through your asynchronous write path
> (and then use Phoenix queries to produce the same kind of dashboards). The
> only piece would be to add the concept of a log level to each metric to
> enable statically controlling which metrics are output.
> With this approach, the SYSTEM.LOG table could be declared immutable and use
> our dense storage format with a single byte for column encoding and get a
> 3-5x perf gain. This would also open the door for users to potentially add
> secondary indexes on the table. See schema identified in the wip2 patch.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
