[
https://issues.apache.org/jira/browse/PHOENIX-4701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451930#comment-16451930
]
Ankit Singhal commented on PHOENIX-4701:
----------------------------------------
{quote} If possible, the SYSTEM.LOG table would benefit greatly (3-5x perf
gain) from being declared as immutable with a column encoding of 1 byte and a
storage format of SINGLE_CELL_ARRAY_WITH_OFFSETS.
{quote}
{quote}I've attached a wip2 patch that writes using Phoenix APIs so that you
can have a composite row key to allow querying, use salting, have the table be
column encoded, and potentially add secondary indexes. It needs a bit more
work, though, because the column values are set in various places for the same
query.I think we should log a single line per query - we can cover the
failed/exception case and the success case. I think having so many individual
Put RPCs won't scale well and even if it did, having a table with only the
query ID in the PK means every query would be a full table scan.
{quote}
Agreed [~jamestaylor] , this is a good idea, I thought of using Phoenix API but
it was just impacting the extensibility as we can't prepare multiple upsert
statement before hand for the set of attributes originating at a different part
of the code. We need the logging to start as soon as the query is submitted, so
to reduce no. of RPC, the best we can do here is to do one UPSERT for every
QueryLogState after merging logging information in LogWriter(using Merger)
received from different part of code as per query log state. Let me know if you
are fine with this, I can put up the patch soon for the same.
{quote} But do you think we can run into some kind of infinite loop by using
the Phoenix API for writing to the SYSTEM.LOG table? If so, we may need to do
something similar like what our tracing framework does where it makes sure
writes to SYSTEM.TRACE table do not generate traces themselves.
{quote}
[~samarthjain] , in addition to what [~elserj] said, as we are only logging
SELECT queries, so writing to SYSTEM.LOG will not generate any new log.
> Improve schema of SYSTEM.LOG table
> ----------------------------------
>
> Key: PHOENIX-4701
> URL: https://issues.apache.org/jira/browse/PHOENIX-4701
> Project: Phoenix
> Issue Type: Bug
> Reporter: James Taylor
> Assignee: James Taylor
> Priority: Major
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-4701_wip1.patch, PHOENIX-4701_wip2.patch
>
>
> If possible, the SYSTEM.LOG table would benefit greatly (3-5x perf gain)
> from being declared as immutable with a column encoding of 1 byte and a
> storage format of SINGLE_CELL_ARRAY_WITH_OFFSETS.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
