[
https://issues.apache.org/jira/browse/TRAFODION-2927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347836#comment-16347836
]
ASF GitHub Bot commented on TRAFODION-2927:
-------------------------------------------
GitHub user DaveBirdsall opened a pull request:
https://github.com/apache/trafodion/pull/1429
[TRAFODION-2927] Refactor and improve UPDATE STATISTICS logging
This set of changes achieves two things:
1. It provides a mechanism for UPDATE STATISTICS to capture log information
for each command, and retain that log in the event of a severe error.
(Otherwise the log is deleted.)
2. It refactors the UPDATE STATISTICS logging facility to use the
QRLogger/CommonLogger/log4cxx mechanism used by the rest of the SQL engine.
Externals:
The UPDATE STATISTICS LOG statement has been extended as follows:
UPDATE STATISTICS LOG ON -- Starts continuous logging of all UPDATE
STATISTICS and SHOWSTATS activity into one log file. Logging continues until
the session has ended or a different UPDATE STATISTICS LOG command is entered.
This is the same behavior as today, except the log file will be written to the
$TRAF_HOME/logs directory like other SQL log files.
UPDATE STATISTICS LOG OFF -- Stops logging. Nothing further is logged until
a different UPDATE STATISTICS LOG command is entered. This is the same behavior
as today.
UPDATE STATISTICS LOG SYSTEM -- The "SYSTEM" option is new. When specified,
each UPDATE STATISTICS statement (but not SHOWSTATS) is logged to a separate
log file. If a severe error occurs (e.g. an internal error), the log file is
retained. Otherwise it is deleted at the end of the UPDATE STATISTICS command.
A CQD has been added. CQD USTAT_AUTOMATIC_LOGGING, having values of 'ON'
and 'OFF'. If 'ON', then UPDATE STATISTICS LOG SYSTEM semantics are turned on
by default. If 'OFF', then UPDATE STATISTICS LOG OFF semantics are turned on by
default. (Note that no equivalent of UPDATE STATISTICS LOG ON is provided as
that is remarkably noisy; if one really wants to log continuously, then one
must use UPDATE STATISTICS LOG ON directly.)
The default for this CQD is 'OFF'. So by default no automatic logging is
done. On large systems (e.g. scalability testing, or production systems with
large tables), it is recommended to set this CQD to 'ON' in the system defaults
table. In this way, logs will be captured for long-running UPDATE STATISTICS
operations that fail with severe errors.
In terms of design: The existing QRLogger/CommonLogger routines assume a
model of one log file per category, which doesn't meet the needs for UPDATE
STATISTICS. So, QRLogger has been extended to add methods to explicitly
configure log files on a per process, even per command basis. For such files,
we don't bother logging the process ID and node number in the individual log
messages, as all of the messages are coming from one process. Instead, UPDATE
STATISTICS puts the node number, PID and Linux timestamp of file creation into
the log file name itself.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/DaveBirdsall/trafodion Trafodion2927
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/trafodion/pull/1429.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1429
----
commit f473c0cd5a1d895dd8a40d2ce2ec24c12a5b076d
Author: Dave Birdsall <dbirdsall@...>
Date: 2018-02-01T00:37:49Z
[TRAFODION-2927] Refactor and improve UPDATE STATISTICS logging
----
> Keep log information for UPDATE STATISTICS in case of errors
> ------------------------------------------------------------
>
> Key: TRAFODION-2927
> URL: https://issues.apache.org/jira/browse/TRAFODION-2927
> Project: Apache Trafodion
> Issue Type: Bug
> Components: sql-cmp
> Affects Versions: 2.3
> Reporter: David Wayne Birdsall
> Assignee: David Wayne Birdsall
> Priority: Major
>
> Presently, UPDATE STATISTICS keeps a detailed log of its internal activities
> if one has specifed "update statistics log on" in advance.
> In production scenarios, this is typically not done. That means when a
> long-running UPDATE STATISTICS command fails on a large table, one has to
> redo it with logging turned on in order to troubleshoot.
> A better practice might be to always log, and delete the log if the operation
> succeeds.
> Another issue with UPDATE STATISTICS logs is their location. The directory is
> different than other Trafodion logs and is sometimes hard to find. As part of
> this JIRA, consideration should be given to writing the logs to the Trafodion
> logs directory instead.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
