[
https://issues.apache.org/jira/browse/HIVE-22733?focusedWorklogId=374384&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-374384
]
ASF GitHub Bot logged work on HIVE-22733:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 20/Jan/20 03:10
Start Date: 20/Jan/20 03:10
Worklog Time Spent: 10m
Work Description: maheshk114 commented on pull request #881: HIVE-22733 :
After disable operation log property in hive, still HS2 saving the operation log
URL: https://github.com/apache/hive/pull/881#discussion_r368355162
##########
File path:
itests/hive-unit/src/test/java/org/apache/hive/service/cli/operation/TestOperationLoggingLayout.java
##########
@@ -132,22 +133,47 @@ private void
appendHushableRandomAccessFileAppender(Appender queryAppender) {
}
}
- @Test
- public void testSwitchLogLayout() throws Exception {
+ private void executeWithOperationLog(String query, boolean queryLogEnabled)
throws Exception {
// verify whether the sql operation log is generated and fetch correctly.
- OperationHandle operationHandle = client.executeStatement(sessionHandle,
sqlCntStar, null);
+ OperationHandle operationHandle = client.executeStatement(sessionHandle,
query, null);
RowSet rowSetLog = client.fetchResults(operationHandle,
FetchOrientation.FETCH_FIRST, 1000,
- FetchType.LOG);
- String queryId = getQueryId(rowSetLog);
- Assert.assertNotNull("Could not find query id, perhaps a logging message
changed", queryId);
+ FetchType.LOG);
+ String queryId = "";
+ boolean expectedStopped = true;
+ if (queryLogEnabled) {
+ queryId = getQueryId(rowSetLog);
+ expectedStopped = false;
+ Assert.assertNotNull("Could not find query id, perhaps a logging message
changed", queryId);
+ } else {
+ Assert.assertEquals("Operation log is generated even if query logging is
disabled", rowSetLog.numRows(), 0);
+ Assert.assertNull("Query id present even if logging is disabled.",
getQueryId(rowSetLog));
+ }
- checkAppenderState("before operation close ",
LogDivertAppender.QUERY_ROUTING_APPENDER, queryId, false);
- checkAppenderState("before operation close ",
LogDivertAppenderForTest.TEST_QUERY_ROUTING_APPENDER, queryId, false);
+ checkAppenderState("before operation close ",
LogDivertAppender.QUERY_ROUTING_APPENDER, queryId, expectedStopped);
+ checkAppenderState("before operation close ",
LogDivertAppenderForTest.TEST_QUERY_ROUTING_APPENDER, queryId, expectedStopped);
client.closeOperation(operationHandle);
checkAppenderState("after operation close ",
LogDivertAppender.QUERY_ROUTING_APPENDER, queryId, true);
checkAppenderState("after operation close ",
LogDivertAppenderForTest.TEST_QUERY_ROUTING_APPENDER, queryId, true);
}
+ @Test
+ public void testSwitchLogLayout() throws Exception {
+ executeWithOperationLog(sqlCntStar, true);
+ }
+
+ @Test
+ public void testQueryLogDisabled() throws Exception {
+ OperationHandle operationHandle = client.executeStatement(sessionHandle,
+ "set hive.server2.logging.operation.enabled=false", null);
+ client.closeOperation(operationHandle);
+
+ executeWithOperationLog(sqlCntStar, false);
+
+ operationHandle = client.executeStatement(sessionHandle,
+ "set hive.server2.logging.operation.enabled=true", null);
+ client.closeOperation(operationHandle);
+ }
+
Review comment:
you mean ..without the above changes ? just back porting HIVE-22115 is
sufficient ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 374384)
Time Spent: 1h (was: 50m)
> After disable operation log property in hive, still HS2 saving the operation
> log
> --------------------------------------------------------------------------------
>
> Key: HIVE-22733
> URL: https://issues.apache.org/jira/browse/HIVE-22733
> Project: Hive
> Issue Type: Bug
> Reporter: mahesh kumar behera
> Assignee: mahesh kumar behera
> Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22733.01.patch
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> There are few issues in this area.
> 1. If logging is disabled using hive.server2.logging.operation.enabled, then
> operation logs for the queries should not be generated. But the
> registerLoggingContext method in LogUtils, registers the logging context
> even if the operation log is disabled. This causes the logs to be added by
> logger. The registration of query context should be done only if operation
> logging is enabled.
> {code:java}
> public static void registerLoggingContext(Configuration conf) {
> - MDC.put(SESSIONID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVESESSIONID));
> - MDC.put(QUERYID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVEQUERYID));
> if (HiveConf.getBoolVar(conf,
> HiveConf.ConfVars.HIVE_SERVER2_LOGGING_OPERATION_ENABLED)) {
> + MDC.put(SESSIONID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVESESSIONID));
> + MDC.put(QUERYID_LOG_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVEQUERYID));
> MDC.put(OPERATIONLOG_LEVEL_KEY, HiveConf.getVar(conf,
> HiveConf.ConfVars.HIVE_SERVER2_LOGGING_OPERATION_LEVEL));{code}
>
> 2. In case of failed query, we close the operations and that deletes the
> logging context (appender and route) from logger for that query. But if any
> log is added after that, the query logs are getting added and new operation
> log file is getting generated for the query. This looks like issue with MCD
> clear. MCD clear is not removing the keys from the map. If remove is used
> instead of clear, its working fine.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
