Naveen Gangam created HIVE-14784:
------------------------------------

             Summary: Operation logs are disabled automatically if the parent 
directory does not exist.
                 Key: HIVE-14784
                 URL: https://issues.apache.org/jira/browse/HIVE-14784
             Project: Hive
          Issue Type: Bug
          Components: HiveServer2
    Affects Versions: 1.1.0
            Reporter: Naveen Gangam
            Assignee: Naveen Gangam


Operation logging is disabled automatically for the query if for some reason 
the parent directory (named after the hive session id) that gets created when 
the session is established gets deleted (for any reason). For ex: if the 
operation logdir is /tmp which automatically can get purged at a configured 
interval by the OS.

Running a query from that session leads to
{code}
2016-09-15 15:09:16,723 WARN org.apache.hive.service.cli.operation.Operation: 
Unable to create operation log file: 
/tmp/hive/operation_logs/b8809985-6b38-47ec-a49b-6158a67cd9fc/d35414f7-2418-426c-8489-c6f643ca4599
java.io.IOException: No such file or directory
        at java.io.UnixFileSystem.createFileExclusively(Native Method)
        at java.io.File.createNewFile(File.java:1012)
        at 
org.apache.hive.service.cli.operation.Operation.createOperationLog(Operation.java:195)
        at 
org.apache.hive.service.cli.operation.Operation.beforeRun(Operation.java:237)
        at 
org.apache.hive.service.cli.operation.Operation.run(Operation.java:255)
        at 
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:398)
        at 
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:385)
        at 
org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:271)
        at 
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:490)
        at 
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)
        at 
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at 
org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:692)
        at 
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

{code}

This later leads to errors like (more prominent when using HUE as HUE does not 
close hive sessions and attempts to retrieve the operations logs days after 
they were created).
{code}
WARN org.apache.hive.service.cli.thrift.ThriftCLIService: Error fetching 
results: 
org.apache.hive.service.cli.HiveSQLException: Couldn't find log associated with 
operation handle: OperationHandle [opType=EXECUTE_STATEMENT, 
getHandleIdentifier()=d35414f7-2418-426c-8489-c6f643ca4599]
        at 
org.apache.hive.service.cli.operation.OperationManager.getOperationLogRowSet(OperationManager.java:259)
        at 
org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:701)
        at 
org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:451)
        at 
org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:676)
        at 
org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1553)
        at 
org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1538)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at 
org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:692)
        at 
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745) 
{code}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to