Rajesh Balamohan created HIVE-24290:
---------------------------------------
Summary: Explain analyze can be slow in cloud storage
Key: HIVE-24290
URL: https://issues.apache.org/jira/browse/HIVE-24290
Project: Hive
Issue Type: Improvement
Reporter: Rajesh Balamohan
"explain analyze" takes a lot longer time to exit with the following path,
specifically in cloud environments (where it could be EFS vol). HIVE-24270 is
a related ticket as well.
{noformat}
at java.io.UnixFileSystem.delete0(Native Method)
at java.io.UnixFileSystem.delete(UnixFileSystem.java:265)
at java.io.File.delete(File.java:1043)
at org.apache.hadoop.fs.FileUtil.deleteImpl(FileUtil.java:229)
at org.apache.hadoop.fs.FileUtil.fullyDeleteContents(FileUtil.java:270)
at org.apache.hadoop.fs.FileUtil.fullyDelete(FileUtil.java:182)
at org.apache.hadoop.fs.FileUtil.fullyDelete(FileUtil.java:153)
at
org.apache.hadoop.fs.RawLocalFileSystem.delete(RawLocalFileSystem.java:453)
at
org.apache.hadoop.fs.ChecksumFileSystem.delete(ChecksumFileSystem.java:685)
at
org.apache.hadoop.hive.ql.stats.fs.FSStatsAggregator.closeConnection(FSStatsAggregator.java:115)
at
org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.aggregateStats(ExplainSemanticAnalyzer.java:261)
at
org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:156)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288)
at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:221)
at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:188)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:600)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:546)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:540)
at
org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:127)
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
