[jira] [Commented] (HIVE-9721) Hadoop23Shims.setFullFileStatus should check for null
[
https://issues.apache.org/jira/browse/HIVE-9721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352381#comment-14352381
]
Brock Noland commented on HIVE-9721:
+1
Hadoop23Shims.setFullFileStatus should check for null
-
Key: HIVE-9721
URL: https://issues.apache.org/jira/browse/HIVE-9721
Project: Hive
Issue Type: Bug
Reporter: Brock Noland
Assignee: Sergio Peña
Attachments: HIVE-9721.1.patch
{noformat}
2015-02-18 22:46:10,209 INFO org.apache.hadoop.hive.shims.HadoopShimsSecure:
Skipping ACL inheritance: File system for path
file:/tmp/hive/f1a28dee-70e8-4bc3-bd35-9be13834d1fc/hive_2015-02-18_22-46-10_065_3348083202601156561-1
does not support ACLs but dfs.namenode.acls.enabled is set to true:
java.lang.UnsupportedOperationException: RawLocalFileSystem doesn't support
getAclStatus
java.lang.UnsupportedOperationException: RawLocalFileSystem doesn't support
getAclStatus
at org.apache.hadoop.fs.FileSystem.getAclStatus(FileSystem.java:2429)
at
org.apache.hadoop.fs.FilterFileSystem.getAclStatus(FilterFileSystem.java:562)
at
org.apache.hadoop.hive.shims.Hadoop23Shims.getFullFileStatus(Hadoop23Shims.java:645)
at org.apache.hadoop.hive.common.FileUtils.mkdir(FileUtils.java:524)
at org.apache.hadoop.hive.ql.Context.getStagingDir(Context.java:234)
at
org.apache.hadoop.hive.ql.Context.getExtTmpPathRelTo(Context.java:424)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFileSinkPlan(SemanticAnalyzer.java:6290)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:9069)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:8961)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9807)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:9700)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genOPTree(SemanticAnalyzer.java:10136)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:284)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10147)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:190)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:222)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:421)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:307)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1112)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1106)
at
org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:101)
at
org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:172)
at
org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:379)
at
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:366)
at
org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:271)
at
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:415)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)
at
org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:692)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-02-18 17:30:58,753 INFO org.apache.hadoop.hive.shims.HadoopShimsSecure:
Skipping ACL inheritance: File system for path
file:/tmp/hive/e3eb01f0-bb58-45a8-b773-8f4f3420457c/hive_2015-02-18_17-30-58_346_5020255420422913166-1/-mr-1
does not support ACLs but dfs.namenode.acls.enabled is set to true:
java.lang.NullPointerException
java.lang.NullPointerException
at
org.apache.hadoop.hive.shims.Hadoop23Shims.setFullFileStatus(Hadoop23Shims.java:668)
at
[jira] [Commented] (HIVE-9889) Merge trunk to Spark branch 3/6/2015 [Spark Branch]
[ https://issues.apache.org/jira/browse/HIVE-9889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352380#comment-14352380 ] Brock Noland commented on HIVE-9889: [~spena] should probably take a look first. Merge trunk to Spark branch 3/6/2015 [Spark Branch] --- Key: HIVE-9889 URL: https://issues.apache.org/jira/browse/HIVE-9889 Project: Hive Issue Type: Task Components: Spark Reporter: Xuefu Zhang Assignee: Xuefu Zhang Attachments: HIVE-9889.1-spark.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9882) Add jar/file doesn't work with yarn-cluster mode [Spark Branch]
[
https://issues.apache.org/jira/browse/HIVE-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-9882:
-
Attachment: HIVE-9882.1-spark.patch
Upload same patch to trigger test
Add jar/file doesn't work with yarn-cluster mode [Spark Branch]
---
Key: HIVE-9882
URL: https://issues.apache.org/jira/browse/HIVE-9882
Project: Hive
Issue Type: Sub-task
Components: Hive, spark-branch
Affects Versions: spark-branch
Reporter: Xiaomin Zhang
Assignee: Rui Li
Attachments: HIVE-9882.1-spark.patch, HIVE-9882.1-spark.patch
It seems current fix for HIVE-9425 only uploads the Jar/Files to HDFS,
however, they are not accessible by the Driver/Executor.
I found below in the AM log:
{noformat}
15/02/26 15:10:36 INFO Configuration.deprecation: mapred.min.split.size is
deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
15/02/26 15:10:36 INFO client.SparkClientUtilities: Added
jar[file:/data/hadoop-devel/data/nm/usercache/user/appcache/application_1424933948132_0002/container_1424933948132_0002_01_01/hdfs:/localhost:8020/tmp/hive/user/47040bca-1da4-49b6-b2c7-69be9bc92855/hive-exec-1.2.0-SNAPSHOT.jar]
to classpath.
15/02/26 15:10:36 INFO client.SparkClientUtilities: Added
jar[file:/data/hadoop-devel/data/nm/usercache/user/appcache/application_1424933948132_0002/container_1424933948132_0002_01_01/hdfs:/localhost:8020/tmp/hive/user/47040bca-1da4-49b6-b2c7-69be9bc92855/opennlp-maxent-3.0.3.jar]
to classpath.
15/02/26 15:10:36 INFO client.SparkClientUtilities: Added
jar[file:/data/hadoop-devel/data/nm/usercache/user/appcache/application_1424933948132_0002/container_1424933948132_0002_01_01/hdfs:/localhost:8020/tmp/hive/user/47040bca-1da4-49b6-b2c7-69be9bc92855/bigbenchqueriesmr.jar]
to classpath.
15/02/26 15:10:36 INFO client.SparkClientUtilities: Added
jar[file:/data/hadoop-devel/data/nm/usercache/user/appcache/application_1424933948132_0002/container_1424933948132_0002_01_01/hdfs:/localhost:8020/tmp/hive/user/47040bca-1da4-49b6-b2c7-69be9bc92855/opennlp-tools-1.5.3.jar]
to classpath.
15/02/26 15:10:36 INFO client.SparkClientUtilities: Added
jar[file:/data/hadoop-devel/data/nm/usercache/user/appcache/application_1424933948132_0002/container_1424933948132_0002_01_01/hdfs:/localhost:8020/tmp/hive/user/47040bca-1da4-49b6-b2c7-69be9bc92855/jcl-over-slf4j-1.7.5.jar]
to classpath.
15/02/26 15:10:36 INFO client.RemoteDriver: Failed to run job
6886df05-f430-456c-a0ff-c7621db712d6
org.apache.hive.com.esotericsoftware.kryo.KryoException: Unable to find
class: de.bankmark.bigbench.queries.q10.SentimentUDF
{noformat}
As above shows, the file path which was attempted to add to Classpath is
invalid, so actually all uploaded Jars/Files are still not available for
Driver/Executor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-1374) Query compile-only option
[ https://issues.apache.org/jira/browse/HIVE-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352479#comment-14352479 ] ASF GitHub Bot commented on HIVE-1374: -- Github user navis commented on the pull request: https://github.com/apache/tajo/pull/400#issuecomment-77793864 @jinossy If it will be not deprecated in near future, can we still get some usefulness in this? Fixed test fails, anyway. Query compile-only option - Key: HIVE-1374 URL: https://issues.apache.org/jira/browse/HIVE-1374 Project: Hive Issue Type: New Feature Affects Versions: 0.6.0 Reporter: Paul Yang Assignee: Siying Dong A compile-only option might be useful for helping users quickly prototype queries, fix errors, and do test runs. The proposed change would be adding a -c switch that behaves like -e but only compiles the specified query. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HIVE-9819) Add timeout check inside the HMS server
[ https://issues.apache.org/jira/browse/HIVE-9819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Chen updated HIVE-9819: Attachment: HIVE-9819.patch Test seems not running. Re-attach to trigger test. Thanks for your review! [~brocknoland] Add timeout check inside the HMS server --- Key: HIVE-9819 URL: https://issues.apache.org/jira/browse/HIVE-9819 Project: Hive Issue Type: Sub-task Components: Metastore Reporter: Dong Chen Assignee: Dong Chen Attachments: HIVE-9819.patch, HIVE-9819.patch In HIVE-9253, a timeout check mechanism is added for long running methods in HMS server. We should add this check to each of the inner loops inside the HMS server. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9895) Update hive people page with recent changes
[ https://issues.apache.org/jira/browse/HIVE-9895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352588#comment-14352588 ] Brock Noland commented on HIVE-9895: [~chengxiang li] - any chance you could review this? Update hive people page with recent changes --- Key: HIVE-9895 URL: https://issues.apache.org/jira/browse/HIVE-9895 Project: Hive Issue Type: Task Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-9895.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9891) LLAP: disable plan caching
[ https://issues.apache.org/jira/browse/HIVE-9891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352337#comment-14352337 ] Siddharth Seth commented on HIVE-9891: -- Would be nice if the plan were immutable - I'm guessing that's a big change and is an item for later. Caching the plan and cloning it for each execution, rather than deserializing it each time may be another option. LLAP: disable plan caching -- Key: HIVE-9891 URL: https://issues.apache.org/jira/browse/HIVE-9891 Project: Hive Issue Type: Sub-task Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Attachments: HIVE-9891.1.patch Can't share the same plan objects in LLAP as they are used concurrently. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
