[
https://issues.apache.org/jira/browse/HIVE-18086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Olson updated HIVE-18086:
--------------------------------
Description:
First, some background info: A non-native table can be created with partition
columns defined. However, the existence of partition columns for a non-native
table is problematic. Nothing disallows the table creation, and the
documentation [1] does not mention that non-native tables cannot have partition
columns. In fact, it suggests that "PARTITIONED BY" can be specified.
When launching the MR job for any query a {{NullPointerException}} is thrown. A
null path value originates in
{{org.apache.hadoop.hive.ql.exec.Utilities#createDummyFileForEmptyTable(...)}}
because "if this isn't a hive table we can't create an empty file for it" and
is added to the list of input paths that the
{{org.apache.hadoop.hive.ql.exec.mr.ExecDriver}} provides to the
{{FileInputFormat}}, ultimately causing the NPE in the {{Path}} constructor
because the child path is null.
{noformat}
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
java.lang.NullPointerException
at org.apache.hadoop.fs.Path.<init>(Path.java:115)
at
org.apache.hadoop.mapred.FileInputFormat.setInputPaths(FileInputFormat.java:438)
at
org.apache.hadoop.hive.ql.exec.Utilities.setInputPaths(Utilities.java:3550)
at
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:371)
at
org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1690)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1449)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1235)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1053)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1043)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:209)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:161)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:372)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:617)
{noformat}
We encountered this bug using the {{HBaseStorageHandler}}.
[1]
https://cwiki.apache.org/confluence/display/Hive/StorageHandlers#StorageHandlers-DDL
was:
A non-native table can be created with partition columns defined. However, the
existence of partition columns for a non-native table is problematic. Nothing
disallows the table creation, and the documentation [1] does not mention that
non-native tables cannot have partition columns. In fact, it suggests that
"PARTITIONED BY" can be specified.
When launching the MR job for any query a NullPointerException is thrown. A
null path value originates in
{{org.apache.hadoop.hive.ql.exec.Utilities#createDummyFileForEmptyTable(...)}}
because "if this isn't a hive table we can't create an empty file for it" and
is added to the list of input paths that the
{{org.apache.hadoop.hive.ql.exec.mr.ExecDriver}} provides to the
FileInputFormat, ultimately causing the NPE in the Path constructor because the
child path is null.
{noformat}
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
java.lang.NullPointerException
at org.apache.hadoop.fs.Path.<init>(Path.java:115)
at
org.apache.hadoop.mapred.FileInputFormat.setInputPaths(FileInputFormat.java:438)
at
org.apache.hadoop.hive.ql.exec.Utilities.setInputPaths(Utilities.java:3550)
at
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:371)
at
org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1690)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1449)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1235)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1053)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1043)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:209)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:161)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:372)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:617)
{noformat}
We encountered this bug using the HBaseStorageHandler.
[1]
https://cwiki.apache.org/confluence/display/Hive/StorageHandlers#StorageHandlers-DDL
> NullPointerException initializing query job when non-native table has
> partition columns
> ---------------------------------------------------------------------------------------
>
> Key: HIVE-18086
> URL: https://issues.apache.org/jira/browse/HIVE-18086
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Environment: CDH 5.5.5 on RHEL 6.8
> Reporter: Andrew Olson
>
> First, some background info: A non-native table can be created with partition
> columns defined. However, the existence of partition columns for a non-native
> table is problematic. Nothing disallows the table creation, and the
> documentation [1] does not mention that non-native tables cannot have
> partition columns. In fact, it suggests that "PARTITIONED BY" can be
> specified.
> When launching the MR job for any query a {{NullPointerException}} is thrown.
> A null path value originates in
> {{org.apache.hadoop.hive.ql.exec.Utilities#createDummyFileForEmptyTable(...)}}
> because "if this isn't a hive table we can't create an empty file for it"
> and is added to the list of input paths that the
> {{org.apache.hadoop.hive.ql.exec.mr.ExecDriver}} provides to the
> {{FileInputFormat}}, ultimately causing the NPE in the {{Path}} constructor
> because the child path is null.
> {noformat}
> Total jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks is set to 0 since there's no reduce operator
> java.lang.NullPointerException
> at org.apache.hadoop.fs.Path.<init>(Path.java:115)
> at
> org.apache.hadoop.mapred.FileInputFormat.setInputPaths(FileInputFormat.java:438)
> at
> org.apache.hadoop.hive.ql.exec.Utilities.setInputPaths(Utilities.java:3550)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:371)
> at
> org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1690)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1449)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1235)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1053)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1043)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:209)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:161)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:372)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:759)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:617)
> {noformat}
> We encountered this bug using the {{HBaseStorageHandler}}.
> [1]
> https://cwiki.apache.org/confluence/display/Hive/StorageHandlers#StorageHandlers-DDL
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
