[
https://issues.apache.org/jira/browse/HIVE-18086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Olson updated HIVE-18086:
--------------------------------
Description:
A non-native table can be created with partition columns defined. However, the
existence of partition columns for a non-native table is problematic. Nothing
disallows the table creation, and the documentation [1] does not mention that
non-native tables cannot have partition columns. In fact, it suggests that
"PARTITIONED BY" can be specified.
When launching the MR job for any query a NullPointerException is thrown. A
null path value originates in
{{org.apache.hadoop.hive.ql.exec.Utilities#createDummyFileForEmptyTable(...)}}
because "if this isn't a hive table we can't create an empty file for it" and
is added to the list of input paths that the
{{org.apache.hadoop.hive.ql.exec.mr.ExecDriver}} provides to the
FileInputFormat, ultimately causing the NPE in the Path constructor because the
child path is null.
{noformat}
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
java.lang.NullPointerException
at org.apache.hadoop.fs.Path.<init>(Path.java:115)
at
org.apache.hadoop.mapred.FileInputFormat.setInputPaths(FileInputFormat.java:438)
at
org.apache.hadoop.hive.ql.exec.Utilities.setInputPaths(Utilities.java:3550)
at
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:371)
at
org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1690)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1449)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1235)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1053)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1043)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:209)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:161)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:372)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:617)
{noformat}
We encountered this bug using the HBaseStorageHandler.
[1]
https://cwiki.apache.org/confluence/display/Hive/StorageHandlers#StorageHandlers-DDL
was:
A non-native table can be created with partition columns defined. However, the
existence of partition columns for a non-native table is problematic. Nothing
disallows the table creation, and the documentation [1] does not mention that
non-native tables cannot have partition columns. In fact, it suggests that
PARTITIONED BY can be specified.
When launching the MR job for any query a NullPointerException is thrown. A
null path value originates in
{{org.apache.hadoop.hive.ql.exec.Utilities#createDummyFileForEmptyTable(...)}}
because "if this isn't a hive table we can't create an empty file for it" and
is added to the list of input paths that the
org.apache.hadoop.hive.ql.exec.mr.ExecDriver provides to the FileInputFormat,
ultimately causing the NPE in the Path constructor because the child path is
null.
{noformat}
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
java.lang.NullPointerException
at org.apache.hadoop.fs.Path.<init>(Path.java:115)
at
org.apache.hadoop.mapred.FileInputFormat.setInputPaths(FileInputFormat.java:438)
at
org.apache.hadoop.hive.ql.exec.Utilities.setInputPaths(Utilities.java:3550)
at
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:371)
at
org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1690)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1449)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1235)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1053)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1043)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:209)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:161)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:372)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:617)
{noformat}
We encountered this bug using the HBaseStorageHandler.
[1]
https://cwiki.apache.org/confluence/display/Hive/StorageHandlers#StorageHandlers-DDL
> NullPointerException initializing query job when non-native table has
> partition columns
> ---------------------------------------------------------------------------------------
>
> Key: HIVE-18086
> URL: https://issues.apache.org/jira/browse/HIVE-18086
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
> Environment: CDH 5.5.5 on RHEL 6.8
> Reporter: Andrew Olson
>
> A non-native table can be created with partition columns defined. However,
> the existence of partition columns for a non-native table is problematic.
> Nothing disallows the table creation, and the documentation [1] does not
> mention that non-native tables cannot have partition columns. In fact, it
> suggests that "PARTITIONED BY" can be specified.
> When launching the MR job for any query a NullPointerException is thrown. A
> null path value originates in
> {{org.apache.hadoop.hive.ql.exec.Utilities#createDummyFileForEmptyTable(...)}}
> because "if this isn't a hive table we can't create an empty file for it"
> and is added to the list of input paths that the
> {{org.apache.hadoop.hive.ql.exec.mr.ExecDriver}} provides to the
> FileInputFormat, ultimately causing the NPE in the Path constructor because
> the child path is null.
> {noformat}
> Total jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks is set to 0 since there's no reduce operator
> java.lang.NullPointerException
> at org.apache.hadoop.fs.Path.<init>(Path.java:115)
> at
> org.apache.hadoop.mapred.FileInputFormat.setInputPaths(FileInputFormat.java:438)
> at
> org.apache.hadoop.hive.ql.exec.Utilities.setInputPaths(Utilities.java:3550)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:371)
> at
> org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1690)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1449)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1235)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1053)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1043)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:209)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:161)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:372)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:759)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:617)
> {noformat}
> We encountered this bug using the HBaseStorageHandler.
> [1]
> https://cwiki.apache.org/confluence/display/Hive/StorageHandlers#StorageHandlers-DDL
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
