[
https://issues.apache.org/jira/browse/HIVE-11091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599283#comment-14599283
]
Nishant Kelkar commented on HIVE-11091:
---------------------------------------
I did a diff of Hive 0.11 vs. Hive 0.14 for the piece of code within MoveTask
that is causing this error:
Hive-0.11:
{code}
Table table = db.getTable(tbd.getTable().getTableName());
if (work.getCheckFileFormat()) {
// Get all files from the src directory
FileStatus[] dirs;
ArrayList<FileStatus> files;
FileSystem fs;
try {
fs = FileSystem.get(table.getDataLocation(), conf);
dirs = fs.globStatus(new Path(tbd.getSourceDir()));
files = new ArrayList<FileStatus>();
for (int i = 0; (dirs != null && i < dirs.length); i++) {
files.addAll(Arrays.asList(fs.listStatus(dirs[i].getPath())));
// We only check one file, so exit the loop when we have at least
// one.
if (files.size() > 0) {
break;
}
}
} catch (IOException e) {
throw new HiveException(
"addFiles: filesystem error in check phase", e);
}
if (HiveConf.getBoolVar(conf, HiveConf.ConfVars.HIVECHECKFILEFORMAT))
{
// Check if the file format of the file matches that of the table.
boolean flag = HiveFileFormatUtils.checkInputFormat(
fs, conf, tbd.getTable().getInputFileFormatClass(), files);
if (!flag) {
throw new HiveException(
"Wrong file format. Please check the file's format.");
}
}
}
{code}
Hive-0.14:
{code}
Table table = db.getTable(tbd.getTable().getTableName());
if (work.getCheckFileFormat()) {
// Get all files from the src directory
FileStatus[] dirs;
ArrayList<FileStatus> files;
FileSystem srcFs; // source filesystem
try {
srcFs = tbd.getSourcePath().getFileSystem(conf);
dirs = srcFs.globStatus(tbd.getSourcePath());
files = new ArrayList<FileStatus>();
for (int i = 0; (dirs != null && i < dirs.length); i++) {
files.addAll(Arrays.asList(srcFs.listStatus(dirs[i].getPath(),
FileUtils.HIDDEN_FILES_PATH_FILTER)));
// We only check one file, so exit the loop when we have at least
// one.
if (files.size() > 0) {
break;
}
}
} catch (IOException e) {
throw new HiveException(
"addFiles: filesystem error in check phase", e);
}
if (HiveConf.getBoolVar(conf, HiveConf.ConfVars.HIVECHECKFILEFORMAT))
{
// Check if the file format of the file matches that of the table.
boolean flag = HiveFileFormatUtils.checkInputFormat(
srcFs, conf, tbd.getTable().getInputFileFormatClass(), files);
if (!flag) {
throw new HiveException(
"Wrong file format. Please check the file's format.");
}
}
}
{code}
> Unable to load data into hive table using "Load data local inapth" command
> from unix named pipe
> -----------------------------------------------------------------------------------------------
>
> Key: HIVE-11091
> URL: https://issues.apache.org/jira/browse/HIVE-11091
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Affects Versions: 0.14.0
> Environment: Unix,MacOS
> Reporter: Manoranjan Sahoo
> Priority: Blocker
>
> Unable to load data into hive table from unix named pipe in Hive 0.14.0
> Please find below the execution details in env ( Hadoop2.6.0 + Hive 0.14.0):
> ----------------------------------------------------------------------------------------------------
> $ mkfifo /tmp/test.txt
> $ hive
> hive> create table test(id bigint,name string);
> OK
> Time taken: 1.018 seconds
> hive> LOAD DATA LOCAL INPATH '/tmp/test.txt' OVERWRITE INTO TABLE test;
> Loading data to table default.test
> Failed with exception addFiles: filesystem error in check phase
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MoveTask
> But in Hadoop 1.3 and hive 0.11.0 it works fine:
> hive> LOAD DATA LOCAL INPATH '/tmp/test.txt' OVERWRITE INTO TABLE test;
> Copying data from file:/tmp/test.txt
> Copying file: file:/tmp/test.txt
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
