[
https://issues.apache.org/jira/browse/HADOOP-13510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15426869#comment-15426869
]
David Sidlo commented on HADOOP-13510:
--------------------------------------
The issue may be based on the total dataset size.
The following command does not work. But, will work with the addition of "/*".
The resulting file size would be 4G and 17 files get merged (when it works).
> hdfs dfs -getmerge hdfs://production/apps/hive/warehouse/dgs_tmp.db xxx
1013 hdfs dfs -getmerge hdfs://production/apps/hive/warehouse/dgs_tmp.db/* xxx
1019 hdfs dfs -getmerge hdfs://production/user/ds_dsidlo xxx
1028* hdfs dfs -getmerge hdfs://production/tmp/ds_dsidlo
1029 hdfs dfs -getmerge hdfs://production/tmp/ds_dsidlo.xx xxx
The following works, but the file size is only 1k.
> hdfs dfs -getmerge hdfs://production/user/ds_dsidlo xxx
1013 hdfs dfs -getmerge hdfs://production/apps/hive/warehouse/dgs_tmp.db/* xxx
1019 hdfs dfs -getmerge hdfs://production/user/ds_dsidlo xxx
1028* hdfs dfs -getmerge hdfs://production/tmp/ds_dsidlo
1029 hdfs dfs -getmerge hdfs://production/tmp/ds_dsidlo.xx xxx
So, it may be that the total data set size makes a difference.
> "hadoop fs -getmerge" docs, .../dir does not work, .../dir/* works.
> -------------------------------------------------------------------
>
> Key: HADOOP-13510
> URL: https://issues.apache.org/jira/browse/HADOOP-13510
> Project: Hadoop Common
> Issue Type: Bug
> Affects Versions: 2.7.1
> Environment: HDP 2.4.2
> Reporter: David Sidlo
> Priority: Minor
> Labels: dfs, fs, getmerge, hadoop, hdfs
>
> Docs indicate that the following command would work...
> hadoop fs -getmerge -nl /src /opt/output.txt
> For me, it results in a zero-length file /opt/output.txt.
> But the following does work...
> hadoop fs -getmerge -nl /src/* /opt/output.txt
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
