[jira] [Commented] (HADOOP-13510) "hadoop fs -getmerge" docs, .../dir does not work, .../dir/* works.

[David Sidlo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HADOOP-13510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15426869#comment-15426869
 ] 

David Sidlo commented on HADOOP-13510:
--

The issue may be based on the total dataset size.

The following command does not work. But, will work with the addition of "/*". 
The resulting file size would be 4G and 17 files get merged (when it works).
> hdfs dfs -getmerge hdfs://production/apps/hive/warehouse/dgs_tmp.db xxx

 1013  hdfs dfs -getmerge hdfs://production/apps/hive/warehouse/dgs_tmp.db/* xxx
 1019  hdfs dfs -getmerge hdfs://production/user/ds_dsidlo xxx
 1028* hdfs dfs -getmerge hdfs://production/tmp/ds_dsidlo
 1029  hdfs dfs -getmerge hdfs://production/tmp/ds_dsidlo.xx xxx


The following works, but the file size is only 1k. 
> hdfs dfs -getmerge hdfs://production/user/ds_dsidlo xxx

 1013  hdfs dfs -getmerge hdfs://production/apps/hive/warehouse/dgs_tmp.db/* xxx
 1019  hdfs dfs -getmerge hdfs://production/user/ds_dsidlo xxx
 1028* hdfs dfs -getmerge hdfs://production/tmp/ds_dsidlo
 1029  hdfs dfs -getmerge hdfs://production/tmp/ds_dsidlo.xx xxx


So, it may be that the total data set size makes a difference.


> "hadoop fs -getmerge" docs, .../dir does not work, .../dir/* works.
> ---
>
> Key: HADOOP-13510
> URL: https://issues.apache.org/jira/browse/HADOOP-13510
> Project: Hadoop Common
>  Issue Type: Bug
>Affects Versions: 2.7.1
> Environment: HDP 2.4.2
>Reporter: David Sidlo
>Priority: Minor
>  Labels: dfs, fs, getmerge, hadoop, hdfs
>
> Docs indicate that the following command would work...
>hadoop fs -getmerge -nl /src /opt/output.txt
> For me, it results in a zero-length file /opt/output.txt.
> But the following does work...
>hadoop fs -getmerge -nl /src/* /opt/output.txt



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-13510) "hadoop fs -getmerge" docs, .../dir does not work, .../dir/* works.

[Rushabh S Shah (JIRA)]

[ 
https://issues.apache.org/jira/browse/HADOOP-13510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15426502#comment-15426502
 ] 

Rushabh S Shah commented on HADOOP-13510:
-

The following command works for me.
I am running hadoop 2.7 with some internal fixes.
But none of the fixes are related to getmerge command.
{noformat}
hadoop fs -getmerge -nl /user/rushabhs/tmp1 /tmp/merge.java
{noformat}
May be if you can attach all the files in src folder (or a subset of the files 
which are enough to reproduce this failure) then I can try running the command 
with that files and check whether I encounter the bug.
But according to me it should work just fine.
Correct me if I am missing something here.

> "hadoop fs -getmerge" docs, .../dir does not work, .../dir/* works.
> ---
>
> Key: HADOOP-13510
> URL: https://issues.apache.org/jira/browse/HADOOP-13510
> Project: Hadoop Common
>  Issue Type: Bug
>Affects Versions: 2.7.1
> Environment: HDP 2.4.2
>Reporter: David Sidlo
>Priority: Minor
>  Labels: dfs, fs, getmerge, hadoop, hdfs
>
> Docs indicate that the following command would work...
>hadoop fs -getmerge -nl /src /opt/output.txt
> For me, it results in a zero-length file /opt/output.txt.
> But the following does work...
>hadoop fs -getmerge -nl /src/* /opt/output.txt



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org