Modifying the script to precede the put with a mkdir (which creates the missing subdirectories) fixes the issue.thanksJohn
Date: Wed, 16 Jan 2013 12:36:20 -0500 Subject: Re: does "fs -put " create subdirectories? From: [email protected] To: [email protected] Good point Harsh. As a Linux Admin, I prefer the behavior of 2.x. It allows me to see if I've made a mistake in my planned placement of files instead of blindly writing. On Wed, Jan 16, 2013 at 12:05 PM, Harsh J <[email protected]> wrote: On 1.x, -put does mkdir the parent directories if they are non existent (akin to mkdir -p). On 2.x, -put does not do this - we need the dir to pre-exist or mkdir'd before -put. ➜ ~ hadoop fs -put foo a/bc/dfe/fsdf.tput: `a/bc/dfe/fsdf.t': No such file or directory On Wed, Jan 16, 2013 at 10:28 PM, Jean-Marc Spaggiari <[email protected]> wrote: Yes it does, you can just try ;) hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/ Found 1 items drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt /user/test/CHANGES.txt hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/ Found 2 items drwxr-xr-x - hbase supergroup 0 2013-01-03 09:54 /user/hbase drwxr-xr-x - hadoop supergroup 0 2013-01-16 11:56 /user/test hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test/ Found 1 items -rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:56 /user/test/CHANGES.txt hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -put CHANGES.txt /user/test2/subdir/anotherone/CHANGES.txt hadoop@node3:~/hadoop-1.0.3$ bin/hadoop fs -ls /user/test2/subdir/anotherone/ Found 1 items -rw-r--r-- 3 hadoop supergroup 446615 2013-01-16 11:58 /user/test2/subdir/anotherone/CHANGES.txt 2013/1/16, John Meza <[email protected]>: > Does "hadoop fs -put mmddyyyy.tsv t1/2012/01/01/mmddyyyy.tsv" create the > necassary subdirectories in hdfs?thanksJohn -- Harsh J
