Hi all,
Just want to share with you my experience on using hdfs's glob syntax in
shell, it takes me a long time to figure out the problem.
I have a program that will accept an input path-pattern, and I will run
command like this "hadoop jar MainClass -input /2011/08/{01,02}"
And if you run this command, shell will expand this command to "hadoop jar
MainClass -input /2011/08/01 /2011/08/02", then you will find that you will
always load the data of /2011/08/01 and the data of /2011/08/02 won't be
loaded
One solution to fix this problem is putting quotation around the
path-pattern to prevent shell syntax expansion, invoke command like this
"hadoop jar MainClass -input '/2011/08/{01,02}' "
--
Best Regards
Jeff Zhang
