Yes, I can use, it works with pig-0.4 -----Original Message----- From: Richard Ding [mailto:[email protected]] Sent: Monday, May 10, 2010 5:13 PM To: [email protected] Subject: RE: List of directories in Load
I think you can use Hadoop globbing (as in Scott's example) with Pig 0.4. Thanks, -Richard -----Original Message----- From: Katukuri, Jay [mailto:[email protected]] Sent: Monday, May 10, 2010 10:07 AM To: [email protected] Subject: RE: List of directories in Load Does this work in Pig 0.40 also? -----Original Message----- From: Scott Carey [mailto:[email protected]] Sent: Sunday, May 09, 2010 11:36 PM To: [email protected] Subject: Re: List of directories in Load Also, pig passes on the string to HDFS, which supports globs, so this works: raw = LOAD '/input_data/dir{1,2,3}' On May 7, 2010, at 4:14 PM, Richard Ding wrote: > This feature is supported since 0.6 (PIG-1071). But the correct form is > > raw = LOAD '/input_data/dir1,/input_data/dir2,/input_data/dir3' USING > PigStorage ('\t') AS (.....); > > Thanks, > -Richard > -----Original Message----- > From: Katukuri, Jay [mailto:[email protected]] > Sent: Friday, May 07, 2010 3:53 PM > To: [email protected] > Subject: List of directories in Load > > Hi all, > Is it possible to specify multiple HDFS directories in 'Load' function. > Ex: raw = LOAD '/input_data/dir1', '/input_data/dir2', > '/input_data/dir3' USING PigStorage ('\t') AS (.....); > > Thanks, > Jay
