Re: Hive error when loading csv data.

2012-06-27 Thread ramakanth reddy
Hi Can any help me how to start working with hadoop in single Node and cluster environment,please send me some useful links. On Wed, Jun 27, 2012 at 4:50 PM, Subir S wrote: > Pig has this CSVExcelStorage [1] and CSVLoader [2] as part of PiggyBank. It > may help. > > [1] > > http://pig.apache.or

Re: Hive error when loading csv data.

2012-06-27 Thread Subir S
Pig has this CSVExcelStorage [1] and CSVLoader [2] as part of PiggyBank. It may help. [1] http://pig.apache.org/docs/r0.9.2/api/org/apache/pig/piggybank/storage/CSVExcelStorage.html [2] http://pig.apache.org/docs/r0.9.2/api/org/apache/pig/piggybank/storage/CSVLoader.html CCed pig user-list also.

Re: Hive error when loading csv data.

2012-06-26 Thread Sandeep Reddy P
Thanks Michael Sorry i didnt get that soon. I'll try that and reply you back. On Tue, Jun 26, 2012 at 10:13 PM, Michel Segel wrote: > Sorry, > I was saying that you can write a python script that replaces the > delimiter with a | and ignore the commas within quotes. > > > Sent from a remote devi

Re: Hive error when loading csv data.

2012-06-26 Thread Michel Segel
Sorry, I was saying that you can write a python script that replaces the delimiter with a | and ignore the commas within quotes. Sent from a remote device. Please excuse any typos... Mike Segel On Jun 26, 2012, at 8:58 PM, Sandeep Reddy P wrote: > If i do that my data will be d|"abc|def"|a

Re: Hive error when loading csv data.

2012-06-26 Thread Michel Segel
What I am suggesting is to write a simple script , maybe using python, where you replace the commas that are used as field delimiter Sent from a remote device. Please excuse any typos... Mike Segel On Jun 26, 2012, at 8:58 PM, Sandeep Reddy P wrote: > If i do that my data will be d|"abc|def"

Re: Hive error when loading csv data.

2012-06-26 Thread Sandeep Reddy P
If i do that my data will be d|"abc|def"|abcd my problem is not solved On Tue, Jun 26, 2012 at 6:48 PM, Michel Segel wrote: > Yup. I just didnt add the quotes. > > Sent from a remote device. Please excuse any typos... > > Mike Segel > > On Jun 26, 2012, at 4:30 PM, Sandeep Reddy P > wrote: > > >

Re: Hive error when loading csv data.

2012-06-26 Thread Michel Segel
Yup. I just didnt add the quotes. Sent from a remote device. Please excuse any typos... Mike Segel On Jun 26, 2012, at 4:30 PM, Sandeep Reddy P wrote: > Thanks for the reply. > I didnt get that Michael. My f2 should be "abc,def" > > On Tue, Jun 26, 2012 at 4:00 PM, Michael Segel > wrote: >

Re: Hive error when loading csv data.

2012-06-26 Thread Hitesh Shah
Michael's suggestion was to change your data to: c|zxy|xyz d|abc,def|abcd and then use "|" as the delimiter. -- Hitesh On Jun 26, 2012, at 2:30 PM, Sandeep Reddy P wrote: > Thanks for the reply. > I didnt get that Michael. My f2 should be "abc,def" > > On Tue, Jun 26, 2012 at 4:00 PM, Michae

Re: Hive error when loading csv data.

2012-06-26 Thread Sandeep Reddy P
Thanks for the reply. I didnt get that Michael. My f2 should be "abc,def" On Tue, Jun 26, 2012 at 4:00 PM, Michael Segel wrote: > Alternatively you could write a simple script to convert the csv to a pipe > delimited file so that "abc,def" will be abc,def. > > On Jun 26, 2012, at 2:51 PM, Harsh J

Re: Hive error when loading csv data.

2012-06-26 Thread Michael Segel
Alternatively you could write a simple script to convert the csv to a pipe delimited file so that "abc,def" will be abc,def. On Jun 26, 2012, at 2:51 PM, Harsh J wrote: > Hive's delimited-fields-format record reader does not handle quoted > text that carry the same delimiter within them. Excel s

Re: Hive error when loading csv data.

2012-06-26 Thread Harsh J
Hive's delimited-fields-format record reader does not handle quoted text that carry the same delimiter within them. Excel supports such records, so it reads it fine. You will need to create your table with a custom InputFormat class that can handle this (Try using OpenCSV readers, they support thi