Hi, The HiveColumnarLoader can only read files written by hive or the hive API(s), and has its own InputFormat returning the HiveRCRecordReader.
Are you trying to read a plain text format? Under the hood the HiveRCRecordReader uses the hive specific rc reader to read the input file and throws an error either if the file is not hive rc or is a corrupt hiverc. If what you want is a Loader that loads all types of files, have a look at the AllLoader (latest piggybank trunk). It uses configuration that you set in the pig.properties to decide on the fly what loader to use for what files (does extension, content and path matching), it also has the hive style path partitioning for dates etc. Using this loader you can point it at a directoy with lzo, gz, bz2 hiverc etc files in it and if you setup the loaders correctly it will load each file with its preconfigured loader. The javadoc in the class explains how to configure it. Cheers, Gerrit -----Original Message----- From: Jae Lee [mailto:jae....@forward.co.uk] Sent: Wednesday, December 01, 2010 12:33 PM To: dev@pig.apache.org Subject: has anyone tried using HiveColumnarLoader over TextFile fileformat? Hi everyone. I've tried using HiveColumnarLoader and getting java.io.IOException: hdfs://file_path not a RCFile I've noticed HiveColumnarLoader is expecting HiveRCRecordReader from prepareToRead method.. Could you guys give any guidance how possible it is to modify HiveRCRecordReader to support any RecordReader? J
