Custom Loadfunc problem!

2009-10-25 Thread RichardGUO Fei
Hi, I implemented a custom loadfunc class to read data from a database. This class implements both Slicer and LoadFunc. It also generates the slices properly. The problem is that when I perform this query: A = LOAD 'table_name' USING CustomStorage; DUMP A; The PIG gives the

RE: Custom Loadfunc problem!

2009-10-26 Thread RichardGUO Fei
Hi Richard, I am not sure what's your problem, but I suggest you can debug your program in pig's local mode. And you can even debug it in pig's mapreduce mode locally Jeff zhang 2009/10/25 RichardGUO Fei gladiato...@hotmail.com Hi, I implemented a custom loadfunc

RE: Custom Loadfunc problem!

2009-10-26 Thread RichardGUO Fei
, while dump and store will execute it. So I think there must be something wrong with your custom slicer. I suggest you set breakpoint in your slicer and debug it in map reduce mode locally -Original Message- From: RichardGUO Fei [mailto:gladiato...@hotmail.com] Sent: 20091026 0:43

RE: Custom Loadfunc problem!

2009-10-26 Thread RichardGUO Fei
? -D 2009/10/26 RichardGUO Fei gladiato...@hotmail.com: Hi, This is the rough source codes of the slicer/loadfunc: public class HadoopStoreStorage extends Utf8StorageConverter implements LoadFunc, Slicer { private static final Log LOG = LogFactory.getLog

RE: Custom Loadfunc problem!

2009-10-28 Thread RichardGUO Fei
the job, while dump and store will execute it. So I think there must be something wrong with your custom slicer. I suggest you set breakpoint in your slicer and debug it in map reduce mode locally -Original Message- From: RichardGUO Fei [mailto:gladiato...@hotmail.com] Sent

RE: Custom Loadfunc problem!

2009-10-28 Thread RichardGUO Fei
if you are doing something that's not file-based. Try changing your code to expect the location string to start with some scheme, such as abcd:/ (or perhaps something more meaningful) and things should work. -D 2009/10/28 RichardGUO Fei gladiato...@hotmail.com: It seems that the real

RE: RequiredFields contents

2009-11-05 Thread RichardGUO Fei
Hi, I am also curious. I studied this and I guess that it could be the input index. For example, foreach A generate Here A's index is 0 in the inputs of the operator foreach. Let me know if I am wrong. Thanks, Richard Date: Fri, 6 Nov 2009 00:04:36 -0500 Subject: RequiredFields