54 -0400
> Subject: Re: Custom Loadfunc problem!
> From: dvrya...@gmail.com
> To: pig-dev@hadoop.apache.org
>
> Richard,
> I would strongly advise you against trying to mess with PigInputFormat
> and JobControlCompiler. We know that slicers work (after all, people
> manage to wri
ut the
> path passed to my storage is actually a table name rather than a file path! I
> think that the LoadFunc should be more flexible. I am trying to fix this at
> this moment.
>
>
>
> Thanks,
>
> Richard
>
>> From: zjf...@gmail.com
>> To: pig-dev@h
this at this
moment.
Thanks,
Richard
> From: zjf...@gmail.com
> To: pig-dev@hadoop.apache.org
> Subject: RE: Custom Loadfunc problem!
> Date: Tue, 27 Oct 2009 23:40:43 -0800
>
>
> I mean hadoop's local mode not pig's own local mode
>
>
> -Orig
I mean hadoop's local mode not pig's own local mode
-Original Message-
From: Dmitriy Ryaboy [mailto:dvrya...@gmail.com]
Sent: 2009年10月26日 6:33
To: pig-dev@hadoop.apache.org; pig-dev@hadoop.apache.org
Subject: RE: Custom Loadfunc problem!
Jeff,
Slicers dont work in local mode
ublic void readFields(DataInput is) throws IOException {
partitionIndex = is.readInt();
path = is.readUTF();
}
public void write(DataOutput os) throws IOException {
os.writeInt(partitionIndex);
os.writeUTF(path);
}
}
Thanks,
Richard
> Date: Mon, 2
void validate(DataStorage store, String location) throws
> IOException {
>LOG.info("Enter validate");
>
>if (!FileLocalizer.isDirectory(location, store)
> || !FileLocalizer.fileExists(location +
> GlobalParameter.TABLE_SCHEMA_RELATIVE_PATH, store)) {
>
ase let me
know if you see any problem.
I really appreciate your help!
Thanks,
Richard
> From: dvrya...@gmail.com
> Subject: RE: Custom Loadfunc problem!
> Date: Mon, 26 Oct 2009 10:33:21 -0400
> To: pig-dev@hadoop.apache.org; pig-dev@hadoop.apache.org
>
> Jeff,
>
: "zjffdu"
To: pig-dev@hadoop.apache.org
Sent: 10/27/2009 1:45 AM
Subject: RE: Custom Loadfunc problem!
Illustrate will not execute the job, while dump and store will execute it.
So I think there must be something wrong with your custom slicer. I suggest
you set breakpoint in your slicer an
...@hotmail.com]
Sent: 2009年10月26日 0:43
To: pig-dev@hadoop.apache.org
Subject: RE: Custom Loadfunc problem!
Hi,
Btw, my program works with ILLUSTRATE but not with DUMP or STORE. It could
be a problem of the slicer or POLoad? Please enlighten me.
Thanks,
Richard
> Date: Mon, 26 Oct 2009
Hi,
Btw, my program works with ILLUSTRATE but not with DUMP or STORE. It could be a
problem of the slicer or POLoad? Please enlighten me.
Thanks,
Richard
> Date: Mon, 26 Oct 2009 11:02:04 +0800
> Subject: Re: Custom Loadfunc problem!
> From: zjf...@gmail.com
>
executed?
Since Pig outputs the JobConf to a jar file, it is rather difficult to debug.
Is there any way that I can expose more information?
Thanks,
Richard
> Date: Mon, 26 Oct 2009 11:02:04 +0800
> Subject: Re: Custom Loadfunc problem!
> From: zjf...@gmail.com
>
Hi Richard,
I am not sure what's your problem, but I suggest you can debug your program
in pig's local mode.
And you can even debug it in pig's mapreduce mode locally
Jeff zhang
2009/10/25 RichardGUO Fei
>
> Hi,
>
>
>
> I implemented a custom loadfunc class to read data from a database. Th
Hi,
I implemented a custom loadfunc class to read data from a database. This class
implements both Slicer and LoadFunc. It also generates the slices properly. The
problem is that when I perform this query:
A = LOAD 'table_name' USING CustomStorage;
DUMP A;
The PIG gives the followin
13 matches
Mail list logo
