Re: Error "ERROR 2088: Fetch failed. Couldn't retrieve result" happened during "HCatLoader() then DUMP"

Lorand Bendig Thu, 16 Oct 2014 11:30:16 -0700

Hi Lulynn,

I've opened PIG-4238 and HIVE-8484.


--Lorand

On 16/10/14 04:55, lulynn_2008 wrote:

Hi Lorand,

Have you opened JIRA for Hive side?

Thanks









At 2014-10-15 15:53:41, "Lorand Bendig" <lben...@gmail.com> wrote:

Hi Rohini and Lulynn,

The exception occurs when DUMP is executed:
X = load 'junit_unparted_basic' using
org.apache.hive.hcatalog.pig.HCatLoader();
dump X

I made some tests on Hive 0.14 / Pig 0.14 and found a bug.
When HCatLoader#setLocation(String location, Job job) is called on the
Hive side, it
can't set the outputschema because
HCatUtil#checkJobContextIfRunningFromBackend(job)
always returns false. This is because in fetch mode we don't have a
mapred.task.id.
A null outputschema will raise an exception when
HCatBaseLoader#getNext() calls
PigHCatUtil#transformToTuple(hr, outputSchema);

I can think of two solutions:
1. At Hive side:
    If a job is fetchable then we set
PigImplConstants.CONVERTED_TO_FETCH in the PigContext,
    which we could check in
HCatUtil#checkJobContextIfRunningFromBackend(job) .

2. At Pig side:
    Set a dummy mapred.task.id when fetch is initialized so that
    HCatUtil#checkJobContextIfRunningFromBackend(job) can return true.

I'd go with the first one.

Thanks,
Lorand

On 15/10/14 06:55, Rohini Palaniswamy wrote:

Lorand,
     Isn't fetch optimization supposed to be only for DUMP and not STORE ?

-Rohini

On Tue, Oct 14, 2014 at 6:47 PM, lulynn_2008 <lulynn_2...@163.com> wrote:

Hi Lorand,
The query run fine is I disable fetch. Thanks for your help. Could you
tell why we need to disable fetch?
BTW, I was using pig-0.13.0 and hive-0.13.1.









At 2014-10-14 21:03:13, "Lorand Bendig" <lben...@gmail.com> wrote:

Hi,

Which Pig and Hive versions do you use? Does the query run fine if you
disable fetch (set opt.fetch false) ?

Thanks,
Lorand

On 14/10/14 10:50, lulynn_2008 wrote:

Hi All,
I was running HCatStore and HCatLoader in pig grunt. But encounter

"ERROR 2088: Fetch failed. Couldn't retrieve result". Please help give a
glance and give your suggestions. Thanks.

Test case:
1. Create table in hive:
create table junit_unparted_basic(a int, b string) stored as RCFILE

tblproperties('hcat.isd'='org.apache.hive.hcatalog.rcfile.RCFileInputDriver','hcat.osd'='org.apache.hive.hcatalog.rcfile.RCFileOutputDriver');

2. copy basic.input.data file into hdfs, here is the content in file:
1    S1S
1    S2S
1    S3S
2    S1S
2    S2S
2    S3S
3    S1S
3    S2S
3    S3S

3. run Pig: pig -useHCatalog
4. grunt> A = load 'basic.input.data' as (a:int, b:chararray);
5. grunt> store A into 'junit_unparted_basic' using

org.apache.hive.hcatalog.pig.HCatStorer();

6. X = load 'junit_unparted_basic' using

org.apache.hive.hcatalog.pig.HCatLoader();

7. grunt> dump X

Error Log:

================================================================================

Pig Stack Trace
---------------
ERROR 2088: Fetch failed. Couldn't retrieve result

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable

to open iterator for alias X

           at org.apache.pig.PigServer.openIterator(PigServer.java:912)
           at

org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:752)

at

org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:372)

at

org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:228)

at

org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:203)

           at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:66)
           at org.apache.pig.Main.run(Main.java:542)
           at org.apache.pig.Main.main(Main.java:156)
           at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
           at

sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94)

at

sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)

           at java.lang.reflect.Method.invoke(Method.java:619)
           at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store

alias X

           at org.apache.pig.PigServer.storeEx(PigServer.java:1015)
           at org.apache.pig.PigServer.store(PigServer.java:974)
           at org.apache.pig.PigServer.openIterator(PigServer.java:887)
           ... 12 more
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR

2088: Fetch failed. Couldn't retrieve result

at

org.apache.pig.backend.hadoop.executionengine.fetch.FetchLauncher.runPipeline(FetchLauncher.java:180)

at

org.apache.pig.backend.hadoop.executionengine.fetch.FetchLauncher.launchPig(FetchLauncher.java:81)

at

org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.launchPig(HExecutionEngine.java:275)

           at org.apache.pig.PigServer.launchPlan(PigServer.java:1367)
           at

org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1352)

           at org.apache.pig.PigServer.storeEx(PigServer.java:1011)
           ... 14 more

================================================================================

Re: Error "ERROR 2088: Fetch failed. Couldn't retrieve result" happened during "HCatLoader() then DUMP"

Reply via email to