As for #1, pushdownProject() is called only if it's applicable- https://github.com/apache/pig/blob/trunk/src/org/apache/pig/newplan/optimizer/PlanOptimizer.java#L108
Set a breakpoint in ColumnMapKeyPrune.java and see whether check() returns true or false- https://github.com/apache/pig/blob/trunk/src/org/apache/pig/newplan/logical/rules/ColumnMapKeyPrune.java#L85 It probably returns false in your case, and that's why your pushProjection() is never called. On Thu, Dec 19, 2013 at 6:09 AM, Costin Leau <[email protected]> wrote: > Forgot to specify the aforementioned thread [1] > > [1] http://www.mail-archive.com/[email protected]/msg06285.html > > > On 19/12/2013 4:08 PM, Costin Leau wrote: > >> Hi, >> >> I'm trying to get a hold of the schema specified for a loader through >> 'AS' using Apache Pig 0.12 : >> >> A = LOAD 'pig/tupleartists' USING MyStorage() AS (name: chararray, links: >> (url:chararray, picture:chararray)); >> B = FOREACH A GENERATE name, links.url; >> DUMP B; >> >> 1. >> My loader implements LoadPushDown#pushProjection() which does not seem to >> be called at all (tried breakpoints, >> System.out - nothing). The API docs and this thread [1] suggest it should >> be call yet in my tests (using a local >> PigServer) this does not happen. Am I missing something? >> >> 2. >> As an alternative, I'm loading the POStore objects (from pig.map.store >> and pig.reduce.store) but the schema that I'm >> getting is incorrect, namely: >> "(name: chararray, url: charray)" without any mention of the "links" >> field. Is there any way to recreate/retrieve the >> actual schema defined by the user or at least determine which fields are >> nested ("links.url") as oppose to the top level >> ones ("name")? >> >> Thanks, >> > > -- > Costin >
