Just closing the loop here. This was an oversight on my part related to v1 vs v2 MapReduce APIs (thanks +Andrew Thorbur <[email protected]> for tracking this down).
I've opened https://issues.apache.org/jira/browse/HAWQ-1161 to help minimize others from getting bit by oversights similar. The latest commit <https://github.com/kdunn926/incubator-hawq/pull/1/files> shows preliminary success reading native HAWQ 1.x data (AO only, for now) with a HAWQ 2 PXF plugin. -Kyle On Tue, Nov 15, 2016 at 2:01 PM Kyle Dunn <[email protected]> wrote: > Hello, > > I'm working on a new PXF plugin to read HAWQ 1.x data using > HAWQInputFormat (aka hawq-mapreduce). The APIs seems to align relatively > well but I'm having issues with the Accessor implementation. > > The pattern needed (I think) seems to follow well with plugins using a > RecordReader class (e.g. PXF-JSON). Here is a PR for tracking the code > under development: https://github.com/kdunn926/incubator-hawq/pull/1 > > The issue I'm having is more Java-specific than PXF but maybe someone can > see something obvious I'm missing. The current Fragmenter instantiates fine > but the Accessor class is failing to compile: > > :pxf-hawqinputformat1x:compileJava > /Users/kdunn/repos/incubator-hawq/pxf/pxf-hawqinputformat1x/src/main/java/org/apache/hawq/pxf/plugins/hawqinputformat/HAWQInputFormatAccessor.java:55: > error: *HAWQInputFormatAccessor is not abstract and does not override > abstract method getReader(JobConf,InputSplit) in HdfsSplittableDataAccessor* > public class HAWQInputFormatAccessor extends HdfsSplittableDataAccessor { > ^ > /Users/kdunn/repos/incubator-hawq/pxf/pxf-hawqinputformat1x/src/main/java/org/apache/hawq/pxf/plugins/hawqinputformat/HAWQInputFormatAccessor.java:116: > error: method does not override or implement a method from a supertype > @Override > ^ > 2 errors > :pxf-hawqinputformat1x:compileJava FAILED > > The signature for *getReader()* in the derived class ( > HAWQInputFormatAccessor) is identical to the one declared abstract in the > base class (HdfsSplittableDataAccessor) so I can't wrap my head around > what's going on. Anyone have any ideas? > > > > Thanks, > Kyle > -- > *Kyle Dunn | Data Engineering | Pivotal* > Direct: 303.905.3171 <3039053171> | Email: [email protected] > -- *Kyle Dunn | Data Engineering | Pivotal* Direct: 303.905.3171 <3039053171> | Email: [email protected]
