Hi, Gagang It seems you use HA hdfs cluster? I am not sure if HAWQ can work like this. Can any HAWQ developer clarify this condition? If so, you can try a non-HA hdfs cluster with direct IP access. All PXF services are working perfect here.
On Fri, Mar 11, 2016 at 10:25 AM, Gagan Brahmi <[email protected]> wrote: > Thank you Ting! > > That was the problem. It seemed to have worked, but now I am stuck > with a different error. > > gagan=# INSERT into ext_put_foo VALUES (1, 'Gagan'); > ERROR: failed sending to remote component (libchurl.c:574) (seg0 > my-hadoop-cluster:40000 pid=24563) (dispatcher.c:1753) > > This certainly mean that the back ground service has stopped serving > connection for some reason. > > I check the namenode and find this. > > 2016-03-10 19:28:11,759 INFO hdfs.StateChange > (FSNamesystem.java:completeFile(3503)) - DIR* completeFile: > /tmp/foo_bar/1350_0 is closed by DFSClient_NONMAPREDUCE_-244490296_23 > > I have a single node installation with a HDFS replication factor of 1 > (both in hdfs-site and hdfs-client for hawq). > > I have also tried to update the connectTimeout value to 60 secs in the > server.xml file for pxf webapp. > > A normal write to HDFS works fine. I see file being created in the > directory foor_bar but are 0 bytes in size. > > -rw-r--r-- 1 pxf hdfs 0 2016-03-10 19:08 /tmp/foo_bar/1336_0 > -rw-r--r-- 1 pxf hdfs 0 2016-03-10 19:27 /tmp/foo_bar/1349_0 > -rw-r--r-- 1 pxf hdfs 0 2016-03-10 19:28 /tmp/foo_bar/1350_0 > > Not sure if someone has encountered this before. Would appreciate any > inputs. > > > Regards, > Gagan Brahmi > > On Thu, Mar 10, 2016 at 11:45 AM, Ting(Goden) Yao <[email protected]> wrote: > > Your table definition: > > ('pxf://my-hadoop-cluster:*50070*/foo_bar?profile=HdfsTextSimple') > > if you installed pxf on 51200, you need to use the port 51200 > > > > > > On Thu, Mar 10, 2016 at 10:34 AM Gagan Brahmi <[email protected]> > wrote: > > > >> Hi Team, > >> > >> I was wondering if someone has encountered this problem before. > >> > >> While trying to work with PXF on hawq 2.0 I am encountering the > following > >> error: > >> > >> gagan=# CREATE EXTERNAL TABLE ext_get_foo (i int, bar text) LOCATION > >> ('pxf://my-hadoop-cluster:50070/foo_bar?profile=HdfsTextSimple') > >> FORMAT 'text' (delimiter '|' null 'null'); > >> > >> gagan=# SELECT * FROM ext_get_foo ; > >> ERROR: remote component error (404): PXF service could not be > >> reached. PXF is not running in the tomcat container (libchurl.c:878) > >> > >> The same happens when I try to write to an external table using PXF. > >> > >> I believe the above error signifies that PXF service isn't running or > >> unavailable. But PXF is running on port 51200. > >> > >> Curl response works fine as well: > >> > >> # curl -s http://localhost:51200/pxf/v0 > >> Wrong version v0, supported version is v14 > >> > >> PXF is build using gradlew and installed as RPM files. I also have > >> tomcat 7.0.62 installed with the PXF packages. > >> > >> The following is how PXF is running on the instance: > >> > >> pxf 21405 0.3 2.8 825224 115164 ? Sl 02:07 0:10 > >> /usr/java/latest/bin/java > >> > >> > -Djava.util.logging.config.file=/var/pxf/pxf-service/conf/logging.properties > >> -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager > >> -Xmx512M -Xss256K -Djava.endorsed.dirs=/var/pxf/pxf-service/endorsed > >> -classpath > >> > /var/pxf/pxf-service/bin/bootstrap.jar:/var/pxf/pxf-service/bin/tomcat-juli.jar > >> -Dcatalina.base=/var/pxf/pxf-service > >> -Dcatalina.home=/var/pxf/pxf-service > >> -Djava.io.tmpdir=/var/pxf/pxf-service/temp > >> org.apache.catalina.startup.Bootstrap start > >> > >> I do not have apache-tomcat running. Not sure how are the two > >> interrelated. But the RPM file created by gradlew requires tomcat for > >> pxf-service. > >> > >> I would appreciate any inputs into this problem. > >> > >> > >> Regards, > >> Gagan Brahmi > >> >
