[GitHub] incubator-hawq pull request #1335: Hawq 1536

[yk-st]

GitHub user yk-st opened a pull request:

https://github.com/apache/incubator-hawq/pull/1335

Hawq 1536

As Cloudera has mentioned below

https://www.cloudera.com/documentation/enterprise/5-12-x/topics/cm_ig_parcels.html

Hadoop jar was installed in /opt/cloudera/parcels/CDH/ from the Cloudera 
Manager(parcel), from yum or somthing(package) , installed in /usr/lib/hadoop.
To deal with both parcel and package, Replace CDH_HADOOP_JAR with specific 
value.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/yk-st/incubator-hawq HAWQ-1536

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/1335.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1335


commit 46b118bbbebd93098a1f7cb0ba763d644f36fb7e
Author: yuki 
Date:   2018-01-29T14:02:03Z

HAWQ-1536 work in progress

commit 8990a2259abd8328e55b214f0aefaa89991dc35d
Author: yuki saito 
Date:   2018-01-31T03:54:29Z

HAWQ-1536 add setting for cloudera manager

commit b4697bbf616aa24149da7a09a72b2398137b41e2
Author: yuki saito 
Date:   2018-01-31T04:07:22Z

HAWQ-1536 add setting for cloudera manager




---

[GitHub] incubator-hawq issue #1334: Don't ignore exceptions during bridge.endIterati...

[radarwave]

Github user radarwave commented on the issue:

https://github.com/apache/incubator-hawq/pull/1334
  
Any Jira number for this PR and commit? Thanks.


---

[GitHub] incubator-hawq issue #1329: HAWQ-1579. fix logging issue with null Metadata

[radarwave]

Github user radarwave commented on the issue:

https://github.com/apache/incubator-hawq/pull/1329
  
@deem0n Thanks for the contribution, please help to close this PR since 
it's already been merged. 


---

[GitHub] incubator-hawq pull request #1334: Don't ignore exceptions during bridge.end...

[denalex]

Github user denalex commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1334#discussion_r164928664
  
--- Diff: 
pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/rest/WritableResource.java
 ---
@@ -159,20 +159,16 @@ private static Response writeResponse(Bridge bridge,
 ++totalWritten;
 }
 } catch (ClientAbortException e) {
-LOG.debug("Remote connection closed by HAWQ", e);
+LOG.error("Remote connection closed by HAWQ", e);
 } catch (Exception ex) {
-LOG.debug("totalWritten so far " + totalWritten + " to " + 
path);
+LOG.error("totalWritten so far " + totalWritten + " to " + 
path);
 throw ex;
 } finally {
-try {
-bridge.endIteration();
-} catch (Exception e) {
-// ignore ... any significant errors should already have 
been handled
-}
+bridge.endIteration();
--- End diff --

I do not see exception being logged: `LOG.error("totalWritten so far " + 
totalWritten + " to " + path);` and only the latest one will be returned to 
client, not the original which potentially caused the issue


---

[GitHub] incubator-hawq pull request #1334: Don't ignore exceptions during bridge.end...

[lavjain]

Github user lavjain commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1334#discussion_r164928261
  
--- Diff: 
pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/rest/WritableResource.java
 ---
@@ -159,20 +159,16 @@ private static Response writeResponse(Bridge bridge,
 ++totalWritten;
 }
 } catch (ClientAbortException e) {
-LOG.debug("Remote connection closed by HAWQ", e);
+LOG.error("Remote connection closed by HAWQ", e);
 } catch (Exception ex) {
-LOG.debug("totalWritten so far " + totalWritten + " to " + 
path);
+LOG.error("totalWritten so far " + totalWritten + " to " + 
path);
 throw ex;
 } finally {
-try {
-bridge.endIteration();
-} catch (Exception e) {
-// ignore ... any significant errors should already have 
been handled
-}
+bridge.endIteration();
--- End diff --

Both (original catch exception as well endIteration) are logging at error 
level. Any exception would be propagated to the client.


---

[GitHub] incubator-hawq pull request #1334: Don't ignore exceptions during bridge.end...

[denalex]

Github user denalex commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1334#discussion_r164913643
  
--- Diff: 
pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/rest/WritableResource.java
 ---
@@ -159,20 +159,16 @@ private static Response writeResponse(Bridge bridge,
 ++totalWritten;
 }
 } catch (ClientAbortException e) {
-LOG.debug("Remote connection closed by HAWQ", e);
+LOG.error("Remote connection closed by HAWQ", e);
 } catch (Exception ex) {
-LOG.debug("totalWritten so far " + totalWritten + " to " + 
path);
+LOG.error("totalWritten so far " + totalWritten + " to " + 
path);
 throw ex;
 } finally {
-try {
-bridge.endIteration();
-} catch (Exception e) {
-// ignore ... any significant errors should already have 
been handled
-}
+bridge.endIteration();
--- End diff --

if Exception happened in try and is rethrown by catch and also endIteration 
throws an exception, then the original one will be forgotten and the latest one 
form endIteration will be thrown. If this is ok, we should at least log at 
error level the original one in catch block. Alternatively, have an error flag 
set by catch and then ignore exception here only if the flag is set to have the 
original one propagate to the client.


---

[GitHub] incubator-hawq pull request #1334: Don't ignore exceptions during bridge.end...

[denalex]

Github user denalex commented on a diff in the pull request:

https://github.com/apache/incubator-hawq/pull/1334#discussion_r164912937
  
--- Diff: 
pxf/pxf-service/src/main/java/org/apache/hawq/pxf/service/rest/WritableResource.java
 ---
@@ -159,20 +159,16 @@ private static Response writeResponse(Bridge bridge,
 ++totalWritten;
 }
 } catch (ClientAbortException e) {
-LOG.debug("Remote connection closed by HAWQ", e);
+LOG.error("Remote connection closed by HAWQ", e);
--- End diff --

this was not considered an error, for some reason, we are not throwing 
exception in this case, so maybe warning level ?


---

[GitHub] incubator-hawq pull request #1334: Don't ignore exceptions during bridge.end...

[lavjain]

GitHub user lavjain opened a pull request:

https://github.com/apache/incubator-hawq/pull/1334

Don't ignore exceptions during bridge.endIteration when writing to HDFS



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/lavjain/incubator-hawq fix_writable

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-hawq/pull/1334.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1334


commit 36eaff0ed2aea5428aca86331fa515824e4fc328
Author: lavjain 
Date:   2018-01-30T07:16:14Z

Don't ignore exceptions during bridge.endIteration when writing to HDFS




---

[GitHub] incubator-hawq pull request #1309: HAWQ-1545. Fix pxf-private.classpath file...

[lavjain]

Github user lavjain closed the pull request at:

https://github.com/apache/incubator-hawq/pull/1309


---

[GitHub] incubator-hawq pull request #1330: HAWQ-1580. Change PXF port number to 5888...

[lavjain]

Github user lavjain closed the pull request at:

https://github.com/apache/incubator-hawq/pull/1330


---

[jira] [Updated] (HAWQ-1575) Implement readable Parquet profile

[Radar Lei (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HAWQ-1575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Radar Lei updated HAWQ-1575:

Fix Version/s: (was: 2.3.0.0-incubating)
   2.4.0.0-incubating

> Implement readable Parquet profile
> --
>
> Key: HAWQ-1575
> URL: https://issues.apache.org/jira/browse/HAWQ-1575
> Project: Apache HAWQ
>  Issue Type: New Feature
>  Components: PXF
>Reporter: Oleksandr Diachenko
>Assignee: Ed Espino
>Priority: Major
> Fix For: 2.4.0.0-incubating
>
>
> PXF should be able to read data from Parquet files stored in HDFS.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)