[
https://issues.apache.org/jira/browse/HCATALOG-314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246753#comment-13246753
]
[email protected] commented on HCATALOG-314:
--------------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4623/#review6687
-----------------------------------------------------------
/trunk/src/java/org/apache/hcatalog/pig/HCatLoader.java
<https://reviews.apache.org/r/4623/#comment14492>
you prolly shouldn't repopulate the context on backend. So the first part
of the condition is not needed.
/trunk/src/java/org/apache/hcatalog/pig/HCatLoader.java
<https://reviews.apache.org/r/4623/#comment14495>
add comment about storing the delta.
/trunk/src/java/org/apache/hcatalog/pig/HCatLoader.java
<https://reviews.apache.org/r/4623/#comment14494>
This could be replaced with:
for(Map.Entry<String,String> keyValue : job.getConfiguration())
/trunk/src/java/org/apache/hcatalog/pig/HCatStorer.java
<https://reviews.apache.org/r/4623/#comment14498>
don't need to run this repopulate on the back end. the else statement
shouldn't run on the backend either. propbably throw an exception if
LOCATION_SET is not set and you're in the backend.
/trunk/src/java/org/apache/hcatalog/pig/HCatStorer.java
<https://reviews.apache.org/r/4623/#comment14497>
credentials shouldn't be stored in udfProps....as this gets passed to the
backend most likely in plaintext via jobconf which is unsecure. It should stay
on the frontend and just repopulate the job object. Take a look at
Security.java on how to do this.
/trunk/src/java/org/apache/hcatalog/pig/HCatStorer.java
<https://reviews.apache.org/r/4623/#comment14499>
I don't think you're gonna need this if statement anymore?
- Francis
On 2012-04-03 21:31:04, Vandana Ayyalasomayajula wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4623/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-04-03 21:31:04)
bq.
bq.
bq. Review request for hcatalog, Francis Liu and Rohini Palaniswamy.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. HCatStorer.setStoreLocation() is called more than once by pig. Which ends
up calling HCatOutputFormat.setOutput multiple times.
HCatOutputFormat.setOutput should be called only once, the values that it
changed must be tracked and reapplied to job.getconfiguration() the succeeding
times it is called. HCatLoader may have the same issue and should be
investigated as well.
bq.
bq.
bq. This addresses bug hcatalog-314.
bq. https://issues.apache.org/jira/browse/hcatalog-314
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. /trunk/src/java/org/apache/hcatalog/common/HCatConstants.java 1309001
bq. /trunk/src/java/org/apache/hcatalog/pig/HCatLoader.java 1309001
bq. /trunk/src/java/org/apache/hcatalog/pig/HCatStorer.java 1309001
bq.
bq. Diff: https://reviews.apache.org/r/4623/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. The trunk builds are currently failing. I will run the entire test suite
once the build is fine.
bq.
bq.
bq. Thanks,
bq.
bq. Vandana
bq.
bq.
> HCatOutputFormat.setOutput is called more than once by HCatStorer
> -----------------------------------------------------------------
>
> Key: HCATALOG-314
> URL: https://issues.apache.org/jira/browse/HCATALOG-314
> Project: HCatalog
> Issue Type: Bug
> Affects Versions: 0.4
> Reporter: Francis Liu
> Assignee: Vandana Ayyalasomayajula
>
> HCatStorer.setStoreLocation() is called more than once by pig. Which ends up
> calling HCatOutputFormat.setOutput multiple times. HCatOutputFormat.setOutput
> should be called only once, the values that it changed must be tracked and
> reapplied to job.getconfiguration() the succeeding times it is called.
> HCatLoader may have the same issue and should be investigated as well.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
