[
https://issues.apache.org/jira/browse/HCATALOG-308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228678#comment-13228678
]
[email protected] commented on HCATALOG-308:
--------------------------------------------------------
bq. On 2012-03-13 16:33:21, Francis Liu wrote:
bq. > trunk/src/java/org/apache/hcatalog/mapreduce/InitializeInput.java, line
206
bq. > <https://reviews.apache.org/r/4309/diff/1/?file=91633#file91633line206>
bq. >
bq. > Not related. Didn't know this was still here.
bq.
bq. Sushanth Sowmyan wrote:
bq. Should be removed as part of another patch that was scheduled to
remove storagedriver remnants as it's likely to affect some test cases, not
addressing in this patch.
bq.
bq. Sushanth Sowmyan wrote:
bq. Actually, looking at HCatUtil.configureOutputStorageHandler , other
behaviour needs a change first - this is better handled in a separate jira, I'm
going to create one for it.
bq.
bq. In that function, you seem to copy all the entries of JobContext into
tableDesc.jobProperties. Why is that? That really shouldn't be happening as
that bloats up jobProperties significantly, also, assuming you instantiated the
storageHandler using ReflectionUtils, don't you already get the conf set on it?
I guess it didn't matter when TableDesc was created and thrown away, but not if
used for a unified jobProperties.
bq.
bq.
Edit - ugh, ignore above comment, replied to the wrong comment.
bq. On 2012-03-13 16:33:21, Francis Liu wrote:
bq. > trunk/src/java/org/apache/hcatalog/mapreduce/HCatStorageHandler.java,
line 74
bq. > <https://reviews.apache.org/r/4309/diff/1/?file=91632#file91632line74>
bq. >
bq. > we should change output at the same time to keep things consistent.
bq.
bq. Sushanth Sowmyan wrote:
bq. Oh, I thought that was the subject of another jira that you'd taken on
at the end of the other meeting we had. Let me see if I can quickly change this
too.
Actually, looking at HCatUtil.configureOutputStorageHandler , other behaviour
needs a change first - this is better handled in a separate jira, I'm going to
create one for it.
In that function, you seem to copy all the entries of JobContext into
tableDesc.jobProperties. Why is that? That really shouldn't be happening as
that bloats up jobProperties significantly, also, assuming you instantiated the
storageHandler using ReflectionUtils, don't you already get the conf set on it?
I guess it didn't matter when TableDesc was created and thrown away, but not if
used for a unified jobProperties.
- Sushanth
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4309/#review5876
-----------------------------------------------------------
On 2012-03-13 12:09:33, Sushanth Sowmyan wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4309/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-03-13 12:09:33)
bq.
bq.
bq. Review request for Alan Gates and Francis Liu.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. + changing signature of configureInputJobProperties to take only
TableDesc, and to use its jobProperties
bq. + calling configureInputJobProperties at a table/job level, not at a
partition level.
bq.
bq.
bq. This addresses bug HCATALOG-308.
bq. https://issues.apache.org/jira/browse/HCATALOG-308
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/src/java/org/apache/hcatalog/common/HCatUtil.java 1299936
bq. trunk/src/java/org/apache/hcatalog/mapred/HCatMapredInputFormat.java
1299936
bq. trunk/src/java/org/apache/hcatalog/mapred/HCatMapredOutputFormat.java
1299936
bq. trunk/src/java/org/apache/hcatalog/mapred/HiveHCatSplitWrapper.java
1299936
bq. trunk/src/java/org/apache/hcatalog/mapreduce/FosterStorageHandler.java
1299936
bq. trunk/src/java/org/apache/hcatalog/mapreduce/HCatStorageHandler.java
1299936
bq. trunk/src/java/org/apache/hcatalog/mapreduce/InitializeInput.java
1299936
bq. trunk/src/java/org/apache/hcatalog/mapreduce/InputJobInfo.java 1299936
bq.
trunk/src/java/org/apache/hcatalog/storagehandler/HCatStorageHandlerImpl.java
1299936
bq. trunk/src/test/org/apache/hcatalog/cli/DummyStorageHandler.java 1299936
bq.
trunk/storage-handlers/hbase/src/java/org/apache/hcatalog/hbase/HBaseHCatStorageHandler.java
1299936
bq.
bq. Diff: https://reviews.apache.org/r/4309/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Sushanth
bq.
bq.
> configureInputJobProperties(TableDesc) related changes
> ------------------------------------------------------
>
> Key: HCATALOG-308
> URL: https://issues.apache.org/jira/browse/HCATALOG-308
> Project: HCatalog
> Issue Type: Sub-task
> Reporter: Sushanth Sowmyan
> Assignee: Sushanth Sowmyan
> Fix For: 0.4
>
> Attachments: HCATALOG-308.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
