[
https://issues.apache.org/jira/browse/HCATALOG-239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13215793#comment-13215793
]
[email protected] commented on HCATALOG-239:
--------------------------------------------------------
bq. On 2012-02-24 17:36:48, Vikram Dixit Kumaraswamy wrote:
bq. > trunk/src/java/org/apache/hcatalog/mapreduce/HCatBaseInputFormat.java,
line 171
bq. > <https://reviews.apache.org/r/3945/diff/2/?file=85622#file85622line171>
bq. >
bq. > Isn't the JobContext a local copy and not affect future computations
even if modified at this stage?
stuff that's set in jobcontext here gets passed downstream if not then all the
setting jobconf here would be all for nothing :-).
bq. On 2012-02-24 17:36:48, Vikram Dixit Kumaraswamy wrote:
bq. > trunk/src/java/org/apache/hcatalog/mapreduce/HCatBaseInputFormat.java,
line 144
bq. > <https://reviews.apache.org/r/3945/diff/2/?file=85622#file85622line144>
bq. >
bq. > PartInfo.jobProperties is set as part of InitializeInput for each
partition. I am not sure of partInfo will get the location of previous
partInfo.. We should chat sometime today about this.
Hmmm it's a bit hard to explain, can you see the difference between the two? In
any case this is file specific stuff it would be cleaner to put setInputPath in
FosterStorageHandler.
Job localJob = new Job(conf);
setInputPath(jobContext, partitionInfo.getLocation());
and
setInputPath(jobContext, partitionInfo.getLocation());
Job localJob = new Job(conf);
- Francis
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3945/#review5321
-----------------------------------------------------------
On 2012-02-24 03:32:30, Vikram Dixit Kumaraswamy wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3945/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-02-24 03:32:30)
bq.
bq.
bq. Review request for Sushanth Sowmyan and Francis Liu.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Incorporated Francis' comments.
bq.
bq.
bq. This addresses bug HCATALOG-239.
bq. https://issues.apache.org/jira/browse/HCATALOG-239
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/src/java/org/apache/hcatalog/common/ErrorType.java 1292521
bq. trunk/src/java/org/apache/hcatalog/common/HCatUtil.java 1292521
bq. trunk/src/java/org/apache/hcatalog/data/DataType.java 1292521
bq.
trunk/src/java/org/apache/hcatalog/data/HCatRecordObjectInspectorFactory.java
1292521
bq. trunk/src/java/org/apache/hcatalog/data/LazyHCatRecord.java 1292521
bq.
trunk/src/java/org/apache/hcatalog/mapreduce/DefaultOutputCommitterContainer.java
1292521
bq.
trunk/src/java/org/apache/hcatalog/mapreduce/FileOutputCommitterContainer.java
1292521
bq.
trunk/src/java/org/apache/hcatalog/mapreduce/FileOutputFormatContainer.java
1292521
bq. trunk/src/java/org/apache/hcatalog/mapreduce/FosterStorageHandler.java
1292521
bq. trunk/src/java/org/apache/hcatalog/mapreduce/HCatBaseInputFormat.java
1292521
bq. trunk/src/java/org/apache/hcatalog/mapreduce/HCatBaseOutputFormat.java
1292521
bq. trunk/src/java/org/apache/hcatalog/mapreduce/HCatOutputFormat.java
1292521
bq. trunk/src/java/org/apache/hcatalog/mapreduce/HCatRecordReader.java
1292521
bq. trunk/src/java/org/apache/hcatalog/mapreduce/HCatSplit.java 1292521
bq. trunk/src/java/org/apache/hcatalog/mapreduce/HCatTableInfo.java 1292521
bq. trunk/src/java/org/apache/hcatalog/mapreduce/InitializeInput.java
1292521
bq. trunk/src/java/org/apache/hcatalog/mapreduce/InputJobInfo.java 1292521
bq. trunk/src/java/org/apache/hcatalog/mapreduce/InternalUtil.java 1292521
bq. trunk/src/java/org/apache/hcatalog/mapreduce/PartInfo.java 1292521
bq. trunk/src/java/org/apache/hcatalog/pig/HCatLoader.java 1292521
bq. trunk/src/test/org/apache/hcatalog/data/TestHCatRecordSerDe.java 1292521
bq. trunk/src/test/org/apache/hcatalog/data/TestLazyHCatRecord.java 1292521
bq.
trunk/src/test/org/apache/hcatalog/mapreduce/TestHCatHiveCompatibility.java
1292521
bq. trunk/src/test/org/apache/hcatalog/pig/TestHCatLoaderComplexSchema.java
1292521
bq.
bq. Diff: https://reviews.apache.org/r/3945/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Vikram
bq.
bq.
> Changes to HCatInputFormat to make it use SerDes instead of StorageDrivers
> --------------------------------------------------------------------------
>
> Key: HCATALOG-239
> URL: https://issues.apache.org/jira/browse/HCATALOG-239
> Project: HCatalog
> Issue Type: Sub-task
> Components: storage handlers
> Affects Versions: 0.4
> Reporter: Alan Gates
> Assignee: Vikram Dixit K
> Fix For: 0.4
>
> Attachments: 239.patch, 239_1.patch, 239_2.patch, 239_3.patch,
> 239_4.patch, 239_5.patch, 239_6.patch, 239_7.patch
>
>
> This JIRA covers changes to HCatInputFormat and InputJobInfo. See
> HCATALOG-237 for details and design notes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
