[
https://issues.apache.org/jira/browse/HCATALOG-240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207937#comment-13207937
]
[email protected] commented on HCATALOG-240:
--------------------------------------------------------
bq. On 2012-02-07 19:01:36, Alan Gates wrote:
bq. > src/java/org/apache/hcatalog/mapreduce/FileRecordWriterContainer.java,
line 264
bq. > <https://reviews.apache.org/r/3775/diff/1/?file=72857#file72857line264>
bq. >
bq. > We should add this before we commit the patch, as it should speed
things up quite a bit.
bq.
bq. Francis Liu wrote:
bq. I was hoping to make that change during integration with Sushanth's
HCatRecord patch. Should be easy enough to do before that as well.
Object Inspectors, if fetched from serdes, are usually cached by their
corresponding Factories, and do not need to be cached again. However, if it's a
single object inspector for a record type, you might as well cache it, rather
than calling the function to fetch it every time - that has little overhead,
but an overhead nonetheless. If you reference the object returned, rest assured
you are not making a copy of the object, ObjectInspectors tend to be allocated
from a static cache by type.
- Sushanth
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3775/#review4867
-----------------------------------------------------------
On 2012-02-09 07:34:56, Francis Liu wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/3775/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-02-09 07:34:56)
bq.
bq.
bq. Review request for hcatalog, Alan Gates and Sushanth Sowmyan.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. First drop. See HCATALOG-240.
bq.
bq.
bq. This addresses bug HCATALOG-240.
bq. https://issues.apache.org/jira/browse/HCATALOG-240
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. src/java/org/apache/hadoop/mapred/HCatMapRedUtil.java PRE-CREATION
bq. src/java/org/apache/hcatalog/cli/SemanticAnalysis/CreateTableHook.java
9d98f50
bq. src/java/org/apache/hcatalog/common/HCatUtil.java 13b56fd
bq. src/java/org/apache/hcatalog/mapred/HCatMapredOutputFormat.java 32ede94
bq.
src/java/org/apache/hcatalog/mapreduce/DefaultOutputCommitterContainer.java
9c5fdd3
bq. src/java/org/apache/hcatalog/mapreduce/DefaultOutputFormatContainer.java
13fa8ac
bq. src/java/org/apache/hcatalog/mapreduce/DefaultRecordWriterContainer.java
7c90737
bq. src/java/org/apache/hcatalog/mapreduce/FileOutputCommitterContainer.java
f37c5fd
bq. src/java/org/apache/hcatalog/mapreduce/FileOutputFormatContainer.java
e91ed0f
bq. src/java/org/apache/hcatalog/mapreduce/FileOutputStorageDriver.java
6596264
bq. src/java/org/apache/hcatalog/mapreduce/FileRecordWriterContainer.java
c72cb4f
bq. src/java/org/apache/hcatalog/mapreduce/FosterStorageHandler.java
PRE-CREATION
bq. src/java/org/apache/hcatalog/mapreduce/HCatBaseOutputFormat.java 1b2f9a4
bq. src/java/org/apache/hcatalog/mapreduce/HCatEximInputFormat.java 90c5671
bq. src/java/org/apache/hcatalog/mapreduce/HCatEximInputFormat.java.broken
PRE-CREATION
bq. src/java/org/apache/hcatalog/mapreduce/HCatEximOutputCommitter.java
1ba8ccd
bq.
src/java/org/apache/hcatalog/mapreduce/HCatEximOutputCommitter.java.broken
PRE-CREATION
bq. src/java/org/apache/hcatalog/mapreduce/HCatEximOutputFormat.java ed1ba66
bq. src/java/org/apache/hcatalog/mapreduce/HCatEximOutputFormat.java.broken
PRE-CREATION
bq. src/java/org/apache/hcatalog/mapreduce/HCatOutputFormat.java d7eab9e
bq. src/java/org/apache/hcatalog/mapreduce/HCatOutputStorageDriver.java
566943f
bq. src/java/org/apache/hcatalog/mapreduce/HCatStorageHandler.java
PRE-CREATION
bq. src/java/org/apache/hcatalog/mapreduce/HCatTableInfo.java e65f1d0
bq. src/java/org/apache/hcatalog/mapreduce/InternalUtil.java PRE-CREATION
bq. src/java/org/apache/hcatalog/mapreduce/OutputCommitterContainer.java
5f0585a
bq. src/java/org/apache/hcatalog/mapreduce/OutputFormatContainer.java
f73a6dc
bq. src/java/org/apache/hcatalog/mapreduce/OutputJobInfo.java 9830190
bq. src/java/org/apache/hcatalog/mapreduce/ProgressReporter.java
PRE-CREATION
bq. src/java/org/apache/hcatalog/mapreduce/RecordWriterContainer.java
9a93bfa
bq. src/java/org/apache/hcatalog/mapreduce/StorerInfo.java 8785ffb
bq. src/java/org/apache/hcatalog/pig/HCatEximLoader.java a36f808
bq. src/java/org/apache/hcatalog/pig/HCatEximLoader.java.broken PRE-CREATION
bq. src/java/org/apache/hcatalog/pig/HCatEximStorer.java b7bee44
bq. src/java/org/apache/hcatalog/pig/HCatEximStorer.java.broken PRE-CREATION
bq. src/java/org/apache/hcatalog/pig/HCatStorer.java f480321
bq.
src/test/e2e/hcatalog/udfs/java/org/apache/hcatalog/utils/GroupByAge.java
21e780c
bq.
src/test/e2e/hcatalog/udfs/java/org/apache/hcatalog/utils/HBaseReadWrite.java
d28f29b
bq. src/test/e2e/hcatalog/udfs/java/org/apache/hcatalog/utils/ReadWrite.java
1a8e6a6
bq.
src/test/e2e/hcatalog/udfs/java/org/apache/hcatalog/utils/StoreComplex.java
b08bc4b
bq. src/test/e2e/hcatalog/udfs/java/org/apache/hcatalog/utils/StoreDemo.java
6ca634e
bq.
src/test/e2e/hcatalog/udfs/java/org/apache/hcatalog/utils/StoreNumbers.java
b01d810
bq. src/test/e2e/hcatalog/udfs/java/org/apache/hcatalog/utils/WriteJson.java
037610c
bq. src/test/e2e/hcatalog/udfs/java/org/apache/hcatalog/utils/WriteRC.java
5577182
bq. src/test/e2e/hcatalog/udfs/java/org/apache/hcatalog/utils/WriteText.java
a64527f
bq. src/test/org/apache/hcatalog/cli/DummyStorageHandler.java 0e9565c
bq. src/test/org/apache/hcatalog/cli/TestPermsGrp.java f067ebc
bq. src/test/org/apache/hcatalog/cli/TestStorageHandlerProperties.java
df8f5a5
bq.
src/test/org/apache/hcatalog/cli/TestStorageHandlerProperties.java.broken
PRE-CREATION
bq. src/test/org/apache/hcatalog/mapred/TestHiveHCatInputFormat.java 28bac09
bq. src/test/org/apache/hcatalog/mapred/TestHiveHCatInputFormat.java.broken
PRE-CREATION
bq. src/test/org/apache/hcatalog/mapreduce/HCatMapReduceTest.java 1fb3cf5
bq. src/test/org/apache/hcatalog/mapreduce/TestHCatDynamicPartitioned.java
c1080ce
bq. src/test/org/apache/hcatalog/mapreduce/TestHCatEximInputFormat.java
9d94c7b
bq.
src/test/org/apache/hcatalog/mapreduce/TestHCatEximInputFormat.java.broken
PRE-CREATION
bq. src/test/org/apache/hcatalog/mapreduce/TestHCatEximOutputFormat.java
99ec02b
bq.
src/test/org/apache/hcatalog/mapreduce/TestHCatEximOutputFormat.java.broken
PRE-CREATION
bq. src/test/org/apache/hcatalog/mapreduce/TestHCatOutputFormat.java 5df0146
bq. src/test/org/apache/hcatalog/pig/TestHCatEximLoader.java 7691fb9
bq. src/test/org/apache/hcatalog/pig/TestHCatEximLoader.java.broken
PRE-CREATION
bq. src/test/org/apache/hcatalog/pig/TestHCatEximStorer.java c3ba19d
bq. src/test/org/apache/hcatalog/pig/TestHCatEximStorer.java.broken
PRE-CREATION
bq. src/test/org/apache/hcatalog/pig/TestPermsInheritance.java 78e353a
bq. src/test/org/apache/hcatalog/pig/TestPermsInheritance.java.broken
PRE-CREATION
bq. src/test/org/apache/hcatalog/pig/TestPigStorageDriver.java 36c1539
bq. src/test/org/apache/hcatalog/pig/TestPigStorageDriver.java.broken
PRE-CREATION
bq. src/test/org/apache/hcatalog/rcfile/TestRCFileOutputStorageDriver.java
eda2875
bq. storage-drivers/build.xml efd26d0
bq.
bq. Diff: https://reviews.apache.org/r/3775/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Francis
bq.
bq.
> Changes to HCatOutputFormat to make it use SerDes instead of StorageDriver
> --------------------------------------------------------------------------
>
> Key: HCATALOG-240
> URL: https://issues.apache.org/jira/browse/HCATALOG-240
> Project: HCatalog
> Issue Type: Sub-task
> Components: storage handlers
> Affects Versions: 0.4
> Reporter: Alan Gates
> Assignee: Francis Liu
> Fix For: 0.4
>
> Attachments: HCATALOG-240.patch, HCATALOG-240_2.patch,
> HCATALOG-240_3.patch
>
>
> This JIRA covers changes to HCatOutputFormat, OutputJobInfo, and
> HCatOutputCommitter. See HCATALOG-237 for design notes and details.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
