> On 2012-02-07 19:01:36, Alan Gates wrote: > > src/java/org/apache/hcatalog/mapreduce/FileRecordWriterContainer.java, line > > 264 > > <https://reviews.apache.org/r/3775/diff/1/?file=72857#file72857line264> > > > > We should add this before we commit the patch, as it should speed > > things up quite a bit. > > Francis Liu wrote: > I was hoping to make that change during integration with Sushanth's > HCatRecord patch. Should be easy enough to do before that as well.
Object Inspectors, if fetched from serdes, are usually cached by their corresponding Factories, and do not need to be cached again. However, if it's a single object inspector for a record type, you might as well cache it, rather than calling the function to fetch it every time - that has little overhead, but an overhead nonetheless. If you reference the object returned, rest assured you are not making a copy of the object, ObjectInspectors tend to be allocated from a static cache by type. - Sushanth ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/3775/#review4867 ----------------------------------------------------------- On 2012-02-09 07:34:56, Francis Liu wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/3775/ > ----------------------------------------------------------- > > (Updated 2012-02-09 07:34:56) > > > Review request for hcatalog, Alan Gates and Sushanth Sowmyan. > > > Summary > ------- > > First drop. See HCATALOG-240. > > > This addresses bug HCATALOG-240. > https://issues.apache.org/jira/browse/HCATALOG-240 > > > Diffs > ----- > > src/java/org/apache/hadoop/mapred/HCatMapRedUtil.java PRE-CREATION > src/java/org/apache/hcatalog/cli/SemanticAnalysis/CreateTableHook.java > 9d98f50 > src/java/org/apache/hcatalog/common/HCatUtil.java 13b56fd > src/java/org/apache/hcatalog/mapred/HCatMapredOutputFormat.java 32ede94 > src/java/org/apache/hcatalog/mapreduce/DefaultOutputCommitterContainer.java > 9c5fdd3 > src/java/org/apache/hcatalog/mapreduce/DefaultOutputFormatContainer.java > 13fa8ac > src/java/org/apache/hcatalog/mapreduce/DefaultRecordWriterContainer.java > 7c90737 > src/java/org/apache/hcatalog/mapreduce/FileOutputCommitterContainer.java > f37c5fd > src/java/org/apache/hcatalog/mapreduce/FileOutputFormatContainer.java > e91ed0f > src/java/org/apache/hcatalog/mapreduce/FileOutputStorageDriver.java 6596264 > src/java/org/apache/hcatalog/mapreduce/FileRecordWriterContainer.java > c72cb4f > src/java/org/apache/hcatalog/mapreduce/FosterStorageHandler.java > PRE-CREATION > src/java/org/apache/hcatalog/mapreduce/HCatBaseOutputFormat.java 1b2f9a4 > src/java/org/apache/hcatalog/mapreduce/HCatEximInputFormat.java 90c5671 > src/java/org/apache/hcatalog/mapreduce/HCatEximInputFormat.java.broken > PRE-CREATION > src/java/org/apache/hcatalog/mapreduce/HCatEximOutputCommitter.java 1ba8ccd > src/java/org/apache/hcatalog/mapreduce/HCatEximOutputCommitter.java.broken > PRE-CREATION > src/java/org/apache/hcatalog/mapreduce/HCatEximOutputFormat.java ed1ba66 > src/java/org/apache/hcatalog/mapreduce/HCatEximOutputFormat.java.broken > PRE-CREATION > src/java/org/apache/hcatalog/mapreduce/HCatOutputFormat.java d7eab9e > src/java/org/apache/hcatalog/mapreduce/HCatOutputStorageDriver.java 566943f > src/java/org/apache/hcatalog/mapreduce/HCatStorageHandler.java PRE-CREATION > src/java/org/apache/hcatalog/mapreduce/HCatTableInfo.java e65f1d0 > src/java/org/apache/hcatalog/mapreduce/InternalUtil.java PRE-CREATION > src/java/org/apache/hcatalog/mapreduce/OutputCommitterContainer.java > 5f0585a > src/java/org/apache/hcatalog/mapreduce/OutputFormatContainer.java f73a6dc > src/java/org/apache/hcatalog/mapreduce/OutputJobInfo.java 9830190 > src/java/org/apache/hcatalog/mapreduce/ProgressReporter.java PRE-CREATION > src/java/org/apache/hcatalog/mapreduce/RecordWriterContainer.java 9a93bfa > src/java/org/apache/hcatalog/mapreduce/StorerInfo.java 8785ffb > src/java/org/apache/hcatalog/pig/HCatEximLoader.java a36f808 > src/java/org/apache/hcatalog/pig/HCatEximLoader.java.broken PRE-CREATION > src/java/org/apache/hcatalog/pig/HCatEximStorer.java b7bee44 > src/java/org/apache/hcatalog/pig/HCatEximStorer.java.broken PRE-CREATION > src/java/org/apache/hcatalog/pig/HCatStorer.java f480321 > src/test/e2e/hcatalog/udfs/java/org/apache/hcatalog/utils/GroupByAge.java > 21e780c > > src/test/e2e/hcatalog/udfs/java/org/apache/hcatalog/utils/HBaseReadWrite.java > d28f29b > src/test/e2e/hcatalog/udfs/java/org/apache/hcatalog/utils/ReadWrite.java > 1a8e6a6 > src/test/e2e/hcatalog/udfs/java/org/apache/hcatalog/utils/StoreComplex.java > b08bc4b > src/test/e2e/hcatalog/udfs/java/org/apache/hcatalog/utils/StoreDemo.java > 6ca634e > src/test/e2e/hcatalog/udfs/java/org/apache/hcatalog/utils/StoreNumbers.java > b01d810 > src/test/e2e/hcatalog/udfs/java/org/apache/hcatalog/utils/WriteJson.java > 037610c > src/test/e2e/hcatalog/udfs/java/org/apache/hcatalog/utils/WriteRC.java > 5577182 > src/test/e2e/hcatalog/udfs/java/org/apache/hcatalog/utils/WriteText.java > a64527f > src/test/org/apache/hcatalog/cli/DummyStorageHandler.java 0e9565c > src/test/org/apache/hcatalog/cli/TestPermsGrp.java f067ebc > src/test/org/apache/hcatalog/cli/TestStorageHandlerProperties.java df8f5a5 > src/test/org/apache/hcatalog/cli/TestStorageHandlerProperties.java.broken > PRE-CREATION > src/test/org/apache/hcatalog/mapred/TestHiveHCatInputFormat.java 28bac09 > src/test/org/apache/hcatalog/mapred/TestHiveHCatInputFormat.java.broken > PRE-CREATION > src/test/org/apache/hcatalog/mapreduce/HCatMapReduceTest.java 1fb3cf5 > src/test/org/apache/hcatalog/mapreduce/TestHCatDynamicPartitioned.java > c1080ce > src/test/org/apache/hcatalog/mapreduce/TestHCatEximInputFormat.java 9d94c7b > src/test/org/apache/hcatalog/mapreduce/TestHCatEximInputFormat.java.broken > PRE-CREATION > src/test/org/apache/hcatalog/mapreduce/TestHCatEximOutputFormat.java > 99ec02b > src/test/org/apache/hcatalog/mapreduce/TestHCatEximOutputFormat.java.broken > PRE-CREATION > src/test/org/apache/hcatalog/mapreduce/TestHCatOutputFormat.java 5df0146 > src/test/org/apache/hcatalog/pig/TestHCatEximLoader.java 7691fb9 > src/test/org/apache/hcatalog/pig/TestHCatEximLoader.java.broken > PRE-CREATION > src/test/org/apache/hcatalog/pig/TestHCatEximStorer.java c3ba19d > src/test/org/apache/hcatalog/pig/TestHCatEximStorer.java.broken > PRE-CREATION > src/test/org/apache/hcatalog/pig/TestPermsInheritance.java 78e353a > src/test/org/apache/hcatalog/pig/TestPermsInheritance.java.broken > PRE-CREATION > src/test/org/apache/hcatalog/pig/TestPigStorageDriver.java 36c1539 > src/test/org/apache/hcatalog/pig/TestPigStorageDriver.java.broken > PRE-CREATION > src/test/org/apache/hcatalog/rcfile/TestRCFileOutputStorageDriver.java > eda2875 > storage-drivers/build.xml efd26d0 > > Diff: https://reviews.apache.org/r/3775/diff > > > Testing > ------- > > > Thanks, > > Francis > >
