> On Jan. 13, 2015, 10:52 p.m., Rohini Palaniswamy wrote: > > trunk/test/org/apache/pig/test/TestMultiQueryLocal.java, line 447 > > <https://reviews.apache.org/r/29644/diff/1/?file=808163#file808163line447> > > > > If the location is different why do we have to pass two diffent keys > > for two different output formats to get the test passing? > > Daniel Dai wrote: > This test is to make sure different storer is using different context. So > I use different key for different storer for the test. There is a difference > which makes the test pass in MR. In tez, OutputFormat and Storer is using the > same context, while MR is not. I don't think this is a big issue. > > Rohini Palaniswamy wrote: > Don't we clone the context and use it for different > Storers/OutputFormats? May be that is missing in Tez. We should fix it as > using of same context by different Storers will cause settings to be > overwritten. > > Daniel Dai wrote: > Didn't get it. Different Storer/OutputFormat should use independent > context. That is, if you store some config entry in Storer1/OutputFormat1, > Storer2/OutputFormat2 will not see. That is same in both MR and Tez. What's > different is, in MR, also provide separation between Storer1/OutputFormat1. > If you store config in OutputFormat1, Storer1 will not see. But that does not > seems to be a required feature.
Oh. That should be fine. Infact HCat folks were asking me for the same thing. They were adding something to config in RecordWriter and wanted that to be available to commitJob, but was not possible with MR - Rohini ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29644/#review67958 ----------------------------------------------------------- On Jan. 20, 2015, 10:08 p.m., Daniel Dai wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/29644/ > ----------------------------------------------------------- > > (Updated Jan. 20, 2015, 10:08 p.m.) > > > Review request for pig and Rohini Palaniswamy. > > > Repository: pig > > > Description > ------- > > See PIG-4359 > > > Diffs > ----- > > trunk/shims/test/hadoop23/org/apache/pig/test/MiniCluster.java 1652696 > trunk/shims/test/hadoop23/org/apache/pig/test/TezMiniCluster.java 1652696 > > trunk/src/org/apache/pig/backend/hadoop/executionengine/tez/plan/TezCompiler.java > 1652696 > trunk/test/excluded-tests-20 1652696 > trunk/test/org/apache/pig/test/MiniGenericCluster.java 1652696 > trunk/test/org/apache/pig/test/TestMultiQueryLocal.java 1652696 > trunk/test/org/apache/pig/test/TestPOPartialAggPlan.java 1652696 > trunk/test/org/apache/pig/test/TestPOPartialAggPlanMR.java PRE-CREATION > trunk/test/org/apache/pig/test/TestStore.java 1652696 > trunk/test/org/apache/pig/test/TestStoreBase.java PRE-CREATION > trunk/test/org/apache/pig/test/TestStoreLocal.java PRE-CREATION > trunk/test/org/apache/pig/tez/TestPOPartialAggPlanTez.java PRE-CREATION > trunk/test/org/apache/pig/tez/TezUtil.java PRE-CREATION > trunk/test/tez-local-tests 1652696 > > Diff: https://reviews.apache.org/r/29644/diff/ > > > Testing > ------- > > > Thanks, > > Daniel Dai > >
