Filed https://issues.apache.org/jira/browse/TEZ-2935 to partially address this. To add some clarity to the issue, I have a bunch of pig on MR jobs that define the slow start property. Currently there is no translation to pick that setting up in ShuffleVertexManager. Filed TEZ-2935 to address this. However, the vertex manager is a user defined property. In particular, I have some VertexManager that are a subclass of ShuffleVertexManager that should pick up the new config. The scenario I'm wondering about is where there is a user defined Vertex Manager that will have to add its own translation.
My question is whether the scenario exists more generally, so that we should provide a recommended way of doing this or perhaps even extending MRHelpers so that user code can register its translations. Jon On Fri, Nov 6, 2015 at 10:41 PM, Bikas Saha <[email protected]> wrote: > Hi, > I am not sure I clearly understand what you want to do?Maybe an example > would help. > Bikas > > Date: Fri, 6 Nov 2015 13:01:47 -0600 > > Subject: User defined configuration deprecations > > From: [email protected] > > To: [email protected] > > > > Devs, > > > > What is the recommended way to go about adding deprecations for user > > defined deprecations. For example, vertex managers. You could add them in > > DeprecatedKeys for know vertex managers like ShuffleVertexManager, but > that > > has not been done historically. > > > > In my particular case, I want to translate mapreduce slow start setting > to > > shuffleVertexManager slow start settings. I also want to add them for > > custom defined vertex managers. Hopefully this has a pretty simple > answer, > > but just wanted to get thoughts? > > > > Jon > >
