Thanks guys, good to know.
On Thu, May 23, 2013 at 1:00 PM, Vinod Kumar Vavilapalli < [email protected]> wrote: > > In all likelihood, we will add them back in MR2 to avoid MR AM crashes. > The only question in when. > > Thanks, > +Vinod > > On May 23, 2013, at 12:57 PM, Harsh J wrote: > > > There is a limit in MR1, mapred.user.jobconf.limit per > > http://hadoop.apache.org/docs/stable/mapred-default.html, that limits > it to > > 5 MB (but this is applied at the JT level). I am not aware of any > > serialization-time limits and think there are none as I've seen Hive use > > the same code to write enormous sized files. > > > > Worth noting that MR2, suitable to its service-less architecture, has no > > such limits on jobconf size and the property isn't present in it anymore. > > -- Director of Data Science Cloudera <http://www.cloudera.com> Twitter: @josh_wills <http://twitter.com/josh_wills>
