So is there a good way to flush a buffer accumulated by putNext? I was hoping it was possible in cleanUpOnSuccess, but that apparently isn't going to work. This is horrible for something talking to a store such as MySql, as it means you have to do updates one-at-a-time.
Patrick On Sun, Dec 15, 2013 at 12:41 AM, Cheolsoo Park <[email protected]>wrote: > >> putNext and cleanUpOnSuccess will be called in the same execution > context? > > putNext() is called on the backend during the job execution, whereas > cleanUpOnSuccess() is called on the frontend after the job is finished. So > they won't be executed by the same object. From the comment, I also doubt > that you can share properties between them via JobConf. > > See MapReduceLauncher.java as for how cleanUpOnSuccess() is used. > > On Thu, Dec 5, 2013 at 11:10 AM, Patrick Thompson < > [email protected]> wrote: > > > It's not clear from the docs where the various StoreFuncInterface > functions > > get called. There are some hints in the API > > docs<http://pig.apache.org/docs/r0.12.0/api/>, > > but I am left wondering, does pig guarantee that, for example, putNext > and > > cleanUpOnSuccess will be called in the same execution context? > > > > Is this documented somewhere? Maybe someone can provide an answer? It > would > > save me a lot of time experimenting and spelunking in the code. > > > > Thanks > > > > Patrick > > > -- fun and games - a blog <http://funazonki.blogspot.com/>, a word game<http://1.whatwouldwho.appspot.com/wwws.html>and CanCan <http://www.standingwaiting.com/CanCan/Game.html>
