Gerrit,
what you are saying is much more serious than just being reentrant. Are you
saying you have not been able to run same grunt or pig server instance thru
a bunch of scripts and OOM happened eventually? can you share which version
of pig that was?
I so far haven't actually seen that issue but then i run only 4 pig jobs a
day and the process i ran has been up for perhaps couple of weeks only
Thanks.
-Dmitriy
On Wed, Aug 4, 2010 at 11:27 PM, Gerrit van Vuuren <
gvanvuu...@specificmedia.com> wrote:
> Yep I can confirm that if you call it enough times within the same java
> process you will run out of memory eventually. I've tried this before,
> monitored this with jconsole and saw the memory gradually increasing over 50
> or so iterations, each iteration also created its own set of threads that
> never died but this might be in the hadoop client itself.
> I even tried using a whole different set of classloaders to try and unload
> classes after each call but this did not work either
>
>
> ----- Original Message -----
> From: Vincent Barat <vba...@ubikod.com>
> To: pig-user@hadoop.apache.org <pig-user@hadoop.apache.org>;
> pig-user@hadoop.apache.org <pig-user@hadoop.apache.org>
> Sent: Thu Aug 05 07:08:06 2010
> Subject: Re: Call Pig from Java
>
> No. PigServer is not reentrant at this time, afaik, and even if you create
> several pigserver objects you will run into trouble, as there is a small set
> of global data shared between them. It may work for a time, but it will fail
> at a point. The only way is to create different processes to handle your
> requests.
>
> Wenhao Xu <xuwenhao2...@gmail.com> a écrit :
>
> >Thanks!
> >Can PigServer handle concurrent requests? Because the store is a
> >synchronous interface, is there any asynchronous one?
> >
> >cheers,
> >W.
> >
> >On Wed, Aug 4, 2010 at 9:01 PM, Harsh J <qwertyman...@gmail.com> wrote:
> >
> >> You need to use the class PigServer.
> >>
> >> PigServer pigServer = new PigServer("mapreduce"); // Or "local" for
> local
> >> mode
> >> pigServer.registerQuery("A = LOAD ...");
> >> (...) // Your statements here.
> >> pigServer.store("A", "filename");
> >>
> >> On Thu, Aug 5, 2010 at 9:26 AM, Wenhao Xu <xuwenhao2...@gmail.com>
> wrote:
> >> > Hi all,
> >> > I am new to pig. I am wondering is there any recommended way to call
> >> Pig
> >> > code from Java?
> >> > Is there any Java interface which can be called directly from Java
> and
> >> > makes them work smoothly? It seems each keyword (filter, group,
> cogrape,
> >> > generate) and data types in Pig can have a counterpart in Java by
> using
> >> > Class, interface and data type. Is these Java interface available to
> Java
> >> > programmers to use? If not, why not?
> >> > Thanks very much for help!
> >> >
> >> > regards,
> >> > Wenhao
> >> >
> >> > --
> >> > ~_~
> >> >
> >>
> >>
> >>
> >> --
> >> Harsh J
> >> www.harshj.com
> >>
> >
> >
> >
> >--
> >~_~
>
