On 2018/09/06 05:31:45, Jean-Baptiste Onofré <[email protected]> wrote:
> Hi,
>
> AFAIU you are using dataflow runner right ?
>
> Regards
> JB
>
> On 06/09/2018 00:03, [email protected] wrote:
> > Hi
> >
> > I am doing some processing and creating local list inside a Pardo function.
> > Total data size is 2 GB which is executing inside a single instance. I am
> > running this code on highmem-64 machine
> >
> > and It gives this error
> > Shutting down JVM after 8 consecutive periods of measured GC thrashing.
> > Memory is used/total/max = 24239/26759/260864 MB, GC last/max = 67.00/99.00
> > %, #pushbacks=0, gc thrashing=true. Heap dump written to
> > '/var/log/dataflow/heap_dump.hprof'.
> >
> > It seems looking at this error - even though Max heap size is much more but
> > 'total memory' can't reach 'max possible' memory and it gives heap issues.
> >
> > Can you please advise how to make total memory go upto max possible memory.
> >
>
> --
> Jean-Baptiste Onofré
> [email protected]
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
yes runner is DataFlow.
Thanks
Aniruddh
