Re: Disk full errors in local-dirs, what data is stored in yarn.nodemanager.local-dirs?

2017-04-12 Thread Sidharth Kumar
Hi, Can you paste the output of "df -h" command here. Regards Sidharth On Wednesday, April 12, 2017, Albert Chu wrote: > Hi, > > I have a cluster where we have a parallel networked file system for our > major data storage and our nodes have ~750G of local SSD space. To > speed up things, we c

Disk full errors in local-dirs, what data is stored in yarn.nodemanager.local-dirs?

2017-04-11 Thread Albert Chu
Hi, I have a cluster where we have a parallel networked file system for our major data storage and our nodes have ~750G of local SSD space. To speed up things, we configure yarn.nodemanager.local-dirs to use the local SSD for local caching. Recently, I've been trying to do a terasort of 2 teraby