Hi everyone,
I am running spark jobs on EMR (using pyspark). I noticed that after
running jobs, the size of the usercache (basically the filecache folder)
keeps on increasing (with directory names as 1,2,3,4,5,...).
    Directory location: */mnt/yarn/usercache/hadoop/**filecache/*
Is there a way to avoid creating these directories or automatically
clearing the usercache/filecache after a job/periodically?
-- 
--Thanks,
Shuporno Choudhury

Reply via email to