I think there are two problems: 1. Sort allows you to specify a temp dir, but it seems BytesRefSorter does not respect this? 2. each jvm when running tests should get its own sandboxed tempdir (i think it should be in the build/ directory with other temp test stuff).
On Sun, Jul 8, 2012 at 12:29 PM, Uwe Schindler <[email protected]> wrote: > Yes, ist the offline sorting on disk of Suggester. But: > > The code uses File.createTempFile() without any chance to configure it! So we > should make this configureable when initializing the sorter and configure > this to local test directory on tests. > > Especially the always failing SolrSuggesterTest fills my RAM-resident /tmp. > > Uwe > > ----- > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: [email protected] > > >> -----Original Message----- >> From: Robert Muir [mailto:[email protected]] >> Sent: Sunday, July 08, 2012 2:07 PM >> To: [email protected] >> Subject: Re: RefSorter files in /tmp? >> >> when suggester tests fail maybe this happens? >> >> >> On Sun, Jul 8, 2012 at 5:07 AM, Uwe Schindler <[email protected]> wrote: >> > When reviewing my Jenkins installation (because slave was taken >> > offline, which was config bug), I found out that /tmp is filled by >> > Jenkins with the following files: >> > >> > -rw-r--r-- 1 jenkins nogroup 12433 Jul 5 21:14 >> > RefSorter-1839005885812820606.sorted >> > -rw-r--r-- 1 jenkins nogroup 13574 Jul 5 19:26 >> > RefSorter-2799526995307200478.sorted >> > -rw-r--r-- 1 jenkins nogroup 12600 Jul 5 17:14 >> > RefSorter-2841491891429925756.sorted >> > -rw-r--r-- 1 jenkins nogroup 11697 Jul 5 19:57 >> > RefSorter-3302954184439492426.sorted >> > -rw-r--r-- 1 jenkins nogroup 13496 Jul 5 16:30 >> > RefSorter-3738422482066276549.sorted >> > -rw-r--r-- 1 jenkins nogroup 13781 Jul 5 15:36 >> > RefSorter-4235756851148318773.sorted >> > -rw-r--r-- 1 jenkins nogroup 12719 Jul 5 18:54 >> > RefSorter-4530019493984469514.sorted >> > -rw-r--r-- 1 jenkins nogroup 12696 Jul 5 16:04 >> > RefSorter-5245195867837976219.sorted >> > -rw-r--r-- 1 jenkins nogroup 13879 Jul 5 18:27 >> > RefSorter-5977302780601133089.sorted >> > -rw-r--r-- 1 jenkins nogroup 12712 Jul 5 21:39 >> > RefSorter-6336186633027300753.sorted >> > -rw-r--r-- 1 jenkins nogroup 12820 Jul 5 20:30 >> > RefSorter-6447286760971372233.sorted >> > -rw-r--r-- 1 jenkins nogroup 12105 Jul 5 17:48 >> > RefSorter-6532780916605441895.sorted >> > -rw-r--r-- 1 jenkins nogroup 13505 Jul 5 20:53 >> > RefSorter-7247901917320979657.sorted >> > -rw-r--r-- 1 jenkins nogroup 12688 Jul 5 22:10 >> > RefSorter-7796370222379929612.sorted >> > -rw-r--r-- 1 jenkins nogroup 19 Jul 5 18:54 >> > sort1277839437346448611partition >> > -rw-r--r-- 1 jenkins nogroup 21299752 Jul 5 15:35 >> > sort1362726822297484023intermediate >> > -rw-r--r-- 1 jenkins nogroup 21300496 Jul 5 17:48 >> > sort1435680293746542872intermediate >> > -rw-r--r-- 1 jenkins nogroup 19 Jul 5 16:30 >> > sort1498884601796138622partition >> > -rw-r--r-- 1 jenkins nogroup 21300869 Jul 5 20:30 >> > sort1634015425760928615intermediate >> > -rw-r--r-- 1 jenkins nogroup 19 Jul 5 20:30 >> > sort1954741677243403383partition >> > -rw-r--r-- 1 jenkins nogroup 21300802 Jul 5 20:53 >> > sort2203784121687916561intermediate >> > -rw-r--r-- 1 jenkins nogroup 21300493 Jul 5 22:10 >> > sort24154414907891444intermediate >> > -rw-r--r-- 1 jenkins nogroup 19 Jul 5 22:10 >> > sort2816986454022083882partition >> > -rw-r--r-- 1 jenkins nogroup 21300111 Jul 5 18:27 >> > sort285022281545547041intermediate >> > -rw-r--r-- 1 jenkins nogroup 19 Jul 5 18:28 >> > sort295507558144077223partition >> > -rw-r--r-- 1 jenkins nogroup 21300569 Jul 5 16:30 >> > sort3013772538520090513intermediate >> > -rw-r--r-- 1 jenkins nogroup 21300574 Jul 5 17:14 >> > sort3297463807520676013intermediate >> > -rw-r--r-- 1 jenkins nogroup 19 Jul 5 21:14 >> > sort3364874175018276528partition >> > -rw-r--r-- 1 jenkins nogroup 19 Jul 5 17:14 >> > sort3846182021346233750partition >> > -rw-r--r-- 1 jenkins nogroup 21300204 Jul 5 19:26 >> > sort4397860673342757974intermediate >> > -rw-r--r-- 1 jenkins nogroup 21300050 Jul 5 16:04 >> > sort4474792189525490476intermediate >> > -rw-r--r-- 1 jenkins nogroup 21300825 Jul 5 18:54 >> > sort4518448528614283778intermediate >> > -rw-r--r-- 1 jenkins nogroup 19 Jul 5 21:39 >> > sort4756172476965226743partition >> > -rw-r--r-- 1 jenkins nogroup 19 Jul 5 20:53 >> > sort5416699953867843402partition >> > -rw-r--r-- 1 jenkins nogroup 19 Jul 5 19:26 >> > sort5558474409634346477partition >> > -rw-r--r-- 1 jenkins nogroup 19 Jul 5 17:48 >> > sort6281513108922200314partition >> > -rw-r--r-- 1 jenkins nogroup 21300155 Jul 5 21:39 >> > sort6639309492804635005intermediate >> > -rw-r--r-- 1 jenkins nogroup 19 Jul 5 19:57 >> > sort6777765458777941142partition >> > -rw-r--r-- 1 jenkins nogroup 21301369 Jul 5 19:57 >> > sort6973021800616034113intermediate >> > -rw-r--r-- 1 jenkins nogroup 21300341 Jul 5 21:14 >> > sort7260811068342958052intermediate >> > -rw-r--r-- 1 jenkins nogroup 19 Jul 5 16:04 >> > sort852078170643422390partition >> > -rw-r--r-- 1 jenkins nogroup 19 Jul 5 15:35 >> > sort8857722113319559286partition >> > >> > The pattern "RefSorter-" I found in Lucene's source code, so it must >> > come from tests. Interstingly, all files are from July 5, so not >> > appearing on every run. Why are they not cleaned up and why do we need >> > those files? Would a RamDirectory not be enough for this? >> > >> > On the Jenkins machine, /tmp is a maximum size 2 GB tmpfs (like on >> > Solaris), so it's limited (previously it was maximum 1 GB and Jenkins >> > took slaves offline when size < 1 GB -> fixed). >> > >> > # /tmp as 2048 MB tmpfs like on Slowlaris /tmp /tmp tmpfs >> > defaults,size=2048m,mode=1777 0 0 >> > >> > Uwe >> > >> > ----- >> > Uwe Schindler >> > H.-H.-Meier-Allee 63, D-28213 Bremen >> > http://www.thetaphi.de >> > eMail: [email protected] >> > >> > >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: [email protected] For >> > additional commands, e-mail: [email protected] >> > >> >> >> >> -- >> lucidimagination.com >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] For additional >> commands, e-mail: [email protected] > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > -- lucidimagination.com --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
