Hello, We are using lucene 3.6.0. We ran optimize on a large collection (250 GB before optimization), during which we ran out of disk space. After adding more disk space, we re-ran the optimization, and the process completed successfully. However, the index size didn't change very much, and looking at the existing segments - it seems that the old files are still there: [Notice the optimization ran on Sep 23, but there are files from May]
total 250G -rw-r----- 1 prodedm edm 907M Jun 6 08:05 _16mx.fdt -rw-r----- 1 prodedm edm 13M Jun 6 08:05 _16mx.fdx -rw-r----- 1 prodedm edm 154 Jun 6 08:05 _16mx.fnm -rw-r----- 1 prodedm edm 2.2G Jun 6 08:41 _16mx.frq -rw-r----- 1 prodedm edm 13M Jun 6 08:41 _16mx.nrm -rw-r----- 1 prodedm edm 23G Jun 6 08:41 _16mx.prx -rw-r----- 1 prodedm edm 18M Jun 6 08:41 _16mx.tii -rw-r----- 1 prodedm edm 1.7G Jun 6 08:41 _16mx.tis -rw-r----- 1 prodedm edm 44 Jun 6 22:34 _16mx_5.del -rw-r----- 1 prodedm edm 5.2G Jun 8 22:32 _186v.cfs -rw-r----- 1 prodedm edm 636 Jun 8 22:32 _186v_1.del -rw-r----- 1 prodedm edm 195M May 6 15:09 _1a1.fdt -rw-r----- 1 prodedm edm 2.6M May 6 15:09 _1a1.fdx -rw-r----- 1 prodedm edm 154 May 6 15:09 _1a1.fnm -rw-r----- 1 prodedm edm 573M May 6 15:15 _1a1.frq -rw-r----- 1 prodedm edm 2.6M May 6 15:15 _1a1.nrm -rw-r----- 1 prodedm edm 6.3G May 6 15:15 _1a1.prx -rw-r----- 1 prodedm edm 6.1M May 6 15:15 _1a1.tii -rw-r----- 1 prodedm edm 589M May 6 15:15 _1a1.tis -rw-r----- 1 prodedm edm 78 May 7 13:30 _1a1_7.del -rw-r----- 1 prodedm edm 11G Jun 10 11:25 _1anj.cfs -rw-r----- 1 prodedm edm 35 Jun 22 15:09 _1anj_4.del -rw-r----- 1 prodedm edm 12G Jun 12 10:58 _1do2.cfs -rw-r----- 1 prodedm edm 70 Jun 13 07:10 _1do2_2.del -rw-r----- 1 prodedm edm 18G Jun 17 02:59 _1i65.cfs -rw-r----- 1 prodedm edm 222 Jul 6 03:41 _1i65_6.del -rw-r----- 1 prodedm edm 4.2G Jun 17 14:21 _1j5h.cfs -rw-r----- 1 prodedm edm 151 Jun 18 07:40 _1j5h_7.del -rw-r----- 1 prodedm edm 560M Sep 23 20:22 _1o1h.cfs -rw-r----- 1 prodedm edm 7.0G Sep 23 20:51 _1o1i.cfs -rw-r----- 1 prodedm edm 12G Sep 23 21:05 _1o1j.cfs -rw-r----- 1 prodedm edm 9.5G Sep 23 20:57 _1o1k.cfs -rw-r----- 1 prodedm edm 9.6G Sep 23 20:59 _1o1l.cfs -rw-r----- 1 prodedm edm 3.1G Sep 23 20:29 _1o1m.cfs -rw-r----- 1 prodedm edm 22G Sep 23 21:27 _1o1n.cfs -rw-r----- 1 prodedm edm 4.6G Sep 23 20:36 _1o1o.cfs -rw-r----- 1 prodedm edm 16G Sep 23 21:10 _1o1p.cfs -rw-r----- 1 prodedm edm 3.8G Sep 23 20:32 _1o1q.cfs -rw-r----- 1 prodedm edm 6.4G Sep 23 20:42 _1o1r.cfs -rw-r----- 1 prodedm edm 17G Sep 23 20:48 _1o1s.cfs -rw-r----- 1 prodedm edm 2.4G Sep 23 20:32 _1o1t.cfs -rw-r----- 1 prodedm edm 103M May 7 09:28 _2lz.fdt -rw-r----- 1 prodedm edm 1.4M May 7 09:28 _2lz.fdx -rw-r----- 1 prodedm edm 154 May 7 09:28 _2lz.fnm -rw-r----- 1 prodedm edm 300M May 7 09:33 _2lz.frq -rw-r----- 1 prodedm edm 1.4M May 7 09:33 _2lz.nrm -rw-r----- 1 prodedm edm 4.0G May 7 09:33 _2lz.prx -rw-r----- 1 prodedm edm 2.6M May 7 09:33 _2lz.tii -rw-r----- 1 prodedm edm 235M May 7 09:33 _2lz.tis -rw-r----- 1 prodedm edm 22K May 8 06:19 _2lz_1i.del -rw-r----- 1 prodedm edm 130M May 9 13:11 _6n0.fdt -rw-r----- 1 prodedm edm 1.7M May 9 13:11 _6n0.fdx -rw-r----- 1 prodedm edm 154 May 9 13:11 _6n0.fnm -rw-r----- 1 prodedm edm 342M May 9 13:19 _6n0.frq -rw-r----- 1 prodedm edm 1.7M May 9 13:19 _6n0.nrm -rw-r----- 1 prodedm edm 3.9G May 9 13:19 _6n0.prx -rw-r----- 1 prodedm edm 3.4M May 9 13:19 _6n0.tii -rw-r----- 1 prodedm edm 322M May 9 13:19 _6n0.tis -rw-r----- 1 prodedm edm 944 May 10 11:50 _6n0_8.del -rw-r----- 1 prodedm edm 120M May 12 11:57 _8ry.fdt -rw-r----- 1 prodedm edm 1.6M May 12 11:57 _8ry.fdx -rw-r----- 1 prodedm edm 154 May 12 11:57 _8ry.fnm -rw-r----- 1 prodedm edm 310M May 12 12:04 _8ry.frq -rw-r----- 1 prodedm edm 1.6M May 12 12:04 _8ry.nrm -rw-r----- 1 prodedm edm 4.2G May 12 12:04 _8ry.prx -rw-r----- 1 prodedm edm 3.1M May 12 12:04 _8ry.tii -rw-r----- 1 prodedm edm 285M May 12 12:04 _8ry.tis -rw-r----- 1 prodedm edm 467 May 13 06:56 _8ry_h.del -rw-r----- 1 prodedm edm 4.4G May 13 02:58 _9ls.cfs -rw-r----- 1 prodedm edm 92 May 13 16:55 _9ls_6.del -rw-r----- 1 prodedm edm 173M May 14 03:31 _be6.fdt -rw-r----- 1 prodedm edm 2.3M May 14 03:31 _be6.fdx -rw-r----- 1 prodedm edm 154 May 14 03:31 _be6.fnm -rw-r----- 1 prodedm edm 443M May 14 03:38 _be6.frq -rw-r----- 1 prodedm edm 2.3M May 14 03:38 _be6.nrm -rw-r----- 1 prodedm edm 5.1G May 14 03:38 _be6.prx -rw-r----- 1 prodedm edm 4.5M May 14 03:38 _be6.tii -rw-r----- 1 prodedm edm 416M May 14 03:38 _be6.tis -rw-r----- 1 prodedm edm 63 May 15 03:05 _be6_6.del -rw-r----- 1 prodedm edm 5.5G May 16 09:26 _fcd.cfs -rw-r----- 1 prodedm edm 797 May 17 08:38 _fcd_5.del -rw-r----- 1 prodedm edm 4.9G May 23 01:45 _n3v.cfs -rw-r----- 1 prodedm edm 94 May 23 22:45 _n3v_c.del -rw-r----- 1 prodedm edm 4.3G May 23 14:49 _o56.cfs -rw-r----- 1 prodedm edm 132 May 24 09:58 _o56_8.del -rw-r----- 1 prodedm edm 4.5G May 27 05:55 _qn1.cfs -rw-r----- 1 prodedm edm 27 May 28 03:55 _qn1_3.del -rw-r----- 1 prodedm edm 5.1G May 29 14:24 _uw3.cfs -rw-r----- 1 prodedm edm 122 May 30 07:39 _uw3_9.del -rw-r----- 1 prodedm edm 5.4G May 30 11:38 _woy.cfs -rw-r----- 1 prodedm edm 61 May 31 10:09 _woy_5.del -rw-r----- 1 prodedm edm 20 Sep 23 21:27 segments.gen -rw-r----- 1 prodedm edm 7.7K Sep 23 21:27 segments_18xk Is there any tool available or any easy way we can remove unnecessary files to clean up the index and save some disk space? Thanks in advance! Shlomit
