Re: [DNG] rm not freeing space (SOLVED)
Quoting Marc Shapiro via Dng (dng@lists.dyne.org):
> On 3/19/21 8:31 PM, tempforever via Dng wrote:
> >
> >To find files last modified in January:
> >(adjust the dates as needed; on Mar. 19 this should locate files dated
> >Jan 1 - 31 but it could be off a day or so)
> >
> >cd /media/archives
> >find -type f -mtime -78 -mtime +46 -ls
>
> Thank you!
>
> That found the files. They were in a hidden directory
> '/media/archives/.Trash-0'.
Aha! For your future convenience, here is a Perl script I get a lot of
use from, when looking for the (individually) largest files within $PWD
and its subtrees:
:r /usr/local/bin/largest20
#!/usr/bin/perl -w
# You can alternatively just do:
# find . -xdev -type f -print0 | xargs -r0 ls -l | sort -rn -k +5 | head -20
# Sometimes also handy: du -cks * | sort -rn
use File::Find;
@ARGV = $ENV{ PWD } unless @ARGV;
find ( sub { $size{ $File::Find::name } = -s if -f; }, @ARGV );
@sorted = sort { $size{ $b } <=> $size{ $a } } keys %size;
splice @sorted, 20 if @sorted > 20;
printf "%10d %s\n", $size{$_}, $_ for @sorted
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space (SOLVED)
On 3/19/21 8:31 PM, tempforever via Dng wrote: To find files last modified in January: (adjust the dates as needed; on Mar. 19 this should locate files dated Jan 1 - 31 but it could be off a day or so) cd /media/archives find -type f -mtime -78 -mtime +46 -ls Thank you! That found the files. They were in a hidden directory '/media/archives/.Trash-0'. I had seen this before and I really thought that I had checked this directory. Possibly, I checked it as myself and not as root. That would have given me a 'permissions denied' error. Although I would normally try again as root, perhaps I missed that step. I have seen posts that suggest this directory is created when deleting files with a file manager, not from the command line. Even so, despite the fact that I usually do my file management from the command line that is not always the case, it seems odd that this is the ONLY such directory that I can find anywhere in my directory tree. If I look at 'Trash' in my file manager (caja) it shows some other files, but not the ones in question. In any case, deleting the files under '/media/archive/.Trash-0/files' and '/media/archive/.Trash-0/info' freed up about 618 GB of space (some files when back to October). Now, '/media/archives' is about 46% used, instead of 84%, and I am much happier. Thank you to all who have responded to my request for help. We can now put this issue to rest. Marc ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
Marc Shapiro via Dng wrote: > > On 3/15/21 6:31 AM, Hendrik Boom wrote: >> On Sun, Mar 14, 2021 at 10:28:32PM -0700, Marc Shapiro via Dng wrote: >>> On 3/14/21 10:09 PM, Ludovic Bellière wrote: I assume you read the man page of fsck, as it's return code is what you want to pay attention to. As for lsof, the correct parameters would be `lsof +aL1 /dev/sdx. It should have thrown an error were you to use `lsof -L1`. If lsof returns nothing, your drive is most likely corrupted. >>> You are correct. I used '+L' NOT '-L'. It may also be possible that the files you removed have other references on your file system, aka. hard links. To find them, you would need to know the inode number, either by using `stat` or `ls -i`. You can then find them using `find -inum`. Since you already removed the files, you most likely can't know the inode number. However you could throw a `find $path -size n[cwbkMG]` to list the files with the matching size. >>> I'm not following you on this. What is this going to do for me? >>> 'find' is >>> only going to show undeleted files. How does this help? >> It is possible for a single file to be hard-linked in several places >> in the >> file system. If so, removing it in one place will still leave it >> accessible >> from another, and therefore not deleted. >> >> Files have reference counts to keep track of this. > > These files should not have any links, hard or soft. None of the other > files in that directory show a reference count above 1. They are backup > files created by fsarchiver. I'm just trying to free up space by > deleting files from January. > > Marc > To find files last modified in January: (adjust the dates as needed; on Mar. 19 this should locate files dated Jan 1 - 31 but it could be off a day or so) cd /media/archives find -type f -mtime -78 -mtime +46 -ls > ___ > Dng mailing list > Dng@lists.dyne.org > https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
On Sat, 20 Mar 2021 15:13:39 +0100 Bernard Rosset via Dng wrote: > Marc, > > So far, the only factual data does not show any problem, and the only > input stating something is wrong is actually you saying that. Hallo Bernard, perhaps you missed these lines within the mess of pasted output, but I had stopped reading exactly here: || On Tue, 16 Mar 2021 07:41:42 -0700 || Marc Shapiro via Dng wrote: || || > root@quixote:/home/marc# du -sc /path/to/mountpoint/ || > du: cannot access '/path/to/mountpoint/': No such file or directory || > 0 total libre Grüße, Florian ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
Marc,
So far, the only factual data does not show any problem, and the only
input stating something is wrong is actually you saying that.
My previous piece of advice with sfill was to kinda force *all* the free
space to be allocated, then released... Kinda desperate measure, which
would indicate something *very* wrong with the ext4 filesystem you use
on that disk.
Before going further, I guess the problem shall factually be
demonstrated. Here is what I suggest:
1°) sudo du --block-size=512 -s /mount/point
2°) sudo find /mount/point -print0 | xargs -0 ls --block-size=512 -ds |
awk '{print $1}' | paste -sd+ | bc
This should compute the size of all resources without dereferencing
links. Make sure no supplemental mount is made inside the hierarchy
you're inspecting.
Hopefully results are the same.
Bernard (Beer) Rosset
https://rosset.net/
___
Dng mailing list
Dng@lists.dyne.org
https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
How do you create those backup archives? Do you use a tool? On Fri, 19 Mar 2021 18:38:49 -0700 Marc Shapiro via Dng wrote: > > These are backup archives. No one is using them after they are > created (unless I screw up and accidentally wipe out an entire > partition, again). > > > Marc pgpXwlY068kay.pgp Description: Signature digitale OpenPGP ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
On 3/16/21 2:06 PM, Ralph Ronnquist via Dng wrote: On Tue, 16 Mar 2021 07:41:42 -0700 Marc Shapiro via Dng wrote: ... tmpfs 2466616 24 2466592 1% /run/user/1001 tmpfs 2466616 24 2466592 1% /run/user/1002 tmpfs 2466616 32 2466584 1% /run/user/1000 Is it possible that one of the users runs some program that holds on to cached directory entries for the file(s) if not the files themselves? I believe there's currently significant effort going into trying to isolate users, including root, from each other so that even sharing of common resources is "hidden" (with virtual file systems and various forms of "unsharing" and whatnot). Ralph. These are backup archives. No one is using them after they are created (unless I screw up and accidentally wipe out an entire partition, again). Marc ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
On 3/16/21 11:17 AM, g4sra via Dng wrote: ‐‐‐ Original Message ‐‐‐ On Tuesday, March 16, 2021 2:32 PM, Marc Shapiro via Dng wrote: On 3/16/21 2:32 AM, g4sra via Dng wrote: --snip-- With your removable drive attached and mounted... Paste the outputs of 'mount' and 'df' when run as root. The drive in question is /dev/sdb1. root:/home/marc# mount /dev/sdb1 on /media/archives type ext4 (rw,nosuid,nodev,noexec,relatime,user) root:/home/marc# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdb1 1663749752 1267718540 311447996 81% /media/archives Good, that answered some of my queries. Now as root again, paste the output of a comparative usage of the directory structure immediately above where the files were that you deleted. For example if you have /media/archives/2021/march/january /media/archives/2021/march/february then execute du -s /media/archives/2021/march/* /media/archives/january /media/archives/february then execute du -s /media/archives/* expect an output that ends with a 'january' root:/home/marc# du -s /media/archives/* 9913216 /media/archives/january 43905428 /media/archives/february 219244/media/archives/march I cannot make the following judgement, hopefully you can. Compare the directories usage, as you know their expected contents determine if it makes sense. An extra 200GB somewhere should stick out like a sore thumb. Once you are certain where the extra usage is, you can work on that directory alone. My very first check would be sorting by size including .dotfiles ls -alrS /media/archives/january The difference in space used as shown by 'ls' and by 'du -s' is very minimal and can be explained by the fact that 'ls' does not take allocation block size into account, but 'du -s' does. Marc ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
On Tue, 16 Mar 2021 07:41:42 -0700 Marc Shapiro via Dng wrote: > ... > tmpfs 2466616 24 2466592 1% /run/user/1001 > tmpfs 2466616 24 2466592 1% /run/user/1002 > tmpfs 2466616 32 2466584 1% /run/user/1000 Is it possible that one of the users runs some program that holds on to cached directory entries for the file(s) if not the files themselves? I believe there's currently significant effort going into trying to isolate users, including root, from each other so that even sharing of common resources is "hidden" (with virtual file systems and various forms of "unsharing" and whatnot). Ralph. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
‐‐‐ Original Message ‐‐‐ On Tuesday, March 16, 2021 2:32 PM, Marc Shapiro via Dng wrote: On 3/16/21 2:32 AM, g4sra via Dng wrote: --snip-- With your removable drive attached and mounted... Paste the outputs of 'mount' and 'df' when run as root. The drive in question is /dev/sdb1. root:/home/marc# mount /dev/sdb1 on /media/archives type ext4 (rw,nosuid,nodev,noexec,relatime,user) root:/home/marc# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdb1 1663749752 1267718540 311447996 81% /media/archives Good, that answered some of my queries. Now as root again, paste the output of a comparative usage of the directory structure immediately above where the files were that you deleted. For example if you have /media/archives/2021/march/january /media/archives/2021/march/february then execute du -s /media/archives/2021/march/* /media/archives/january /media/archives/february then execute du -s /media/archives/* expect an output that ends with a 'january' root:/home/marc# du -s /media/archives/* 9913216 /media/archives/january 43905428 /media/archives/february 219244/media/archives/march I cannot make the following judgement, hopefully you can. Compare the directories usage, as you know their expected contents determine if it makes sense. An extra 200GB somewhere should stick out like a sore thumb. Once you are certain where the extra usage is, you can work on that directory alone. My very first check would be sorting by size including .dotfiles ls -alrS /media/archives/january publickey - g4sra@protonmail.com - 0x42E94623.asc Description: application/pgp-keys signature.asc Description: OpenPGP digital signature ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
On 3/16/21 4:45 AM, Florian Zieboll via Dng wrote: Just to doube check for "human error": Does the output of 'df' match with the output of 'du -sc /path/to/mountpoint/'? The drive in question is /dev/sdb1/ mounteded on /dev/media/. Not an exact match, but not off by 200GB, either. root:/home/marc# df Filesystem 1K-blocks Used Available Use% Mounted on udev 12281428 0 12281428 0% /dev tmpfs 2466620 1116 2465504 1% /run /dev/mapper/vg1-root--devuan 3030800 2089220 767912 74% / /dev/mapper/vg1-usr--ascii 51343840 5063608 43642408 11% /usr tmpfs 5120 4 5116 1% /run/lock tmpfs 4933220 1008 4932212 1% /dev/shm /dev/mapper/vg1-tmp--ascii 5095040 22360 4794152 1% /tmp /dev/mapper/vg1-usrlocal 25671996 17704060 6813536 73% /usr/local /dev/mapper/vg1-photos 10255992 2626160 7174264 27% /usr/local/photos /dev/mapper/vg1-var--ascii 10255636 2180880 7534084 23% /var /dev/mapper/vg1-home 256981444 53203092 190654768 22% /home tmpfs 12333080 0 12333080 0% /sys/fs/cgroup tmpfs 2466616 24 2466592 1% /run/user/1001 tmpfs 2466616 24 2466592 1% /run/user/1002 /dev/mapper/vg1-vdisks 154687468 85293144 61530004 59% /usr/local/vdisks tmpfs 2466616 32 2466584 1% /run/user/1000 /dev/sdb1 1663749752 1267718540 311447996 81% /media/archives root@quixote:/home/marc# root@quixote:/home/marc# du -sc /path/to/mountpoint/ du: cannot access '/path/to/mountpoint/': No such file or directory 0 total root:/home/marc# du -sc /media/archives 1267648384 /media/archives 1267648384 total ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
On 3/16/21 2:32 AM, g4sra via Dng wrote: <--snip--> With your removable drive attached and mounted... Paste the outputs of 'mount' and 'df' when run as root. The drive in question is /dev/sdb1. root:/home/marc# mount sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime) proc on /proc type proc (rw,nosuid,nodev,noexec,relatime) udev on /dev type devtmpfs (rw,nosuid,relatime,size=12281428k,nr_inodes=3070357,mode=755) devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000) tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=2466620k,mode=755) /dev/mapper/vg1-root--devuan on / type ext4 (rw,relatime,errors=remount-ro) /dev/mapper/vg1-usr--ascii on /usr type ext4 (rw,relatime) tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k) pstore on /sys/fs/pstore type pstore (rw,relatime) tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=4933220k) /dev/mapper/vg1-tmp--ascii on /tmp type ext4 (rw,relatime) /dev/mapper/vg1-usrlocal on /usr/local type ext3 (rw,relatime) /dev/mapper/vg1-photos on /usr/local/photos type ext3 (rw,relatime) /dev/mapper/vg1-var--ascii on /var type ext4 (rw,relatime) /dev/mapper/vg1-home on /home type ext4 (rw,relatime) securityfs on /sys/kernel/security type securityfs (rw,relatime) binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,nosuid,nodev,noexec,relatime) tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,mode=755) cgroup2 on /sys/fs/cgroup/unified type cgroup2 (rw,nosuid,nodev,noexec,relatime,nsdelegate) cgroup on /sys/fs/cgroup/elogind type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/elogind/elogind-cgroups-agent,name=elogind) tmpfs on /run/user/1001 type tmpfs (rw,nosuid,nodev,relatime,size=2466616k,mode=700,uid=1001,gid=1001) tmpfs on /run/user/1002 type tmpfs (rw,nosuid,nodev,relatime,size=2466616k,mode=700,uid=1002,gid=1002) /dev/mapper/vg1-vdisks on /usr/local/vdisks type ext3 (rw,relatime) tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,size=2466616k,mode=700,uid=1000,gid=1000) /dev/sdb1 on /media/archives type ext4 (rw,nosuid,nodev,noexec,relatime,user) root:/home/marc# df Filesystem 1K-blocks Used Available Use% Mounted on udev 12281428 0 12281428 0% /dev tmpfs 2466620 1116 2465504 1% /run /dev/mapper/vg1-root--devuan 3030800 2089220 767912 74% / /dev/mapper/vg1-usr--ascii 51343840 5063608 43642408 11% /usr tmpfs 5120 4 5116 1% /run/lock tmpfs 4933220 1008 4932212 1% /dev/shm /dev/mapper/vg1-tmp--ascii 5095040 22360 4794152 1% /tmp /dev/mapper/vg1-usrlocal 25671996 17704060 6813536 73% /usr/local /dev/mapper/vg1-photos 10255992 2626160 7174264 27% /usr/local/photos /dev/mapper/vg1-var--ascii 10255636 2180880 7534084 23% /var /dev/mapper/vg1-home 256981444 53203092 190654768 22% /home tmpfs 12333080 0 12333080 0% /sys/fs/cgroup tmpfs 2466616 24 2466592 1% /run/user/1001 tmpfs 2466616 24 2466592 1% /run/user/1002 /dev/mapper/vg1-vdisks 154687468 85293144 61530004 59% /usr/local/vdisks tmpfs 2466616 32 2466584 1% /run/user/1000 /dev/sdb1 1663749752 1267718540 311447996 81% /media/archives ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
Just to doube check for "human error": Does the output of 'df' match with the output of 'du -sc /path/to/mountpoint/'? -- [message sent otg] ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
On Tue, Mar 16, 2021 at 04:57:57AM +0100, Ludovic Bellière wrote: > > > On Mon, 15 Mar 2021 20:36:28 -0700 > Marc Shapiro via Dng wrote: > > > I'm not familiar with sfill. What does it do and what package is it > > in. It is not currently installed here. > > > > Marc > > sfill is part of secure-delete. rm doesn't actually remove data, it > just remove all links to it leaving the space available to be used by Technically, it removes only one link to it; whic is often the only link. > something else. secure-delete scramble the data with junk (several > times) so nothing can be recovered. sfill looks useful if one is likely to cross international borders. There is software that's perfectly lawful to have and use in some countries and severely illegal in others. Take a backup, leave it at home, then erase it from your hard drive and then really, really erase it irecoverably, all before crossing that border. -- hendrik > ___ > Dng mailing list > Dng@lists.dyne.org > https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
On Tue, Mar 16, 2021 at 04:57:57AM +0100, Ludovic Bellière wrote: > > > On Mon, 15 Mar 2021 20:36:28 -0700 > Marc Shapiro via Dng wrote: > > > I'm not familiar with sfill. What does it do and what package is it > > in. It is not currently installed here. > > > > Marc > > sfill is part of secure-delete. rm doesn't actually remove data, it > just remove all links to it leaving the space available to be used by > something else. secure-delete scramble the data with junk (several > times) so nothing can be recovered. It's not clear how that would help. The problem isn't making sure the data are inaccessible. The problem is that the allegedly free space isn't being recognised as free space; presumably sfill won't know it's free space either. -- hendrik > ___ > Dng mailing list > Dng@lists.dyne.org > https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
<--snip--> With your removable drive attached and mounted... Paste the outputs of 'mount' and 'df' when run as root. publickey - g4sra@protonmail.com - 0x42E94623.asc Description: application/pgp-keys signature.asc Description: OpenPGP digital signature ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
On Mon, 15 Mar 2021 20:36:28 -0700 Marc Shapiro via Dng wrote: > I'm not familiar with sfill. What does it do and what package is it > in. It is not currently installed here. > > Marc sfill is part of secure-delete. rm doesn't actually remove data, it just remove all links to it leaving the space available to be used by something else. secure-delete scramble the data with junk (several times) so nothing can be recovered. pgpgIldYjm8c3.pgp Description: Signature digitale OpenPGP ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
On 3/15/21 2:48 AM, Bernard Rosset via Dng wrote: You are correct. I used '+L' NOT '-L'. I would add -nP -> "lsof -nP +L1" If negative, I would go for the ugly path, grep'ing lsof's output on "deleted" or "(deleted)". Past this point, if space of alleged deleted files is not cleared... I wonder. Even ext2 should do the trick. If not ext4, I would upgrade to it by changing the flags with the help of tune2fs. I am especially surprises by the fact that you stated you could unmount the volume and remount it, still without seeing free space improvement... Are you positive you deleted the real files, and not mere links to them? I would make sure by using du to seek for actual disk usage location. In a last, desperate resort, I would try to force allocation/release of free space through sfill I'm not familiar with sfill. What does it do and what package is it in. It is not currently installed here. Marc ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
On 3/15/21 8:23 PM, Arnt Karlsen wrote: On Mon, 15 Mar 2021 19:54:20 -0700, Marc wrote in message <74553f6b-2616-70e1-e742-1ce9275b3...@gmail.com>: On 3/15/21 6:31 AM, Hendrik Boom wrote: On Sun, Mar 14, 2021 at 10:28:32PM -0700, Marc Shapiro via Dng wrote: On 3/14/21 10:09 PM, Ludovic Bellière wrote: I assume you read the man page of fsck, as it's return code is what you want to pay attention to. As for lsof, the correct parameters would be `lsof +aL1 /dev/sdx. It should have thrown an error were you to use `lsof -L1`. If lsof returns nothing, your drive is most likely corrupted. You are correct. I used '+L' NOT '-L'. It may also be possible that the files you removed have other references on your file system, aka. hard links. To find them, you would need to know the inode number, either by using `stat` or `ls -i`. You can then find them using `find -inum`. Since you already removed the files, you most likely can't know the inode number. However you could throw a `find $path -size n[cwbkMG]` to list the files with the matching size. I'm not following you on this. What is this going to do for me? 'find' is only going to show undeleted files. How does this help? It is possible for a single file to be hard-linked in several places in the file system. If so, removing it in one place will still leave it accessible from another, and therefore not deleted. Files have reference counts to keep track of this. These files should not have any links, hard or soft. None of the other files in that directory show a reference count above 1. They are backup files created by fsarchiver. I'm just trying to free up space by deleting files from January. ..any chance they have white-space-character-only names? E.g. " ", " ", " " etc, or Norwegian æ, ø, å, or some invisible non-Latin alphabet soup not supported by your fonts nor locales? You should still be able to cut-n-paste those and have them show up as high-lighted boxes. No. These are file that had names when I deleted them. I wrote/modified the scripts that generated the filenames, so all of the characters came from my keyboard. I hate names with spaces in them, so there weren't any spaces or hidden characters in them. Marc ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
On Mon, 15 Mar 2021 19:54:20 -0700, Marc wrote in message <74553f6b-2616-70e1-e742-1ce9275b3...@gmail.com>: > On 3/15/21 6:31 AM, Hendrik Boom wrote: > > On Sun, Mar 14, 2021 at 10:28:32PM -0700, Marc Shapiro via Dng > > wrote: > >> On 3/14/21 10:09 PM, Ludovic Bellière wrote: > >>> I assume you read the man page of fsck, as it's return code is > >>> what you want to pay attention to. > >>> > >>> As for lsof, the correct parameters would be `lsof +aL1 /dev/sdx. > >>> It should have thrown an error were you to use `lsof -L1`. If > >>> lsof returns nothing, your drive is most likely corrupted. > >> You are correct. I used '+L' NOT '-L'. > >>> It may also be possible that the files you removed have other > >>> references on your file system, aka. hard links. To find them, you > >>> would need to know the inode number, either by using `stat` or > >>> `ls -i`. You can then find them using `find -inum`. > >>> > >>> Since you already removed the files, you most likely can't know > >>> the inode number. However you could throw a `find $path -size > >>> n[cwbkMG]` to list the files with the matching size. > >> I'm not following you on this. What is this going to do for me? > >> 'find' is only going to show undeleted files. How does this > >> help? > > It is possible for a single file to be hard-linked in several > > places in the file system. If so, removing it in one place will > > still leave it accessible from another, and therefore not deleted. > > > > Files have reference counts to keep track of this. > > These files should not have any links, hard or soft. None of the > other files in that directory show a reference count above 1. They > are backup files created by fsarchiver. I'm just trying to free up > space by deleting files from January. ..any chance they have white-space-character-only names? E.g. " ", " ", " " etc, or Norwegian æ, ø, å, or some invisible non-Latin alphabet soup not supported by your fonts nor locales? You should still be able to cut-n-paste those and have them show up as high-lighted boxes. -- ..med vennlig hilsen = with Kind Regards from Arnt Karlsen ...with a number of polar bear hunters in his ancestry... Scenarios always come in sets of three: best case, worst case, and just in case. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
On 3/15/21 6:31 AM, Hendrik Boom wrote: On Sun, Mar 14, 2021 at 10:28:32PM -0700, Marc Shapiro via Dng wrote: On 3/14/21 10:09 PM, Ludovic Bellière wrote: I assume you read the man page of fsck, as it's return code is what you want to pay attention to. As for lsof, the correct parameters would be `lsof +aL1 /dev/sdx. It should have thrown an error were you to use `lsof -L1`. If lsof returns nothing, your drive is most likely corrupted. You are correct. I used '+L' NOT '-L'. It may also be possible that the files you removed have other references on your file system, aka. hard links. To find them, you would need to know the inode number, either by using `stat` or `ls -i`. You can then find them using `find -inum`. Since you already removed the files, you most likely can't know the inode number. However you could throw a `find $path -size n[cwbkMG]` to list the files with the matching size. I'm not following you on this. What is this going to do for me? 'find' is only going to show undeleted files. How does this help? It is possible for a single file to be hard-linked in several places in the file system. If so, removing it in one place will still leave it accessible from another, and therefore not deleted. Files have reference counts to keep track of this. These files should not have any links, hard or soft. None of the other files in that directory show a reference count above 1. They are backup files created by fsarchiver. I'm just trying to free up space by deleting files from January. Marc ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
On 3/15/21 12:55 AM, Steve Litt wrote: Wait!!! Make sure you run only a non-destructive fsck. If the non-destructive fsck shows no problem, do the following... First run the sync command. The sync command forces all caches to be written to their respective files. If they're on a removeable drive, my next step after sync would be to unmount the drive. Then physically disconnect it. Then physically reconnect it. Then remount it. Then do your df -h. Already did this. See my original post. Also, if your removeable drive is some sort of SSD, you might need to do fstrim, although I swear I remember space deleted but not yet trimmed doesn't count in df. But I could be wrong. Removable spinning disk. SteveT Ludovic Bellière said on Mon, 15 Mar 2021 04:33:37 +0100 Run fsck to make sure your disk isn't corrupted or damaged. Afterward, your lost+found might get populated with the stuff that occupies the space, done so in order for you to review. On Sun, 14 Mar 2021 20:10:03 -0700 Marc Shapiro via Dng wrote: I had some large files (over 200GB in total size) that I no longer needed. So I removed them. They no longer show up in the directory. However, df, still shows the space as being used. I understand that if processes are still using the files they will not be removed until the processes either release them, or are shut down (or killed). I have tried using 'lsof -a -L1 MOUNTPOINT' to list the open, but deleted files. It returns no files at all. These files are on a removable drive. I can unmount the partition and disconnect the drive, then reconnect the drive and remount the partition, but the space still shows as being used. If the files are still open, shouldn't 'umount' give an error? I would rather not have to shut down the system. Is there some other way to determine what is keeping these files open, or otherwise preventing the space from being reclaimed? Marc ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
You are correct. I used '+L' NOT '-L'. I would add -nP -> "lsof -nP +L1" If negative, I would go for the ugly path, grep'ing lsof's output on "deleted" or "(deleted)". Past this point, if space of alleged deleted files is not cleared... I wonder. Even ext2 should do the trick. If not ext4, I would upgrade to it by changing the flags with the help of tune2fs. I am especially surprises by the fact that you stated you could unmount the volume and remount it, still without seeing free space improvement... Are you positive you deleted the real files, and not mere links to them? I would make sure by using du to seek for actual disk usage location. In a last, desperate resort, I would try to force allocation/release of free space through sfill Bernard (Beer) Rosset https://rosset.net/ ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
On Sun, 14 Mar 2021 20:39:51 -0700 Marc Shapiro via Dng wrote: > > On 3/14/21 8:33 PM, Ludovic Bellière wrote: > > Run fsck to make sure your disk isn't corrupted or damaged. > > Afterward, your lost+found might get populated with the stuff that > > occupies the space, done so in order for you to review. > > Already looked in lost+found. Nothing there. > > Tried fsck. It says the partition is clean. Did you "force" the fsck (with '-f'), or did you rely on the filesystem's "marked as clean" flag? libre Grüße, Florian ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
Wait!!! Make sure you run only a non-destructive fsck. If the non-destructive fsck shows no problem, do the following... First run the sync command. The sync command forces all caches to be written to their respective files. If they're on a removeable drive, my next step after sync would be to unmount the drive. Then physically disconnect it. Then physically reconnect it. Then remount it. Then do your df -h. Also, if your removeable drive is some sort of SSD, you might need to do fstrim, although I swear I remember space deleted but not yet trimmed doesn't count in df. But I could be wrong. SteveT Ludovic Bellière said on Mon, 15 Mar 2021 04:33:37 +0100 >Run fsck to make sure your disk isn't corrupted or damaged. Afterward, >your lost+found might get populated with the stuff that occupies the >space, done so in order for you to review. > >On Sun, 14 Mar 2021 20:10:03 -0700 >Marc Shapiro via Dng wrote: > >> I had some large files (over 200GB in total size) that I no longer >> needed. So I removed them. They no longer show up in the >> directory. However, df, still shows the space as being used. I >> understand that if processes are still using the files they will not >> be removed until the processes either release them, or are shut down >> (or killed). I have tried using 'lsof -a -L1 MOUNTPOINT' to list the >> open, but deleted files. It returns no files at all. >> >> These files are on a removable drive. I can unmount the partition >> and disconnect the drive, then reconnect the drive and remount the >> partition, but the space still shows as being used. If the files are >> still open, shouldn't 'umount' give an error? >> >> I would rather not have to shut down the system. Is there some >> other way to determine what is keeping these files open, or >> otherwise preventing the space from being reclaimed? >> >> Marc >> >> ___ >> Dng mailing list >> Dng@lists.dyne.org >> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
On Sun, 14 Mar 2021 22:28:32 -0700 Marc Shapiro via Dng wrote: > > It may also be possible that the files you removed have other > > references on your file system, aka. hard links. To find them, you > > would need to know the inode number, either by using `stat` or `ls > > -i`. You can then find them using `find -inum`. > > > > Since you already removed the files, you most likely can't know the > > inode number. However you could throw a `find $path -size > > n[cwbkMG]` to list the files with the matching size. > > I'm not following you on this. What is this going to do for me? > 'find' is only going to show undeleted files. How does this help? If rm doesn't free space, that means the file isn't gone. Thus you need to find it. The find command can help you in that endeavor. Another software (or kind of software) would be xdiskusage, which will tell you where the big files are. That is, of course, if you drive isn't corrupted. pgpadRY9iMa1r.pgp Description: Signature digitale OpenPGP ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
On 3/14/21 10:09 PM, Ludovic Bellière wrote: I assume you read the man page of fsck, as it's return code is what you want to pay attention to. As for lsof, the correct parameters would be `lsof +aL1 /dev/sdx. It should have thrown an error were you to use `lsof -L1`. If lsof returns nothing, your drive is most likely corrupted. You are correct. I used '+L' NOT '-L'. It may also be possible that the files you removed have other references on your file system, aka. hard links. To find them, you would need to know the inode number, either by using `stat` or `ls -i`. You can then find them using `find -inum`. Since you already removed the files, you most likely can't know the inode number. However you could throw a `find $path -size n[cwbkMG]` to list the files with the matching size. I'm not following you on this. What is this going to do for me? 'find' is only going to show undeleted files. How does this help? Marc ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
I assume you read the man page of fsck, as it's return code is what you want to pay attention to. As for lsof, the correct parameters would be `lsof +aL1 /dev/sdx. It should have thrown an error were you to use `lsof -L1`. If lsof returns nothing, your drive is most likely corrupted. It may also be possible that the files you removed have other references on your file system, aka. hard links. To find them, you would need to know the inode number, either by using `stat` or `ls -i`. You can then find them using `find -inum`. Since you already removed the files, you most likely can't know the inode number. However you could throw a `find $path -size n[cwbkMG]` to list the files with the matching size. On Sun, 14 Mar 2021 20:39:51 -0700 Marc Shapiro via Dng wrote: > On 3/14/21 8:33 PM, Ludovic Bellière wrote: > > Run fsck to make sure your disk isn't corrupted or damaged. > > Afterward, your lost+found might get populated with the stuff that > > occupies the space, done so in order for you to review. > > Already looked in lost+found. Nothing there. > > Tried fsck. It says the partition is clean. > > > Marc pgpj92yggPs5Q.pgp Description: Signature digitale OpenPGP ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
On 3/14/21 8:33 PM, Ludovic Bellière wrote: Run fsck to make sure your disk isn't corrupted or damaged. Afterward, your lost+found might get populated with the stuff that occupies the space, done so in order for you to review. Already looked in lost+found. Nothing there. Tried fsck. It says the partition is clean. Marc On Sun, 14 Mar 2021 20:10:03 -0700 Marc Shapiro via Dng wrote: I had some large files (over 200GB in total size) that I no longer needed. So I removed them. They no longer show up in the directory. However, df, still shows the space as being used. I understand that if processes are still using the files they will not be removed until the processes either release them, or are shut down (or killed). I have tried using 'lsof -a -L1 MOUNTPOINT' to list the open, but deleted files. It returns no files at all. These files are on a removable drive. I can unmount the partition and disconnect the drive, then reconnect the drive and remount the partition, but the space still shows as being used. If the files are still open, shouldn't 'umount' give an error? I would rather not have to shut down the system. Is there some other way to determine what is keeping these files open, or otherwise preventing the space from being reclaimed? Marc ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] rm not freeing space
Run fsck to make sure your disk isn't corrupted or damaged. Afterward, your lost+found might get populated with the stuff that occupies the space, done so in order for you to review. On Sun, 14 Mar 2021 20:10:03 -0700 Marc Shapiro via Dng wrote: > I had some large files (over 200GB in total size) that I no longer > needed. So I removed them. They no longer show up in the > directory. However, df, still shows the space as being used. I > understand that if processes are still using the files they will not > be removed until the processes either release them, or are shut down > (or killed). I have tried using 'lsof -a -L1 MOUNTPOINT' to list the > open, but deleted files. It returns no files at all. > > These files are on a removable drive. I can unmount the partition > and disconnect the drive, then reconnect the drive and remount the > partition, but the space still shows as being used. If the files are > still open, shouldn't 'umount' give an error? > > I would rather not have to shut down the system. Is there some other > way to determine what is keeping these files open, or otherwise > preventing the space from being reclaimed? > > Marc > > ___ > Dng mailing list > Dng@lists.dyne.org > https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng pgpQKNOmwt33o.pgp Description: Signature digitale OpenPGP ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
[DNG] rm not freeing space
I had some large files (over 200GB in total size) that I no longer needed. So I removed them. They no longer show up in the directory. However, df, still shows the space as being used. I understand that if processes are still using the files they will not be removed until the processes either release them, or are shut down (or killed). I have tried using 'lsof -a -L1 MOUNTPOINT' to list the open, but deleted files. It returns no files at all. These files are on a removable drive. I can unmount the partition and disconnect the drive, then reconnect the drive and remount the partition, but the space still shows as being used. If the files are still open, shouldn't 'umount' give an error? I would rather not have to shut down the system. Is there some other way to determine what is keeping these files open, or otherwise preventing the space from being reclaimed? Marc ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
