Re: [ceph-users] problem in cephfs for remove empty directory
On 03/03/2015 14:07, Daniel Takatori Ohara wrote: *$ls test-daniel-old/* total 0 drwx-- 1 rmagalhaes BioInfoHSL Users0 Mar 2 10:52 ./ drwx-- 1 rmagalhaes BioInfoHSL Users 773099838313 Mar 2 11:41 ../ *$rm -rf test-daniel-old/* rm: cannot remove ‘test-daniel-old/’: Directory not empty *$ls test-daniel-old/* ls: cannot access test-daniel-old/M_S8_L001_R1-2_001.fastq.gz_ref.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L001_R1-2_001.fastq.gz_sylvio.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L002_R1-2_001.fastq.gz_ref.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L002_R1-2_001.fastq.gz_sylvio.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L003_R1-2_001.fastq.gz_ref.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L003_R1-2_001.fastq.gz_sylvio.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L004_R1-2_001.fastq.gz_ref.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L004_R1-2_001.fastq.gz_sylvio.sam_fixed.bam: No such file or directory total 0 drwx-- 1 rmagalhaes BioInfoHSL Users0 Mar 2 10:52 ./ drwx-- 1 rmagalhaes BioInfoHSL Users 773099838313 Mar 2 11:41 ../ l? ? ? ? ? ? M_S8_L001_R1-2_001.fastq.gz_ref.sam_fixed.bam l? ? ? ? ? ? M_S8_L001_R1-2_001.fastq.gz_sylvio.sam_fixed.bam l? ? ? ? ? ? M_S8_L002_R1-2_001.fastq.gz_ref.sam_fixed.bam l? ? ? ? ? ? M_S8_L002_R1-2_001.fastq.gz_sylvio.sam_fixed.bam l? ? ? ? ? ? M_S8_L003_R1-2_001.fastq.gz_ref.sam_fixed.bam l? ? ? ? ? ? M_S8_L003_R1-2_001.fastq.gz_sylvio.sam_fixed.bam l? ? ? ? ? ? M_S8_L004_R1-2_001.fastq.gz_ref.sam_fixed.bam l? ? ? ? ? ? M_S8_L004_R1-2_001.fastq.gz_sylvio.sam_fixed.bam You don't say what version of the client (version of kernel, if it's the kernel client) this is. It would appear that the client thinks there are some dentries that don't really exist. You should enable verbose debug logs (with fuse client, debug client = 20) and reproduce this. It looks like you had similar issues (subject: problem for remove files in cephfs) a while back, when Yan Zheng also advised you to get some debug logs. John ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] problem in cephfs for remove empty directory
On Tue, Mar 3, 2015 at 9:24 AM, John Spray john.sp...@redhat.com wrote: On 03/03/2015 14:07, Daniel Takatori Ohara wrote: $ls test-daniel-old/ total 0 drwx-- 1 rmagalhaes BioInfoHSL Users0 Mar 2 10:52 ./ drwx-- 1 rmagalhaes BioInfoHSL Users 773099838313 Mar 2 11:41 ../ $rm -rf test-daniel-old/ rm: cannot remove ‘test-daniel-old/’: Directory not empty $ls test-daniel-old/ ls: cannot access test-daniel-old/M_S8_L001_R1-2_001.fastq.gz_ref.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L001_R1-2_001.fastq.gz_sylvio.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L002_R1-2_001.fastq.gz_ref.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L002_R1-2_001.fastq.gz_sylvio.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L003_R1-2_001.fastq.gz_ref.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L003_R1-2_001.fastq.gz_sylvio.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L004_R1-2_001.fastq.gz_ref.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L004_R1-2_001.fastq.gz_sylvio.sam_fixed.bam: No such file or directory total 0 drwx-- 1 rmagalhaes BioInfoHSL Users0 Mar 2 10:52 ./ drwx-- 1 rmagalhaes BioInfoHSL Users 773099838313 Mar 2 11:41 ../ l? ? ? ? ?? M_S8_L001_R1-2_001.fastq.gz_ref.sam_fixed.bam l? ? ? ? ?? M_S8_L001_R1-2_001.fastq.gz_sylvio.sam_fixed.bam l? ? ? ? ?? M_S8_L002_R1-2_001.fastq.gz_ref.sam_fixed.bam l? ? ? ? ?? M_S8_L002_R1-2_001.fastq.gz_sylvio.sam_fixed.bam l? ? ? ? ?? M_S8_L003_R1-2_001.fastq.gz_ref.sam_fixed.bam l? ? ? ? ?? M_S8_L003_R1-2_001.fastq.gz_sylvio.sam_fixed.bam l? ? ? ? ?? M_S8_L004_R1-2_001.fastq.gz_ref.sam_fixed.bam l? ? ? ? ?? M_S8_L004_R1-2_001.fastq.gz_sylvio.sam_fixed.bam You don't say what version of the client (version of kernel, if it's the kernel client) this is. It would appear that the client thinks there are some dentries that don't really exist. You should enable verbose debug logs (with fuse client, debug client = 20) and reproduce this. It looks like you had similar issues (subject: problem for remove files in cephfs) a while back, when Yan Zheng also advised you to get some debug logs. In particular this is a known bug in older kernels and is fixed in new enough ones. Unfortunately I don't have the bug link handy though. :( -Greg ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] problem in cephfs for remove empty directory
Hi John and Gregory, The version of ceph client is 0.87 and the kernel is 3.13. The debug logs here in attach. I see this problem in a older kernel, but i didn't find the solution in the track. Thanks, Att. --- Daniel Takatori Ohara. System Administrator - Lab. of Bioinformatics Molecular Oncology Center Instituto Sírio-Libanês de Ensino e Pesquisa Hospital Sírio-Libanês Phone: +55 11 3155-0200 (extension 1927) R: Cel. Nicolau dos Santos, 69 São Paulo-SP. 01308-060 http://www.bioinfo.mochsl.org.br On Tue, Mar 3, 2015 at 2:26 PM, Gregory Farnum g...@gregs42.com wrote: On Tue, Mar 3, 2015 at 9:24 AM, John Spray john.sp...@redhat.com wrote: On 03/03/2015 14:07, Daniel Takatori Ohara wrote: $ls test-daniel-old/ total 0 drwx-- 1 rmagalhaes BioInfoHSL Users0 Mar 2 10:52 ./ drwx-- 1 rmagalhaes BioInfoHSL Users 773099838313 Mar 2 11:41 ../ $rm -rf test-daniel-old/ rm: cannot remove ‘test-daniel-old/’: Directory not empty $ls test-daniel-old/ ls: cannot access test-daniel-old/M_S8_L001_R1-2_001.fastq.gz_ref.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L001_R1-2_001.fastq.gz_sylvio.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L002_R1-2_001.fastq.gz_ref.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L002_R1-2_001.fastq.gz_sylvio.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L003_R1-2_001.fastq.gz_ref.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L003_R1-2_001.fastq.gz_sylvio.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L004_R1-2_001.fastq.gz_ref.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L004_R1-2_001.fastq.gz_sylvio.sam_fixed.bam: No such file or directory total 0 drwx-- 1 rmagalhaes BioInfoHSL Users0 Mar 2 10:52 ./ drwx-- 1 rmagalhaes BioInfoHSL Users 773099838313 Mar 2 11:41 ../ l? ? ? ? ?? M_S8_L001_R1-2_001.fastq.gz_ref.sam_fixed.bam l? ? ? ? ?? M_S8_L001_R1-2_001.fastq.gz_sylvio.sam_fixed.bam l? ? ? ? ?? M_S8_L002_R1-2_001.fastq.gz_ref.sam_fixed.bam l? ? ? ? ?? M_S8_L002_R1-2_001.fastq.gz_sylvio.sam_fixed.bam l? ? ? ? ?? M_S8_L003_R1-2_001.fastq.gz_ref.sam_fixed.bam l? ? ? ? ?? M_S8_L003_R1-2_001.fastq.gz_sylvio.sam_fixed.bam l? ? ? ? ?? M_S8_L004_R1-2_001.fastq.gz_ref.sam_fixed.bam l? ? ? ? ?? M_S8_L004_R1-2_001.fastq.gz_sylvio.sam_fixed.bam You don't say what version of the client (version of kernel, if it's the kernel client) this is. It would appear that the client thinks there are some dentries that don't really exist. You should enable verbose debug logs (with fuse client, debug client = 20) and reproduce this. It looks like you had similar issues (subject: problem for remove files in cephfs) a while back, when Yan Zheng also advised you to get some debug logs. In particular this is a known bug in older kernels and is fixed in new enough ones. Unfortunately I don't have the bug link handy though. :( -Greg log_mds.gz Description: GNU Zip compressed data ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] problem in cephfs for remove empty directory
Hi, I have a problem when i will remove a empty directory in cephfs. The directory is empty, but it seems have files crashed in MDS. *$ls test-daniel-old/* total 0 drwx-- 1 rmagalhaes BioInfoHSL Users0 Mar 2 10:52 ./ drwx-- 1 rmagalhaes BioInfoHSL Users 773099838313 Mar 2 11:41 ../ *$rm -rf test-daniel-old/* rm: cannot remove ‘test-daniel-old/’: Directory not empty *$ls test-daniel-old/* ls: cannot access test-daniel-old/M_S8_L001_R1-2_001.fastq.gz_ref.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L001_R1-2_001.fastq.gz_sylvio.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L002_R1-2_001.fastq.gz_ref.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L002_R1-2_001.fastq.gz_sylvio.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L003_R1-2_001.fastq.gz_ref.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L003_R1-2_001.fastq.gz_sylvio.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L004_R1-2_001.fastq.gz_ref.sam_fixed.bam: No such file or directory ls: cannot access test-daniel-old/M_S8_L004_R1-2_001.fastq.gz_sylvio.sam_fixed.bam: No such file or directory total 0 drwx-- 1 rmagalhaes BioInfoHSL Users0 Mar 2 10:52 ./ drwx-- 1 rmagalhaes BioInfoHSL Users 773099838313 Mar 2 11:41 ../ l? ? ? ? ?? M_S8_L001_R1-2_001.fastq.gz_ref.sam_fixed.bam l? ? ? ? ?? M_S8_L001_R1-2_001.fastq.gz_sylvio.sam_fixed.bam l? ? ? ? ?? M_S8_L002_R1-2_001.fastq.gz_ref.sam_fixed.bam l? ? ? ? ?? M_S8_L002_R1-2_001.fastq.gz_sylvio.sam_fixed.bam l? ? ? ? ?? M_S8_L003_R1-2_001.fastq.gz_ref.sam_fixed.bam l? ? ? ? ?? M_S8_L003_R1-2_001.fastq.gz_sylvio.sam_fixed.bam l? ? ? ? ?? M_S8_L004_R1-2_001.fastq.gz_ref.sam_fixed.bam l? ? ? ? ?? M_S8_L004_R1-2_001.fastq.gz_sylvio.sam_fixed.bam Att. --- Daniel Takatori Ohara. System Administrator - Lab. of Bioinformatics Molecular Oncology Center Instituto Sírio-Libanês de Ensino e Pesquisa Hospital Sírio-Libanês Phone: +55 11 3155-0200 (extension 1927) R: Cel. Nicolau dos Santos, 69 São Paulo-SP. 01308-060 http://www.bioinfo.mochsl.org.br ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
