Re: [Cocci] [bug] exists do not work if file group is too big (>49)
On Thu, May 17, 2018 at 10:31:52PM +0200, Julia Lawall wrote: > Concretely, the problem is with LOCK_REQ, which is used as a typedef name > in cifs and as a enum constant in f2fs. > > Would there be any chance of getting rid of the LOCK_REQ typedef? > Typedefs for structure types are discouraged by the Linux kernel coding > style. > > julia I will try to make patches for that, thought not anytime soon i am going on PTO so not before i get back. Thank you for debugging this further. Cheers, Jérôme ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [bug] exists do not work if file group is too big (>49)
Concretely, the problem is with LOCK_REQ, which is used as a typedef name in cifs and as a enum constant in f2fs. Would there be any chance of getting rid of the LOCK_REQ typedef? Typedefs for structure types are discouraged by the Linux kernel coding style. julia ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [bug] exists do not work if file group is too big (>49)
On Wed, 16 May 2018, Jerome Glisse wrote: > When operating on too many files the exists keyword no longer work. For > instance: It has nothing to do with the number of files. The problem is that some property of one file causes a parse error in another file, due to the various heuristics. You can see this with the --verbose-parsing option. I'm not yet sure where the bad interaction comes from, but I get the same bad behavior with the command line: spatch --verbose-parsing --no-includes --sp-file test.cocci fs/ceph/addr.c fs/cifs/cifssmb.c fs/f2fs/gc.c julia > > @exists@ > expression E1; > @@ > set_page_dirty( > +NULL, > E1) > > Using linux kernel as playground and looking at at fs/f2fs/gc.c where > there is set_page_dirty() calls in nested blocks. > > Works: > > spatch --no-includes --sp-file test.spatch > arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c > arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c > drivers/block/zram/zram_drv.c drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > drivers/gpu/drm/drm_gem.c drivers/gpu/drm/exynos/exynos_drm_g2d.c > drivers/gpu/drm/i915/i915_gem.c drivers/gpu/drm/i915/i915_gem_fence_reg.c > drivers/gpu/drm/i915/i915_gem_userptr.c drivers/gpu/drm/radeon/radeon_ttm.c > drivers/gpu/drm/ttm/ttm_tt.c drivers/infiniband/core/umem.c > drivers/infiniband/core/umem_odp.c drivers/infiniband/hw/hfi1/user_pages.c > drivers/infiniband/hw/qib/qib_user_pages.c > drivers/infiniband/hw/usnic/usnic_uiom.c > drivers/media/common/videobuf2/videobuf2-dma-contig.c > drivers/media/common/videobuf2/videobuf2-dma-sg.c > drivers/media/common/videobuf2/videobuf2-vmalloc.c > drivers/misc/genwqe/card_utils.c drivers/misc/vmw_vmci/vmci_queue_pair.c > drivers/mtd/devices/block2mtd.c drivers/platform/goldfish/goldfish_pipe.c > drivers/sbus/! char/oradax.c drivers/staging/lustre/lustre/llite/rw26.c drivers/staging/lustre/lustre/llite/vvp_io.c drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c fs/9p/vfs_addr.c fs/afs/dir.c fs/afs/file.c --include fs/afs/internal.h fs/afs/write.c fs/aio.c fs/block_dev.c fs/btrfs/disk-io.c fs/btrfs/extent_io.c fs/btrfs/file.c fs/btrfs/inode.c fs/btrfs/ioctl.c fs/btrfs/relocation.c fs/buffer.c fs/ceph/addr.c fs/f2fs/gc.c > toto > > > Don't works: > > spatch --no-includes --sp-file test.spatch > arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c > arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c > drivers/block/zram/zram_drv.c drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > drivers/gpu/drm/drm_gem.c drivers/gpu/drm/exynos/exynos_drm_g2d.c > drivers/gpu/drm/i915/i915_gem.c drivers/gpu/drm/i915/i915_gem_fence_reg.c > drivers/gpu/drm/i915/i915_gem_userptr.c drivers/gpu/drm/radeon/radeon_ttm.c > drivers/gpu/drm/ttm/ttm_tt.c drivers/infiniband/core/umem.c > drivers/infiniband/core/umem_odp.c drivers/infiniband/hw/hfi1/user_pages.c > drivers/infiniband/hw/qib/qib_user_pages.c > drivers/infiniband/hw/usnic/usnic_uiom.c > drivers/media/common/videobuf2/videobuf2-dma-contig.c > drivers/media/common/videobuf2/videobuf2-dma-sg.c > drivers/media/common/videobuf2/videobuf2-vmalloc.c > drivers/misc/genwqe/card_utils.c drivers/misc/vmw_vmci/vmci_queue_pair.c > drivers/mtd/devices/block2mtd.c drivers/platform/goldfish/goldfish_pipe.c > drivers/sbus/! char/oradax.c drivers/staging/lustre/lustre/llite/rw26.c drivers/staging/lustre/lustre/llite/vvp_io.c drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c fs/9p/vfs_addr.c fs/afs/dir.c fs/afs/file.c --include fs/afs/internal.h fs/afs/write.c fs/aio.c fs/block_dev.c fs/btrfs/disk-io.c fs/btrfs/extent_io.c fs/btrfs/file.c fs/btrfs/inode.c fs/btrfs/ioctl.c fs/btrfs/relocation.c fs/buffer.c fs/ceph/addr.c fs/cifs/cifssmb.c fs/f2fs/gc.c > toto > > > Exact same command line except the latter have 49 files before > fs/f2fs/gc.c while the former has 48 (and it works then). Note > that it seems that running the script multiple times even when > they are too many files will eventualy work. > > Cheers, > Jérôme > ___ > Cocci mailing list > Cocci@systeme.lip6.fr > https://systeme.lip6.fr/mailman/listinfo/cocci >___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [bug] exists do not work if file group is too big (>49)
On Wed, May 16, 2018 at 10:29:30PM +0200, Julia Lawall wrote: > > [glisse@localhost ~]$ spatch --version > > spatch version 1.0.6 compiled with OCaml version 4.05.0 > > Flags passed to the configure script: --build=x86_64-redhat-linux-gnu > > --host=x86_64-redhat-linux-gnu --program-prefix= > > --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr > > --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc > > --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 > > --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib > > --mandir=/usr/share/man --infodir=/usr/share/info --enable-release=yes > > --with-python=/usr/bin/python3 --with-menhir=/usr/bin/menhir > > Python scripting support: yes > > Syntax of regular expresssions: PCRE > > I don't think this indicates whether it is bytecode or native code, and I > don't know what the fedora people offer. > > My bytecode executable is 41M and my native code executable is 13M. I am guessing native code: -rwxr-xr-x. 1 root root 1.3K May 10 13:51 /usr/bin/spatch Jérôme ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [bug] exists do not work if file group is too big (>49)
> [glisse@localhost ~]$ spatch --version > spatch version 1.0.6 compiled with OCaml version 4.05.0 > Flags passed to the configure script: --build=x86_64-redhat-linux-gnu > --host=x86_64-redhat-linux-gnu --program-prefix= > --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr > --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share > --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec > --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man > --infodir=/usr/share/info --enable-release=yes --with-python=/usr/bin/python3 > --with-menhir=/usr/bin/menhir > Python scripting support: yes > Syntax of regular expresssions: PCRE I don't think this indicates whether it is bytecode or native code, and I don't know what the fedora people offer. My bytecode executable is 41M and my native code executable is 13M. julia ___ Cocci mailing list Cocci@systeme.lip6.fr https://systeme.lip6.fr/mailman/listinfo/cocci
Re: [Cocci] [bug] exists do not work if file group is too big (>49)
On Wed, May 16, 2018 at 10:20:05PM +0200, Julia Lawall wrote: > > > On Wed, 16 May 2018, Jerome Glisse wrote: > > > On Wed, May 16, 2018 at 10:02:08PM +0200, Julia Lawall wrote: > > > > > > > > > On Wed, 16 May 2018, Jerome Glisse wrote: > > > > > > > On Wed, May 16, 2018 at 09:37:39PM +0200, Julia Lawall wrote: > > > > > > > > > > > > > > > On Wed, 16 May 2018, Jerome Glisse wrote: > > > > > > > > > > > When operating on too many files the exists keyword no longer work. > > > > > > For > > > > > > instance: > > > > > > > > > > > > @exists@ > > > > > > expression E1; > > > > > > @@ > > > > > > set_page_dirty( > > > > > > +NULL, > > > > > > E1) > > > > > > > > > > > > Using linux kernel as playground and looking at at fs/f2fs/gc.c > > > > > > where > > > > > > there is set_page_dirty() calls in nested blocks. > > > > > > > > > > > > Works: > > > > > > > > > > > > spatch --no-includes --sp-file test.spatch > > > > > > arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c > > > > > > arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c > > > > > > drivers/block/zram/zram_drv.c > > > > > > drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c drivers/gpu/drm/drm_gem.c > > > > > > drivers/gpu/drm/exynos/exynos_drm_g2d.c > > > > > > drivers/gpu/drm/i915/i915_gem.c > > > > > > drivers/gpu/drm/i915/i915_gem_fence_reg.c > > > > > > drivers/gpu/drm/i915/i915_gem_userptr.c > > > > > > drivers/gpu/drm/radeon/radeon_ttm.c drivers/gpu/drm/ttm/ttm_tt.c > > > > > > drivers/infiniband/core/umem.c drivers/infiniband/core/umem_odp.c > > > > > > drivers/infiniband/hw/hfi1/user_pages.c > > > > > > drivers/infiniband/hw/qib/qib_user_pages.c > > > > > > drivers/infiniband/hw/usnic/usnic_uiom.c > > > > > > drivers/media/common/videobuf2/videobuf2-dma-contig.c > > > > > > drivers/media/common/videobuf2/videobuf2-dma-sg.c > > > > > > drivers/media/common/videobuf2/videobuf2-vmalloc.c > > > > > > drivers/misc/genwqe/card_utils.c > > > > > > drivers/misc/vmw_vmci/vmci_queue_pair.c > > > > > > drivers/mtd/devices/block2mtd.c > > > > > > drivers/platform/goldfish/goldfish_pipe.c drivers/sbus/c > har/ > > > orad > > > > > ax.c drivers/staging/lustre/lustre/llite/rw26.c > > > > > drivers/staging/lustre/lustre/llite/vvp_io.c > > > > > drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c > > > > > drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c > > > > > fs/9p/vfs_addr.c fs/afs/dir.c fs/afs/file.c --include > > > > > fs/afs/internal.h fs/afs/write.c fs/aio.c fs/block_dev.c > > > > > fs/btrfs/disk-io.c fs/btrfs/extent_io.c fs/btrfs/file.c > > > > > fs/btrfs/inode.c fs/btrfs/ioctl.c fs/btrfs/relocation.c fs/buffer.c > > > > > fs/ceph/addr.c fs/f2fs/gc.c > toto > > > > > > > > > > > > > > > > > > Don't works: > > > > > > > > > > > > spatch --no-includes --sp-file test.spatch > > > > > > arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c > > > > > > arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c > > > > > > drivers/block/zram/zram_drv.c > > > > > > drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c drivers/gpu/drm/drm_gem.c > > > > > > drivers/gpu/drm/exynos/exynos_drm_g2d.c > > > > > > drivers/gpu/drm/i915/i915_gem.c > > > > > > drivers/gpu/drm/i915/i915_gem_fence_reg.c > > > > > > drivers/gpu/drm/i915/i915_gem_userptr.c > > > > > > drivers/gpu/drm/radeon/radeon_ttm.c drivers/gpu/drm/ttm/ttm_tt.c > > > > > > drivers/infiniband/core/umem.c drivers/infiniband/core/umem_odp.c > > > > > > drivers/infiniband/hw/hfi1/user_pages.c > > > > > > drivers/infiniband/hw/qib/qib_user_pages.c > > > > > > drivers/infiniband/hw/usnic/usnic_uiom.c > > > > > > drivers/media/common/videobuf2/videobuf2-dma-contig.c > > > > > > drivers/media/common/videobuf2/videobuf2-dma-sg.c > > > > > > drivers/media/common/videobuf2/videobuf2-vmalloc.c > > > > > > drivers/misc/genwqe/card_utils.c > > > > > > drivers/misc/vmw_vmci/vmci_queue_pair.c > > > > > > drivers/mtd/devices/block2mtd.c > > > > > > drivers/platform/goldfish/goldfish_pipe.c drivers/sbus/c > har/ > > > orad > > > > > ax.c drivers/staging/lustre/lustre/llite/rw26.c > > > > > drivers/staging/lustre/lustre/llite/vvp_io.c > > > > > drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c > > > > > drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c > > > > > fs/9p/vfs_addr.c fs/afs/dir.c fs/afs/file.c --include > > > > > fs/afs/internal.h fs/afs/write.c fs/aio.c fs/block_dev.c > > > > > fs/btrfs/disk-io.c fs/btrfs/extent_io.c fs/btrfs/file.c > > > > > fs/btrfs/inode.c fs/btrfs/ioctl.c fs/btrfs/relocation.c fs/buffer.c > > > > > fs/ceph/addr.c fs/cifs/cifssmb.c fs/f2fs/gc.c > toto > > > > > > > > > > > > > > > > > > Exact same command line except the latter have 49 files before > > > > > > fs/f2fs/gc.c while the former has 48 (and it works then). Note > > > > > > that it seems that running the script multiple times even when > > > > > > they are too many files will eventualy work. > > > > >
Re: [Cocci] [bug] exists do not work if file group is too big (>49)
On Wed, 16 May 2018, Jerome Glisse wrote: > On Wed, May 16, 2018 at 10:02:08PM +0200, Julia Lawall wrote: > > > > > > On Wed, 16 May 2018, Jerome Glisse wrote: > > > > > On Wed, May 16, 2018 at 09:37:39PM +0200, Julia Lawall wrote: > > > > > > > > > > > > On Wed, 16 May 2018, Jerome Glisse wrote: > > > > > > > > > When operating on too many files the exists keyword no longer work. > > > > > For > > > > > instance: > > > > > > > > > > @exists@ > > > > > expression E1; > > > > > @@ > > > > > set_page_dirty( > > > > > +NULL, > > > > > E1) > > > > > > > > > > Using linux kernel as playground and looking at at fs/f2fs/gc.c where > > > > > there is set_page_dirty() calls in nested blocks. > > > > > > > > > > Works: > > > > > > > > > > spatch --no-includes --sp-file test.spatch > > > > > arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c > > > > > arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c > > > > > drivers/block/zram/zram_drv.c drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > > > > > drivers/gpu/drm/drm_gem.c drivers/gpu/drm/exynos/exynos_drm_g2d.c > > > > > drivers/gpu/drm/i915/i915_gem.c > > > > > drivers/gpu/drm/i915/i915_gem_fence_reg.c > > > > > drivers/gpu/drm/i915/i915_gem_userptr.c > > > > > drivers/gpu/drm/radeon/radeon_ttm.c drivers/gpu/drm/ttm/ttm_tt.c > > > > > drivers/infiniband/core/umem.c drivers/infiniband/core/umem_odp.c > > > > > drivers/infiniband/hw/hfi1/user_pages.c > > > > > drivers/infiniband/hw/qib/qib_user_pages.c > > > > > drivers/infiniband/hw/usnic/usnic_uiom.c > > > > > drivers/media/common/videobuf2/videobuf2-dma-contig.c > > > > > drivers/media/common/videobuf2/videobuf2-dma-sg.c > > > > > drivers/media/common/videobuf2/videobuf2-vmalloc.c > > > > > drivers/misc/genwqe/card_utils.c > > > > > drivers/misc/vmw_vmci/vmci_queue_pair.c > > > > > drivers/mtd/devices/block2mtd.c > > > > > drivers/platform/goldfish/goldfish_pipe.c drive! rs/sbus/char/ > > orad > > > > ax.c drivers/staging/lustre/lustre/llite/rw26.c > > > > drivers/staging/lustre/lustre/llite/vvp_io.c > > > > drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c > > > > drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c > > > > fs/9p/vfs_addr.c fs/afs/dir.c fs/afs/file.c --include fs/afs/internal.h > > > > fs/afs/write.c fs/aio.c fs/block_dev.c fs/btrfs/disk-io.c > > > > fs/btrfs/extent_io.c fs/btrfs/file.c fs/btrfs/inode.c fs/btrfs/ioctl.c > > > > fs/btrfs/relocation.c fs/buffer.c fs/ceph/addr.c fs/f2fs/gc.c > toto > > > > > > > > > > > > > > > Don't works: > > > > > > > > > > spatch --no-includes --sp-file test.spatch > > > > > arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c > > > > > arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c > > > > > drivers/block/zram/zram_drv.c drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > > > > > drivers/gpu/drm/drm_gem.c drivers/gpu/drm/exynos/exynos_drm_g2d.c > > > > > drivers/gpu/drm/i915/i915_gem.c > > > > > drivers/gpu/drm/i915/i915_gem_fence_reg.c > > > > > drivers/gpu/drm/i915/i915_gem_userptr.c > > > > > drivers/gpu/drm/radeon/radeon_ttm.c drivers/gpu/drm/ttm/ttm_tt.c > > > > > drivers/infiniband/core/umem.c drivers/infiniband/core/umem_odp.c > > > > > drivers/infiniband/hw/hfi1/user_pages.c > > > > > drivers/infiniband/hw/qib/qib_user_pages.c > > > > > drivers/infiniband/hw/usnic/usnic_uiom.c > > > > > drivers/media/common/videobuf2/videobuf2-dma-contig.c > > > > > drivers/media/common/videobuf2/videobuf2-dma-sg.c > > > > > drivers/media/common/videobuf2/videobuf2-vmalloc.c > > > > > drivers/misc/genwqe/card_utils.c > > > > > drivers/misc/vmw_vmci/vmci_queue_pair.c > > > > > drivers/mtd/devices/block2mtd.c > > > > > drivers/platform/goldfish/goldfish_pipe.c drive! rs/sbus/char/ > > orad > > > > ax.c drivers/staging/lustre/lustre/llite/rw26.c > > > > drivers/staging/lustre/lustre/llite/vvp_io.c > > > > drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c > > > > drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c > > > > fs/9p/vfs_addr.c fs/afs/dir.c fs/afs/file.c --include fs/afs/internal.h > > > > fs/afs/write.c fs/aio.c fs/block_dev.c fs/btrfs/disk-io.c > > > > fs/btrfs/extent_io.c fs/btrfs/file.c fs/btrfs/inode.c fs/btrfs/ioctl.c > > > > fs/btrfs/relocation.c fs/buffer.c fs/ceph/addr.c fs/cifs/cifssmb.c > > > > fs/f2fs/gc.c > toto > > > > > > > > > > > > > > > Exact same command line except the latter have 49 files before > > > > > fs/f2fs/gc.c while the former has 48 (and it works then). Note > > > > > that it seems that running the script multiple times even when > > > > > they are too many files will eventualy work. > > > > > > > > I'm surprised that the number of files would have an impact, unless by > > > > not > > > > working you mean that it runs out of memory or doesn't terminate. > > > > > > Don't work == only set_page_dirty in non nested block are modified, > > > set_page_dirty in nested block are not. So it runs and termina
Re: [Cocci] [bug] exists do not work if file group is too big (>49)
On Wed, May 16, 2018 at 10:02:08PM +0200, Julia Lawall wrote: > > > On Wed, 16 May 2018, Jerome Glisse wrote: > > > On Wed, May 16, 2018 at 09:37:39PM +0200, Julia Lawall wrote: > > > > > > > > > On Wed, 16 May 2018, Jerome Glisse wrote: > > > > > > > When operating on too many files the exists keyword no longer work. For > > > > instance: > > > > > > > > @exists@ > > > > expression E1; > > > > @@ > > > > set_page_dirty( > > > > +NULL, > > > > E1) > > > > > > > > Using linux kernel as playground and looking at at fs/f2fs/gc.c where > > > > there is set_page_dirty() calls in nested blocks. > > > > > > > > Works: > > > > > > > > spatch --no-includes --sp-file test.spatch > > > > arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c > > > > arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c > > > > drivers/block/zram/zram_drv.c drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > > > > drivers/gpu/drm/drm_gem.c drivers/gpu/drm/exynos/exynos_drm_g2d.c > > > > drivers/gpu/drm/i915/i915_gem.c > > > > drivers/gpu/drm/i915/i915_gem_fence_reg.c > > > > drivers/gpu/drm/i915/i915_gem_userptr.c > > > > drivers/gpu/drm/radeon/radeon_ttm.c drivers/gpu/drm/ttm/ttm_tt.c > > > > drivers/infiniband/core/umem.c drivers/infiniband/core/umem_odp.c > > > > drivers/infiniband/hw/hfi1/user_pages.c > > > > drivers/infiniband/hw/qib/qib_user_pages.c > > > > drivers/infiniband/hw/usnic/usnic_uiom.c > > > > drivers/media/common/videobuf2/videobuf2-dma-contig.c > > > > drivers/media/common/videobuf2/videobuf2-dma-sg.c > > > > drivers/media/common/videobuf2/videobuf2-vmalloc.c > > > > drivers/misc/genwqe/card_utils.c > > > > drivers/misc/vmw_vmci/vmci_queue_pair.c drivers/mtd/devices/block2mtd.c > > > > drivers/platform/goldfish/goldfish_pipe.c drivers/sbus/char/ > orad > > > ax.c drivers/staging/lustre/lustre/llite/rw26.c > > > drivers/staging/lustre/lustre/llite/vvp_io.c > > > drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c > > > drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c > > > fs/9p/vfs_addr.c fs/afs/dir.c fs/afs/file.c --include fs/afs/internal.h > > > fs/afs/write.c fs/aio.c fs/block_dev.c fs/btrfs/disk-io.c > > > fs/btrfs/extent_io.c fs/btrfs/file.c fs/btrfs/inode.c fs/btrfs/ioctl.c > > > fs/btrfs/relocation.c fs/buffer.c fs/ceph/addr.c fs/f2fs/gc.c > toto > > > > > > > > > > > > Don't works: > > > > > > > > spatch --no-includes --sp-file test.spatch > > > > arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c > > > > arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c > > > > drivers/block/zram/zram_drv.c drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > > > > drivers/gpu/drm/drm_gem.c drivers/gpu/drm/exynos/exynos_drm_g2d.c > > > > drivers/gpu/drm/i915/i915_gem.c > > > > drivers/gpu/drm/i915/i915_gem_fence_reg.c > > > > drivers/gpu/drm/i915/i915_gem_userptr.c > > > > drivers/gpu/drm/radeon/radeon_ttm.c drivers/gpu/drm/ttm/ttm_tt.c > > > > drivers/infiniband/core/umem.c drivers/infiniband/core/umem_odp.c > > > > drivers/infiniband/hw/hfi1/user_pages.c > > > > drivers/infiniband/hw/qib/qib_user_pages.c > > > > drivers/infiniband/hw/usnic/usnic_uiom.c > > > > drivers/media/common/videobuf2/videobuf2-dma-contig.c > > > > drivers/media/common/videobuf2/videobuf2-dma-sg.c > > > > drivers/media/common/videobuf2/videobuf2-vmalloc.c > > > > drivers/misc/genwqe/card_utils.c > > > > drivers/misc/vmw_vmci/vmci_queue_pair.c drivers/mtd/devices/block2mtd.c > > > > drivers/platform/goldfish/goldfish_pipe.c drivers/sbus/char/ > orad > > > ax.c drivers/staging/lustre/lustre/llite/rw26.c > > > drivers/staging/lustre/lustre/llite/vvp_io.c > > > drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c > > > drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c > > > fs/9p/vfs_addr.c fs/afs/dir.c fs/afs/file.c --include fs/afs/internal.h > > > fs/afs/write.c fs/aio.c fs/block_dev.c fs/btrfs/disk-io.c > > > fs/btrfs/extent_io.c fs/btrfs/file.c fs/btrfs/inode.c fs/btrfs/ioctl.c > > > fs/btrfs/relocation.c fs/buffer.c fs/ceph/addr.c fs/cifs/cifssmb.c > > > fs/f2fs/gc.c > toto > > > > > > > > > > > > Exact same command line except the latter have 49 files before > > > > fs/f2fs/gc.c while the former has 48 (and it works then). Note > > > > that it seems that running the script multiple times even when > > > > they are too many files will eventualy work. > > > > > > I'm surprised that the number of files would have an impact, unless by not > > > working you mean that it runs out of memory or doesn't terminate. > > > > Don't work == only set_page_dirty in non nested block are modified, > > set_page_dirty in nested block are not. So it runs and terminate and > > all files are processed (ie it is not a command line size limitation > > in the shell). > > > > > > > > In general, you should only give one file name or one directory name to > > > Coccinelle. If you give multiple files, then they are all treated at > > > once, ie their CF
Re: [Cocci] [bug] exists do not work if file group is too big (>49)
On Wed, 16 May 2018, Jerome Glisse wrote: > On Wed, May 16, 2018 at 09:37:39PM +0200, Julia Lawall wrote: > > > > > > On Wed, 16 May 2018, Jerome Glisse wrote: > > > > > When operating on too many files the exists keyword no longer work. For > > > instance: > > > > > > @exists@ > > > expression E1; > > > @@ > > > set_page_dirty( > > > +NULL, > > > E1) > > > > > > Using linux kernel as playground and looking at at fs/f2fs/gc.c where > > > there is set_page_dirty() calls in nested blocks. > > > > > > Works: > > > > > > spatch --no-includes --sp-file test.spatch > > > arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c > > > arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c > > > drivers/block/zram/zram_drv.c drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > > > drivers/gpu/drm/drm_gem.c drivers/gpu/drm/exynos/exynos_drm_g2d.c > > > drivers/gpu/drm/i915/i915_gem.c drivers/gpu/drm/i915/i915_gem_fence_reg.c > > > drivers/gpu/drm/i915/i915_gem_userptr.c > > > drivers/gpu/drm/radeon/radeon_ttm.c drivers/gpu/drm/ttm/ttm_tt.c > > > drivers/infiniband/core/umem.c drivers/infiniband/core/umem_odp.c > > > drivers/infiniband/hw/hfi1/user_pages.c > > > drivers/infiniband/hw/qib/qib_user_pages.c > > > drivers/infiniband/hw/usnic/usnic_uiom.c > > > drivers/media/common/videobuf2/videobuf2-dma-contig.c > > > drivers/media/common/videobuf2/videobuf2-dma-sg.c > > > drivers/media/common/videobuf2/videobuf2-vmalloc.c > > > drivers/misc/genwqe/card_utils.c drivers/misc/vmw_vmci/vmci_queue_pair.c > > > drivers/mtd/devices/block2mtd.c drivers/platform/goldfish/goldfish_pipe.c > > > drivers/s! bus/char/orad > > ax.c drivers/staging/lustre/lustre/llite/rw26.c > > drivers/staging/lustre/lustre/llite/vvp_io.c > > drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c > > drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c fs/9p/vfs_addr.c > > fs/afs/dir.c fs/afs/file.c --include fs/afs/internal.h fs/afs/write.c > > fs/aio.c fs/block_dev.c fs/btrfs/disk-io.c fs/btrfs/extent_io.c > > fs/btrfs/file.c fs/btrfs/inode.c fs/btrfs/ioctl.c fs/btrfs/relocation.c > > fs/buffer.c fs/ceph/addr.c fs/f2fs/gc.c > toto > > > > > > > > > Don't works: > > > > > > spatch --no-includes --sp-file test.spatch > > > arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c > > > arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c > > > drivers/block/zram/zram_drv.c drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > > > drivers/gpu/drm/drm_gem.c drivers/gpu/drm/exynos/exynos_drm_g2d.c > > > drivers/gpu/drm/i915/i915_gem.c drivers/gpu/drm/i915/i915_gem_fence_reg.c > > > drivers/gpu/drm/i915/i915_gem_userptr.c > > > drivers/gpu/drm/radeon/radeon_ttm.c drivers/gpu/drm/ttm/ttm_tt.c > > > drivers/infiniband/core/umem.c drivers/infiniband/core/umem_odp.c > > > drivers/infiniband/hw/hfi1/user_pages.c > > > drivers/infiniband/hw/qib/qib_user_pages.c > > > drivers/infiniband/hw/usnic/usnic_uiom.c > > > drivers/media/common/videobuf2/videobuf2-dma-contig.c > > > drivers/media/common/videobuf2/videobuf2-dma-sg.c > > > drivers/media/common/videobuf2/videobuf2-vmalloc.c > > > drivers/misc/genwqe/card_utils.c drivers/misc/vmw_vmci/vmci_queue_pair.c > > > drivers/mtd/devices/block2mtd.c drivers/platform/goldfish/goldfish_pipe.c > > > drivers/s! bus/char/orad > > ax.c drivers/staging/lustre/lustre/llite/rw26.c > > drivers/staging/lustre/lustre/llite/vvp_io.c > > drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c > > drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c fs/9p/vfs_addr.c > > fs/afs/dir.c fs/afs/file.c --include fs/afs/internal.h fs/afs/write.c > > fs/aio.c fs/block_dev.c fs/btrfs/disk-io.c fs/btrfs/extent_io.c > > fs/btrfs/file.c fs/btrfs/inode.c fs/btrfs/ioctl.c fs/btrfs/relocation.c > > fs/buffer.c fs/ceph/addr.c fs/cifs/cifssmb.c fs/f2fs/gc.c > toto > > > > > > > > > Exact same command line except the latter have 49 files before > > > fs/f2fs/gc.c while the former has 48 (and it works then). Note > > > that it seems that running the script multiple times even when > > > they are too many files will eventualy work. > > > > I'm surprised that the number of files would have an impact, unless by not > > working you mean that it runs out of memory or doesn't terminate. > > Don't work == only set_page_dirty in non nested block are modified, > set_page_dirty in nested block are not. So it runs and terminate and > all files are processed (ie it is not a command line size limitation > in the shell). > > > > > In general, you should only give one file name or one directory name to > > Coccinelle. If you give multiple files, then they are all treated at > > once, ie their CFGs are all in memory at the same time. If you give the > > name of a directory containing the files, then it will work on one file at > > a time, and you can give the option -j XXX to have it run XXX threads in > > parallel. Giving multiple file names on the command line is only for t
Re: [Cocci] [bug] exists do not work if file group is too big (>49)
On Wed, May 16, 2018 at 09:37:39PM +0200, Julia Lawall wrote: > > > On Wed, 16 May 2018, Jerome Glisse wrote: > > > When operating on too many files the exists keyword no longer work. For > > instance: > > > > @exists@ > > expression E1; > > @@ > > set_page_dirty( > > +NULL, > > E1) > > > > Using linux kernel as playground and looking at at fs/f2fs/gc.c where > > there is set_page_dirty() calls in nested blocks. > > > > Works: > > > > spatch --no-includes --sp-file test.spatch > > arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c > > arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c > > drivers/block/zram/zram_drv.c drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > > drivers/gpu/drm/drm_gem.c drivers/gpu/drm/exynos/exynos_drm_g2d.c > > drivers/gpu/drm/i915/i915_gem.c drivers/gpu/drm/i915/i915_gem_fence_reg.c > > drivers/gpu/drm/i915/i915_gem_userptr.c drivers/gpu/drm/radeon/radeon_ttm.c > > drivers/gpu/drm/ttm/ttm_tt.c drivers/infiniband/core/umem.c > > drivers/infiniband/core/umem_odp.c drivers/infiniband/hw/hfi1/user_pages.c > > drivers/infiniband/hw/qib/qib_user_pages.c > > drivers/infiniband/hw/usnic/usnic_uiom.c > > drivers/media/common/videobuf2/videobuf2-dma-contig.c > > drivers/media/common/videobuf2/videobuf2-dma-sg.c > > drivers/media/common/videobuf2/videobuf2-vmalloc.c > > drivers/misc/genwqe/card_utils.c drivers/misc/vmw_vmci/vmci_queue_pair.c > > drivers/mtd/devices/block2mtd.c drivers/platform/goldfish/goldfish_pipe.c > > drivers/sbus/char/orad > ax.c drivers/staging/lustre/lustre/llite/rw26.c > drivers/staging/lustre/lustre/llite/vvp_io.c > drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c > drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c fs/9p/vfs_addr.c > fs/afs/dir.c fs/afs/file.c --include fs/afs/internal.h fs/afs/write.c > fs/aio.c fs/block_dev.c fs/btrfs/disk-io.c fs/btrfs/extent_io.c > fs/btrfs/file.c fs/btrfs/inode.c fs/btrfs/ioctl.c fs/btrfs/relocation.c > fs/buffer.c fs/ceph/addr.c fs/f2fs/gc.c > toto > > > > > > Don't works: > > > > spatch --no-includes --sp-file test.spatch > > arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c > > arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c > > drivers/block/zram/zram_drv.c drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > > drivers/gpu/drm/drm_gem.c drivers/gpu/drm/exynos/exynos_drm_g2d.c > > drivers/gpu/drm/i915/i915_gem.c drivers/gpu/drm/i915/i915_gem_fence_reg.c > > drivers/gpu/drm/i915/i915_gem_userptr.c drivers/gpu/drm/radeon/radeon_ttm.c > > drivers/gpu/drm/ttm/ttm_tt.c drivers/infiniband/core/umem.c > > drivers/infiniband/core/umem_odp.c drivers/infiniband/hw/hfi1/user_pages.c > > drivers/infiniband/hw/qib/qib_user_pages.c > > drivers/infiniband/hw/usnic/usnic_uiom.c > > drivers/media/common/videobuf2/videobuf2-dma-contig.c > > drivers/media/common/videobuf2/videobuf2-dma-sg.c > > drivers/media/common/videobuf2/videobuf2-vmalloc.c > > drivers/misc/genwqe/card_utils.c drivers/misc/vmw_vmci/vmci_queue_pair.c > > drivers/mtd/devices/block2mtd.c drivers/platform/goldfish/goldfish_pipe.c > > drivers/sbus/char/orad > ax.c drivers/staging/lustre/lustre/llite/rw26.c > drivers/staging/lustre/lustre/llite/vvp_io.c > drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c > drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c fs/9p/vfs_addr.c > fs/afs/dir.c fs/afs/file.c --include fs/afs/internal.h fs/afs/write.c > fs/aio.c fs/block_dev.c fs/btrfs/disk-io.c fs/btrfs/extent_io.c > fs/btrfs/file.c fs/btrfs/inode.c fs/btrfs/ioctl.c fs/btrfs/relocation.c > fs/buffer.c fs/ceph/addr.c fs/cifs/cifssmb.c fs/f2fs/gc.c > toto > > > > > > Exact same command line except the latter have 49 files before > > fs/f2fs/gc.c while the former has 48 (and it works then). Note > > that it seems that running the script multiple times even when > > they are too many files will eventualy work. > > I'm surprised that the number of files would have an impact, unless by not > working you mean that it runs out of memory or doesn't terminate. Don't work == only set_page_dirty in non nested block are modified, set_page_dirty in nested block are not. So it runs and terminate and all files are processed (ie it is not a command line size limitation in the shell). > > In general, you should only give one file name or one directory name to > Coccinelle. If you give multiple files, then they are all treated at > once, ie their CFGs are all in memory at the same time. If you give the > name of a directory containing the files, then it will work on one file at > a time, and you can give the option -j XXX to have it run XXX threads in > parallel. Giving multiple file names on the command line is only for the > case where information collected from one file is needed for the treatment > of another file. Saddly for modification i am doing i need to treat many files as one as otherwise it does not work and even sadder for me is that this groups can get
Re: [Cocci] [bug] exists do not work if file group is too big (>49)
On Wed, 16 May 2018, Jerome Glisse wrote: > When operating on too many files the exists keyword no longer work. For > instance: > > @exists@ > expression E1; > @@ > set_page_dirty( > +NULL, > E1) > > Using linux kernel as playground and looking at at fs/f2fs/gc.c where > there is set_page_dirty() calls in nested blocks. > > Works: > > spatch --no-includes --sp-file test.spatch > arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c > arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c > drivers/block/zram/zram_drv.c drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > drivers/gpu/drm/drm_gem.c drivers/gpu/drm/exynos/exynos_drm_g2d.c > drivers/gpu/drm/i915/i915_gem.c drivers/gpu/drm/i915/i915_gem_fence_reg.c > drivers/gpu/drm/i915/i915_gem_userptr.c drivers/gpu/drm/radeon/radeon_ttm.c > drivers/gpu/drm/ttm/ttm_tt.c drivers/infiniband/core/umem.c > drivers/infiniband/core/umem_odp.c drivers/infiniband/hw/hfi1/user_pages.c > drivers/infiniband/hw/qib/qib_user_pages.c > drivers/infiniband/hw/usnic/usnic_uiom.c > drivers/media/common/videobuf2/videobuf2-dma-contig.c > drivers/media/common/videobuf2/videobuf2-dma-sg.c > drivers/media/common/videobuf2/videobuf2-vmalloc.c > drivers/misc/genwqe/card_utils.c drivers/misc/vmw_vmci/vmci_queue_pair.c > drivers/mtd/devices/block2mtd.c drivers/platform/goldfish/goldfish_pipe.c > drivers/sbus/! char/oradax.c drivers/staging/lustre/lustre/llite/rw26.c drivers/staging/lustre/lustre/llite/vvp_io.c drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c fs/9p/vfs_addr.c fs/afs/dir.c fs/afs/file.c --include fs/afs/internal.h fs/afs/write.c fs/aio.c fs/block_dev.c fs/btrfs/disk-io.c fs/btrfs/extent_io.c fs/btrfs/file.c fs/btrfs/inode.c fs/btrfs/ioctl.c fs/btrfs/relocation.c fs/buffer.c fs/ceph/addr.c fs/f2fs/gc.c > toto > > > Don't works: > > spatch --no-includes --sp-file test.spatch > arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c > arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c > drivers/block/zram/zram_drv.c drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c > drivers/gpu/drm/drm_gem.c drivers/gpu/drm/exynos/exynos_drm_g2d.c > drivers/gpu/drm/i915/i915_gem.c drivers/gpu/drm/i915/i915_gem_fence_reg.c > drivers/gpu/drm/i915/i915_gem_userptr.c drivers/gpu/drm/radeon/radeon_ttm.c > drivers/gpu/drm/ttm/ttm_tt.c drivers/infiniband/core/umem.c > drivers/infiniband/core/umem_odp.c drivers/infiniband/hw/hfi1/user_pages.c > drivers/infiniband/hw/qib/qib_user_pages.c > drivers/infiniband/hw/usnic/usnic_uiom.c > drivers/media/common/videobuf2/videobuf2-dma-contig.c > drivers/media/common/videobuf2/videobuf2-dma-sg.c > drivers/media/common/videobuf2/videobuf2-vmalloc.c > drivers/misc/genwqe/card_utils.c drivers/misc/vmw_vmci/vmci_queue_pair.c > drivers/mtd/devices/block2mtd.c drivers/platform/goldfish/goldfish_pipe.c > drivers/sbus/! char/oradax.c drivers/staging/lustre/lustre/llite/rw26.c drivers/staging/lustre/lustre/llite/vvp_io.c drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c fs/9p/vfs_addr.c fs/afs/dir.c fs/afs/file.c --include fs/afs/internal.h fs/afs/write.c fs/aio.c fs/block_dev.c fs/btrfs/disk-io.c fs/btrfs/extent_io.c fs/btrfs/file.c fs/btrfs/inode.c fs/btrfs/ioctl.c fs/btrfs/relocation.c fs/buffer.c fs/ceph/addr.c fs/cifs/cifssmb.c fs/f2fs/gc.c > toto > > > Exact same command line except the latter have 49 files before > fs/f2fs/gc.c while the former has 48 (and it works then). Note > that it seems that running the script multiple times even when > they are too many files will eventualy work. I'm surprised that the number of files would have an impact, unless by not working you mean that it runs out of memory or doesn't terminate. In general, you should only give one file name or one directory name to Coccinelle. If you give multiple files, then they are all treated at once, ie their CFGs are all in memory at the same time. If you give the name of a directory containing the files, then it will work on one file at a time, and you can give the option -j XXX to have it run XXX threads in parallel. Giving multiple file names on the command line is only for the case where information collected from one file is needed for the treatment of another file. If the set of files you want to work on doesn't correspond to the directory structure, you can make a file with the names of the files to treat one by one each separated by a blank line, and then give the command line argument --file-groups filename. Actually, exists is not relevant to your rule, because it has no ... Exists means that there exists a path through the ... that matches the pattern. Otherwise, all paths have to match the pattern. julia > > Cheers, > Jérôme > ___ > Cocci mailing list > Cocci@systeme.lip6.fr > https://systeme.lip6.fr/mailman/listinf