Re: [Cocci] [bug] exists do not work if file group is too big (>49)

2018-05-17 Thread Jerome Glisse
On Thu, May 17, 2018 at 10:31:52PM +0200, Julia Lawall wrote:
> Concretely, the problem is with LOCK_REQ, which is used as a typedef name
> in cifs and as a enum constant in f2fs.
> 
> Would there be any chance of getting rid of the LOCK_REQ typedef?
> Typedefs for structure types are discouraged by the Linux kernel coding
> style.
> 
> julia

I will try to make patches for that, thought not anytime soon i am going
on PTO so not before i get back.

Thank you for debugging this further.

Cheers,
Jérôme
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci


Re: [Cocci] [bug] exists do not work if file group is too big (>49)

2018-05-17 Thread Julia Lawall
Concretely, the problem is with LOCK_REQ, which is used as a typedef name
in cifs and as a enum constant in f2fs.

Would there be any chance of getting rid of the LOCK_REQ typedef?
Typedefs for structure types are discouraged by the Linux kernel coding
style.

julia
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci


Re: [Cocci] [bug] exists do not work if file group is too big (>49)

2018-05-17 Thread Julia Lawall


On Wed, 16 May 2018, Jerome Glisse wrote:

> When operating on too many files the exists keyword no longer work. For
> instance:

It has nothing to do with the number of files.  The problem is that some
property of one file causes a parse error in another file, due to the
various heuristics.  You can see this with the --verbose-parsing option.
I'm not yet sure where the bad interaction comes from, but I get the same
bad behavior with the command line:

spatch --verbose-parsing --no-includes --sp-file test.cocci fs/ceph/addr.c
fs/cifs/cifssmb.c fs/f2fs/gc.c

julia

>
> @exists@
> expression E1;
> @@
> set_page_dirty(
> +NULL,
> E1)
>
> Using linux kernel as playground and looking at at fs/f2fs/gc.c where
> there is set_page_dirty() calls in nested blocks.
>
> Works:
>
> spatch --no-includes --sp-file test.spatch 
> arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c 
> arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c 
> drivers/block/zram/zram_drv.c drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c 
> drivers/gpu/drm/drm_gem.c drivers/gpu/drm/exynos/exynos_drm_g2d.c 
> drivers/gpu/drm/i915/i915_gem.c drivers/gpu/drm/i915/i915_gem_fence_reg.c 
> drivers/gpu/drm/i915/i915_gem_userptr.c drivers/gpu/drm/radeon/radeon_ttm.c 
> drivers/gpu/drm/ttm/ttm_tt.c drivers/infiniband/core/umem.c 
> drivers/infiniband/core/umem_odp.c drivers/infiniband/hw/hfi1/user_pages.c 
> drivers/infiniband/hw/qib/qib_user_pages.c 
> drivers/infiniband/hw/usnic/usnic_uiom.c 
> drivers/media/common/videobuf2/videobuf2-dma-contig.c 
> drivers/media/common/videobuf2/videobuf2-dma-sg.c 
> drivers/media/common/videobuf2/videobuf2-vmalloc.c 
> drivers/misc/genwqe/card_utils.c drivers/misc/vmw_vmci/vmci_queue_pair.c 
> drivers/mtd/devices/block2mtd.c drivers/platform/goldfish/goldfish_pipe.c 
> drivers/sbus/!
 char/oradax.c drivers/staging/lustre/lustre/llite/rw26.c 
drivers/staging/lustre/lustre/llite/vvp_io.c 
drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c 
drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c fs/9p/vfs_addr.c 
fs/afs/dir.c fs/afs/file.c --include fs/afs/internal.h fs/afs/write.c fs/aio.c 
fs/block_dev.c fs/btrfs/disk-io.c fs/btrfs/extent_io.c fs/btrfs/file.c 
fs/btrfs/inode.c fs/btrfs/ioctl.c fs/btrfs/relocation.c fs/buffer.c 
fs/ceph/addr.c fs/f2fs/gc.c > toto
>
>
> Don't works:
>
> spatch --no-includes --sp-file test.spatch 
> arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c 
> arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c 
> drivers/block/zram/zram_drv.c drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c 
> drivers/gpu/drm/drm_gem.c drivers/gpu/drm/exynos/exynos_drm_g2d.c 
> drivers/gpu/drm/i915/i915_gem.c drivers/gpu/drm/i915/i915_gem_fence_reg.c 
> drivers/gpu/drm/i915/i915_gem_userptr.c drivers/gpu/drm/radeon/radeon_ttm.c 
> drivers/gpu/drm/ttm/ttm_tt.c drivers/infiniband/core/umem.c 
> drivers/infiniband/core/umem_odp.c drivers/infiniband/hw/hfi1/user_pages.c 
> drivers/infiniband/hw/qib/qib_user_pages.c 
> drivers/infiniband/hw/usnic/usnic_uiom.c 
> drivers/media/common/videobuf2/videobuf2-dma-contig.c 
> drivers/media/common/videobuf2/videobuf2-dma-sg.c 
> drivers/media/common/videobuf2/videobuf2-vmalloc.c 
> drivers/misc/genwqe/card_utils.c drivers/misc/vmw_vmci/vmci_queue_pair.c 
> drivers/mtd/devices/block2mtd.c drivers/platform/goldfish/goldfish_pipe.c 
> drivers/sbus/!
 char/oradax.c drivers/staging/lustre/lustre/llite/rw26.c 
drivers/staging/lustre/lustre/llite/vvp_io.c 
drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c 
drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c fs/9p/vfs_addr.c 
fs/afs/dir.c fs/afs/file.c --include fs/afs/internal.h fs/afs/write.c fs/aio.c 
fs/block_dev.c fs/btrfs/disk-io.c fs/btrfs/extent_io.c fs/btrfs/file.c 
fs/btrfs/inode.c fs/btrfs/ioctl.c fs/btrfs/relocation.c fs/buffer.c 
fs/ceph/addr.c fs/cifs/cifssmb.c fs/f2fs/gc.c > toto
>
>
> Exact same command line except the latter have 49 files before
> fs/f2fs/gc.c while the former has 48 (and it works then). Note
> that it seems that running the script multiple times even when
> they are too many files will eventualy work.
>
> Cheers,
> Jérôme
> ___
> Cocci mailing list
> Cocci@systeme.lip6.fr
> https://systeme.lip6.fr/mailman/listinfo/cocci
>___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci


Re: [Cocci] [bug] exists do not work if file group is too big (>49)

2018-05-16 Thread Jerome Glisse
On Wed, May 16, 2018 at 10:29:30PM +0200, Julia Lawall wrote:
> > [glisse@localhost ~]$ spatch --version
> > spatch version 1.0.6 compiled with OCaml version 4.05.0
> > Flags passed to the configure script: --build=x86_64-redhat-linux-gnu 
> > --host=x86_64-redhat-linux-gnu --program-prefix= 
> > --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr 
> > --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc 
> > --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 
> > --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib 
> > --mandir=/usr/share/man --infodir=/usr/share/info --enable-release=yes 
> > --with-python=/usr/bin/python3 --with-menhir=/usr/bin/menhir
> > Python scripting support: yes
> > Syntax of regular expresssions: PCRE
> 
> I don't think this indicates whether it is bytecode or native code, and I
> don't know what the fedora people offer.
> 
> My bytecode executable is 41M and my native code executable is 13M.

I am guessing native code:
-rwxr-xr-x. 1 root root 1.3K May 10 13:51 /usr/bin/spatch

Jérôme
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci


Re: [Cocci] [bug] exists do not work if file group is too big (>49)

2018-05-16 Thread Julia Lawall
> [glisse@localhost ~]$ spatch --version
> spatch version 1.0.6 compiled with OCaml version 4.05.0
> Flags passed to the configure script: --build=x86_64-redhat-linux-gnu 
> --host=x86_64-redhat-linux-gnu --program-prefix= 
> --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr 
> --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share 
> --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec 
> --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man 
> --infodir=/usr/share/info --enable-release=yes --with-python=/usr/bin/python3 
> --with-menhir=/usr/bin/menhir
> Python scripting support: yes
> Syntax of regular expresssions: PCRE

I don't think this indicates whether it is bytecode or native code, and I
don't know what the fedora people offer.

My bytecode executable is 41M and my native code executable is 13M.

julia
___
Cocci mailing list
Cocci@systeme.lip6.fr
https://systeme.lip6.fr/mailman/listinfo/cocci


Re: [Cocci] [bug] exists do not work if file group is too big (>49)

2018-05-16 Thread Jerome Glisse
On Wed, May 16, 2018 at 10:20:05PM +0200, Julia Lawall wrote:
> 
> 
> On Wed, 16 May 2018, Jerome Glisse wrote:
> 
> > On Wed, May 16, 2018 at 10:02:08PM +0200, Julia Lawall wrote:
> > >
> > >
> > > On Wed, 16 May 2018, Jerome Glisse wrote:
> > >
> > > > On Wed, May 16, 2018 at 09:37:39PM +0200, Julia Lawall wrote:
> > > > >
> > > > >
> > > > > On Wed, 16 May 2018, Jerome Glisse wrote:
> > > > >
> > > > > > When operating on too many files the exists keyword no longer work. 
> > > > > > For
> > > > > > instance:
> > > > > >
> > > > > > @exists@
> > > > > > expression E1;
> > > > > > @@
> > > > > > set_page_dirty(
> > > > > > +NULL,
> > > > > > E1)
> > > > > >
> > > > > > Using linux kernel as playground and looking at at fs/f2fs/gc.c 
> > > > > > where
> > > > > > there is set_page_dirty() calls in nested blocks.
> > > > > >
> > > > > > Works:
> > > > > >
> > > > > > spatch --no-includes --sp-file test.spatch 
> > > > > > arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c 
> > > > > > arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c 
> > > > > > drivers/block/zram/zram_drv.c 
> > > > > > drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c drivers/gpu/drm/drm_gem.c 
> > > > > > drivers/gpu/drm/exynos/exynos_drm_g2d.c 
> > > > > > drivers/gpu/drm/i915/i915_gem.c 
> > > > > > drivers/gpu/drm/i915/i915_gem_fence_reg.c 
> > > > > > drivers/gpu/drm/i915/i915_gem_userptr.c 
> > > > > > drivers/gpu/drm/radeon/radeon_ttm.c drivers/gpu/drm/ttm/ttm_tt.c 
> > > > > > drivers/infiniband/core/umem.c drivers/infiniband/core/umem_odp.c 
> > > > > > drivers/infiniband/hw/hfi1/user_pages.c 
> > > > > > drivers/infiniband/hw/qib/qib_user_pages.c 
> > > > > > drivers/infiniband/hw/usnic/usnic_uiom.c 
> > > > > > drivers/media/common/videobuf2/videobuf2-dma-contig.c 
> > > > > > drivers/media/common/videobuf2/videobuf2-dma-sg.c 
> > > > > > drivers/media/common/videobuf2/videobuf2-vmalloc.c 
> > > > > > drivers/misc/genwqe/card_utils.c 
> > > > > > drivers/misc/vmw_vmci/vmci_queue_pair.c 
> > > > > > drivers/mtd/devices/block2mtd.c 
> > > > > > drivers/platform/goldfish/goldfish_pipe.c drivers/sbus/c
>  har/
> > >  orad
> > > > >  ax.c drivers/staging/lustre/lustre/llite/rw26.c 
> > > > > drivers/staging/lustre/lustre/llite/vvp_io.c 
> > > > > drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c 
> > > > > drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c 
> > > > > fs/9p/vfs_addr.c fs/afs/dir.c fs/afs/file.c --include 
> > > > > fs/afs/internal.h fs/afs/write.c fs/aio.c fs/block_dev.c 
> > > > > fs/btrfs/disk-io.c fs/btrfs/extent_io.c fs/btrfs/file.c 
> > > > > fs/btrfs/inode.c fs/btrfs/ioctl.c fs/btrfs/relocation.c fs/buffer.c 
> > > > > fs/ceph/addr.c fs/f2fs/gc.c > toto
> > > > > >
> > > > > >
> > > > > > Don't works:
> > > > > >
> > > > > > spatch --no-includes --sp-file test.spatch 
> > > > > > arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c 
> > > > > > arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c 
> > > > > > drivers/block/zram/zram_drv.c 
> > > > > > drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c drivers/gpu/drm/drm_gem.c 
> > > > > > drivers/gpu/drm/exynos/exynos_drm_g2d.c 
> > > > > > drivers/gpu/drm/i915/i915_gem.c 
> > > > > > drivers/gpu/drm/i915/i915_gem_fence_reg.c 
> > > > > > drivers/gpu/drm/i915/i915_gem_userptr.c 
> > > > > > drivers/gpu/drm/radeon/radeon_ttm.c drivers/gpu/drm/ttm/ttm_tt.c 
> > > > > > drivers/infiniband/core/umem.c drivers/infiniband/core/umem_odp.c 
> > > > > > drivers/infiniband/hw/hfi1/user_pages.c 
> > > > > > drivers/infiniband/hw/qib/qib_user_pages.c 
> > > > > > drivers/infiniband/hw/usnic/usnic_uiom.c 
> > > > > > drivers/media/common/videobuf2/videobuf2-dma-contig.c 
> > > > > > drivers/media/common/videobuf2/videobuf2-dma-sg.c 
> > > > > > drivers/media/common/videobuf2/videobuf2-vmalloc.c 
> > > > > > drivers/misc/genwqe/card_utils.c 
> > > > > > drivers/misc/vmw_vmci/vmci_queue_pair.c 
> > > > > > drivers/mtd/devices/block2mtd.c 
> > > > > > drivers/platform/goldfish/goldfish_pipe.c drivers/sbus/c
>  har/
> > >  orad
> > > > >  ax.c drivers/staging/lustre/lustre/llite/rw26.c 
> > > > > drivers/staging/lustre/lustre/llite/vvp_io.c 
> > > > > drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c 
> > > > > drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c 
> > > > > fs/9p/vfs_addr.c fs/afs/dir.c fs/afs/file.c --include 
> > > > > fs/afs/internal.h fs/afs/write.c fs/aio.c fs/block_dev.c 
> > > > > fs/btrfs/disk-io.c fs/btrfs/extent_io.c fs/btrfs/file.c 
> > > > > fs/btrfs/inode.c fs/btrfs/ioctl.c fs/btrfs/relocation.c fs/buffer.c 
> > > > > fs/ceph/addr.c fs/cifs/cifssmb.c fs/f2fs/gc.c > toto
> > > > > >
> > > > > >
> > > > > > Exact same command line except the latter have 49 files before
> > > > > > fs/f2fs/gc.c while the former has 48 (and it works then). Note
> > > > > > that it seems that running the script multiple times even when
> > > > > > they are too many files will eventualy work.
> > > > >

Re: [Cocci] [bug] exists do not work if file group is too big (>49)

2018-05-16 Thread Julia Lawall


On Wed, 16 May 2018, Jerome Glisse wrote:

> On Wed, May 16, 2018 at 10:02:08PM +0200, Julia Lawall wrote:
> >
> >
> > On Wed, 16 May 2018, Jerome Glisse wrote:
> >
> > > On Wed, May 16, 2018 at 09:37:39PM +0200, Julia Lawall wrote:
> > > >
> > > >
> > > > On Wed, 16 May 2018, Jerome Glisse wrote:
> > > >
> > > > > When operating on too many files the exists keyword no longer work. 
> > > > > For
> > > > > instance:
> > > > >
> > > > > @exists@
> > > > > expression E1;
> > > > > @@
> > > > > set_page_dirty(
> > > > > +NULL,
> > > > > E1)
> > > > >
> > > > > Using linux kernel as playground and looking at at fs/f2fs/gc.c where
> > > > > there is set_page_dirty() calls in nested blocks.
> > > > >
> > > > > Works:
> > > > >
> > > > > spatch --no-includes --sp-file test.spatch 
> > > > > arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c 
> > > > > arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c 
> > > > > drivers/block/zram/zram_drv.c drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c 
> > > > > drivers/gpu/drm/drm_gem.c drivers/gpu/drm/exynos/exynos_drm_g2d.c 
> > > > > drivers/gpu/drm/i915/i915_gem.c 
> > > > > drivers/gpu/drm/i915/i915_gem_fence_reg.c 
> > > > > drivers/gpu/drm/i915/i915_gem_userptr.c 
> > > > > drivers/gpu/drm/radeon/radeon_ttm.c drivers/gpu/drm/ttm/ttm_tt.c 
> > > > > drivers/infiniband/core/umem.c drivers/infiniband/core/umem_odp.c 
> > > > > drivers/infiniband/hw/hfi1/user_pages.c 
> > > > > drivers/infiniband/hw/qib/qib_user_pages.c 
> > > > > drivers/infiniband/hw/usnic/usnic_uiom.c 
> > > > > drivers/media/common/videobuf2/videobuf2-dma-contig.c 
> > > > > drivers/media/common/videobuf2/videobuf2-dma-sg.c 
> > > > > drivers/media/common/videobuf2/videobuf2-vmalloc.c 
> > > > > drivers/misc/genwqe/card_utils.c 
> > > > > drivers/misc/vmw_vmci/vmci_queue_pair.c 
> > > > > drivers/mtd/devices/block2mtd.c 
> > > > > drivers/platform/goldfish/goldfish_pipe.c drive!
 rs/sbus/char/
> >  orad
> > > >  ax.c drivers/staging/lustre/lustre/llite/rw26.c 
> > > > drivers/staging/lustre/lustre/llite/vvp_io.c 
> > > > drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c 
> > > > drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c 
> > > > fs/9p/vfs_addr.c fs/afs/dir.c fs/afs/file.c --include fs/afs/internal.h 
> > > > fs/afs/write.c fs/aio.c fs/block_dev.c fs/btrfs/disk-io.c 
> > > > fs/btrfs/extent_io.c fs/btrfs/file.c fs/btrfs/inode.c fs/btrfs/ioctl.c 
> > > > fs/btrfs/relocation.c fs/buffer.c fs/ceph/addr.c fs/f2fs/gc.c > toto
> > > > >
> > > > >
> > > > > Don't works:
> > > > >
> > > > > spatch --no-includes --sp-file test.spatch 
> > > > > arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c 
> > > > > arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c 
> > > > > drivers/block/zram/zram_drv.c drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c 
> > > > > drivers/gpu/drm/drm_gem.c drivers/gpu/drm/exynos/exynos_drm_g2d.c 
> > > > > drivers/gpu/drm/i915/i915_gem.c 
> > > > > drivers/gpu/drm/i915/i915_gem_fence_reg.c 
> > > > > drivers/gpu/drm/i915/i915_gem_userptr.c 
> > > > > drivers/gpu/drm/radeon/radeon_ttm.c drivers/gpu/drm/ttm/ttm_tt.c 
> > > > > drivers/infiniband/core/umem.c drivers/infiniband/core/umem_odp.c 
> > > > > drivers/infiniband/hw/hfi1/user_pages.c 
> > > > > drivers/infiniband/hw/qib/qib_user_pages.c 
> > > > > drivers/infiniband/hw/usnic/usnic_uiom.c 
> > > > > drivers/media/common/videobuf2/videobuf2-dma-contig.c 
> > > > > drivers/media/common/videobuf2/videobuf2-dma-sg.c 
> > > > > drivers/media/common/videobuf2/videobuf2-vmalloc.c 
> > > > > drivers/misc/genwqe/card_utils.c 
> > > > > drivers/misc/vmw_vmci/vmci_queue_pair.c 
> > > > > drivers/mtd/devices/block2mtd.c 
> > > > > drivers/platform/goldfish/goldfish_pipe.c drive!
 rs/sbus/char/
> >  orad
> > > >  ax.c drivers/staging/lustre/lustre/llite/rw26.c 
> > > > drivers/staging/lustre/lustre/llite/vvp_io.c 
> > > > drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c 
> > > > drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c 
> > > > fs/9p/vfs_addr.c fs/afs/dir.c fs/afs/file.c --include fs/afs/internal.h 
> > > > fs/afs/write.c fs/aio.c fs/block_dev.c fs/btrfs/disk-io.c 
> > > > fs/btrfs/extent_io.c fs/btrfs/file.c fs/btrfs/inode.c fs/btrfs/ioctl.c 
> > > > fs/btrfs/relocation.c fs/buffer.c fs/ceph/addr.c fs/cifs/cifssmb.c 
> > > > fs/f2fs/gc.c > toto
> > > > >
> > > > >
> > > > > Exact same command line except the latter have 49 files before
> > > > > fs/f2fs/gc.c while the former has 48 (and it works then). Note
> > > > > that it seems that running the script multiple times even when
> > > > > they are too many files will eventualy work.
> > > >
> > > > I'm surprised that the number of files would have an impact, unless by 
> > > > not
> > > > working you mean that it runs out of memory or doesn't terminate.
> > >
> > > Don't work == only set_page_dirty in non nested block are modified,
> > > set_page_dirty in nested block are not. So it runs and 

Re: [Cocci] [bug] exists do not work if file group is too big (>49)

2018-05-16 Thread Jerome Glisse
On Wed, May 16, 2018 at 10:02:08PM +0200, Julia Lawall wrote:
> 
> 
> On Wed, 16 May 2018, Jerome Glisse wrote:
> 
> > On Wed, May 16, 2018 at 09:37:39PM +0200, Julia Lawall wrote:
> > >
> > >
> > > On Wed, 16 May 2018, Jerome Glisse wrote:
> > >
> > > > When operating on too many files the exists keyword no longer work. For
> > > > instance:
> > > >
> > > > @exists@
> > > > expression E1;
> > > > @@
> > > > set_page_dirty(
> > > > +NULL,
> > > > E1)
> > > >
> > > > Using linux kernel as playground and looking at at fs/f2fs/gc.c where
> > > > there is set_page_dirty() calls in nested blocks.
> > > >
> > > > Works:
> > > >
> > > > spatch --no-includes --sp-file test.spatch 
> > > > arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c 
> > > > arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c 
> > > > drivers/block/zram/zram_drv.c drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c 
> > > > drivers/gpu/drm/drm_gem.c drivers/gpu/drm/exynos/exynos_drm_g2d.c 
> > > > drivers/gpu/drm/i915/i915_gem.c 
> > > > drivers/gpu/drm/i915/i915_gem_fence_reg.c 
> > > > drivers/gpu/drm/i915/i915_gem_userptr.c 
> > > > drivers/gpu/drm/radeon/radeon_ttm.c drivers/gpu/drm/ttm/ttm_tt.c 
> > > > drivers/infiniband/core/umem.c drivers/infiniband/core/umem_odp.c 
> > > > drivers/infiniband/hw/hfi1/user_pages.c 
> > > > drivers/infiniband/hw/qib/qib_user_pages.c 
> > > > drivers/infiniband/hw/usnic/usnic_uiom.c 
> > > > drivers/media/common/videobuf2/videobuf2-dma-contig.c 
> > > > drivers/media/common/videobuf2/videobuf2-dma-sg.c 
> > > > drivers/media/common/videobuf2/videobuf2-vmalloc.c 
> > > > drivers/misc/genwqe/card_utils.c 
> > > > drivers/misc/vmw_vmci/vmci_queue_pair.c drivers/mtd/devices/block2mtd.c 
> > > > drivers/platform/goldfish/goldfish_pipe.c drivers/sbus/char/
>  orad
> > >  ax.c drivers/staging/lustre/lustre/llite/rw26.c 
> > > drivers/staging/lustre/lustre/llite/vvp_io.c 
> > > drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c 
> > > drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c 
> > > fs/9p/vfs_addr.c fs/afs/dir.c fs/afs/file.c --include fs/afs/internal.h 
> > > fs/afs/write.c fs/aio.c fs/block_dev.c fs/btrfs/disk-io.c 
> > > fs/btrfs/extent_io.c fs/btrfs/file.c fs/btrfs/inode.c fs/btrfs/ioctl.c 
> > > fs/btrfs/relocation.c fs/buffer.c fs/ceph/addr.c fs/f2fs/gc.c > toto
> > > >
> > > >
> > > > Don't works:
> > > >
> > > > spatch --no-includes --sp-file test.spatch 
> > > > arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c 
> > > > arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c 
> > > > drivers/block/zram/zram_drv.c drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c 
> > > > drivers/gpu/drm/drm_gem.c drivers/gpu/drm/exynos/exynos_drm_g2d.c 
> > > > drivers/gpu/drm/i915/i915_gem.c 
> > > > drivers/gpu/drm/i915/i915_gem_fence_reg.c 
> > > > drivers/gpu/drm/i915/i915_gem_userptr.c 
> > > > drivers/gpu/drm/radeon/radeon_ttm.c drivers/gpu/drm/ttm/ttm_tt.c 
> > > > drivers/infiniband/core/umem.c drivers/infiniband/core/umem_odp.c 
> > > > drivers/infiniband/hw/hfi1/user_pages.c 
> > > > drivers/infiniband/hw/qib/qib_user_pages.c 
> > > > drivers/infiniband/hw/usnic/usnic_uiom.c 
> > > > drivers/media/common/videobuf2/videobuf2-dma-contig.c 
> > > > drivers/media/common/videobuf2/videobuf2-dma-sg.c 
> > > > drivers/media/common/videobuf2/videobuf2-vmalloc.c 
> > > > drivers/misc/genwqe/card_utils.c 
> > > > drivers/misc/vmw_vmci/vmci_queue_pair.c drivers/mtd/devices/block2mtd.c 
> > > > drivers/platform/goldfish/goldfish_pipe.c drivers/sbus/char/
>  orad
> > >  ax.c drivers/staging/lustre/lustre/llite/rw26.c 
> > > drivers/staging/lustre/lustre/llite/vvp_io.c 
> > > drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c 
> > > drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c 
> > > fs/9p/vfs_addr.c fs/afs/dir.c fs/afs/file.c --include fs/afs/internal.h 
> > > fs/afs/write.c fs/aio.c fs/block_dev.c fs/btrfs/disk-io.c 
> > > fs/btrfs/extent_io.c fs/btrfs/file.c fs/btrfs/inode.c fs/btrfs/ioctl.c 
> > > fs/btrfs/relocation.c fs/buffer.c fs/ceph/addr.c fs/cifs/cifssmb.c 
> > > fs/f2fs/gc.c > toto
> > > >
> > > >
> > > > Exact same command line except the latter have 49 files before
> > > > fs/f2fs/gc.c while the former has 48 (and it works then). Note
> > > > that it seems that running the script multiple times even when
> > > > they are too many files will eventualy work.
> > >
> > > I'm surprised that the number of files would have an impact, unless by not
> > > working you mean that it runs out of memory or doesn't terminate.
> >
> > Don't work == only set_page_dirty in non nested block are modified,
> > set_page_dirty in nested block are not. So it runs and terminate and
> > all files are processed (ie it is not a command line size limitation
> > in the shell).
> >
> > >
> > > In general, you should only give one file name or one directory name to
> > > Coccinelle.  If you give multiple files, then they are all treated at
> > > once, ie their 

Re: [Cocci] [bug] exists do not work if file group is too big (>49)

2018-05-16 Thread Julia Lawall


On Wed, 16 May 2018, Jerome Glisse wrote:

> When operating on too many files the exists keyword no longer work. For
> instance:
>
> @exists@
> expression E1;
> @@
> set_page_dirty(
> +NULL,
> E1)
>
> Using linux kernel as playground and looking at at fs/f2fs/gc.c where
> there is set_page_dirty() calls in nested blocks.
>
> Works:
>
> spatch --no-includes --sp-file test.spatch 
> arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c 
> arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c 
> drivers/block/zram/zram_drv.c drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c 
> drivers/gpu/drm/drm_gem.c drivers/gpu/drm/exynos/exynos_drm_g2d.c 
> drivers/gpu/drm/i915/i915_gem.c drivers/gpu/drm/i915/i915_gem_fence_reg.c 
> drivers/gpu/drm/i915/i915_gem_userptr.c drivers/gpu/drm/radeon/radeon_ttm.c 
> drivers/gpu/drm/ttm/ttm_tt.c drivers/infiniband/core/umem.c 
> drivers/infiniband/core/umem_odp.c drivers/infiniband/hw/hfi1/user_pages.c 
> drivers/infiniband/hw/qib/qib_user_pages.c 
> drivers/infiniband/hw/usnic/usnic_uiom.c 
> drivers/media/common/videobuf2/videobuf2-dma-contig.c 
> drivers/media/common/videobuf2/videobuf2-dma-sg.c 
> drivers/media/common/videobuf2/videobuf2-vmalloc.c 
> drivers/misc/genwqe/card_utils.c drivers/misc/vmw_vmci/vmci_queue_pair.c 
> drivers/mtd/devices/block2mtd.c drivers/platform/goldfish/goldfish_pipe.c 
> drivers/sbus/!
 char/oradax.c drivers/staging/lustre/lustre/llite/rw26.c 
drivers/staging/lustre/lustre/llite/vvp_io.c 
drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c 
drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c fs/9p/vfs_addr.c 
fs/afs/dir.c fs/afs/file.c --include fs/afs/internal.h fs/afs/write.c fs/aio.c 
fs/block_dev.c fs/btrfs/disk-io.c fs/btrfs/extent_io.c fs/btrfs/file.c 
fs/btrfs/inode.c fs/btrfs/ioctl.c fs/btrfs/relocation.c fs/buffer.c 
fs/ceph/addr.c fs/f2fs/gc.c > toto
>
>
> Don't works:
>
> spatch --no-includes --sp-file test.spatch 
> arch/powerpc/kvm/book3s_64_mmu_radix.c arch/powerpc/kvm/e500_mmu.c 
> arch/s390/kvm/interrupt.c arch/x86/kvm/svm.c block/bio.c 
> drivers/block/zram/zram_drv.c drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c 
> drivers/gpu/drm/drm_gem.c drivers/gpu/drm/exynos/exynos_drm_g2d.c 
> drivers/gpu/drm/i915/i915_gem.c drivers/gpu/drm/i915/i915_gem_fence_reg.c 
> drivers/gpu/drm/i915/i915_gem_userptr.c drivers/gpu/drm/radeon/radeon_ttm.c 
> drivers/gpu/drm/ttm/ttm_tt.c drivers/infiniband/core/umem.c 
> drivers/infiniband/core/umem_odp.c drivers/infiniband/hw/hfi1/user_pages.c 
> drivers/infiniband/hw/qib/qib_user_pages.c 
> drivers/infiniband/hw/usnic/usnic_uiom.c 
> drivers/media/common/videobuf2/videobuf2-dma-contig.c 
> drivers/media/common/videobuf2/videobuf2-dma-sg.c 
> drivers/media/common/videobuf2/videobuf2-vmalloc.c 
> drivers/misc/genwqe/card_utils.c drivers/misc/vmw_vmci/vmci_queue_pair.c 
> drivers/mtd/devices/block2mtd.c drivers/platform/goldfish/goldfish_pipe.c 
> drivers/sbus/!
 char/oradax.c drivers/staging/lustre/lustre/llite/rw26.c 
drivers/staging/lustre/lustre/llite/vvp_io.c 
drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c 
drivers/vhost/vhost.c drivers/video/fbdev/core/fb_defio.c fs/9p/vfs_addr.c 
fs/afs/dir.c fs/afs/file.c --include fs/afs/internal.h fs/afs/write.c fs/aio.c 
fs/block_dev.c fs/btrfs/disk-io.c fs/btrfs/extent_io.c fs/btrfs/file.c 
fs/btrfs/inode.c fs/btrfs/ioctl.c fs/btrfs/relocation.c fs/buffer.c 
fs/ceph/addr.c fs/cifs/cifssmb.c fs/f2fs/gc.c > toto
>
>
> Exact same command line except the latter have 49 files before
> fs/f2fs/gc.c while the former has 48 (and it works then). Note
> that it seems that running the script multiple times even when
> they are too many files will eventualy work.

I'm surprised that the number of files would have an impact, unless by not
working you mean that it runs out of memory or doesn't terminate.

In general, you should only give one file name or one directory name to
Coccinelle.  If you give multiple files, then they are all treated at
once, ie their CFGs are all in memory at the same time.  If you give the
name of a directory containing the files, then it will work on one file at
a time, and you can give the option -j XXX to have it run XXX threads in
parallel.  Giving multiple file names on the command line is only for the
case where information collected from one file is needed for the treatment
of another file.

If the set of files you want to work on doesn't correspond to the
directory structure, you can make a file with the names of the files to
treat one by one each separated by a blank line, and then give the command
line argument --file-groups filename.

Actually, exists is not relevant to your rule, because it has no ...
Exists means that there exists a path through the ... that matches the
pattern.  Otherwise, all paths have to match the pattern.

julia

>
> Cheers,
> Jérôme
> ___
> Cocci mailing list
> Cocci@systeme.lip6.fr
>