Bug#1041588: grep -r fails with "Operation not supported"

[Santiago Ruano Rincón]

Control: tags -1 + pending

El 21/07/23 a las 19:46, Vincent Lefevre escribió:
> Control: tags -1 patch
> 
> On 2023-07-21 19:38:08 +0200, Vincent Lefevre wrote:
> > Indeed, it's a bundled gnulib. I was surprised because there is
> > no "gnulib" directory. Actually these are just files under the m4
> > directory. The bug is in "m4/dirfd.m4". I can see
> > 
> >   if test $ac_cv_func_dirfd = no && test $gl_cv_func_dirfd_macro = no; then
> > HAVE_DIRFD=0
> >   else
> > HAVE_DIRFD=1
> > dnl Replace only if the system declares dirfd already.
> > if test $ac_cv_have_decl_dirfd = yes; then
> >   REPLACE_DIRFD=1
> > fi
> > 
> > where the last 4 lines
> > 
> > dnl Replace only if the system declares dirfd already.
> > if test $ac_cv_have_decl_dirfd = yes; then
> >   REPLACE_DIRFD=1
> > fi
> > 
> > need to be removed to match the upstream gnulib fix.
> 
> And I could check that the corresponding attached patch makes
> the error disappear.

Thank you! Patch and autopkgetst inclucded. As long as I can confirm the
autopkgtest works, I upload the fixed version.

Cheers,

 -- S


signature.asc
Description: PGP signature

Bug#1041588: grep -r fails with "Operation not supported"

[Vincent Lefevre]

Control: tags -1 patch

On 2023-07-21 19:38:08 +0200, Vincent Lefevre wrote:
> Indeed, it's a bundled gnulib. I was surprised because there is
> no "gnulib" directory. Actually these are just files under the m4
> directory. The bug is in "m4/dirfd.m4". I can see
> 
>   if test $ac_cv_func_dirfd = no && test $gl_cv_func_dirfd_macro = no; then
> HAVE_DIRFD=0
>   else
> HAVE_DIRFD=1
> dnl Replace only if the system declares dirfd already.
> if test $ac_cv_have_decl_dirfd = yes; then
>   REPLACE_DIRFD=1
> fi
> 
> where the last 4 lines
> 
> dnl Replace only if the system declares dirfd already.
> if test $ac_cv_have_decl_dirfd = yes; then
>   REPLACE_DIRFD=1
> fi
> 
> need to be removed to match the upstream gnulib fix.

And I could check that the corresponding attached patch makes
the error disappear.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)
diff -Naurd grep-3.11-orig/m4/dirfd.m4 grep-3.11/m4/dirfd.m4
--- grep-3.11~/m4/dirfd.m4	2023-04-29 11:03:00.0 +0200
+++ grep-3.11/m4/dirfd.m4	2023-07-21 19:40:41.345044696 +0200
@@ -40,10 +40,6 @@
 HAVE_DIRFD=0
   else
 HAVE_DIRFD=1
-dnl Replace only if the system declares dirfd already.
-if test $ac_cv_have_decl_dirfd = yes; then
-  REPLACE_DIRFD=1
-fi
 dnl Replace dirfd() on native Windows, to support fdopendir().
 AC_REQUIRE([gl_DIRENT_DIR])
 if test $DIR_HAS_FD_MEMBER = 0; then

Bug#1041588: grep -r fails with "Operation not supported"

[Vincent Lefevre]

On 2023-07-21 13:08:11 -0400, Boyuan Yang wrote:
> BTW: Is grep 3.11-1 using Debian-packaged gnulib anywere? I did not
> see a build-dependency in
> https://sources.debian.org/src/grep/3.11-1/debian/control/ .
> 
> * If grep is indeed using it, please include the build-dep explicitly.
> 
> * If not, your current grep issue is unrelated to Debian-packaged gnulib and
> you will need further debugging.

Indeed, it's a bundled gnulib. I was surprised because there is
no "gnulib" directory. Actually these are just files under the m4
directory. The bug is in "m4/dirfd.m4". I can see

  if test $ac_cv_func_dirfd = no && test $gl_cv_func_dirfd_macro = no; then
HAVE_DIRFD=0
  else
HAVE_DIRFD=1
dnl Replace only if the system declares dirfd already.
if test $ac_cv_have_decl_dirfd = yes; then
  REPLACE_DIRFD=1
fi

where the last 4 lines

dnl Replace only if the system declares dirfd already.
if test $ac_cv_have_decl_dirfd = yes; then
  REPLACE_DIRFD=1
fi

need to be removed to match the upstream gnulib fix.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Bug#1041588: grep -r fails with "Operation not supported"

[Santiago Ruano Rincón]

Hi!

El 21/07/23 a las 11:02, Boyuan Yang escribió:
> Hi,
> 
> On Fri, 21 Jul 2023 10:39:54 +0200 Vincent Lefevre  wrote:
> > Control: severity -1 grave
> > Control: tags -1 - moreinfo
> > Control: retitle -1 grep -r on 10+ files fails with "Operation not
> supported"
> > 
> > On 2023-07-20 23:43:45 -0300, Santiago Ruano Rincón wrote:
> > > El 21/07/23 a las 04:06, Vincent Lefevre escribió:
> > > > On 2023-07-21 03:30:12 +0200, Vincent Lefevre wrote:
> > > > > There is a major regression in grep 3.11-1: I now get an error
> > > > > 
> > > > > cventin:~/Mail> grep -r xxxyyyzzz oldarc
> > > > > grep: oldarc/cur: Operation not supported
> > > > 
> > > > And no such issue with grep from the upstream git.
> > > 
> > > How are you compiling it?
> > 
> > $ ./bootstrap
> > $ ./configure --prefix=$HOME/opt/grep
> > $ make
> > $ make install
> > 
> > > My intuition is this comes from gnulib.
> > 
> > Yes, as in my next message, I said that the error disappeared
> > with the commit that updated gnulib (and did nothing else).
> > 
> > > But a reproducer is needed to confirm.
> > > 
> > > Tagging with moreinfo, and downgrading the severity since I cannot
> > > reproduce it myself:
> > [...]
> > > Please, feel free to bump the severity back again if you are able to
> > > find out a way to reproduce it.
> > 
> > One needs at least 10 files in the directory (9 is not enough).
> > With 10 empty files:
> > 
> > $ mkdir test-dir && for i in `seq 10` ; do : > test-dir/$i ; done
> > $ grep -r x test-dir
> > grep: test-dir: Operation not supported
> 
> Debian gnulib package maintainer here. I briefly checked grep source code,
> and it seems that grep is using the bundled gnulib and not using the packaged
> gnulib. Not 100% sure.

It indeed currently uses a bundled gnulib.

> 
> Whether to use the bundled gnulib is up to the packager's decision. Anyway,
> let me know if I could help.

I suppose we could give the packaged gnulib a try. Patches welcome ;-)

Cheers,

 -- S


signature.asc
Description: PGP signature

Bug#1041588: grep -r fails with "Operation not supported"

[Boyuan Yang]

Hi,

在 2023-07-21星期五的 19:00 +0200，Vincent Lefevre写道：
> On 2023-07-21 17:22:11 +0200, Vincent Lefevre wrote:
> > The best solution would be to fix gnulib on the Debian side, IMHO.
> > I wonder whether other packages are affected too.
> [...]
> > So, if I understand correctly, the fix in gnulib appeared between
> > these two commits. But I don't see which change could affect the
> > result. I could try to investigate.
> 
> This was fixed in the following commit:
> 
> 
> commit d4d8abb39eb02c555f062b1f83ffcfac999c582f
> Author: Bruno Haible 
> Date:   2023-05-05 12:02:49 +0200
> 
>     dirfd: Fix bogus override (regression 2023-04-26).
>     
>     Reported by Bjarni Ingi Gislason  in
>     .
>     
>     * m4/dirfd.m4 (gl_FUNC_DIRFD): Fix mistake in last change.
> 
> diff --git a/ChangeLog b/ChangeLog
> index aaffe12fc1..5f01a52535 100644
> --- a/ChangeLog
> +++ b/ChangeLog
> @@ -1,3 +1,10 @@
> +2023-05-05  Bruno Haible  
> +
> +   dirfd: Fix bogus override (regression 2023-04-26).
> +   Reported by Bjarni Ingi Gislason  in
> +  
> .
> +   * m4/dirfd.m4 (gl_FUNC_DIRFD): Fix mistake in last change.
> +
>  2023-05-04  Bruno Haible  
>  
>     c32swidth: Add tests.
> diff --git a/m4/dirfd.m4 b/m4/dirfd.m4
> index d1ee2c7f61..7968b1287c 100644
> --- a/m4/dirfd.m4
> +++ b/m4/dirfd.m4
> @@ -1,4 +1,4 @@
> -# serial 27   -*- Autoconf -*-
> +# serial 28   -*- Autoconf -*-
>  
>  dnl Find out how to get the file descriptor associated with an open DIR*.
>  
> @@ -40,10 +40,6 @@ AC_DEFUN([gl_FUNC_DIRFD],
>  HAVE_DIRFD=0
>    else
>  HAVE_DIRFD=1
> -    dnl Replace only if the system declares dirfd already.
> -    if test $ac_cv_have_decl_dirfd = yes; then
> -  REPLACE_DIRFD=1
> -    fi
>  dnl Replace dirfd() on native Windows, to support fdopendir().
>  AC_REQUIRE([gl_DIRENT_DIR])
>  if test $DIR_HAS_FD_MEMBER = 0; then
> 
> 
> In the bug-gnulib discussion:
> 
>   https://lists.gnu.org/archive/html/bug-gnulib/2023-05/msg00046.html
> 
> Bernhard Voelker says:
> > Gnulib commit 3f0950f65abb (2023-04-26) not only lead to build time
> > issues, but also made e.g. coreutils' rm(1) fail:
> > 
> > $ mkdir d && (cd d && seq 40 | xargs touch )
> > 
> > $ rm -rf d
> > rm: traversal failed: d: Operation not supported
> > 
> > I can confirm that this gnulib commit d4d8abb39eb0 fixes the issue again.
> 
> This is the same kind of error.
> 
> But I can see:
> 
> gnulib (20230615+stable-1) unstable; urgency=medium
> 
>   * QA upload.
>   * New upstream snapshot from stable branch stable-202301.
> 
>  -- Boyuan Yang   Tue, 20 Jun 2023 14:58:32 -0400
> 
> I've just rebuilt grep 3.11-1 with "debuild -i -us -uc -b",
> but I still get the same issue with it. I'm wondering what
> "stable branch stable-202301" means.

Looking at https://git.savannah.gnu.org/gitweb/?p=gnulib.git , gnulib
upstream is now providing stable branches that only include important
fixes. In Debian's package, I was tracking the stable-202301 branch.
It is likely that this bugfix was not backported onto it.

Anyway, I will look into it and see whether it affects Stable and Sid.
Since gnulib upstream also opened stable-202307 branch, I will likely
switch gnulib in Sid onto that branch, which should have included the
fix that grep now needs.

BTW: Is grep 3.11-1 using Debian-packaged gnulib anywere? I did not
see a build-dependency in
https://sources.debian.org/src/grep/3.11-1/debian/control/ .

* If grep is indeed using it, please include the build-dep explicitly.

* If not, your current grep issue is unrelated to Debian-packaged gnulib and
you will need further debugging.


Thanks,
Boyuan Yang


signature.asc
Description: This is a digitally signed message part

Bug#1041588: grep -r fails with "Operation not supported"

[Vincent Lefevre]

On 2023-07-21 17:22:11 +0200, Vincent Lefevre wrote:
> The best solution would be to fix gnulib on the Debian side, IMHO.
> I wonder whether other packages are affected too.
[...]
> So, if I understand correctly, the fix in gnulib appeared between
> these two commits. But I don't see which change could affect the
> result. I could try to investigate.

This was fixed in the following commit:


commit d4d8abb39eb02c555f062b1f83ffcfac999c582f
Author: Bruno Haible 
Date:   2023-05-05 12:02:49 +0200

dirfd: Fix bogus override (regression 2023-04-26).

Reported by Bjarni Ingi Gislason  in
.

* m4/dirfd.m4 (gl_FUNC_DIRFD): Fix mistake in last change.

diff --git a/ChangeLog b/ChangeLog
index aaffe12fc1..5f01a52535 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,10 @@
+2023-05-05  Bruno Haible  
+
+   dirfd: Fix bogus override (regression 2023-04-26).
+   Reported by Bjarni Ingi Gislason  in
+   .
+   * m4/dirfd.m4 (gl_FUNC_DIRFD): Fix mistake in last change.
+
 2023-05-04  Bruno Haible  
 
c32swidth: Add tests.
diff --git a/m4/dirfd.m4 b/m4/dirfd.m4
index d1ee2c7f61..7968b1287c 100644
--- a/m4/dirfd.m4
+++ b/m4/dirfd.m4
@@ -1,4 +1,4 @@
-# serial 27   -*- Autoconf -*-
+# serial 28   -*- Autoconf -*-
 
 dnl Find out how to get the file descriptor associated with an open DIR*.
 
@@ -40,10 +40,6 @@ AC_DEFUN([gl_FUNC_DIRFD],
 HAVE_DIRFD=0
   else
 HAVE_DIRFD=1
-dnl Replace only if the system declares dirfd already.
-if test $ac_cv_have_decl_dirfd = yes; then
-  REPLACE_DIRFD=1
-fi
 dnl Replace dirfd() on native Windows, to support fdopendir().
 AC_REQUIRE([gl_DIRENT_DIR])
 if test $DIR_HAS_FD_MEMBER = 0; then


In the bug-gnulib discussion:

  https://lists.gnu.org/archive/html/bug-gnulib/2023-05/msg00046.html

Bernhard Voelker says:
| Gnulib commit 3f0950f65abb (2023-04-26) not only lead to build time
| issues, but also made e.g. coreutils' rm(1) fail:
|
| $ mkdir d && (cd d && seq 40 | xargs touch )
|
| $ rm -rf d
| rm: traversal failed: d: Operation not supported
|
| I can confirm that this gnulib commit d4d8abb39eb0 fixes the issue again.

This is the same kind of error.

But I can see:

gnulib (20230615+stable-1) unstable; urgency=medium

  * QA upload.
  * New upstream snapshot from stable branch stable-202301.

 -- Boyuan Yang   Tue, 20 Jun 2023 14:58:32 -0400

I've just rebuilt grep 3.11-1 with "debuild -i -us -uc -b",
but I still get the same issue with it. I'm wondering what
"stable branch stable-202301" means.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Bug#1041588: grep -r fails with "Operation not supported"

[Vincent Lefevre]

On 2023-07-21 11:02:10 -0400, Boyuan Yang wrote:
> Debian gnulib package maintainer here. I briefly checked grep source code,
> and it seems that grep is using the bundled gnulib and not using the packaged
> gnulib. Not 100% sure.
> 
> Whether to use the bundled gnulib is up to the packager's decision. Anyway,
> let me know if I could help.

The best solution would be to fix gnulib on the Debian side, IMHO.
I wonder whether other packages are affected too.

BTW, the grep commit that updated the bundled gnulib:

diff --git a/gnulib b/gnulib
index c2a5953..2e671b8 16
--- a/gnulib
+++ b/gnulib
@@ -1 +1 @@
-Subproject commit c2a5953f179814dd0f4b7e627aae9a009181ed6a
+Subproject commit 2e671b84c8ec5426aee37f4a6656d531929ee430

So, if I understand correctly, the fix in gnulib appeared between
these two commits. But I don't see which change could affect the
result. I could try to investigate.

Knowing the real problem would allow one to make sure that this
is not a bug that is temporarily hidden.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Bug#1041588: grep -r fails with "Operation not supported"

[Boyuan Yang]

Hi,

On Fri, 21 Jul 2023 10:39:54 +0200 Vincent Lefevre  wrote:
> Control: severity -1 grave
> Control: tags -1 - moreinfo
> Control: retitle -1 grep -r on 10+ files fails with "Operation not
supported"
> 
> On 2023-07-20 23:43:45 -0300, Santiago Ruano Rincón wrote:
> > El 21/07/23 a las 04:06, Vincent Lefevre escribió:
> > > On 2023-07-21 03:30:12 +0200, Vincent Lefevre wrote:
> > > > There is a major regression in grep 3.11-1: I now get an error
> > > > 
> > > > cventin:~/Mail> grep -r xxxyyyzzz oldarc
> > > > grep: oldarc/cur: Operation not supported
> > > 
> > > And no such issue with grep from the upstream git.
> > 
> > How are you compiling it?
> 
> $ ./bootstrap
> $ ./configure --prefix=$HOME/opt/grep
> $ make
> $ make install
> 
> > My intuition is this comes from gnulib.
> 
> Yes, as in my next message, I said that the error disappeared
> with the commit that updated gnulib (and did nothing else).
> 
> > But a reproducer is needed to confirm.
> > 
> > Tagging with moreinfo, and downgrading the severity since I cannot
> > reproduce it myself:
> [...]
> > Please, feel free to bump the severity back again if you are able to
> > find out a way to reproduce it.
> 
> One needs at least 10 files in the directory (9 is not enough).
> With 10 empty files:
> 
> $ mkdir test-dir && for i in `seq 10` ; do : > test-dir/$i ; done
> $ grep -r x test-dir
> grep: test-dir: Operation not supported

Debian gnulib package maintainer here. I briefly checked grep source code,
and it seems that grep is using the bundled gnulib and not using the packaged
gnulib. Not 100% sure.

Whether to use the bundled gnulib is up to the packager's decision. Anyway,
let me know if I could help.

Thanks,
Boyuan Yang


signature.asc
Description: This is a digitally signed message part

Bug#1041588: grep -r fails with "Operation not supported"

[Vincent Lefevre]

Control: severity -1 grave
Control: tags -1 - moreinfo
Control: retitle -1 grep -r on 10+ files fails with "Operation not 
supported"

On 2023-07-20 23:43:45 -0300, Santiago Ruano Rincón wrote:
> El 21/07/23 a las 04:06, Vincent Lefevre escribió:
> > On 2023-07-21 03:30:12 +0200, Vincent Lefevre wrote:
> > > There is a major regression in grep 3.11-1: I now get an error
> > > 
> > > cventin:~/Mail> grep -r xxxyyyzzz oldarc
> > > grep: oldarc/cur: Operation not supported
> > 
> > And no such issue with grep from the upstream git.
> 
> How are you compiling it?

$ ./bootstrap
$ ./configure --prefix=$HOME/opt/grep
$ make
$ make install

> My intuition is this comes from gnulib.

Yes, as in my next message, I said that the error disappeared
with the commit that updated gnulib (and did nothing else).

> But a reproducer is needed to confirm.
> 
> Tagging with moreinfo, and downgrading the severity since I cannot
> reproduce it myself:
[...]
> Please, feel free to bump the severity back again if you are able to
> find out a way to reproduce it.

One needs at least 10 files in the directory (9 is not enough).
With 10 empty files:

$ mkdir test-dir && for i in `seq 10` ; do : > test-dir/$i ; done
$ grep -r x test-dir
grep: test-dir: Operation not supported

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Bug#1041588: grep -r fails with "Operation not supported"

[Santiago Ruano Rincón]

Control: tags -1 + moreinfo
Control: severity -1 important

El 21/07/23 a las 04:06, Vincent Lefevre escribió:
> On 2023-07-21 03:30:12 +0200, Vincent Lefevre wrote:
> > There is a major regression in grep 3.11-1: I now get an error
> > 
> > cventin:~/Mail> grep -r xxxyyyzzz oldarc
> > grep: oldarc/cur: Operation not supported
> 
> And no such issue with grep from the upstream git.

How are you compiling it? My intuition is this comes from gnulib. But a
reproducer is needed to confirm.

Tagging with moreinfo, and downgrading the severity since I cannot
reproduce it myself:

santiago@novelo ~/C/Riseup> grep -r xxxyyyzzz INBOX/
INBOX/new/1689905669.3476970_3.novelo,U=46874:2,:cventin:~/Mail> grep -r 
xxxyyyzzz oldarc
INBOX/cur/1689905669.3476970_5.novelo,U=46876:2,S:> cventin:~/Mail> grep -r 
xxxyyyzzz oldarc

Please, feel free to bump the severity back again if you are able to
find out a way to reproduce it.

Cheers,

 -- Santiago


signature.asc
Description: PGP signature

Bug#1041588: grep -r fails with "Operation not supported"

[Vincent Lefevre]

On 2023-07-21 04:06:51 +0200, Vincent Lefevre wrote:
> On 2023-07-21 03:30:12 +0200, Vincent Lefevre wrote:
> > There is a major regression in grep 3.11-1: I now get an error
> > 
> > cventin:~/Mail> grep -r xxxyyyzzz oldarc
> > grep: oldarc/cur: Operation not supported
> 
> And no such issue with grep from the upstream git.

Well, if I recompile from f951840aa510277fe48e5f2579a98f517246ea86
(tag: v3.11), I can reproduce the error.

The error disappears with

commit 6980733869e8a0311c7eb26cb554dde6a1c26b5b
Author: Jim Meyering 
Date:   2023-06-25 02:28:15 +0200

build: update gnulib to latest

So this is a bug in gnulib.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Bug#1041588: grep -r fails with "Operation not supported"

[Vincent Lefevre]

On 2023-07-21 03:30:12 +0200, Vincent Lefevre wrote:
> There is a major regression in grep 3.11-1: I now get an error
> 
> cventin:~/Mail> grep -r xxxyyyzzz oldarc
> grep: oldarc/cur: Operation not supported

And no such issue with grep from the upstream git.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Bug#1041588: grep -r fails with "Operation not supported"

[Vincent Lefevre]

Package: grep
Version: 3.11-1
Severity: grave
Justification: renders package unusable

There is a major regression in grep 3.11-1: I now get an error

cventin:~/Mail> grep -r xxxyyyzzz oldarc
grep: oldarc/cur: Operation not supported

This is just a big maildir mailbox (195700 messages).

The strace output has

[...]
81082 openat(5, "1511962287.17728_1395.cventin:2,S", 
O_RDONLY|O_NOCTTY|O_NONBLOCK|O_NOFOLLOW) = 6
81082 newfstatat(6, "", {st_mode=S_IFREG|0600, st_size=1171, ...}, 
AT_EMPTY_PATH) = 0
81082 read(6, "Return-Path:  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)