Re: [Bug-tar] --files-from and recursive extract (behavior change in 1.27)
On Friday 2015-06-19 14:29, Pavel Raiskup wrote: The patch fix the first issue I reported, but not the second. This patch, which was recently backported to openSUSE's tar-1.28[1] [1] https://build.opensuse.org/package/view_file/Base:System/tar/tar-recursive--files-from.patch?expand=1 breaks creation of tar archives when --no-recursive appears after -T: https://bugzilla.opensuse.org/show_bug.cgi?id=918487 I'm not sure whether this was discussed off-list somehow, but I would call the *new* behavior as expected. There is inconsistency in behavior - yes, but does not seem to be regression, rather a new feature..? It feels very much like a regression, because the behavior changed in such a way that it changed the outcome of scripts making use of that option. By default, people will assume that options are position-independent, that specifying no-argument options multiple times is idempotent unless otherwise specified, and that specifying with-argument options have a last-win strategy unless otherwise specified. TTBOMK, only --verbose is spelled out to change behavior if present multiple times, but there is no word on multiple-presence of position-dependence of --(no-)recursion in tar.info.
Re: [Bug-tar] --files-from and recursive extract (behavior change in 1.27)
On Friday 19 of June 2015 15:20:04 Jan Engelhardt wrote: On Friday 2015-06-19 14:29, Pavel Raiskup wrote: The patch fix the first issue I reported, but not the second. This patch, which was recently backported to openSUSE's tar-1.28[1] [1] https://build.opensuse.org/package/view_file/Base:System/tar/tar-recursive--files-from.patch?expand=1 breaks creation of tar archives when --no-recursive appears after -T: https://bugzilla.opensuse.org/show_bug.cgi?id=918487 I'm not sure whether this was discussed off-list somehow, but I would call the *new* behavior as expected. There is inconsistency in behavior - yes, but does not seem to be regression, rather a new feature..? It feels very much like a regression, because the behavior changed in such a way that it changed the outcome of scripts making use of that option. By default, people will assume that options are position-independent, that specifying no-argument options multiple times is idempotent unless otherwise specified, and that specifying with-argument options have a last-win strategy unless otherwise specified. TTBOMK, only --verbose is spelled out to change behavior if present multiple times, but there is no word on multiple-presence of position-dependence of --(no-)recursion in tar.info. Hmm, actually, I looked at the documentation and it's been documented since 2003 [1], however I failed to find any working tar release (tried v1.16.1 and up). Fix came with thread [3]. [1] http://git.savannah.gnu.org/cgit/tar.git/commit/?id=01d81c7f0fc87 [2] http://www.gnu.org/software/tar/manual/html_section/tar_54.html#recurse [3] https://lists.gnu.org/archive/html/bug-tar/2014-01/msg00020.html Pavel
Re: [Bug-tar] --files-from and recursive extract (behavior change in 1.27)
On Friday 20 of February 2015 10:32:13 Jan Engelhardt wrote:
On Thursday 2014-09-18 21:57, Sergey Poznyakoff wrote:
The patch fix the first issue I reported, but not the second.
Yes, indeed. I have installed the following patch instead, which
fixes both issues.
This patch, which was recently backported to openSUSE's tar-1.28[1]
[1]
https://build.opensuse.org/package/view_file/Base:System/tar/tar-recursive--files-from.patch?expand=1
breaks creation of tar archives when --no-recursive appears after -T:
https://bugzilla.opensuse.org/show_bug.cgi?id=918487
I'm not sure whether this was discussed off-list somehow, but I would call
the *new* behavior as expected. There is inconsistency in behavior - yes,
but does not seem to be regression, rather a new feature..?
The options --{no-,}recursive are kind of positional (I'm not sure how
to call that), so those affect only following members specified on
commandline. You may do:
$ mkdir dir{1,2,3}
$ touch dir{1,2,3}/{a,b}
$ tar -cvf a.tar dir1 --no-recursion dir2 --recursion dir3
dir1/
dir1/b
dir1/a
dir2/
dir3/
dir3/b
dir3/a
The older behavior was the-last-wins but the new is more powerful.
There are other options, e.g. -C, --directory that allows you to work
with members similar way. Like 'tar -cf A -C dirA a ../dirB b'.
IMHO, from the long term perspective, it would be fine if other options
like '--owner', '--group', '--xattrs', .. worked the same way. This is
now possible only by --files-from (and thus the classic option usage and
the -T option are not completely consistent):
$ cat T
dir1
--xattrs --owner root --group root
dir2
--no-xattrs --owner apache --group apache
dir3
$ tar --posix -cf A --files-from T
$ tar --xattrs-include='*' -tvvf A
drwxrwxr-x praiskup/praiskup 0 2015-06-19 13:51 dir1/
-rw-rw-r-- praiskup/praiskup 0 2015-06-19 13:51 dir1/b
-rw-rw-r-- praiskup/praiskup 0 2015-06-19 13:51 dir1/a
drwxrwxr-x* root/root 0 2015-06-19 13:51 dir2/
x: 36 security.selinux
-rw-rw-r--* root/root 0 2015-06-19 13:51 dir2/b
x: 36 security.selinux
-rw-rw-r--* root/root 0 2015-06-19 13:51 dir2/a
x: 36 security.selinux
drwxrwxr-x apache/apache 0 2015-06-19 13:51 dir3/
-rw-rw-r-- apache/apache 0 2015-06-19 13:51 dir3/b
-rw-rw-r-- apache/apache 0 2015-06-19 13:51 dir3/a
Pavel
Re: [Bug-tar] --files-from and recursive extract (behavior change in 1.27)
On Thursday 2014-09-18 21:57, Sergey Poznyakoff wrote: The patch fix the first issue I reported, but not the second. Yes, indeed. I have installed the following patch instead, which fixes both issues. This patch, which was recently backported to openSUSE's tar-1.28[1] [1] https://build.opensuse.org/package/view_file/Base:System/tar/tar-recursive--files-from.patch?expand=1 breaks creation of tar archives when --no-recursive appears after -T: https://bugzilla.opensuse.org/show_bug.cgi?id=918487
Re: [Bug-tar] --files-from and recursive extract (behavior change in 1.27)
Thanks Sergey for the fast fix.
Both issues are fixed.
Jean-Louis
On 09/18/2014 03:57 PM, Sergey Poznyakoff wrote:
The patch fix the first issue I reported, but not the second.
Yes, indeed. I have installed the following patch instead, which
fixes both issues.
Regards,
Sergey
* src/common.h (name_add_file): Change signature.
* src/names.c (name_elt_alloc_matflags): New function.
(name_add_name): Use name_elt_alloc_matflags.
(name_add_file): Take matching flags as third argument.
(read_next_name): Remove trailing slashes.
* src/tar.c (parse_opt): Pass matching_flags to name_add_file.
* tests/T-dir00.at: New file.
* tests/T-dir01.at: New file.
* tests/Makefile.am: Add new testcases.
* tests/testsuite.at: Likewise.
---
src/common.h | 2 +-
src/names.c| 56 ++
src/tar.c | 2 +-
tests/Makefile.am | 2 ++
tests/T-dir00.at | 45 +++
tests/T-dir01.at | 45 +++
tests/testsuite.at | 2 ++
7 files changed, 131 insertions(+), 23 deletions(-)
create mode 100644 tests/T-dir00.at
create mode 100644 tests/T-dir01.at
diff --git a/src/common.h b/src/common.h
index edf787c..3cc2011 100644
--- a/src/common.h
+++ b/src/common.h
@@ -725,7 +725,7 @@ int uname_to_uid (char const *uname, uid_t *puid);
void name_init (void);
void name_add_name (const char *name, int matching_flags);
void name_add_dir (const char *name);
-void name_add_file (const char *name, int term);
+void name_add_file (const char *name, int term, int matching_flags);
void name_term (void);
const char *name_next (int change_dirs);
void name_gather (void);
diff --git a/src/names.c b/src/names.c
index 594e7fd..e3e145a 100644
--- a/src/names.c
+++ b/src/names.c
@@ -258,6 +258,21 @@ name_elt_alloc (void)
return elt;
}
+static struct name_elt *
+name_elt_alloc_matflags (int matflags)
+{
+ static int prev_flags = 0; /* FIXME: Or EXCLUDE_ANCHORED? */
+ struct name_elt *ep = name_elt_alloc ();
+ if (prev_flags != matflags)
+{
+ ep-type = NELT_FMASK;
+ ep-v.matching_flags = matflags;
+ prev_flags = matflags;
+ ep = name_elt_alloc ();
+}
+ return ep;
+}
+
static void
name_list_adjust (void)
{
@@ -276,20 +291,13 @@ name_list_advance (void)
free (elt);
}
-/* Add to name_array the file NAME with fnmatch options MATCHING_FLAGS */
+
+/* Add to name_array the file NAME with fnmatch options MATFLAGS */
void
-name_add_name (const char *name, int matching_flags)
+name_add_name (const char *name, int matflags)
{
- static int prev_flags = 0; /* FIXME: Or EXCLUDE_ANCHORED? */
- struct name_elt *ep = name_elt_alloc ();
+ struct name_elt *ep = name_elt_alloc_matflags (matflags);
- if (prev_flags != matching_flags)
-{
- ep-type = NELT_FMASK;
- ep-v.matching_flags = matching_flags;
- prev_flags = matching_flags;
- ep = name_elt_alloc ();
-}
ep-type = NELT_NAME;
ep-v.name = name;
name_count++;
@@ -305,9 +313,10 @@ name_add_dir (const char *name)
}
void
-name_add_file (const char *name, int term)
+name_add_file (const char *name, int term, int matflags)
{
- struct name_elt *ep = name_elt_alloc ();
+ struct name_elt *ep = name_elt_alloc_matflags (matflags);
+
ep-type = NELT_FILE;
ep-v.file.name = name;
ep-v.file.term = term;
@@ -389,6 +398,15 @@ add_file_id (const char *filename)
file_id_list = p;
return 0;
}
+
+/* Chop trailing slashes. */
+static void
+chopslash (char *str)
+{
+ char *p = str + strlen (str) - 1;
+ while (p str ISSLASH (*p))
+*p-- = '\0';
+}
enum read_file_list_state /* Result of reading file name from the list file
*/
{
@@ -428,7 +446,7 @@ read_name_from_file (struct name_elt *ent)
if (counter == name_buffer_length)
name_buffer = x2realloc (name_buffer, name_buffer_length);
name_buffer[counter] = 0;
-
+ chopslash (name_buffer);
return (counter == 0 c == EOF) ? file_list_end : file_list_success;
}
@@ -518,7 +536,6 @@ copy_name (struct name_elt *ep)
{
const char *source;
size_t source_len;
- char *cursor;
source = ep-v.name;
source_len = strlen (source);
@@ -536,11 +553,7 @@ copy_name (struct name_elt *ep)
name_buffer = xmalloc(name_buffer_length + 2);
}
strcpy (name_buffer, source);
-
- /* Zap trailing slashes. */
- cursor = name_buffer + strlen (name_buffer) - 1;
- while (cursor name_buffer ISSLASH (*cursor))
-*cursor-- = '\0';
+ chopslash (name_buffer);
}
@@ -553,7 +566,8 @@ static int matching_flags; /* exclude_fnmatch options */
the request to change to the given directory.
Entries of type NELT_FMASK cause updates of the matching_flags
- value. */
+ value.
+*/
static struct name_elt *
name_next_elt (int change_dirs)
{
diff --git a/src/tar.c b/src/tar.c
index cd32379..225c624 100644
--- a/src/tar.c
[Bug-tar] --files-from and recursive extract (behavior change in 1.27)
Hi, --files-from do not recursively extract since tar-1.27 In tar-1.26, i could put a directory name in a 'list' file and run ' tar -xvf archive.tar --files-from list' to extract the directory recursively. In tar-1.27 and tar-1.28 it extract only the directory, nothing inside it. How to reproduce: mkdir dir1 touch dir1/file1.1 dir1/file1.2 tar cf archive.tar dir1 echo dir1 list rm -rf dir1 tar xvf archive.tar --files-from list $../tar-1.26/src/tar xvf archive.tar --files-from list dir1/ dir1/file1.2 dir1/file1.1 $../tar-1.27/src/tar xvf archive.tar --files-from list dir1/ $../tar-1.28/src/tar xvf archive.tar --files-from list dir1/ How to get the same behavior as tar-1.26? Also, if I put 'dir1/' (with the /) in the list file tar-1.27 and 1.28 report: ../tar-1.27/src/tar: dir1/: Not found in archive ../tar-1.27/src/tar: Exiting with failure status due to previous errors tar-1.26 correctly extract it. Jean-Louis
Re: [Bug-tar] --files-from and recursive extract (behavior change in 1.27)
Hi Jean-Louis,
--files-from do not recursively extract since tar-1.27
Thanks for reporting. Please try the attached patch.
Regards,
Sergey
From 890a81d753b506d6dfd2031d6633d3f5be38a966 Mon Sep 17 00:00:00 2001
From: Sergey Poznyakoff g...@gnu.org.ua
Date: Thu, 18 Sep 2014 18:06:40 +0300
Subject: [PATCH] Bugfix: entries read from the -T file did not get proper
matching_flag.
* src/common.h (name_add_file): Change signature.
* src/names.c (name_elt.file) matching_flags: New member.
(name_add_file): Take matching flags as third argument.
(read_next_name): Set matching_flags and
recursion_option after opening the file.
* src/tar.c (parse_opt): Pass matching_flags to name_add_file.
---
src/common.h | 2 +-
src/names.c | 16
src/tar.c| 2 +-
3 files changed, 14 insertions(+), 6 deletions(-)
diff --git a/src/common.h b/src/common.h
index edf787c..3cc2011 100644
--- a/src/common.h
+++ b/src/common.h
@@ -725,7 +725,7 @@ int uname_to_uid (char const *uname, uid_t *puid);
void name_init (void);
void name_add_name (const char *name, int matching_flags);
void name_add_dir (const char *name);
-void name_add_file (const char *name, int term);
+void name_add_file (const char *name, int term, int matching_flags);
void name_term (void);
const char *name_next (int change_dirs);
void name_gather (void);
diff --git a/src/names.c b/src/names.c
index 594e7fd..7e920df 100644
--- a/src/names.c
+++ b/src/names.c
@@ -229,6 +229,7 @@ struct name_elt/* A name_array element. */
const char *name;/* File name */
int term;/* File name terminator in the list */
FILE *fp;
+ int matching_flags; /* fnmatch options */
} file;
} v;
};
@@ -305,13 +306,14 @@ name_add_dir (const char *name)
}
void
-name_add_file (const char *name, int term)
+name_add_file (const char *name, int term, int matching_flags)
{
struct name_elt *ep = name_elt_alloc ();
ep-type = NELT_FILE;
ep-v.file.name = name;
ep-v.file.term = term;
ep-v.file.fp = NULL;
+ ep-v.file.matching_flags = matching_flags;
}
/* Names from external name file. */
@@ -456,6 +458,8 @@ handle_option (const char *str)
return 0;
}
+static int matching_flags; /* exclude_fnmatch options */
+
static int
read_next_name (struct name_elt *ent, struct name_elt *ret)
{
@@ -476,6 +480,8 @@ read_next_name (struct name_elt *ent, struct name_elt *ret)
if ((ent-v.file.fp = fopen (ent-v.file.name, r)) == NULL)
open_fatal (ent-v.file.name);
}
+ matching_flags = ent-v.file.matching_flags;
+ recursion_option = matching_flags FNM_LEADING_DIR;
}
while (1)
@@ -544,8 +550,6 @@ copy_name (struct name_elt *ep)
}
-static int matching_flags; /* exclude_fnmatch options */
-
/* Get the next NELT_NAME element from name_array. Result is in
static storage and can't be relied upon across two calls.
@@ -553,7 +557,11 @@ static int matching_flags; /* exclude_fnmatch options */
the request to change to the given directory.
Entries of type NELT_FMASK cause updates of the matching_flags
- value. */
+ value.
+
+ Entries of type NELT_FILE cause updates of the matching_flags
+ once, before opening the file.
+*/
static struct name_elt *
name_next_elt (int change_dirs)
{
diff --git a/src/tar.c b/src/tar.c
index cd32379..225c624 100644
--- a/src/tar.c
+++ b/src/tar.c
@@ -1641,7 +1641,7 @@ parse_opt (int key, char *arg, struct argp_state *state)
break;
case 'T':
- name_add_file (arg, filename_terminator);
+ name_add_file (arg, filename_terminator, MAKE_INCL_OPTIONS (args));
/* Indicate we've been given -T option. This is for backward
compatibility only, so that `tar cfT archive /dev/null will
succeed */
--
1.7.12.1
Re: [Bug-tar] --files-from and recursive extract (behavior change in 1.27)
Hi Sergey, The patch fix the first issue I reported, but not the second. mkdir dir1 touch dir1/file1.1 dir1/file1.2 tar cf archive.tar dir1 echo dir1/ list ## NOTE the / rm -rf dir1 tar xvf archive.tar --files-from list $ ../tar-1.26/src/tar xvf archive.tar --files-from list dir1/ dir1/file-1.2 dir1/file1-1 $ ../tar-1.28/src/tar xvf archive.tar --files-from list ../tar-1.28/src/tar: dir1/: Not found in archive ../tar-1.28/src/tar: Exiting with failure status due to previous errors Why it doesn't accept the trailing slash in a files-from, it works on command line: $ ../tar-1.28/src/tar xvf archive.tar dir1/ dir1/ dir1/file-1.2 dir1/file1-1 Jean-Louis On 09/18/2014 11:02 AM, Sergey Poznyakoff wrote: Hi Jean-Louis, --files-from do not recursively extract since tar-1.27 Thanks for reporting. Please try the attached patch. Regards, Sergey
Re: [Bug-tar] --files-from and recursive extract (behavior change in 1.27)
The patch fix the first issue I reported, but not the second.
Yes, indeed. I have installed the following patch instead, which
fixes both issues.
Regards,
Sergey
* src/common.h (name_add_file): Change signature.
* src/names.c (name_elt_alloc_matflags): New function.
(name_add_name): Use name_elt_alloc_matflags.
(name_add_file): Take matching flags as third argument.
(read_next_name): Remove trailing slashes.
* src/tar.c (parse_opt): Pass matching_flags to name_add_file.
* tests/T-dir00.at: New file.
* tests/T-dir01.at: New file.
* tests/Makefile.am: Add new testcases.
* tests/testsuite.at: Likewise.
---
src/common.h | 2 +-
src/names.c| 56 ++
src/tar.c | 2 +-
tests/Makefile.am | 2 ++
tests/T-dir00.at | 45 +++
tests/T-dir01.at | 45 +++
tests/testsuite.at | 2 ++
7 files changed, 131 insertions(+), 23 deletions(-)
create mode 100644 tests/T-dir00.at
create mode 100644 tests/T-dir01.at
diff --git a/src/common.h b/src/common.h
index edf787c..3cc2011 100644
--- a/src/common.h
+++ b/src/common.h
@@ -725,7 +725,7 @@ int uname_to_uid (char const *uname, uid_t *puid);
void name_init (void);
void name_add_name (const char *name, int matching_flags);
void name_add_dir (const char *name);
-void name_add_file (const char *name, int term);
+void name_add_file (const char *name, int term, int matching_flags);
void name_term (void);
const char *name_next (int change_dirs);
void name_gather (void);
diff --git a/src/names.c b/src/names.c
index 594e7fd..e3e145a 100644
--- a/src/names.c
+++ b/src/names.c
@@ -258,6 +258,21 @@ name_elt_alloc (void)
return elt;
}
+static struct name_elt *
+name_elt_alloc_matflags (int matflags)
+{
+ static int prev_flags = 0; /* FIXME: Or EXCLUDE_ANCHORED? */
+ struct name_elt *ep = name_elt_alloc ();
+ if (prev_flags != matflags)
+{
+ ep-type = NELT_FMASK;
+ ep-v.matching_flags = matflags;
+ prev_flags = matflags;
+ ep = name_elt_alloc ();
+}
+ return ep;
+}
+
static void
name_list_adjust (void)
{
@@ -276,20 +291,13 @@ name_list_advance (void)
free (elt);
}
-/* Add to name_array the file NAME with fnmatch options MATCHING_FLAGS */
+
+/* Add to name_array the file NAME with fnmatch options MATFLAGS */
void
-name_add_name (const char *name, int matching_flags)
+name_add_name (const char *name, int matflags)
{
- static int prev_flags = 0; /* FIXME: Or EXCLUDE_ANCHORED? */
- struct name_elt *ep = name_elt_alloc ();
+ struct name_elt *ep = name_elt_alloc_matflags (matflags);
- if (prev_flags != matching_flags)
-{
- ep-type = NELT_FMASK;
- ep-v.matching_flags = matching_flags;
- prev_flags = matching_flags;
- ep = name_elt_alloc ();
-}
ep-type = NELT_NAME;
ep-v.name = name;
name_count++;
@@ -305,9 +313,10 @@ name_add_dir (const char *name)
}
void
-name_add_file (const char *name, int term)
+name_add_file (const char *name, int term, int matflags)
{
- struct name_elt *ep = name_elt_alloc ();
+ struct name_elt *ep = name_elt_alloc_matflags (matflags);
+
ep-type = NELT_FILE;
ep-v.file.name = name;
ep-v.file.term = term;
@@ -389,6 +398,15 @@ add_file_id (const char *filename)
file_id_list = p;
return 0;
}
+
+/* Chop trailing slashes. */
+static void
+chopslash (char *str)
+{
+ char *p = str + strlen (str) - 1;
+ while (p str ISSLASH (*p))
+*p-- = '\0';
+}
enum read_file_list_state /* Result of reading file name from the list file */
{
@@ -428,7 +446,7 @@ read_name_from_file (struct name_elt *ent)
if (counter == name_buffer_length)
name_buffer = x2realloc (name_buffer, name_buffer_length);
name_buffer[counter] = 0;
-
+ chopslash (name_buffer);
return (counter == 0 c == EOF) ? file_list_end : file_list_success;
}
@@ -518,7 +536,6 @@ copy_name (struct name_elt *ep)
{
const char *source;
size_t source_len;
- char *cursor;
source = ep-v.name;
source_len = strlen (source);
@@ -536,11 +553,7 @@ copy_name (struct name_elt *ep)
name_buffer = xmalloc(name_buffer_length + 2);
}
strcpy (name_buffer, source);
-
- /* Zap trailing slashes. */
- cursor = name_buffer + strlen (name_buffer) - 1;
- while (cursor name_buffer ISSLASH (*cursor))
-*cursor-- = '\0';
+ chopslash (name_buffer);
}
@@ -553,7 +566,8 @@ static int matching_flags; /* exclude_fnmatch options */
the request to change to the given directory.
Entries of type NELT_FMASK cause updates of the matching_flags
- value. */
+ value.
+*/
static struct name_elt *
name_next_elt (int change_dirs)
{
diff --git a/src/tar.c b/src/tar.c
index cd32379..225c624 100644
--- a/src/tar.c
+++ b/src/tar.c
@@ -1641,7 +1641,7 @@ parse_opt (int key, char *arg, struct argp_state *state)
break;
case 'T':
- name_add_file (arg, filename_terminator);
+ name_add_file
