Re: [OE-core] [PATCH] libarchive: enable non-recursive extract/list

[Patrick Ohly]

On Thu, 2016-12-01 at 10:40 +0100, Patrick Ohly wrote:
> Required for meta-swupd performance enhancements: in meta-swupd, the
> so called "mega" image contains a rootfs with all files that can
> potentially be installed on a device. Other virtual image recipes need
> a subset of those files or directories, and a partial extraction from
> a single tar archive is faster than letting all virtual image recipes
> share access to a directory under a single pseudo instance.
> 
> It may be necessary to extract a directory with all of its attributes
> without the content of the directory, hence this patch. Upstream
> agreed to consider merging such a patch (see
> https://groups.google.com/forum/#!topic/libarchive-discuss/JO3hqSaAVfs)
> but has been slow in actually commenting on it, so for now it has
> to be carried as distro patch.

Ping?

Alexander, you touched libarchive most recently. Any thoughts on this?

I could carry the patch also in meta-swupd, if that's preferred. It just
doesn't feel very clean for such a layer to modify a core recipe like
that.

-- 
Best Regards, Patrick Ohly

The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.



-- 
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

[OE-core] [PATCH] libarchive: enable non-recursive extract/list

[Patrick Ohly]

Required for meta-swupd performance enhancements: in meta-swupd, the
so called "mega" image contains a rootfs with all files that can
potentially be installed on a device. Other virtual image recipes need
a subset of those files or directories, and a partial extraction from
a single tar archive is faster than letting all virtual image recipes
share access to a directory under a single pseudo instance.

It may be necessary to extract a directory with all of its attributes
without the content of the directory, hence this patch. Upstream
agreed to consider merging such a patch (see
https://groups.google.com/forum/#!topic/libarchive-discuss/JO3hqSaAVfs)
but has been slow in actually commenting on it, so for now it has
to be carried as distro patch.

Signed-off-by: Patrick Ohly 
---
 .../files/non-recursive-extract-and-list.patch | 153 +
 .../libarchive/libarchive_3.2.2.bb |   1 +
 2 files changed, 154 insertions(+)
 create mode 100644 
meta/recipes-extended/libarchive/files/non-recursive-extract-and-list.patch

diff --git 
a/meta/recipes-extended/libarchive/files/non-recursive-extract-and-list.patch 
b/meta/recipes-extended/libarchive/files/non-recursive-extract-and-list.patch
new file mode 100644
index 000..61f8f3e
--- /dev/null
+++ 
b/meta/recipes-extended/libarchive/files/non-recursive-extract-and-list.patch
@@ -0,0 +1,153 @@
+From d6271709d2deb980804f92e75f9b5cb600dc42ed Mon Sep 17 00:00:00 2001
+From: Patrick Ohly 
+Date: Mon, 24 Oct 2016 12:54:48 +0200
+Subject: [PATCH 1/2] non-recursive extract and list
+
+Sometimes it makes sense to extract or list a directory contained in
+an archive without also doing the same for the content of the
+directory, i.e. allowing -n (= --no-recursion) in combination with the
+x and t modes.
+
+bsdtar uses the match functionality in libarchive to track include
+matches. A new libarchive API call
+archive_match_include_directories_recursively() gets introduced to
+influence the matching behavior, with the default behavior as before.
+
+Non-recursive matching can be achieved by anchoring the path match at
+both start and end. Asking for a directory which itself isn't in the
+archive when in non-recursive mode is an error and handled by the
+existing mechanism for tracking unused inclusion entries.
+
+Upstream-Status: Submitted [https://github.com/libarchive/libarchive/pull/812]
+
+Signed-off-by: Patrick Ohly 
+
+---
+ libarchive/archive.h   |  2 ++
+ libarchive/archive_match.c | 30 +-
+ tar/bsdtar.1   |  3 +--
+ tar/bsdtar.c   | 12 ++--
+ 4 files changed, 42 insertions(+), 5 deletions(-)
+
+diff --git a/libarchive/archive.h b/libarchive/archive.h
+index ff401e9..38d8746 100644
+--- a/libarchive/archive.h
 b/libarchive/archive.h
+@@ -1085,6 +1085,8 @@ __LA_DECL intarchive_match_excluded(struct archive *,
+  */
+ __LA_DECL int archive_match_path_excluded(struct archive *,
+   struct archive_entry *);
++/* Control recursive inclusion of directory content when directory is 
included. Default on. */
++__LA_DECL int archive_match_include_directories_recursively(struct archive *, 
int _enabled);
+ /* Add exclusion pathname pattern. */
+ __LA_DECL int archive_match_exclude_pattern(struct archive *, const char *);
+ __LA_DECL int archive_match_exclude_pattern_w(struct archive *,
+diff --git a/libarchive/archive_match.c b/libarchive/archive_match.c
+index 0719cbd..6d03a65 100644
+--- a/libarchive/archive_match.c
 b/libarchive/archive_match.c
+@@ -93,6 +93,9 @@ struct archive_match {
+   /* exclusion/inclusion set flag. */
+   int  setflag;
+ 
++  /* Recursively include directory content? */
++  int  recursive_include;
++
+   /*
+* Matching filename patterns.
+*/
+@@ -223,6 +226,7 @@ archive_match_new(void)
+   return (NULL);
+   a->archive.magic = ARCHIVE_MATCH_MAGIC;
+   a->archive.state = ARCHIVE_STATE_NEW;
++  a->recursive_include = 1;
+   match_list_init(&(a->inclusions));
+   match_list_init(&(a->exclusions));
+   __archive_rb_tree_init(&(a->exclusion_tree), _ops_mbs);
+@@ -471,6 +475,28 @@ archive_match_path_excluded(struct archive *_a,
+ }
+ 
+ /*
++ * When recursive inclusion of directory content is enabled,
++ * an inclusion pattern that matches a directory will also
++ * include everything beneath that directory. Enabled by default.
++ *
++ * For compatibility with GNU tar, exclusion patterns always
++ * match if a subset of the full patch matches (i.e., they are
++ * are not rooted at the beginning of the path) and thus there
++ * is no corresponding non-recursive exclusion mode.
++ */
++int
++archive_match_include_directories_recursively(struct archive *_a, int 
_enabled)
++{
++  struct archive_match *a;
++
++  archive_check_magic(_a,