Hello community, here is the log from the commit of package libguestfs.12741 for openSUSE:Leap:15.1:Update checked in at 2020-05-29 20:16:16 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Leap:15.1:Update/libguestfs.12741 (Old) and /work/SRC/openSUSE:Leap:15.1:Update/.libguestfs.12741.new.3606 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "libguestfs.12741" Fri May 29 20:16:16 2020 rev:1 rq:809096 version:1.38.0 Changes: -------- New Changes file: --- /dev/null 2020-05-28 02:28:09.919761697 +0200 +++ /work/SRC/openSUSE:Leap:15.1:Update/.libguestfs.12741.new.3606/libguestfs.changes 2020-05-29 20:16:18.156151431 +0200 @@ -0,0 +1,1784 @@ +------------------------------------------------------------------- +Thu Mar 19 10:25:01 MDT 2020 - [email protected] + +- bsc#1167131 - virt-v2v fails importing ubuntu 18.04 LTS ova + 500acb15-v2v-linux-fix-kernel-detection-when-split-in-different-packages.patch + +------------------------------------------------------------------- +Tue Sep 10 09:59:19 UTC 2019 - Christophe Giboudeaux <[email protected]> + +- Use -ffat-lto-objects when building static libraries. + +------------------------------------------------------------------- +Mon Sep 9 16:32:56 UTC 2019 - Larry Dewey <[email protected]> + +- Added lvm2 to Requires for using virt-resize of xfs images. + (bsc#1138860) + +------------------------------------------------------------------- +Wed Aug 28 20:10:11 UTC 2019 - Guilherme Moro <[email protected]> + +- Add sysconfig-netconfig to appliance (boo#1136878) + Patches added: + netconfig.patch + +------------------------------------------------------------------- +Thu Aug 22 13:39:11 UTC 2019 - [email protected] + +- Remove empty /etc/libguestfs-tools.conf + +------------------------------------------------------------------- +Wed Jun 12 14:22:29 UTC 2019 - Dominique Leuenberger <[email protected]> + +- BuildRequire pkgconfig(systemd) instead of systemd: allow OBS to + shortcut the build queues by allowing usage of systemd-mini + +------------------------------------------------------------------- +Wed Apr 17 17:37:04 UTC 2019 - Larry Dewey <[email protected]> + +- Fixing issue with virt-customize in SLES and openSUSE guests + (boo#1132790) + Patches added: + 28bd06227b-inspect-handle-os-release-opensuse-tumbleweed-as-ope.patch + 70407cd622-inspection-Parse-os-release-opensuse-leap-as-opensus.patch + +------------------------------------------------------------------- +Mon Apr 8 18:50:31 UTC 2019 - Larry Dewey <[email protected]> + +- Fixing issue with virt-customize uninstall and '-l' + (bnc#1131342) + Patches added: + fd43730e-error-with-uninstall-option-on-SUSE.patch + +------------------------------------------------------------------- +Mon Jun 11 13:39:04 UTC 2018 - [email protected] + +- Remove zerofree from BuildRequires because only supermin will + consider this feature at runtime. + +------------------------------------------------------------------- +Thu Apr 19 09:07:54 UTC 2018 - [email protected] + +- Build against libtirpc + +------------------------------------------------------------------- +Tue Feb 27 13:14:35 UTC 2018 - [email protected] + +- Remove unneeded createrepo build dependency (bsc#1082689) +- builder-repository: fix SLES short id computation + Patch added: + 0a55098f-builder-repository-fix-compute_short_id-for-sles-X.0.patch +- Fix handling of Latin-1 encoding in rpmdb (bsc#1075803) + Patches added: + 0001-Introduce-a-wrapper-around-xmlParseURI.patch + 0002-common-extract-UTF-8-conversion-function.patch + 0003-inspector-rpm-summary-and-description-may-not-be-utf.patch + +------------------------------------------------------------------- +Fri Feb 23 08:53:30 UTC 2018 - [email protected] + +- Remove dependency on wodim for openSUSE:Factory and use cdrtools + (bnc#1081739) + +------------------------------------------------------------------- +Fri Feb 9 19:39:31 UTC 2018 - [email protected] + +- update to version 1.38.0: + * Virt-builder-repository is a new tool allowing end users to + create and update virt-builder repositories. (fate#318952) + * Virt-rescue has been substantially rewritten, implementing job + control, -m and -i options, escape keys, etc. + * Virt-builder planner has been improved so that faster and more + efficient build plans are chosen for complex cases, especially when + either the tmpdir or output is on networked storage. + * Virt-customize now sets a random /etc/machine-id for Linux guests, if + one is not already set. + * Virt-df now works correctly on filesystems with block sizes smaller + than 1K. + * Virt-dib has further compatibility enhancements with diskimage-builder. + * Virt-sysprep removes "DHCP_HOSTNAME" from ifcfg-* files. + * Virt-resize now correctly copies GPT partition attributes from the + source to the destination. (bsc#1074585) + * Bash tab completion implemented or enhanced for: virt-win-reg, + virt-v2v-copy-to-local. + * Both virt-v2v and virt-p2v are now able to pass through the source CPU + vendor, model and topology. + * Virt-v2v now supports encrypted guests. + * Virt-v2v now detects the special Linux Xen PV-only kernels correctly + * Virt-v2v -o glance now generates the right properties for UEFI guests + * Virt-v2v -o null now avoids spooling the guest to a temporary file, + instead it writes to the qemu "null block device". This makes it + faster and use almost no disk space. + * Virt-v2v -i libvirtxml can now open network disks over http or https. + * Virt-v2v will now give a warning about host passthrough devices + * Inspection support was rewritten in OCaml and included inside the + daemon. This makes inspection considerably faster, more robust and + more easily extensible in future. + * The libguestfs API is now thread-safe (although not parallel). You can + call APIs on the same handle from multiple threads without needing to + take a lock. +- Removed patches: + 531316cc-build-improve-and-simplify-distro-detection.patch + 9d25b4e5-python-add-simple-wrappers-for-PyObject-string-funct.patch + f3f99a09-python-use-right-func-when-PyString_FromStringAndSiz.patch +- Added patches: + d0e5a819-python-Fix-missing-additional-backslashes.patch + + +------------------------------------------------------------------- +Mon Dec 4 17:18:57 UTC 2017 - [email protected] + +- evtxdump is in python3-evtx, drop python2 dependency + (fate#323526) +- add hwinfo to the appliance packages for ntfs support. + (bsc#1070976) +- Don't run fdupes as it symlinks some of the python3 binding + files to their python2 equivalent + +------------------------------------------------------------------- +Wed Nov 22 16:22:39 UTC 2017 - [email protected] + +- Add db48-utils dependency: needed to inspect the installed + packages. +- Remove useless patch: 0000-hotfix.patch + +------------------------------------------------------------------- +Wed Oct 25 13:24:50 UTC 2017 - [email protected] + +- Move guestfs-data to libguestfs0. bsc#1064399 +- Force distro detection + Removed: + * use-idlike.patch + Added: + * 531316cc-build-improve-and-simplify-distro-detection.patch + +------------------------------------------------------------------- +Thu Oct 12 14:37:40 CEST 2017 - [email protected] + +- Use "OCaml" in RPM group exactly as all other packages do. + +------------------------------------------------------------------- +Thu Sep 21 13:25:16 UTC 2017 - [email protected] + +- Update summaries and RPM groups. Trim description for size. +- Replace old $RPM_ shell vars by macros. + Remove redundant %clean section. + +------------------------------------------------------------------- +Wed Sep 20 17:18:24 UTC 2017 - [email protected] + +- Add windows support files to the appliance for SLES (fate#316274) +- Merge SLES and openSUSE spec files. + +------------------------------------------------------------------- +Mon Sep 4 19:15:25 UTC 2017 - [email protected] + +- Tweak configuration before building for python3 or PyString_* + symbols will still be missing + +------------------------------------------------------------------- +Thu Aug 31 15:25:06 UTC 2017 - [email protected] + +- Fix python binding for python3 + add patches: + * 9d25b4e5-python-add-simple-wrappers-for-PyObject-string-funct.patch + * f3f99a09-python-use-right-func-when-PyString_FromStringAndSiz.patch +- Better fix for SUSE distro detection using ID_LIKE as fallback: + * use-idlike.patch + +------------------------------------------------------------------- +Tue Jul 18 14:07:54 UTC 2017 - [email protected] + +- Add missing dependency on guestfs-data (bsc#1048848) +- Add conflict with old virt-v2v (bsc#1050069) +- Fix python binding package + * Replace python-libguestfs by python[23]-libguestfs packages + * Add egg-info to python[23]-libguestfs package + ++++ 1587 more lines (skipped) ++++ between /dev/null ++++ and /work/SRC/openSUSE:Leap:15.1:Update/.libguestfs.12741.new.3606/libguestfs.changes New: ---- 0001-Introduce-a-wrapper-around-xmlParseURI.patch 0002-common-extract-UTF-8-conversion-function.patch 0003-inspector-rpm-summary-and-description-may-not-be-utf.patch 0a55098f-builder-repository-fix-compute_short_id-for-sles-X.0.patch 28bd06227b-inspect-handle-os-release-opensuse-tumbleweed-as-ope.patch 500acb15-v2v-linux-fix-kernel-detection-when-split-in-different-packages.patch 70407cd622-inspection-Parse-os-release-opensuse-leap-as-opensus.patch Pod-Simple-3.23.tar.xz README appliance.patch d0e5a819-python-Fix-missing-additional-backslashes.patch fd43730e-error-with-uninstall-option-on-SUSE.patch libguestfs-1.38.0.tar.gz libguestfs.changes libguestfs.rpmlintrc libguestfs.spec libguestfs.test.simple.create-opensuse-guest-crypt-on-lvm.sh libguestfs.test.simple.create-opensuse-guest.sh libguestfs.test.simple.create-sles12-guest-crypt-on-lvm.sh libguestfs.test.simple.create-sles12-guest.sh libguestfs.test.simple.run-libugestfs-test-tool.sh mount-rootfs-and-chroot.sh netconfig.patch ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ libguestfs.spec ++++++ ++++ 882 lines (skipped) ++++++ 0001-Introduce-a-wrapper-around-xmlParseURI.patch ++++++ >From 66dbffd38377abeb64144990421e52293613840a Mon Sep 17 00:00:00 2001 From: "Richard W.M. Jones" <[email protected]> Date: Thu, 15 Feb 2018 15:55:35 +0000 Subject: [PATCH 1/3] Introduce a wrapper around xmlParseURI. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit We only use xmlParseURI to parse our own "homebrew" URIs, for example the ones used by guestfish --add or virt-v2v. Unfortunately xmlParseURI cannot handle URIs with spaces or other non-RFC-compliant characters so simple commands like these fail: $ guestfish -a 'ssh://example.com/virtual machine.img' guestfish: --add: could not parse URI 'ssh://example.com/virtual machine.img' $ guestfish -a 'ssh://example.com/バーチャルマシン.img' guestfish: --add: could not parse URI 'ssh://example.com/バーチャルマシン.img' This is a usability problem. However since these are not expected to be generic RFC-compliant URIs we can perform the required percent-escaping ourselves instead of demanding that the user does this. Note that the wrapper function should not be used on real URLs or libvirt URLs. --- common/mlxml/Makefile.am | 1 + common/mlxml/xml-c.c | 45 +++++++++-- common/mlxml/xml.ml | 1 + common/mlxml/xml.mli | 4 + common/options/uri.c | 5 +- common/utils/Makefile.am | 2 + common/utils/libxml2-utils.c | 178 +++++++++++++++++++++++++++++++++++++++++++ common/utils/libxml2-utils.h | 27 +++++++ v2v/input_vmx.ml | 8 +- v2v/virt-v2v.pod | 5 +- 10 files changed, 258 insertions(+), 18 deletions(-) create mode 100644 common/utils/libxml2-utils.c create mode 100644 common/utils/libxml2-utils.h diff --git a/common/mlxml/Makefile.am b/common/mlxml/Makefile.am index 083c7a64b..739b58ae4 100644 --- a/common/mlxml/Makefile.am +++ b/common/mlxml/Makefile.am @@ -53,6 +53,7 @@ libmlxml_a_CPPFLAGS = \ -I. \ -I$(top_builddir) \ -I$(top_srcdir)/gnulib/lib -I$(top_builddir)/gnulib/lib \ + -I$(top_srcdir)/common/utils -I$(top_builddir)/common/utils \ -I$(shell $(OCAMLC) -where) libmlxml_a_CFLAGS = \ $(WARN_CFLAGS) $(WERROR_CFLAGS) \ diff --git a/common/mlxml/xml-c.c b/common/mlxml/xml-c.c index 3ebecb25e..6dcdb5ccb 100644 --- a/common/mlxml/xml-c.c +++ b/common/mlxml/xml-c.c @@ -27,17 +27,21 @@ #include <stdlib.h> #include <string.h> #include <unistd.h> +#include <errno.h> #include <caml/alloc.h> #include <caml/custom.h> #include <caml/fail.h> #include <caml/memory.h> #include <caml/mlvalues.h> +#include <caml/unixsupport.h> #include <libxml/xpath.h> #include <libxml/xpathInternals.h> #include <libxml/uri.h> +#include "libxml2-utils.h" + #pragma GCC diagnostic ignored "-Wmissing-prototypes" /* xmlDocPtr type */ @@ -426,16 +430,11 @@ mllib_xml_doc_get_root_element (value docv) } } -value -mllib_xml_parse_uri (value strv) +static value +Val_uri (xmlURIPtr uri) { - CAMLparam1 (strv); + CAMLparam0 (); CAMLlocal3 (rv, sv, ov); - xmlURIPtr uri; - - uri = xmlParseURI (String_val (strv)); - if (uri == NULL) - caml_invalid_argument ("parse_uri: unable to parse URI"); rv = caml_alloc_tuple (9); @@ -514,7 +513,37 @@ mllib_xml_parse_uri (value strv) else ov = Val_int (0); Store_field (rv, 8, ov); + CAMLreturn (rv); +} + +value +mllib_xml_parse_uri (value strv) +{ + CAMLparam1 (strv); + CAMLlocal1 (rv); + xmlURIPtr uri; + + uri = xmlParseURI (String_val (strv)); + if (uri == NULL) + caml_invalid_argument ("parse_uri: unable to parse URI"); + + rv = Val_uri (uri); xmlFreeURI (uri); + CAMLreturn (rv); +} +value +mllib_xml_parse_nonstandard_uri (value strv) +{ + CAMLparam1 (strv); + CAMLlocal1 (rv); + xmlURIPtr uri; + + uri = guestfs_int_parse_nonstandard_uri (String_val (strv)); + if (uri == NULL) + unix_error (errno, (char *) "Xml.parse_uri", strv); + + rv = Val_uri (uri); + xmlFreeURI (uri); CAMLreturn (rv); } diff --git a/common/mlxml/xml.ml b/common/mlxml/xml.ml index 5b5c09c00..faeea35ee 100644 --- a/common/mlxml/xml.ml +++ b/common/mlxml/xml.ml @@ -162,3 +162,4 @@ type uri = { } external parse_uri : string -> uri = "mllib_xml_parse_uri" +external parse_nonstandard_uri : string -> uri = "mllib_xml_parse_nonstandard_uri" diff --git a/common/mlxml/xml.mli b/common/mlxml/xml.mli index f561bd673..73c2fdd4b 100644 --- a/common/mlxml/xml.mli +++ b/common/mlxml/xml.mli @@ -115,3 +115,7 @@ val parse_uri : string -> uri Note this is different from the {!URI} module which is specialized for parsing the [-a] parameter on the command line. This function exposes the full [xmlParseURI] interface. *) + +val parse_nonstandard_uri : string -> uri +(** Similar to {!parse_uri} but only for use with our non-standard + URIs. See [guestfs_int_parse_nonstandard_uri] in [common/utils]. *) diff --git a/common/options/uri.c b/common/options/uri.c index ac36bccb2..88a5f0560 100644 --- a/common/options/uri.c +++ b/common/options/uri.c @@ -38,6 +38,7 @@ #include "guestfs.h" #include "guestfs-utils.h" +#include "libxml2-utils.h" #include "uri.h" static int is_uri (const char *arg); @@ -114,9 +115,9 @@ parse (const char *arg, char **path_ret, char **protocol_ret, CLEANUP_FREE char *socket = NULL; char *path; - uri = xmlParseURI (arg); + uri = guestfs_int_parse_nonstandard_uri (arg); if (!uri) { - fprintf (stderr, _("%s: --add: could not parse URI ‘%s’\n"), + fprintf (stderr, _("%s: --add: could not parse URI ‘%s’: %m\n"), getprogname (), arg); return -1; } diff --git a/common/utils/Makefile.am b/common/utils/Makefile.am index 143e2c141..1fa98f992 100644 --- a/common/utils/Makefile.am +++ b/common/utils/Makefile.am @@ -26,6 +26,8 @@ libutils_la_SOURCES = \ gnulib-cleanups.c \ guestfs-utils.h \ libxml2-cleanups.c \ + libxml2-utils.c \ + libxml2-utils.h \ utils.c libutils_la_CPPFLAGS = \ -DGUESTFS_WARN_DEPRECATED=1 \ diff --git a/common/utils/libxml2-utils.c b/common/utils/libxml2-utils.c new file mode 100644 index 000000000..8a05aa5b1 --- /dev/null +++ b/common/utils/libxml2-utils.c @@ -0,0 +1,178 @@ +/* libguestfs + * Copyright (C) 2017 Red Hat Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +/** + * Utility functions using libxml2. + * + * These functions these I<must not> call internal library functions + * such as C<safe_*>, C<error> or C<perrorf>, or any C<guestfs_int_*>. + */ + +#include <config.h> + +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <errno.h> +#include <locale.h> +#include <langinfo.h> +#include <iconv.h> + +#include <libxml/uri.h> + +#include "c-ctype.h" + +/* NB: MUST NOT include "guestfs-internal.h". */ +#include "guestfs.h" +#include "guestfs-utils.h" +#include "libxml2-utils.h" + +static char *local_string_to_utf8 (/* const */ char *input); + +/** + * This is a wrapper around C<xmlParseURI>. That function cannot + * handle spaces and some non-ASCII characters found in URIs. This + * wrapper URI-encodes those before calling C<xmlParseURI> and returns + * the URI structure. + * + * This function should B<only> be called for the URIs that libguestfs + * has invented, for things like guestfish I<--add> and virt-v2v. + * + * For real URIs or libvirt URIs this may cause corruption in corner + * cases. (See L<https://news.ycombinator.com/item?id=11673058> + * describing some of the complexity involved in dealing with real + * URI). + * + * On error, returns C<NULL> and sets C<errno> appropriately. + * + * Caller must call C<xmlFreeURI> on the returned structure or use the + * C<CLEANUP_XMLFREEURI> cleanup macro. + */ +xmlURIPtr +guestfs_int_parse_nonstandard_uri (const char *arg) +{ + CLEANUP_FREE char *uri = NULL; + CLEANUP_FREE char *escaped_uri = NULL; + static const char hexdigit[] = "0123456789abcdef"; + size_t i, j, len; + xmlURIPtr ret; + + /* Convert the string to UTF-8. */ + uri = local_string_to_utf8 ((char *) arg); + if (uri == NULL) + return NULL; + + /* Since we know the URI is in well-formed UTF-8 we can iterate over + * the bytes to do the escaping. The output of this will never be + * more than 3 times larger (each byte might be rewritten as ‘%XX’). + */ + len = strlen (uri); + escaped_uri = malloc (3*len + 1); + if (escaped_uri == NULL) + return NULL; + + for (i = j = 0; i < strlen (uri); ++i) { + /* See RFC 3986 appendix A. Note this leaves existing %-encoded + * escapes alone. + */ + if (c_isalnum (uri[i]) || + strchr ("%-._~:/?#[]@!$&'()*+,;=", uri[i]) != NULL) + escaped_uri[j++] = uri[i]; + else { + escaped_uri[j++] = '%'; + escaped_uri[j++] = hexdigit [(((unsigned char) uri[i]) >> 4) & 0xf]; + escaped_uri[j++] = hexdigit [((unsigned char) uri[i]) & 0xf]; + } + } + escaped_uri[j++] = '\0'; + + /* libxml2 xmlParseURI does not reliably set errno, so it's likely + * best to ignore whatever errno is returned and overwrite it with + * EINVAL. + */ + ret = xmlParseURI (escaped_uri); + if (ret == NULL) { + errno = EINVAL; + return NULL; + } + + return ret; +} + +/* Would be const, but the interface to iconv is not const-correct on + * all platforms. The input string is not touched. + */ +static char * +local_string_to_utf8 (/* const */ char *input) +{ + iconv_t ic; + size_t len, inlen, outlen, outalloc, r, prev; + int err; + char *out, *inp, *outp; + + /* Convert from input locale to UTF-8. */ + ic = iconv_open ("UTF-8", nl_langinfo (CODESET)); + if (ic == (iconv_t) -1) + return NULL; + + len = strlen (input); + outalloc = len; /* Initial guess. */ + + again: + inlen = len; + outlen = outalloc; + out = malloc (outlen + 1); + if (out == NULL) { + err = errno; + iconv_close (ic); + errno = err; + return NULL; + } + inp = input; + outp = out; + + r = iconv (ic, (ICONV_CONST char **) &inp, &inlen, &outp, &outlen); + if (r == (size_t) -1) { + if (errno == E2BIG) { + err = errno; + prev = outalloc; + /* Try again with a larger output buffer. */ + free (out); + outalloc *= 2; + if (outalloc < prev) { + iconv_close (ic); + errno = err; + return NULL; + } + goto again; + } + else { + /* Else some other conversion failure, eg. EILSEQ, EINVAL. */ + err = errno; + iconv_close (ic); + free (out); + errno = err; + return NULL; + } + } + + *outp = '\0'; + iconv_close (ic); + + return out; +} diff --git a/common/utils/libxml2-utils.h b/common/utils/libxml2-utils.h new file mode 100644 index 000000000..d9916ea58 --- /dev/null +++ b/common/utils/libxml2-utils.h @@ -0,0 +1,27 @@ +/* libguestfs + * Copyright (C) 2017 Red Hat Inc. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + */ + +#ifndef GUESTFS_LIBXML2_UTILS_H_ +#define GUESTFS_LIBXML2_UTILS_H_ + +#include <libxml/uri.h> + +/* libxml2-utils.c */ +extern xmlURIPtr guestfs_int_parse_nonstandard_uri (const char *uri); + +#endif /* GUESTFS_LIBXML2_UTILS_H_ */ diff --git a/v2v/input_vmx.ml b/v2v/input_vmx.ml index a8b33f66f..b97fac700 100644 --- a/v2v/input_vmx.ml +++ b/v2v/input_vmx.ml @@ -38,11 +38,11 @@ type vmx_source = let vmx_source_of_arg input_transport arg = match input_transport, arg with | None, arg -> File arg - | Some `SSH, arg -> + | Some `SSH, uri -> let uri = - try Xml.parse_uri arg - with Invalid_argument _ -> - error (f_"remote vmx ‘%s’ could not be parsed as a URI") arg in + try Xml.parse_nonstandard_uri uri + with exn -> + error (f_"could not parse URI: %s") (Printexc.to_string exn) in if uri.Xml.uri_scheme <> None && uri.Xml.uri_scheme <> Some "ssh" then error (f_"vmx URI start with ‘ssh://...’"); if uri.Xml.uri_server = None then diff --git a/v2v/virt-v2v.pod b/v2v/virt-v2v.pod index e30cc03fb..c67b67e48 100644 --- a/v2v/virt-v2v.pod +++ b/v2v/virt-v2v.pod @@ -1458,10 +1458,7 @@ authorized_keys. When using the SSH input transport you must specify a remote C<ssh://...> URI pointing to the VMX file. A typical URI looks like: - ssh://[email protected]/vmfs/volumes/datastore1/my%20guest/my%20guest.vmx - -Any space must be escaped with C<%20> and other non-ASCII characters -may also need to be URI-escaped. + ssh://[email protected]/vmfs/volumes/datastore1/my guest/my guest.vmx The username is not required if it is the same as your local username. -- 2.16.1 ++++++ 0002-common-extract-UTF-8-conversion-function.patch ++++++ >From 10d1eacdac98575d0d8ce81bc04d74b12bf43cab Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?C=C3=A9dric=20Bosdonnat?= <[email protected]> Date: Thu, 15 Feb 2018 17:38:19 +0100 Subject: [PATCH 2/3] common: extract UTF-8 conversion function libxml2-utils.c local_string_to_utf8() function could easily be reused in other places. This commit extracts it with a new parameter to allow giving the encoding of the input string and publishes it in guestfs-utils.h as guestfs_int_string_to_utf8() --- common/utils/guestfs-utils.h | 11 +++++++ common/utils/libxml2-utils.c | 69 +------------------------------------------- common/utils/utils.c | 64 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 76 insertions(+), 68 deletions(-) diff --git a/common/utils/guestfs-utils.h b/common/utils/guestfs-utils.h index 90e7c3dd9..86da693bc 100644 --- a/common/utils/guestfs-utils.h +++ b/common/utils/guestfs-utils.h @@ -33,6 +33,7 @@ #define GUESTFS_UTILS_H_ #include <stdbool.h> +#include <langinfo.h> #include "guestfs-internal-all.h" #include "cleanups.h" @@ -70,6 +71,16 @@ extern int guestfs_int_is_fifo (int64_t mode); extern int guestfs_int_is_lnk (int64_t mode); extern int guestfs_int_is_sock (int64_t mode); extern char *guestfs_int_full_path (const char *dir, const char *name); +extern char *guestfs_int_string_to_utf8 (/* const */ char *input, const char *encoding); + +/* Would be const, but the interface to iconv is not const-correct on + * all platforms. The input string is not touched. + */ +static inline char * +guestfs_int_local_string_to_utf8 (/* const */ char *input) +{ + return guestfs_int_string_to_utf8 (input, nl_langinfo (CODESET)); +} /* Not all language bindings know how to deal with Pointer arguments. * Those that don't will use this macro which complains noisily and diff --git a/common/utils/libxml2-utils.c b/common/utils/libxml2-utils.c index 8a05aa5b1..a71db30dd 100644 --- a/common/utils/libxml2-utils.c +++ b/common/utils/libxml2-utils.c @@ -30,8 +30,6 @@ #include <string.h> #include <errno.h> #include <locale.h> -#include <langinfo.h> -#include <iconv.h> #include <libxml/uri.h> @@ -42,8 +40,6 @@ #include "guestfs-utils.h" #include "libxml2-utils.h" -static char *local_string_to_utf8 (/* const */ char *input); - /** * This is a wrapper around C<xmlParseURI>. That function cannot * handle spaces and some non-ASCII characters found in URIs. This @@ -73,7 +69,7 @@ guestfs_int_parse_nonstandard_uri (const char *arg) xmlURIPtr ret; /* Convert the string to UTF-8. */ - uri = local_string_to_utf8 ((char *) arg); + uri = guestfs_int_local_string_to_utf8 ((char *) arg); if (uri == NULL) return NULL; @@ -113,66 +109,3 @@ guestfs_int_parse_nonstandard_uri (const char *arg) return ret; } - -/* Would be const, but the interface to iconv is not const-correct on - * all platforms. The input string is not touched. - */ -static char * -local_string_to_utf8 (/* const */ char *input) -{ - iconv_t ic; - size_t len, inlen, outlen, outalloc, r, prev; - int err; - char *out, *inp, *outp; - - /* Convert from input locale to UTF-8. */ - ic = iconv_open ("UTF-8", nl_langinfo (CODESET)); - if (ic == (iconv_t) -1) - return NULL; - - len = strlen (input); - outalloc = len; /* Initial guess. */ - - again: - inlen = len; - outlen = outalloc; - out = malloc (outlen + 1); - if (out == NULL) { - err = errno; - iconv_close (ic); - errno = err; - return NULL; - } - inp = input; - outp = out; - - r = iconv (ic, (ICONV_CONST char **) &inp, &inlen, &outp, &outlen); - if (r == (size_t) -1) { - if (errno == E2BIG) { - err = errno; - prev = outalloc; - /* Try again with a larger output buffer. */ - free (out); - outalloc *= 2; - if (outalloc < prev) { - iconv_close (ic); - errno = err; - return NULL; - } - goto again; - } - else { - /* Else some other conversion failure, eg. EILSEQ, EINVAL. */ - err = errno; - iconv_close (ic); - free (out); - errno = err; - return NULL; - } - } - - *outp = '\0'; - iconv_close (ic); - - return out; -} diff --git a/common/utils/utils.c b/common/utils/utils.c index 22af62b0f..faef7c089 100644 --- a/common/utils/utils.c +++ b/common/utils/utils.c @@ -35,6 +35,7 @@ #include <sys/types.h> #include <sys/wait.h> #include <libintl.h> +#include <iconv.h> /* NB: MUST NOT require linking to gnulib, because that will break the * Python 'sdist' which includes a copy of this file. It's OK to @@ -733,3 +734,66 @@ guestfs_int_full_path (const char *dir, const char *name) return path; } + +/* Would be const, but the interface to iconv is not const-correct on + * all platforms. The input string is not touched. + */ +char * +guestfs_int_string_to_utf8 (/* const */ char *input, const char *encoding) +{ + iconv_t ic; + size_t len, inlen, outlen, outalloc, r, prev; + int err; + char *out, *inp, *outp; + + /* Convert from input encoding to UTF-8. */ + ic = iconv_open ("UTF-8", encoding); + if (ic == (iconv_t) -1) + return NULL; + + len = strlen (input); + outalloc = len; /* Initial guess. */ + + again: + inlen = len; + outlen = outalloc; + out = malloc (outlen + 1); + if (out == NULL) { + err = errno; + iconv_close (ic); + errno = err; + return NULL; + } + inp = input; + outp = out; + + r = iconv (ic, (ICONV_CONST char **) &inp, &inlen, &outp, &outlen); + if (r == (size_t) -1) { + if (errno == E2BIG) { + err = errno; + prev = outalloc; + /* Try again with a larger output buffer. */ + free (out); + outalloc *= 2; + if (outalloc < prev) { + iconv_close (ic); + errno = err; + return NULL; + } + goto again; + } + else { + /* Else some other conversion failure, eg. EILSEQ, EINVAL. */ + err = errno; + iconv_close (ic); + free (out); + errno = err; + return NULL; + } + } + + *outp = '\0'; + iconv_close (ic); + + return out; +} -- 2.16.1 ++++++ 0003-inspector-rpm-summary-and-description-may-not-be-utf.patch ++++++ >From 2a20ad737e4682b9f304b6c3ba6116f4cc195541 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?C=C3=A9dric=20Bosdonnat?= <[email protected]> Date: Wed, 14 Feb 2018 10:21:42 +0100 Subject: [PATCH 3/3] inspector: rpm summary and description may not be utf-8 The application inspection code assumes the data in the RPM database are encoded in UTF-8. However this is not always the case. As a basic workaround, try to parse the string to UTF-8 and if that fails, try converting it from latin-1. --- inspector/expected-fedora.img.xml | 4 ++++ lib/inspect-apps.c | 30 +++++++++++++++++++++++---- test-data/phony-guests/fedora-packages.db.txt | 4 ++-- 3 files changed, 32 insertions(+), 6 deletions(-) diff --git a/inspector/expected-fedora.img.xml b/inspector/expected-fedora.img.xml index df6060a73..c29f9770e 100644 --- a/inspector/expected-fedora.img.xml +++ b/inspector/expected-fedora.img.xml @@ -34,12 +34,16 @@ <version>1.0</version> <release>1.fc14</release> <arch>x86_64</arch> + <summary>summary with ö</summary> + <description>description with ö</description> </application> <application> <name>test2</name> <version>2.0</version> <release>2.fc14</release> <arch>x86_64</arch> + <summary>summary with ö</summary> + <description>description with ö</description> </application> <application> <name>test3</name> diff --git a/lib/inspect-apps.c b/lib/inspect-apps.c index f0cf16b38..fdea85188 100644 --- a/lib/inspect-apps.c +++ b/lib/inspect-apps.c @@ -22,6 +22,7 @@ #include <stdlib.h> #include <unistd.h> #include <string.h> +#include <iconv.h> #ifdef HAVE_ENDIAN_H #include <endian.h> @@ -43,6 +44,7 @@ #include "guestfs.h" #include "guestfs-internal.h" #include "guestfs-internal-actions.h" +#include "guestfs-utils.h" #include "structs-cleanups.h" #ifdef DB_DUMP @@ -251,7 +253,7 @@ get_rpm_header_tag (guestfs_h *g, const unsigned char *header_start, /* This function parses the RPM header structure to pull out various * tag strings (version, release, arch, etc.). For more detail on the * header format, see: - * http://www.rpm.org/max-rpm/s1-rpm-file-format-rpm-file-format.html#S2-RPM-FILE-FORMAT-HEADER + * http://rpm.org/devel_doc/file_format.html#24-header-format */ /* The minimum header size that makes sense here is 24 bytes. Four @@ -301,6 +303,20 @@ struct read_package_data { struct guestfs_application2_list *apps; }; +static char * +to_utf8 (guestfs_h *g, char *input) +{ + char *out = NULL; + + out = guestfs_int_string_to_utf8 (input, "UTF-8"); + if (!out) { + out = guestfs_int_string_to_utf8 (input, "ISO-8859-1"); + perrorf (g, "Not an UTF-8 or latin-1 string: '%s'", input); + } + + return out; +} + static int read_package (guestfs_h *g, const unsigned char *key, size_t keylen, @@ -311,7 +327,7 @@ read_package (guestfs_h *g, struct rpm_name nkey, *entry; CLEANUP_FREE char *version = NULL, *release = NULL, *epoch_str = NULL, *arch = NULL, *url = NULL, *summary = NULL, - *description = NULL; + *description = NULL, *summary_raw = NULL, *description_raw = NULL; int32_t epoch; /* This function reads one (key, value) pair from the Packages @@ -342,8 +358,14 @@ read_package (guestfs_h *g, epoch_str = get_rpm_header_tag (g, value, valuelen, RPMTAG_EPOCH, 'i'); arch = get_rpm_header_tag (g, value, valuelen, RPMTAG_ARCH, 's'); url = get_rpm_header_tag (g, value, valuelen, RPMTAG_URL, 's'); - summary = get_rpm_header_tag (g, value, valuelen, RPMTAG_SUMMARY, 's'); - description = get_rpm_header_tag (g, value, valuelen, RPMTAG_DESCRIPTION, 's'); + summary_raw = get_rpm_header_tag (g, value, valuelen, RPMTAG_SUMMARY, 's'); + description_raw = get_rpm_header_tag (g, value, valuelen, RPMTAG_DESCRIPTION, 's'); + + /* Try (not too hard) to get UTF-8 */ + if (summary_raw) + summary = to_utf8 (g, summary_raw); + if (description_raw) + description = to_utf8 (g, description_raw); /* The epoch is stored as big-endian integer. */ if (epoch_str) diff --git a/test-data/phony-guests/fedora-packages.db.txt b/test-data/phony-guests/fedora-packages.db.txt index f16a5aa76..927d6eb5f 100644 --- a/test-data/phony-guests/fedora-packages.db.txt +++ b/test-data/phony-guests/fedora-packages.db.txt @@ -5,9 +5,9 @@ h_nelem=3 db_pagesize=4096 HEADER=END \01\00\00\00 - \00\00\00\03\00\00\00\11\00\00\03\e9\00\00\00\00\00\00\00\00\00\00\00\00\00\00\03\ea\00\00\00\00\00\00\00\04\00\00\00\00\00\00\03\fe\00\00\00\00\00\00\00\0b\00\00\00\001.0\001.fc14\00x86_64\00 + \00\00\00\05\00\00\00\33\00\00\03\e9\00\00\00\00\00\00\00\00\00\00\00\00\00\00\03\ea\00\00\00\00\00\00\00\04\00\00\00\00\00\00\03\fe\00\00\00\00\00\00\00\0b\00\00\00\00\00\00\03\ec\00\00\00\00\00\00\00\12\00\00\00\00\00\00\03\ed\00\00\00\00\00\00\00\21\00\00\00\001.0\001.fc14\00x86_64\00summary with \f6\00description with \f6\00 \02\00\00\00 - \00\00\00\03\00\00\00\11\00\00\03\e9\00\00\00\00\00\00\00\00\00\00\00\00\00\00\03\ea\00\00\00\00\00\00\00\04\00\00\00\00\00\00\03\fe\00\00\00\00\00\00\00\0b\00\00\00\002.0\002.fc14\00x86_64\00 + \00\00\00\05\00\00\00\35\00\00\03\e9\00\00\00\00\00\00\00\00\00\00\00\00\00\00\03\ea\00\00\00\00\00\00\00\04\00\00\00\00\00\00\03\fe\00\00\00\00\00\00\00\0b\00\00\00\00\00\00\03\ec\00\00\00\00\00\00\00\12\00\00\00\00\00\00\03\ed\00\00\00\00\00\00\00\22\00\00\00\002.0\002.fc14\00x86_64\00summary with \c3\b6\00description with \c3\b6\00 \03\00\00\00 \00\00\00\03\00\00\00\11\00\00\03\e9\00\00\00\00\00\00\00\00\00\00\00\00\00\00\03\ea\00\00\00\00\00\00\00\04\00\00\00\00\00\00\03\fe\00\00\00\00\00\00\00\0b\00\00\00\003.0\003.fc14\00x86_64\00 DATA=END -- 2.16.1 ++++++ 0a55098f-builder-repository-fix-compute_short_id-for-sles-X.0.patch ++++++ >From 0a55098f661457b3a4e17b8f5cdd7540d53aec32 Mon Sep 17 00:00:00 2001 From: Pino Toscano <[email protected]> Date: Thu, 22 Feb 2018 18:39:37 +0100 Subject: [PATCH] builder-repository: fix compute_short_id for sles X.0 It needs to check for the minor version, not major version. Fixes commit a442d2c3217f709128f0e377f88649fb6ba90f45. --- builder/repository_main.ml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/builder/repository_main.ml b/builder/repository_main.ml index bb440563b..9932fbae4 100644 --- a/builder/repository_main.ml +++ b/builder/repository_main.ml @@ -178,7 +178,7 @@ let compute_short_id distro major minor = sprintf "%s%d" distro major | ("fedora"|"mageia") -> sprintf "%s%d" distro major - | "sles" when major = 0 -> + | "sles" when minor = 0 -> sprintf "%s%d" distro major | "sles" -> sprintf "%s%dsp%d" distro major minor -- 2.16.1 ++++++ 28bd06227b-inspect-handle-os-release-opensuse-tumbleweed-as-ope.patch ++++++ >From 2e6b8af52042eadc0ca9f8cac3a8c384671e157b Mon Sep 17 00:00:00 2001 From: Pino Toscano <[email protected]> Date: Fri, 30 Nov 2018 12:41:03 +0100 Subject: [PATCH 2/2] inspect: handle os-release "opensuse-tumbleweed" as opensuse Followup of commit 70407cd622dda6f088a0876e1e1ae669e9f8a281 for openSUSE Thumbleweed. (cherry picked from commit 28bd06227b1633fa08c073fe8dbe65d013d7dc9e) Signed-off-by: Larry Dewey <[email protected]> --- daemon/inspect_fs_unix.ml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/daemon/inspect_fs_unix.ml b/daemon/inspect_fs_unix.ml index c0c0a75a4..b6a9af707 100644 --- a/daemon/inspect_fs_unix.ml +++ b/daemon/inspect_fs_unix.ml @@ -142,7 +142,8 @@ and distro_of_os_release_id = function | "frugalware" -> Some DISTRO_FRUGALWARE | "mageia" -> Some DISTRO_MAGEIA | "neokylin" -> Some DISTRO_NEOKYLIN - | "opensuse" | "opensuse-leap" -> Some DISTRO_OPENSUSE + | "opensuse" -> Some DISTRO_OPENSUSE + | s when String.is_prefix s "opensuse-" -> Some DISTRO_OPENSUSE | "pld" -> Some DISTRO_PLD_LINUX | "rhel" -> Some DISTRO_RHEL | "sles" | "sled" -> Some DISTRO_SLES -- 2.21.0 ++++++ 500acb15-v2v-linux-fix-kernel-detection-when-split-in-different-packages.patch ++++++ Subject: v2v: linux: fix kernel detection when split in different packages From: Pino Toscano [email protected] Tue May 22 10:46:21 2018 +0200 Date: Tue May 22 10:46:21 2018 +0200: Git: 500acb15f8f777e9fe99a60c4216daf84a92aae3 The current detection code for Linux kernels assumes that a kernel package contains everything in it, i.e. the kernel itself, its modules, and its configuration. However, since recent Ubuntu versions (e.g. starting from 18.04) modules & config (with few more files) are split in an own package, thus not detecting the modpath from installed vmlinuz files. To overcome this situation, rework this detection a bit: 1) find the vmlinuz file as before, but then immediately make sure it exists by stat'ing it 2) find the modules path from the package as before: 2a) if found, extract the version in the same step 2b) if not found, get the kernel version from the vmlinuz filename, and use it to detect the modules path 3) check that the modules path exists The detection done in (2b) is based on the current packaging scheme found in the most important Linux distributions (Fedora, RHEL, CentOS, Debian, Ubuntu, openSUSE, AltLinux, and possibly more). The notable exception is Arch Linux. As additional change, do not assume the config file is in the same package as vmlinuz, but directly look into the filesystem using the version we already have. diff --git a/v2v/linux_kernels.ml b/v2v/linux_kernels.ml index c047d6deb..24f61429d 100644 --- a/v2v/linux_kernels.ml +++ b/v2v/linux_kernels.ml @@ -103,27 +103,42 @@ let detect_kernels (g : G.guestfs) inspect family bootloader = None ) else ( - (* Which of these is the kernel itself? *) + (* Which of these is the kernel itself? Also, make sure to check + * it exists by stat'ing it. + *) let vmlinuz = List.find ( fun filename -> String.is_prefix filename "/boot/vmlinuz-" ) files in - (* Which of these is the modpath? *) - let modpath = List.find ( - fun filename -> - String.length filename >= 14 && - String.is_prefix filename "/lib/modules/" - ) files in - - (* Check vmlinuz & modpath exist. *) - if not (g#is_dir ~followsymlinks:true modpath) then - raise Not_found; let vmlinuz_stat = try g#statns vmlinuz with G.Error _ -> raise Not_found in - (* Get/construct the version. XXX Read this from kernel file. *) - let version = - let prefix_len = String.length "/lib/modules/" in - String.sub modpath prefix_len (String.length modpath - prefix_len) in + (* Determine the modpath from the package, falling back to the + * version in the vmlinuz file name. + *) + let modpath, version = + let prefix = "/lib/modules/" in + try + let prefix_len = String.length prefix in + List.find_map ( + fun filename -> + let filename_len = String.length filename in + if filename_len > prefix_len && + String.is_prefix filename prefix then ( + let version = String.sub filename prefix_len + (filename_len - prefix_len) in + Some (filename, version) + ) else + None + ) files + with Not_found -> + let version = + String.sub vmlinuz 14 (String.length vmlinuz - 14) in + let modpath = prefix ^ version in + modpath, version in + + (* Check that the modpath exists. *) + if not (g#is_dir ~followsymlinks:true modpath) then + raise Not_found; (* Find the initramfs which corresponds to the kernel. * Since the initramfs is built at runtime, and doesn't have @@ -188,7 +203,7 @@ let detect_kernels (g : G.guestfs) inspect family bootloader = let config_file = let cfg = "/boot/config-" ^ version in - if List.mem cfg files then Some cfg + if g#is_file ~followsymlinks:true cfg then Some cfg else None in let kernel_supports what kconf = ++++++ 70407cd622-inspection-Parse-os-release-opensuse-leap-as-opensus.patch ++++++ >From d22e678ba961b0b23376db2b290340700e1c2c9c Mon Sep 17 00:00:00 2001 From: "Richard W.M. Jones" <[email protected]> Date: Mon, 1 Oct 2018 09:51:15 +0100 Subject: [PATCH 1/2] inspection: Parse os-release "opensuse-leap" as opensuse (RHBZ#1634248). (cherry picked from commit 70407cd622dda6f088a0876e1e1ae669e9f8a281) Signed-off-by: Larry Dewey <[email protected]> --- daemon/inspect_fs_unix.ml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/daemon/inspect_fs_unix.ml b/daemon/inspect_fs_unix.ml index d0323af3f..c0c0a75a4 100644 --- a/daemon/inspect_fs_unix.ml +++ b/daemon/inspect_fs_unix.ml @@ -142,7 +142,7 @@ and distro_of_os_release_id = function | "frugalware" -> Some DISTRO_FRUGALWARE | "mageia" -> Some DISTRO_MAGEIA | "neokylin" -> Some DISTRO_NEOKYLIN - | "opensuse" -> Some DISTRO_OPENSUSE + | "opensuse" | "opensuse-leap" -> Some DISTRO_OPENSUSE | "pld" -> Some DISTRO_PLD_LINUX | "rhel" -> Some DISTRO_RHEL | "sles" | "sled" -> Some DISTRO_SLES -- 2.21.0 ++++++ README ++++++ The ntfs-3g tools contained in this archive are provided for the exclusive use of guestfs tools. Any use of these ntfs-3g tools outside the guestfs tools context is unsupported and strictly forbidden. ++++++ appliance.patch ++++++ Index: libguestfs-1.38.0/appliance/init =================================================================== --- libguestfs-1.38.0.orig/appliance/init +++ libguestfs-1.38.0/appliance/init @@ -234,7 +234,8 @@ else echo "Note: The contents of / (root) are the rescue appliance." if ! test -d "/sysroot/dev"; then echo "You have to mount the guest’s partitions under /sysroot" - echo "before you can examine them." + echo "before you can examine them. A helper script for that exists:" + echo "mount-rootfs-and-chroot.sh /dev/sda2" else echo "Use 'cd /sysroot' or 'chroot /sysroot' to see guest filesystems." fi ++++++ d0e5a819-python-Fix-missing-additional-backslashes.patch ++++++ >From d0e5a819e8b16b38c22cb7309e88bf49a6fdcc4a Mon Sep 17 00:00:00 2001 From: "Richard W.M. Jones" <[email protected]> Date: Fri, 9 Feb 2018 15:55:38 +0000 Subject: [PATCH] python: Fix missing & additional backslashes which broke python sdist. Fixes commit e6c89f96316c3eda6049d0c3ed4de4cda6f4f973. --- python/Makefile.am | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/python/Makefile.am b/python/Makefile.am index 5d2716e20..ef500d65d 100644 --- a/python/Makefile.am +++ b/python/Makefile.am @@ -101,9 +101,9 @@ stamp-extra-files: \ cleanups.h \ config.h \ guestfs-internal-all.h \ - guestfs-utils.h + guestfs-utils.h \ ignore-value.h \ - utils.c \ + utils.c touch $@ config.h: -- 2.15.1 ++++++ fd43730e-error-with-uninstall-option-on-SUSE.patch ++++++ >From a4f65bf89d955f585f141fd3402e51eec53ba562 Mon Sep 17 00:00:00 2001 From: Sebastian Meyer <[email protected]> Date: Thu, 23 Aug 2018 16:45:27 +0200 Subject: [PATCH 1/1] Fix error with --uninstall option on SUSE The `-l` option for some `zypper` subcommands is short for `--auto-agree-with-licenses` and not available on the `zypper rm` command. (cherry picked from commit fd43730e2bfff17a365ffcf768b6c8db3089b524) [BSC#1131342] Signed-off-by: Larry Dewey <[email protected]> --- customize/customize_run.ml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/customize/customize_run.ml b/customize/customize_run.ml index f92e9a199..3eacdaca0 100644 --- a/customize/customize_run.ml +++ b/customize/customize_run.ml @@ -181,7 +181,7 @@ exec >>%s 2>&1 | "urpmi" -> sprintf "urpme %s" quoted_args | "xbps" -> sprintf "xbps-remove -Sy %s" quoted_args | "yum" -> sprintf "yum -y remove %s" quoted_args - | "zypper" -> sprintf "zypper -n rm -l %s" quoted_args + | "zypper" -> sprintf "zypper -n rm %s" quoted_args | "unknown" -> error_unknown_package_manager (s_"--uninstall") -- 2.21.0 ++++++ libguestfs.rpmlintrc ++++++ addFilter("shlib-policy-nonversioned-dir") addFilter("shlib-policy-missing-lib") # This script is supposed to be run from within the appliance only addFilter("non-executable-script /usr/share/virt-p2v/launch-virt-p2v") addFilter("non-standard-group Development/Languages") ++++++ libguestfs.test.simple.create-opensuse-guest-crypt-on-lvm.sh ++++++ #!/bin/bash # Create an openSUSE image with lvm on dm-crypt partition # # Theory of operation: # This script uses zypper from the host to resolve dependencies # for zypper which runs within the appliance. If zypper on the host # is too old, it will be unable to handle repo data from 13.1: # http://lists.opensuse.org/zypp-devel/2013-11/msg00000.html # "[zypp-devel] Package conflicting with itself" # For this reason zypper from 12.3 can be used to install the pattern # of the final repo. # First the dependencies of zypper are resolved, the required packages # are downloaded and extracted with unrpm. Now the guest is started and # the partitions in the diskimage are prepared. Then the extracted # package content is copied into the guest. Once that is done zypper # inside the guest will install the base pattern and a few extra packages. # Finally the bootloader grub is configured. Once all that is done # kvm is started. If all goes well a login prompt appears. # The password for the crypted partition is "123456". # The password for root is "root". # The guest has also network access to the outside. # # Expected runtime: ca. 200 seconds # Requires at least 1.24.5 because this includes the required crypt modules # # Expected output: # guest should start # no "obvious" errors should be shown during the disk operation # at the end kvm is started with the generated disk image # login should be possible # set -e unset LANG unset ${!LC_*} cpus=`grep -Ec 'cpu[0-9]' /proc/stat || echo 1` output_diskimage=/dev/shm/$LOGNAME/testcase.img final_repo=http://download.opensuse.org/distribution/13.1/repo/oss/ initial_repo=http://download.opensuse.org/distribution/12.3/repo/oss/ force=false guest_zypper_in__pattern_name="base" guest_zypper_in__package_list=" grub less master-boot-code nfs-utils parted vim " guest_root_password="root" guest_crypt_password="123456" diskname_inside_vm=/dev/sda case "$0" in /*) progname="$0" ;; *) progname="$PWD/$0" ;; esac _exit() { echo "Exiting '$0 $*'." exit 1 } _unrpm() { CPIO_OPTS="--extract --unconditional --preserve-modification-time --make-directories" FILES="$@" for f in $FILES; do echo -ne "$f:\t" rpm2cpio $f | cpio ${CPIO_OPTS} done } until test $# -lt 1 do case "$1" in --unrpm) shift ; _unrpm "$@" ; exit 0 ;; -n) diskname_inside_vm="$2" ; shift ;; -o) output_diskimage="$2" ; shift ;; -R) initial_repo="$2" ; shift ;; -r) final_repo="$2" ; shift ;; -f) force=true ;; -x) set -x ;; *) echo "Unknown option '$1'" ; exit 1 ;; esac shift done if test -z "${initial_repo}" then echo "URL to initial repo required. Wrong -R option." _exit fi if test -z "${final_repo}" then echo "URL to final repo required. Wrong -r option." _exit fi if test -z "${output_diskimage}" then echo "Filename for temporary disk image required. Wrong -o option." _exit fi if test -e "${output_diskimage}" then if test "${force}" = "false" then echo "Output diskimage '${output_diskimage}' exists." echo "It will not be overwritten. Option '-f' exists to force overwrite." _exit fi fi zypper --version cpio --version guestfish --version kvm="qemu-system-`uname -m`" if $kvm --version then : good else kvm="qemu-kvm" if $kvm --version then : else echo "No qemu-kvm found." _exit fi fi guestfish_version="`guestfish --version | awk '{print \$2}'`" case "${guestfish_version}" in 1.20*) _exit ;; 1.21*) _exit ;; 1.22*) _exit ;; 1.23*) _exit ;; 1.24.[0-4]) _exit ;; *) ;; esac mkdir -vp "${output_diskimage%/*}" td=`mktemp -d --tmpdir=/dev/shm/${LOGNAME}` tf=`mktemp --tmpdir=/dev/shm/${LOGNAME}` _exit() { rm -rf "$tf" rm -rf "$td" } trap _exit EXIT dir_repo=${td}/repos.d dir_root=${td}/root dir_cache=${td}/cache mkdir -vp \ ${dir_root} \ ${dir_cache} \ ${dir_repo} cat > ${tf} <<EOF [main] reposdir = ${dir_repo} EOF cat > ${dir_repo}/tmp.repo <<EOF [tmp] name=tmp enabled=1 autorefresh=1 keeppackages=0 baseurl=${initial_repo} EOF packages=" curl zypper " head ${dir_repo}/tmp.repo ${tf} zypper \ --verbose \ --verbose \ --config ${tf} \ --root ${dir_root} \ --reposd-dir ${dir_repo} \ --cache-dir ${dir_cache} \ --gpg-auto-import-keys \ --no-gpg-checks \ --non-interactive \ lr -d zypper \ --verbose \ --verbose \ --config ${tf} \ --root ${dir_root} \ --reposd-dir ${dir_repo} \ --cache-dir ${dir_cache} \ --gpg-auto-import-keys \ --no-gpg-checks \ --non-interactive \ install \ --auto-agree-with-licenses \ --no-recommends \ --dry-run \ --download-only \ ${packages} cd ${dir_root} find ${dir_cache} -xdev -name "*.rpm" -print0 | sort -z | xargs -0 -n 1 -P ${cpus} bash "${progname}" --unrpm mkdir -vp etc/zypp/repos.d grep -w search /etc/resolv.conf >> etc/resolv.conf echo nameserver 169.254.2.3 >> etc/resolv.conf grep -w root /etc/passwd > etc/passwd grep -w root /etc/group > etc/group echo 'root::15209::::::' > etc/shadow cat > etc/fstab <<EOF LABEL=SWAP swap swap defaults 0 0 LABEL=ROOT / ext4 noatime 1 2 EOF mkdir -p boot/grub cat > etc/grub.conf <<EOF setup --stage2=/boot/grub/stage2 --force-lba (hd0,1) (hd0,1) quit EOF echo "(hd0) ${diskname_inside_vm}" > boot/grub/device.map cat > boot/grub/menu.lst <<EOF serial --unit=0 --speed=115200 terminal --timeout=10 console serial title ${0} $* kernel /boot/vmlinuz panic=9 quiet video=800x600 initrd /boot/initrd EOF du -sm . find ${dir_cache} -xdev -name "*.rpm" -delete ( echo "${guest_crypt_password}" echo "${guest_crypt_password}" ) | \ guestfish \ -x \ --keys-from-stdin \ \ sparse ${output_diskimage} 2048M : \ set-smp 2 : \ set-memsize 1024 : \ set-network true : \ run : \ list-devices : \ part-init ${diskname_inside_vm} mbr : \ part-add ${diskname_inside_vm} primary 1 $(( ((1024*1024)* 256)/512 - 1)) : \ part-add ${diskname_inside_vm} primary $(( ((1024*1024)* 256)/512 )) $(( ((1024*1024)*1024)/512 - 1)) : \ part-add ${diskname_inside_vm} primary $(( ((1024*1024)*1024)/512 )) $(( ((1024*1024)*1555)/512 - 1)) : \ part-add ${diskname_inside_vm} primary $(( ((1024*1024)*1555)/512 )) $(( ((1024*1024)*2024)/512 - 1)) : \ part-list ${diskname_inside_vm} : \ mkswap-opts ${diskname_inside_vm}1 label:SWAP : \ mke2fs ${diskname_inside_vm}2 label:ROOT fstype:ext4 blocksize:1024 : \ pvcreate ${diskname_inside_vm}3 : \ vgcreate uncrypted ${diskname_inside_vm}3 : \ luks-format ${diskname_inside_vm}4 0 : \ luks-open ${diskname_inside_vm}4 crypt_part4 : \ pvcreate /dev/mapper/crypt_part4 : \ vgcreate crypted /dev/mapper/crypt_part4 : \ lvcreate-free root uncrypted 50 : \ lvcreate-free work uncrypted 50 : \ lvcreate-free home crypted 50 : \ lvcreate-free mail crypted 50 : \ list-devices : \ list-partitions : \ pvs-full : \ vgs-full : \ lvs-full : \ mke2fs /dev/uncrypted/root label:LV_ROOT fstype:ext4 blocksize:1024 : \ mke2fs /dev/uncrypted/work label:LV_WORK fstype:ext4 blocksize:1024 : \ mke2fs /dev/crypted/home label:LV_HOME fstype:ext4 blocksize:1024 : \ mke2fs /dev/crypted/mail label:LV_MAIL fstype:ext4 blocksize:1024 : \ part-set-bootable ${diskname_inside_vm} 2 true : \ list-filesystems : \ swapon-label SWAP : \ mount-options discard ${diskname_inside_vm}2 / : \ set-verbose false : \ copy-in `echo *` / : \ set-verbose true : \ command /sbin/ldconfig : \ cat /etc/resolv.conf : \ command "ip a" : \ command "curl google.com" : \ command "zypper help" : \ command "zypper -v -v ar -c -K -f ${final_repo} tmp" : \ sh "(set -x -e ; z_in='zypper -v -v --gpg-auto-import-keys --no-gpg-checks --non-interactive in --auto-agree-with-licenses --no-recommends' ; \$z_in -t pattern ${guest_zypper_in__pattern_name} ; chkstat --set /etc/permissions /etc/permissions.easy ; echo root:${guest_root_password} | chpasswd ; \$z_in `eval echo ${guest_zypper_in__package_list}` ) 2>&1 " : \ sh "depmod -a \$(get_kernel_version /boot/vmlinuz) ; mkinitrd -B" : \ sh "dd if=/usr/lib/boot/MBR of=${diskname_inside_vm}" : \ sh "cp --verbose --sparse=never --remove-destination --target-directory=/boot/grub /usr/lib/grub/*" : \ sh "grub --batch --verbose < /etc/grub.conf" : \ sh "echo crypt_part4 ${diskname_inside_vm}4 none luks,timeout=0 >> /etc/crypttab" : \ mkdir /LV_ROOT : \ sh "echo LABEL=LV_ROOT /LV_ROOT ext4 noatime 1 2 >> /etc/fstab" : \ mkdir /LV_WORK : \ sh "echo LABEL=LV_WORK /LV_WORK ext4 noatime 1 2 >> /etc/fstab" : \ mkdir /LV_HOME : \ sh "echo LABEL=LV_HOME /LV_HOME ext4 noatime 1 2 >> /etc/fstab" : \ mkdir /LV_MAIL : \ sh "echo LABEL=LV_MAIL /LV_MAIL ext4 noatime 1 2 >> /etc/fstab" : \ sh "echo BOOTPROTO='dhcp' >> /etc/sysconfig/network/ifcfg-eth0" : \ sh "echo STARTMODE='auto' >> /etc/sysconfig/network/ifcfg-eth0" : \ sh "echo 'Password for User root is: ${guest_root_password}' >> /etc/issue" : \ cat /etc/fstab : \ quit ls -lhsS "${output_diskimage}" : ${diskname_inside_vm} case "${diskname_inside_vm}" in *vda*) qemu_drive_options=" -drive file=${output_diskimage},cache=writeback,id=hd0,if=none \ -device virtio-blk-pci,drive=hd0 \ " ;; *sda*) qemu_drive_options=" -device virtio-scsi-pci,id=scsi \ -drive file=${output_diskimage},cache=unsafe,format=raw,id=hd0,if=none \ -device scsi-hd,drive=hd0 \ " ;; *) echo "${diskname_inside_vm} not handled" _exit esac $kvm -enable-kvm \ -global virtio-blk-pci.scsi=off \ -enable-fips \ -machine accel=kvm:tcg \ -cpu host,+kvmclock \ -m 500 \ -no-reboot \ -no-hpet \ ${qemu_drive_options} \ -device virtio-serial-pci \ -serial stdio \ -device sga \ -netdev user,id=usernet,net=169.254.0.0/16 \ -device virtio-net-pci,netdev=usernet exit 0 ++++++ libguestfs.test.simple.create-opensuse-guest.sh ++++++ #!/bin/bash # Create an openSUSE image with just enough packages to allow boot to login prompt # # Theory of operation: # This script uses zypper from the host to resolve dependencies # for zypper which runs within the appliance. If zypper on the host # is too old, it will be unable to handle repo data from 13.1: # http://lists.opensuse.org/zypp-devel/2013-11/msg00000.html # "[zypp-devel] Package conflicting with itself" # For this reason zypper from 12.3 can be used to install the pattern # of the final repo. # First the dependencies of zypper are resolved, the required packages # are downloaded and extracted with unrpm. Now the guest is started and # the partitions in the diskimage are prepared. Then the extracted # package content is copied into the guest. Once that is done zypper # inside the guest will install the base pattern and a few extra packages. # Finally the bootloader grub is configured. Once all that is done # kvm is started. If all goes well a login prompt appears. # The password for root is "root". # The guest has also network access to the outside. # # Expected runtime: ca. 180 seconds # # Expected output: # guest should start # no "obvious" errors should be shown during the disk operation # at the end kvm is started with the generated disk image # login should be possible # set -e unset LANG unset ${!LC_*} cpus=`grep -Ec 'cpu[0-9]' /proc/stat || echo 1` output_diskimage=/dev/shm/$LOGNAME/testcase.img final_repo=http://download.opensuse.org/distribution/13.1/repo/oss/ initial_repo=http://download.opensuse.org/distribution/12.3/repo/oss/ force=false guest_zypper_in__pattern_name="base" guest_zypper_in__package_list=" grub less master-boot-code nfs-utils parted vim " guest_root_password="root" diskname_inside_vm=/dev/sda case "$0" in /*) progname="$0" ;; *) progname="$PWD/$0" ;; esac _exit() { echo "Exiting '$0 $*'." exit 1 } _unrpm() { CPIO_OPTS="--extract --unconditional --preserve-modification-time --make-directories" FILES="$@" for f in $FILES; do echo -ne "$f:\t" rpm2cpio $f | cpio ${CPIO_OPTS} done } until test $# -lt 1 do case "$1" in --unrpm) shift ; _unrpm "$@" ; exit 0 ;; -n) diskname_inside_vm="$2" ; shift ;; -o) output_diskimage="$2" ; shift ;; -R) initial_repo="$2" ; shift ;; -r) final_repo="$2" ; shift ;; -f) force=true ;; -x) set -x ;; *) echo "Unknown option '$1'" ; exit 1 ;; esac shift done if test -z "${initial_repo}" then echo "URL to initial repo required. Wrong -R option." _exit fi if test -z "${final_repo}" then echo "URL to final repo required. Wrong -r option." _exit fi if test -z "${output_diskimage}" then echo "Filename for temporary disk image required. Wrong -o option." _exit fi if test -e "${output_diskimage}" then if test "${force}" = "false" then echo "Output diskimage '${output_diskimage}' exists." echo "It will not be overwritten. Option '-f' exists to force overwrite." _exit fi fi zypper --version cpio --version guestfish --version kvm="qemu-system-`uname -m`" if $kvm --version then : good else kvm="qemu-kvm" if $kvm --version then : else echo "No qemu-kvm found." _exit fi fi mkdir -vp "${output_diskimage%/*}" td=`mktemp -d --tmpdir=/dev/shm/${LOGNAME}` tf=`mktemp --tmpdir=/dev/shm/${LOGNAME}` _exit() { rm -rf "$tf" rm -rf "$td" } trap _exit EXIT dir_repo=${td}/repos.d dir_root=${td}/root dir_cache=${td}/cache mkdir -vp \ ${dir_root} \ ${dir_cache} \ ${dir_repo} cat > ${tf} <<EOF [main] reposdir = ${dir_repo} EOF cat > ${dir_repo}/tmp.repo <<EOF [tmp] name=tmp enabled=1 autorefresh=1 keeppackages=0 baseurl=${initial_repo} EOF packages=" curl zypper " head ${dir_repo}/tmp.repo ${tf} zypper \ --verbose \ --verbose \ --config ${tf} \ --root ${dir_root} \ --reposd-dir ${dir_repo} \ --cache-dir ${dir_cache} \ --gpg-auto-import-keys \ --no-gpg-checks \ --non-interactive \ lr -d zypper \ --verbose \ --verbose \ --config ${tf} \ --root ${dir_root} \ --reposd-dir ${dir_repo} \ --cache-dir ${dir_cache} \ --gpg-auto-import-keys \ --no-gpg-checks \ --non-interactive \ install \ --auto-agree-with-licenses \ --no-recommends \ --dry-run \ --download-only \ ${packages} cd ${dir_root} find ${dir_cache} -xdev -name "*.rpm" -print0 | sort -z | xargs -0 -n 1 -P ${cpus} bash "${progname}" --unrpm mkdir -vp etc/zypp/repos.d grep -w search /etc/resolv.conf >> etc/resolv.conf echo nameserver 169.254.2.3 >> etc/resolv.conf grep -w root /etc/passwd > etc/passwd grep -w root /etc/group > etc/group echo 'root::15209::::::' > etc/shadow cat > etc/fstab <<EOF LABEL=SWAP swap swap defaults 0 0 LABEL=ROOT / ext4 noatime 1 2 EOF mkdir -p boot/grub cat > etc/grub.conf <<EOF setup --stage2=/boot/grub/stage2 --force-lba (hd0,1) (hd0,1) quit EOF echo "(hd0) ${diskname_inside_vm}" > boot/grub/device.map cat > boot/grub/menu.lst <<EOF serial --unit=0 --speed=115200 terminal --timeout=10 console serial title ${0} $* kernel /boot/vmlinuz panic=9 quiet video=800x600 initrd /boot/initrd EOF du -sm . find ${dir_cache} -xdev -name "*.rpm" -delete guestfish \ -x \ \ sparse ${output_diskimage} 1024M : \ set-smp 2 : \ set-memsize 1024 : \ set-network true : \ run : \ list-devices : \ part-init ${diskname_inside_vm} mbr : \ part-add ${diskname_inside_vm} primary 1 $(( ((1024*1024)* 256)/512 - 1)) : \ part-add ${diskname_inside_vm} primary $(( ((1024*1024)* 256)/512 )) $(( ((1024*1024)*1024)/512 - 1)) : \ part-list ${diskname_inside_vm} : \ mkswap-opts ${diskname_inside_vm}1 label:SWAP : \ mke2fs ${diskname_inside_vm}2 label:ROOT fstype:ext4 blocksize:1024 : \ list-devices : \ list-partitions : \ part-set-bootable ${diskname_inside_vm} 2 true : \ list-filesystems : \ swapon-label SWAP : \ mount-options discard ${diskname_inside_vm}2 / : \ set-verbose false : \ copy-in `echo *` / : \ set-verbose true : \ command /sbin/ldconfig : \ cat /etc/resolv.conf : \ command "ip a" : \ command "curl google.com" : \ command "zypper help" : \ command "zypper -v -v ar -c -K -f ${final_repo} tmp" : \ sh "(set -x -e ; z_in='zypper -v -v --gpg-auto-import-keys --no-gpg-checks --non-interactive in --auto-agree-with-licenses --no-recommends' ; \$z_in -t pattern ${guest_zypper_in__pattern_name} ; chkstat --set /etc/permissions /etc/permissions.easy ; echo root:${guest_root_password} | chpasswd ; \$z_in `eval echo ${guest_zypper_in__package_list}` ) 2>&1 " : \ sh "depmod -a \$(get_kernel_version /boot/vmlinuz) ; mkinitrd -B" : \ sh "dd if=/usr/lib/boot/MBR of=${diskname_inside_vm}" : \ sh "cp --verbose --sparse=never --remove-destination --target-directory=/boot/grub /usr/lib/grub/*" : \ sh "grub --batch --verbose < /etc/grub.conf" : \ sh "echo BOOTPROTO='dhcp' >> /etc/sysconfig/network/ifcfg-eth0" : \ sh "echo STARTMODE='auto' >> /etc/sysconfig/network/ifcfg-eth0" : \ sh "echo 'Password for User root is: ${guest_root_password}' >> /etc/issue" : \ cat /etc/fstab : \ quit ls -lhsS "${output_diskimage}" : ${diskname_inside_vm} case "${diskname_inside_vm}" in *vda*) qemu_drive_options=" -drive file=${output_diskimage},cache=writeback,id=hd0,if=none \ -device virtio-blk-pci,drive=hd0 \ " ;; *sda*) qemu_drive_options=" -device virtio-scsi-pci,id=scsi \ -drive file=${output_diskimage},cache=unsafe,format=raw,id=hd0,if=none \ -device scsi-hd,drive=hd0 \ " ;; *) echo "${diskname_inside_vm} not handled" _exit esac $kvm -enable-kvm \ -global virtio-blk-pci.scsi=off \ -enable-fips \ -machine accel=kvm:tcg \ -cpu host,+kvmclock \ -m 500 \ -no-reboot \ -no-hpet \ ${qemu_drive_options} \ -device virtio-serial-pci \ -serial stdio \ -device sga \ -netdev user,id=usernet,net=169.254.0.0/16 \ -device virtio-net-pci,netdev=usernet exit 0 ++++++ libguestfs.test.simple.create-sles12-guest-crypt-on-lvm.sh ++++++ #!/bin/bash # Create an openSUSE image with lvm on dm-crypt partition # # Theory of operation: # This script uses zypper from the host to resolve dependencies # for zypper which runs within the appliance. If zypper on the host # is too old, it will be unable to handle repo data from 13.1: # http://lists.opensuse.org/zypp-devel/2013-11/msg00000.html # "[zypp-devel] Package conflicting with itself" # For this reason zypper from 12.3 can be used to install the pattern # of the final repo. # First the dependencies of zypper are resolved, the required packages # are downloaded and extracted with unrpm. Now the guest is started and # the partitions in the diskimage are prepared. Then the extracted # package content is copied into the guest. Once that is done zypper # inside the guest will install the base pattern and a few extra packages. # Finally the bootloader grub is configured. Once all that is done # kvm is started. If all goes well a login prompt appears. # The password for the crypted partition is "123456". # The password for root is "root". # The guest has also network access to the outside. # # Expected runtime: ca. 200 seconds # Requires at least 1.24.5 because this includes the required crypt modules # # Expected output: # guest should start # no "obvious" errors should be shown during the disk operation # at the end kvm is started with the generated disk image # login should be possible # set -e unset LANG unset ${!LC_*} cpus=`grep -Ec 'cpu[0-9]' /proc/stat || echo 1` output_diskimage=/dev/shm/$LOGNAME/testcase.img final_repo=http://dist.suse.de/install/SLP/SLE-12-Server-LATEST/x86_64/DVD1 initial_repo=http://dist.suse.de/install/SLP/SLE-12-Server-LATEST/x86_64/DVD1 force=false guest_zypper_in__pattern_name="base" guest_zypper_in__package_list=" grub2 kernel-default less master-boot-code nfs-utils parted vim " guest_root_password="root" guest_crypt_password="123456" diskname_inside_vm=/dev/sda case "$0" in /*) progname="$0" ;; *) progname="$PWD/$0" ;; esac _exit() { echo "Exiting '$0 $*'." exit 1 } _unrpm() { CPIO_OPTS="--extract --unconditional --preserve-modification-time --make-directories" FILES="$@" for f in $FILES; do echo -ne "$f:\t" rpm2cpio $f | cpio ${CPIO_OPTS} done } until test $# -lt 1 do case "$1" in --unrpm) shift ; _unrpm "$@" ; exit 0 ;; -n) diskname_inside_vm="$2" ; shift ;; -o) output_diskimage="$2" ; shift ;; -R) initial_repo="$2" ; shift ;; -r) final_repo="$2" ; shift ;; -f) force=true ;; -x) set -x ;; *) echo "Unknown option '$1'" ; exit 1 ;; esac shift done if test -z "${initial_repo}" then echo "URL to initial repo required. Wrong -R option." _exit fi if test -z "${final_repo}" then echo "URL to final repo required. Wrong -r option." _exit fi if test -z "${output_diskimage}" then echo "Filename for temporary disk image required. Wrong -o option." _exit fi if test -e "${output_diskimage}" then if test "${force}" = "false" then echo "Output diskimage '${output_diskimage}' exists." echo "It will not be overwritten. Option '-f' exists to force overwrite." _exit fi fi zypper --version cpio --version guestfish --version kvm="qemu-system-`uname -m`" if $kvm --version then : good else kvm="qemu-kvm" if $kvm --version then : else echo "No qemu-kvm found." _exit fi fi guestfish_version="`guestfish --version | awk '{print \$2}'`" case "${guestfish_version}" in 1.20*) _exit ;; 1.21*) _exit ;; 1.22*) _exit ;; 1.23*) _exit ;; 1.24.[0-4]) _exit ;; *) ;; esac mkdir -vp "${output_diskimage%/*}" td=`mktemp -d --tmpdir=/dev/shm/${LOGNAME}` tf=`mktemp --tmpdir=/dev/shm/${LOGNAME}` _exit() { rm -rf "$tf" rm -rf "$td" } trap _exit EXIT dir_repo=${td}/repos.d dir_root=${td}/root dir_cache=${td}/cache mkdir -vp \ ${dir_root} \ ${dir_cache} \ ${dir_repo} cat > ${tf} <<EOF [main] reposdir = ${dir_repo} EOF cat > ${dir_repo}/tmp.repo <<EOF [tmp] name=tmp enabled=1 autorefresh=1 keeppackages=0 baseurl=${initial_repo} EOF packages=" curl iproute2 zypper " head ${dir_repo}/tmp.repo ${tf} zypper \ --verbose \ --verbose \ --config ${tf} \ --root ${dir_root} \ --reposd-dir ${dir_repo} \ --cache-dir ${dir_cache} \ --gpg-auto-import-keys \ --no-gpg-checks \ --non-interactive \ lr -d zypper \ --verbose \ --verbose \ --config ${tf} \ --root ${dir_root} \ --reposd-dir ${dir_repo} \ --cache-dir ${dir_cache} \ --gpg-auto-import-keys \ --no-gpg-checks \ --non-interactive \ install \ --auto-agree-with-licenses \ --no-recommends \ --dry-run \ --download-only \ ${packages} cd ${dir_root} find ${dir_cache} -xdev -name "*.rpm" -print0 | sort -z | xargs -0 -n 1 -P ${cpus} bash "${progname}" --unrpm mkdir -vp etc/zypp/repos.d grep -w search /etc/resolv.conf >> etc/resolv.conf echo nameserver 169.254.2.3 >> etc/resolv.conf grep -w root /etc/passwd > etc/passwd grep -w root /etc/group > etc/group echo 'root::15209::::::' > etc/shadow cat > etc/fstab <<EOF LABEL=SWAP swap swap defaults 0 0 LABEL=ROOT / ext4 noatime 1 2 EOF du -sm . find ${dir_cache} -xdev -name "*.rpm" -delete ( echo "${guest_crypt_password}" echo "${guest_crypt_password}" ) | \ guestfish \ -x \ --keys-from-stdin \ \ sparse ${output_diskimage} 2048M : \ set-smp 2 : \ set-memsize 1024 : \ set-network true : \ run : \ list-devices : \ part-init ${diskname_inside_vm} mbr : \ part-add ${diskname_inside_vm} primary 1 $(( ((1024*1024)* 64)/512 - 1)) : \ part-add ${diskname_inside_vm} primary $(( ((1024*1024)* 64)/512 )) $(( ((1024*1024)*1024)/512 - 1)) : \ part-add ${diskname_inside_vm} primary $(( ((1024*1024)*1024)/512 )) $(( ((1024*1024)*1555)/512 - 1)) : \ part-add ${diskname_inside_vm} primary $(( ((1024*1024)*1555)/512 )) $(( ((1024*1024)*2024)/512 - 1)) : \ part-list ${diskname_inside_vm} : \ mkswap-opts ${diskname_inside_vm}1 label:SWAP : \ mke2fs ${diskname_inside_vm}2 label:ROOT fstype:ext4 blocksize:1024 : \ pvcreate ${diskname_inside_vm}3 : \ vgcreate uncrypted ${diskname_inside_vm}3 : \ luks-format ${diskname_inside_vm}4 0 : \ luks-open ${diskname_inside_vm}4 crypt_part4 : \ pvcreate /dev/mapper/crypt_part4 : \ vgcreate crypted /dev/mapper/crypt_part4 : \ lvcreate-free root uncrypted 50 : \ lvcreate-free work uncrypted 50 : \ lvcreate-free home crypted 50 : \ lvcreate-free mail crypted 50 : \ list-devices : \ list-partitions : \ pvs-full : \ vgs-full : \ lvs-full : \ mke2fs /dev/uncrypted/root label:LV_ROOT fstype:ext4 blocksize:1024 : \ mke2fs /dev/uncrypted/work label:LV_WORK fstype:ext4 blocksize:1024 : \ mke2fs /dev/crypted/home label:LV_HOME fstype:ext4 blocksize:1024 : \ mke2fs /dev/crypted/mail label:LV_MAIL fstype:ext4 blocksize:1024 : \ part-set-bootable ${diskname_inside_vm} 2 true : \ list-filesystems : \ swapon-label SWAP : \ mount-options discard ${diskname_inside_vm}2 / : \ set-verbose false : \ copy-in `echo *` / : \ set-verbose true : \ command /sbin/ldconfig : \ cat /etc/resolv.conf : \ command "ip a" : \ command "curl google.com" : \ command "zypper help" : \ command "zypper -v -v ar -c -K -f ${final_repo} tmp" : \ sh "(set -x -e ; z_in='zypper -v -v --gpg-auto-import-keys --no-gpg-checks --non-interactive in --auto-agree-with-licenses --no-recommends' ; \$z_in -t pattern ${guest_zypper_in__pattern_name} ; chkstat --set /etc/permissions /etc/permissions.easy ; echo root:${guest_root_password} | chpasswd ; \$z_in `eval echo ${guest_zypper_in__package_list}` ) 2>&1 " : \ sh "depmod -a \$(get_kernel_version /boot/vmlinuz) ; mkinitrd -B" : \ sh "dd if=/usr/lib/boot/MBR of=${diskname_inside_vm}" : \ sh "echo GRUB_DISABLE_OS_PROBER=true >> /etc/default/grub " : \ sh "echo GRUB_DISABLE_LINUX_RECOVERY=true >> /etc/default/grub " : \ sh "echo GRUB_CMDLINE_LINUX_DEFAULT=\'quiet panic=9 video=800x600 \' >> /etc/default/grub " : \ sh "grub2-mkconfig > /boot/grub2/grub.cfg " : \ sh "grub2-install --force --verbose ${diskname_inside_vm}2 " : \ sh "echo crypt_part4 ${diskname_inside_vm}4 none luks,timeout=0 >> /etc/crypttab" : \ mkdir /LV_ROOT : \ sh "echo LABEL=LV_ROOT /LV_ROOT ext4 noatime 1 2 >> /etc/fstab" : \ mkdir /LV_WORK : \ sh "echo LABEL=LV_WORK /LV_WORK ext4 noatime 1 2 >> /etc/fstab" : \ mkdir /LV_HOME : \ sh "echo LABEL=LV_HOME /LV_HOME ext4 noatime 1 2 >> /etc/fstab" : \ mkdir /LV_MAIL : \ sh "echo LABEL=LV_MAIL /LV_MAIL ext4 noatime 1 2 >> /etc/fstab" : \ sh "echo BOOTPROTO='dhcp' >> /etc/sysconfig/network/ifcfg-eth0" : \ sh "echo STARTMODE='auto' >> /etc/sysconfig/network/ifcfg-eth0" : \ sh "echo 'Password for User root is: ${guest_root_password}' >> /etc/issue" : \ sh "echo >> /etc/issue" : \ cat /etc/fstab : \ quit ls -lhsS "${output_diskimage}" : ${diskname_inside_vm} case "${diskname_inside_vm}" in *vda*) qemu_drive_options=" -drive file=${output_diskimage},cache=writeback,id=hd0,if=none \ -device virtio-blk-pci,drive=hd0 \ " ;; *sda*) qemu_drive_options=" -device virtio-scsi-pci,id=scsi \ -drive file=${output_diskimage},cache=unsafe,format=raw,id=hd0,if=none \ -device scsi-hd,drive=hd0 \ " ;; *) echo "${diskname_inside_vm} not handled" _exit esac $kvm -enable-kvm \ -global virtio-blk-pci.scsi=off \ -enable-fips \ -machine accel=kvm:tcg \ -cpu host,+kvmclock \ -m 500 \ -no-reboot \ -no-hpet \ ${qemu_drive_options} \ -device virtio-serial-pci \ -serial stdio \ -device VGA \ -netdev user,id=usernet,net=169.254.0.0/16 \ -device virtio-net-pci,netdev=usernet exit 0 ++++++ libguestfs.test.simple.create-sles12-guest.sh ++++++ #!/bin/bash # Create an openSUSE image with just enough packages to allow boot to login prompt # # Theory of operation: # This script uses zypper from the host to resolve dependencies # for zypper which runs within the appliance. If zypper on the host # is too old, it will be unable to handle repo data from 13.1: # http://lists.opensuse.org/zypp-devel/2013-11/msg00000.html # "[zypp-devel] Package conflicting with itself" # For this reason zypper from 12.3 can be used to install the pattern # of the final repo. # First the dependencies of zypper are resolved, the required packages # are downloaded and extracted with unrpm. Now the guest is started and # the partitions in the diskimage are prepared. Then the extracted # package content is copied into the guest. Once that is done zypper # inside the guest will install the base pattern and a few extra packages. # Finally the bootloader grub is configured. Once all that is done # kvm is started. If all goes well a login prompt appears. # The password for root is "root". # The guest has also network access to the outside. # # Expected runtime: ca. 180 seconds # # Expected output: # guest should start # no "obvious" errors should be shown during the disk operation # at the end kvm is started with the generated disk image # login should be possible # set -e unset LANG unset ${!LC_*} cpus=`grep -Ec 'cpu[0-9]' /proc/stat || echo 1` output_diskimage=/dev/shm/$LOGNAME/testcase.img final_repo=http://dist.suse.de/install/SLP/SLE-12-Server-LATEST/x86_64/DVD1 initial_repo=http://dist.suse.de/install/SLP/SLE-12-Server-LATEST/x86_64/DVD1 force=false guest_zypper_in__pattern_name="base" guest_zypper_in__package_list=" grub2 kernel-default less master-boot-code nfs-utils parted vim " guest_root_password="root" diskname_inside_vm=/dev/sda case "$0" in /*) progname="$0" ;; *) progname="$PWD/$0" ;; esac _exit() { echo "Exiting '$0 $*'." exit 1 } _unrpm() { CPIO_OPTS="--extract --unconditional --preserve-modification-time --make-directories" FILES="$@" for f in $FILES; do echo -ne "$f:\t" rpm2cpio $f | cpio ${CPIO_OPTS} done } until test $# -lt 1 do case "$1" in --unrpm) shift ; _unrpm "$@" ; exit 0 ;; -n) diskname_inside_vm="$2" ; shift ;; -o) output_diskimage="$2" ; shift ;; -R) initial_repo="$2" ; shift ;; -r) final_repo="$2" ; shift ;; -f) force=true ;; -x) set -x ;; *) echo "Unknown option '$1'" ; exit 1 ;; esac shift done if test -z "${initial_repo}" then echo "URL to initial repo required. Wrong -R option." _exit fi if test -z "${final_repo}" then echo "URL to final repo required. Wrong -r option." _exit fi if test -z "${output_diskimage}" then echo "Filename for temporary disk image required. Wrong -o option." _exit fi if test -e "${output_diskimage}" then if test "${force}" = "false" then echo "Output diskimage '${output_diskimage}' exists." echo "It will not be overwritten. Option '-f' exists to force overwrite." _exit fi fi zypper --version cpio --version guestfish --version kvm="qemu-system-`uname -m`" if $kvm --version then : good else kvm="qemu-kvm" if $kvm --version then : else echo "No qemu-kvm found." _exit fi fi mkdir -vp "${output_diskimage%/*}" td=`mktemp -d --tmpdir=/dev/shm/${LOGNAME}` tf=`mktemp --tmpdir=/dev/shm/${LOGNAME}` _exit() { rm -rf "$tf" rm -rf "$td" } trap _exit EXIT dir_repo=${td}/repos.d dir_root=${td}/root dir_cache=${td}/cache mkdir -vp \ ${dir_root} \ ${dir_cache} \ ${dir_repo} cat > ${tf} <<EOF [main] reposdir = ${dir_repo} EOF cat > ${dir_repo}/tmp.repo <<EOF [tmp] name=tmp enabled=1 autorefresh=1 keeppackages=0 baseurl=${initial_repo} EOF packages=" curl iproute2 zypper " head ${dir_repo}/tmp.repo ${tf} zypper \ --verbose \ --verbose \ --config ${tf} \ --root ${dir_root} \ --reposd-dir ${dir_repo} \ --cache-dir ${dir_cache} \ --gpg-auto-import-keys \ --no-gpg-checks \ --non-interactive \ lr -d zypper \ --verbose \ --verbose \ --config ${tf} \ --root ${dir_root} \ --reposd-dir ${dir_repo} \ --cache-dir ${dir_cache} \ --gpg-auto-import-keys \ --no-gpg-checks \ --non-interactive \ install \ --auto-agree-with-licenses \ --no-recommends \ --dry-run \ --download-only \ ${packages} cd ${dir_root} find ${dir_cache} -xdev -name "*.rpm" -print0 | sort -z | xargs -0 -n 1 -P ${cpus} bash "${progname}" --unrpm mkdir -vp etc/zypp/repos.d grep -w search /etc/resolv.conf >> etc/resolv.conf echo nameserver 169.254.2.3 >> etc/resolv.conf grep -w root /etc/passwd > etc/passwd grep -w root /etc/group > etc/group echo 'root::15209::::::' > etc/shadow cat > etc/fstab <<EOF LABEL=SWAP swap swap defaults 0 0 LABEL=ROOT / ext4 noatime 1 2 EOF du -sm . find ${dir_cache} -xdev -name "*.rpm" -delete guestfish \ -x \ \ sparse ${output_diskimage} 1024M : \ set-smp 2 : \ set-memsize 1024 : \ set-network true : \ run : \ list-devices : \ part-init ${diskname_inside_vm} mbr : \ part-add ${diskname_inside_vm} primary 1 $(( ((1024*1024)* 64)/512 - 1)) : \ part-add ${diskname_inside_vm} primary $(( ((1024*1024)* 64)/512 )) $(( ((1024*1024)*1024)/512 - 1)) : \ part-list ${diskname_inside_vm} : \ mkswap-opts ${diskname_inside_vm}1 label:SWAP : \ mke2fs ${diskname_inside_vm}2 label:ROOT fstype:ext4 blocksize:1024 : \ list-devices : \ list-partitions : \ part-set-bootable ${diskname_inside_vm} 2 true : \ list-filesystems : \ swapon-label SWAP : \ mount-options noatime ${diskname_inside_vm}2 / : \ set-verbose false : \ copy-in `echo *` / : \ set-verbose true : \ command /sbin/ldconfig : \ cat /etc/resolv.conf : \ command "ip a" : \ command "curl google.com" : \ command "zypper help" : \ command "zypper -v -v ar -c -K -f ${final_repo} tmp" : \ sh "(set -x -e ; z_in='zypper -v -v --gpg-auto-import-keys --no-gpg-checks --non-interactive in --auto-agree-with-licenses --no-recommends ' ; \$z_in -t pattern ${guest_zypper_in__pattern_name} ; chkstat --set /etc/permissions /etc/permissions.easy ; echo root:${guest_root_password} | chpasswd ; \$z_in `eval echo ${guest_zypper_in__package_list}` ) < /dev/null &>/dev/kmsg " : \ sh "depmod -a \$(get_kernel_version /boot/vmlinuz) ; mkinitrd -B" : \ sh "dd if=/usr/lib/boot/MBR of=${diskname_inside_vm}" : \ sh "echo GRUB_DISABLE_OS_PROBER=true >> /etc/default/grub " : \ sh "echo GRUB_DISABLE_LINUX_RECOVERY=true >> /etc/default/grub " : \ sh "echo GRUB_CMDLINE_LINUX_DEFAULT=\'quiet panic=9 video=800x600 \' >> /etc/default/grub " : \ sh "grub2-mkconfig > /boot/grub2/grub.cfg " : \ sh "grub2-install --force --verbose ${diskname_inside_vm}2 " : \ sh "echo BOOTPROTO='dhcp' >> /etc/sysconfig/network/ifcfg-eth0" : \ sh "echo STARTMODE='auto' >> /etc/sysconfig/network/ifcfg-eth0" : \ sh "echo 'Password for User root is: ${guest_root_password}' >> /etc/issue" : \ sh "echo >> /etc/issue" : \ cat /etc/fstab : \ quit ls -lhsS "${output_diskimage}" : ${diskname_inside_vm} case "${diskname_inside_vm}" in *vda*) qemu_drive_options=" -drive file=${output_diskimage},cache=writeback,id=hd0,if=none \ -device virtio-blk-pci,drive=hd0 \ " ;; *sda*) qemu_drive_options=" -device virtio-scsi-pci,id=scsi \ -drive file=${output_diskimage},cache=unsafe,format=raw,id=hd0,if=none \ -device scsi-hd,drive=hd0 \ " ;; *) echo "${diskname_inside_vm} not handled" _exit esac $kvm -enable-kvm \ -global virtio-blk-pci.scsi=off \ -enable-fips \ -machine accel=kvm:tcg \ -cpu host,+kvmclock \ -m 500 \ -no-reboot \ -no-hpet \ ${qemu_drive_options} \ -device virtio-serial-pci \ -serial stdio \ -device VGA \ -netdev user,id=usernet,net=169.254.0.0/16 \ -device virtio-net-pci,netdev=usernet exit 0 ++++++ libguestfs.test.simple.run-libugestfs-test-tool.sh ++++++ #!/bin/bash # libguestfs-test-tool starts its temporary guest using a dummy disk image # It creates a partition, a filesystem, mounts it and touches a file # Once it is done the dummy image is removed again # Per default it runs in --verbose mode, and our version trace the API calls # # Expected runtime: ca. 10 seconds # # Expected output: # guest should start # no "obvious" errors should be shown during the disk operation # Somewhere at the end of the verbose output lines like this are expected: # ... # libguestfs: trace: touch "/hello" # ... # libguestfs: trace: touch = 0 # # set -x set -e unset LANG unset ${!LC_*} cpus=`grep -Ec 'cpu[0-9]' /proc/stat || echo 1` libguestfs-test-tool -V time libguestfs-test-tool ++++++ mount-rootfs-and-chroot.sh ++++++ #!/bin/bash # Usage: $0 /dev/sda5 rootfs=$1 mnt=/sysroot mounts= if test -b "${rootfs}" then mkdir -v -p "${mnt}" if mount -v "${rootfs}" "${mnt}" then for i in dev dev/pts proc sys selinux do if test -d /${i} && test -d "${mnt}/${i}" && test "`stat -c %D /`" != "`stat -c %D ${i}`" then mount -v --bind /${i} "${mnt}/${i}" fi done chroot "${mnt}" su - while read b m rest do case "${m}" in ${mnt}*) mounts="${m} ${mounts}" ;; esac done <<-EOF ` cat < /proc/mounts ` EOF for i in ${mounts} do umount -v "${i}" done fi fi ++++++ netconfig.patch ++++++ diff -up ./appliance/packagelist.in.netconfig ./appliance/packagelist.in --- ./appliance/packagelist.in.netconfig 2019-08-28 19:26:50.290462768 +0100 +++ ./appliance/packagelist.in 2019-08-28 19:27:05.530462557 +0100 @@ -161,6 +161,7 @@ ifelse(SUSE,1, ntfs-3g reiserfs squashfs + sysconfig-netconfig systemd vim xz
