ANNOUNCE: forthcoming retirement of virt-tools mailing list

2023-10-17 Thread Daniel P . Berrangé
Hello Virt Tools Community

For the past 14-ish years this mailing list has served as a place for
accepting patches and having discussions for various virtualization
userspace projects that didn't justify their own dedicated mailing
list. Mostly this was virt-manager and virt-viewer, but also some
other smaller tools.

Unfortunately as a consequence of infrastructure changes, the
redhat.com Mailman service is scheduled to be decomissioned, and
in the very near future ($low weeks) will cease to be able to send /
receive mail for this list.

In recent years development for the virt-viewer and virt-manager projects
has largely shifted over to the git forges, via a combination of merge
requests for patch submission, and issues for bug/feature discussions.
The remaining traffic on this list isn't high enough to justify the setup
of new hosting for its continued usage.

With this in mind, when the redhat.com Mailman service ends, this mailing
list will be retired. For the sake of historical record, the archives will
be preserved.

As a reminder the primary project websites are:

- https://libvirt.org
- https://libguestfs.org
- https://virt-manager.org
- https://libosinfo.org

and their assocaited respositories

- https://gitlab.com/libvirt
- https://gitlab.com/libosinfo
- https://github.com/virt-manager
- https://gitlab.com/virt-viewer
- https://github.com/libguestfs
- https://gitlab.com/nbdkit

Note, not all of the above use a merge/pull request workflow. Check
the individual websites / repositories for guidance.

With regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: Does Xen dom0 count as a VM?

2023-07-19 Thread Daniel P . Berrangé
On Wed, Jul 19, 2023 at 10:12:07AM +, Sergio Gelato wrote:
> Does this project share the view expressed in
> https://github.com/systemd/systemd/issues/28113#issuecomment-1621986461
> that "when it comes to systemd's PoV xen dom0 is not a VM. i.e. the VM
> that owns the hardware is not a VM in our eyes"?

That is correct. Domain 0 in a Xen-like hypervisor is special, it
is the equivalent of the Lnux host OS in KVM world and clearly
should not be considered a VM.

> If so, virt-what has a similar bug to the above-mentioned systemd issue.

There is a code comment in one probing scenario that it cannot
distinguish domu/dom0.

With regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



[PATCH 1/5] virt-what-cvm: check if hypervisor bit is set

2023-06-29 Thread Daniel P . Berrangé
Before doing any probes for a confidential VM, check that the
tool is running under a hypervisor, rather than bare metal

Signed-off-by: Daniel P. Berrangé 
---
 virt-what-cvm.c | 27 +++
 1 file changed, 27 insertions(+)

diff --git a/virt-what-cvm.c b/virt-what-cvm.c
index 407efb4..f184768 100644
--- a/virt-what-cvm.c
+++ b/virt-what-cvm.c
@@ -35,6 +35,9 @@ static bool dodebug = false;
 
 #define debug(...) do { if (dodebug) fprintf(stderr, __VA_ARGS__); } while(0)
 
+
+#define CPUID_PROCESSOR_INFO_AND_FEATURE_BITS 0x1
+
 /*
  * AMD64 Architecture Programmer’s Manual Volume 3:
  * General-Purpose and System Instructions.
@@ -72,6 +75,9 @@ static bool dodebug = false;
 #define CPUID_SIG_INTEL "GenuineIntel"
 #define CPUID_SIG_INTEL_TDX "IntelTDX"
 
+/* ecx bit 31: set => hyperpvisor, unset => bare metal */
+#define CPUID_FEATURE_HYPERVISOR (1 << 31)
+
 /*
  * This TPM NV data format is not explicitly documented anywhere,
  * but the header definition is present in code at:
@@ -335,11 +341,32 @@ cpu_sig_intel (void)
 puts ("intel-tdx");
 }
 
+static bool
+cpu_is_hv (void)
+{
+  uint32_t eax, ebx, ecx, edx;
+  bool is_hv;
+
+  eax = CPUID_PROCESSOR_INFO_AND_FEATURE_BITS;
+  ebx = ecx = edx = 0;
+
+  cpuid(, , , );
+
+  is_hv = ecx & CPUID_FEATURE_HYPERVISOR;
+
+  debug ("CPUID is hypervisor: %s\n", is_hv ? "yes" : "no");
+  return is_hv;
+}
+
 static void
 cpu_sig (void)
 {
   char sig[13];
 
+  /* Skip everything on bare metal */
+  if (!cpu_is_hv ())
+return;
+
   memset (sig, 0, sizeof sig);
   cpuid_leaf (0, sig);
 
-- 
2.40.1



[PATCH 0/5] virt-what-cvm: improve SEV-SNP detection on Azure/HyperV

2023-06-29 Thread Daniel P . Berrangé
My first attempt used a dirty hack of looking at the TPM NV index for
existance of a SEV-SNP attestation report.

I've since learnt of some HyperV specific CPUID leafs that can be used
to detect SEV-SNP in a much simlper/saner manner.

Thus this series drops the TPM code and replaces it with CPUID.

The fact name is also changed to 'hyperv-' instead of 'azure-' since
it is really a property of the hypervisor rather than cloud service.

Daniel P. Berrangé (5):
  virt-what-cvm: check if hypervisor bit is set
  virt-what-cvm: support alternative cpuid leaf ordering
  virt-what-cvm: probe for SNP/HCL on HyperV/Azure via CPUID
  virt-what-cvm: drop TPM logic for detecting SNP on HyperV/Azure
  virt-what-cvm: rename 'azure-hcl' fact to 'hyperv-hcl'

 Makefile.am   |   3 -
 configure.ac  |   2 -
 virt-what-cvm.c   | 231 +-
 virt-what-cvm.pod |   6 +-
 4 files changed, 88 insertions(+), 154 deletions(-)

-- 
2.40.1



[PATCH 4/5] virt-what-cvm: drop TPM logic for detecting SNP on HyperV/Azure

2023-06-29 Thread Daniel P . Berrangé
Now we have proper CPUID detection, we no longer need the TPM
hacks.

Signed-off-by: Daniel P. Berrangé 
---
 Makefile.am |   3 -
 configure.ac|   2 -
 virt-what-cvm.c | 161 ++--
 3 files changed, 6 insertions(+), 160 deletions(-)

diff --git a/Makefile.am b/Makefile.am
index 2050bef..b68540f 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -30,9 +30,6 @@ if HOST_CPU_IA64
 libexec_PROGRAMS += virt-what-ia64-xen-rdtsc-test
 endif
 
-virt_what_cvm_LDADD = $(TPM2_TSS_LIBS)
-virt_what_cvm_CFLAGS = $(TPM2_TSS_CFLAGS)
-
 if HAVE_POD2MAN
 
 CLEANFILES += virt-what.1 virt-what-cvm.1 virt-what.txt virt-what-cvm.txt
diff --git a/configure.ac b/configure.ac
index 4418483..ab8eb75 100644
--- a/configure.ac
+++ b/configure.ac
@@ -32,8 +32,6 @@ dnl Architecture we are compiling for.
 AC_CANONICAL_HOST
 AM_CONDITIONAL([HOST_CPU_IA64], [ test "x$host_cpu" = "xia64" ])
 
-PKG_HAVE_DEFINE_WITH_MODULES(TPM2_TSS, tss2-esys, [tpm2-tss package])
-
 
 dnl List of tests.
 tests="\
diff --git a/virt-what-cvm.c b/virt-what-cvm.c
index a7a224f..8b8a4df 100644
--- a/virt-what-cvm.c
+++ b/virt-what-cvm.c
@@ -26,10 +26,6 @@
 #include 
 #include 
 #include 
-#ifdef HAVE_TPM2_TSS
-#include 
-#include 
-#endif
 
 static bool dodebug = false;
 
@@ -97,121 +93,8 @@ static bool dodebug = false;
 #define CPUID_HYPERV_ISOLATION_TYPE_MASK 0xf
 #define CPUID_HYPERV_ISOLATION_TYPE_SNP 2
 
-/*
- * This TPM NV data format is not explicitly documented anywhere,
- * but the header definition is present in code at:
- *
- * 
https://github.com/kinvolk/azure-cvm-tooling/blob/main/az-snp-vtpm/src/hcl.rs
- */
-#define TPM_AZURE_HCLA_REPORT_INDEX 0x0141
-
-struct TPMAzureHCLAHeader {
-  uint32_t signature;
-  uint32_t version;
-  uint32_t report_len;
-  uint32_t report_type;
-  uint32_t unknown[4];
-};
-
-/* The bytes for "HCLA" */
-#define TPM_AZURE_HCLA_SIGNATURE 0x414C4348
-#define TPM_AZURE_HCLA_VERSION 0x1
-#define TPM_AZURE_HCLA_REPORT_TYPE_SNP 0x2
-
 #if defined(__x86_64__)
 
-#ifdef HAVE_TPM2_TSS
-static char *
-tpm_nvread(uint32_t nvindex, size_t *retlen)
-{
-  TSS2_RC rc;
-  ESYS_CONTEXT *ctx = NULL;
-  ESYS_TR primary = ESYS_TR_NONE;
-  ESYS_TR session = ESYS_TR_NONE;
-  ESYS_TR nvobj = ESYS_TR_NONE;
-  TPM2B_NV_PUBLIC *pubData = NULL;
-  TPMT_SYM_DEF sym = {
-.algorithm = TPM2_ALG_AES,
-.keyBits = { .aes = 128 },
-.mode = { .aes = TPM2_ALG_CFB }
-  };
-  char *ret;
-  size_t retwant;
-
-  rc = Esys_Initialize(, NULL, NULL);
-  if (rc != TSS2_RC_SUCCESS)
-return NULL;
-
-  rc = Esys_Startup(ctx, TPM2_SU_CLEAR);
-  debug("tpm startup %d\n", rc);
-  if (rc != TSS2_RC_SUCCESS)
-goto error;
-
-  rc = Esys_StartAuthSession(ctx, ESYS_TR_NONE, ESYS_TR_NONE,
-ESYS_TR_NONE, ESYS_TR_NONE, ESYS_TR_NONE,
-NULL, 0,
-, TPM2_ALG_SHA256, );
-  debug("tpm auth session %d\n", rc);
-  if (rc != TSS2_RC_SUCCESS)
-goto error;
-
-  rc = Esys_TR_FromTPMPublic(ctx, nvindex, ESYS_TR_NONE,
-ESYS_TR_NONE, ESYS_TR_NONE, );
-  debug("tpm from public %d\n", rc);
-  if (rc != TSS2_RC_SUCCESS)
-goto error;
-
-  rc = Esys_NV_ReadPublic(ctx, nvobj, ESYS_TR_NONE,
- ESYS_TR_NONE, ESYS_TR_NONE,
- , NULL);
-  debug("tpm read public %d\n", rc);
-  if (rc != TPM2_RC_SUCCESS)
-goto error;
-
-  retwant = pubData->nvPublic.dataSize;
-  free(pubData);
-  *retlen = 0;
-  ret = malloc(retwant);
-  assert(ret);
-  while (*retlen < retwant) {
-size_t want = retwant - *retlen;
-TPM2B_MAX_NV_BUFFER *data = NULL;
-if (want > 1024)
-  want = 1024;
-rc = Esys_NV_Read(ctx,  ESYS_TR_RH_OWNER, nvobj, session, ESYS_TR_NONE, 
ESYS_TR_NONE,
- want, *retlen, );
-debug("tpm nv read %d\n", rc);
-if (rc != TPM2_RC_SUCCESS) {
-  free(ret);
-  goto error;
-}
-
-memcpy(ret + *retlen, data->buffer, data->size);
-*retlen += data->size;
-free(data);
-  }
-
-  return ret;
-
- error:
-  if (nvobj != ESYS_TR_NONE)
-Esys_FlushContext(ctx, nvobj);
-  if (session != ESYS_TR_NONE)
-Esys_FlushContext(ctx, session);
-  if (primary != ESYS_TR_NONE)
-Esys_FlushContext(ctx, primary);
-  Esys_Finalize();
-  *retlen = 0;
-  return NULL;
-}
-#else /* ! HAVE_TPM2_TSS */
-static char *
-tpm_nvread(uint32_t nvindex, size_t *retlen)
-{
-  return NULL;
-}
-#endif /* ! HAVE_TPM2_TSS */
-
 /* Copied from the Linux kernel definition in
  * arch/x86/include/asm/processor.h
  */
@@ -263,34 +146,6 @@ msr (off_t index)
   return ret;
 }
 
-bool
-cpu_sig_amd_azure (void)
-{
-  size_t datalen = 0;
-  char *data = tpm_nvread(TPM_AZURE_HCLA_REPORT_INDEX, );
-  struct TPMAzureHCLAHeader *header = (struct TPMAzureHCLAHeader *)data;
-  bool ret;
-
-  if (!data)
-return false;
-
-  if (datalen < sizeof(struct TPM

[PATCH 5/5] virt-what-cvm: rename 'azure-hcl' fact to 'hyperv-hcl'

2023-06-29 Thread Daniel P . Berrangé
Azure is a cloud service that uses the HyperV platform, so we
should refer to the fact as 'hyperv-hcl', not 'azure-hcl'.

Signed-off-by: Daniel P. Berrangé 
---
 virt-what-cvm.c   | 2 +-
 virt-what-cvm.pod | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/virt-what-cvm.c b/virt-what-cvm.c
index 8b8a4df..52b3426 100644
--- a/virt-what-cvm.c
+++ b/virt-what-cvm.c
@@ -214,7 +214,7 @@ cpu_sig_amd (void)
 
 if (cpu_sig_amd_hyperv ()) {
   puts ("amd-sev-snp");
-  puts ("azure-hcl");
+  puts ("hyperv-hcl");
 } else {
   debug("No hyperv CPUID\n");
 }
diff --git a/virt-what-cvm.pod b/virt-what-cvm.pod
index 12cfc6a..0f90765 100644
--- a/virt-what-cvm.pod
+++ b/virt-what-cvm.pod
@@ -52,11 +52,11 @@ This is a confidential guest running with Intel TDX 
technology
 
 Status: tested on Microsoft Azure TDX CVM (preview)
 
-=item B
+=item B
 
 This is a confidential guest running unenlightened under the
-Azure HCL (Host Compatibility Layer). This will be paired with
-B.
+HyperV (Azure) HCL (Host Compatibility Layer). This will be
+paired with B.
 
 Status: tested on Microsoft Azure SEV-SNP CVM
 
-- 
2.40.1



[PATCH 2/5] virt-what-cvm: support alternative cpuid leaf ordering

2023-06-29 Thread Daniel P . Berrangé
The HyperV CPUID leaf for reporting the vendor string has an
alternative ordering of ecx/edx.

Signed-off-by: Daniel P. Berrangé 
---
 virt-what-cvm.c | 11 +++
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/virt-what-cvm.c b/virt-what-cvm.c
index f184768..1e7c50b 100644
--- a/virt-what-cvm.c
+++ b/virt-what-cvm.c
@@ -209,11 +209,14 @@ cpuid (uint32_t *eax, uint32_t *ebx, uint32_t *ecx, 
uint32_t *edx)
 
 
 static uint32_t
-cpuid_leaf (uint32_t eax, char *sig)
+cpuid_leaf (uint32_t eax, char *sig, bool swapped)
 {
   uint32_t *sig32 = (uint32_t *) sig;
 
-  cpuid (, [0], [2], [1]);
+  if (swapped)
+cpuid (, [0], [2], [1]);
+  else
+cpuid (, [0], [1], [2]);
   sig[12] = 0; /* \0-terminate the string to make string comparison possible */
   debug("CPUID sig %s\n", sig);
   return eax;
@@ -335,7 +338,7 @@ cpu_sig_intel (void)
 return;
 
   memset (sig, 0, sizeof sig);
-  cpuid_leaf (CPUID_INTEL_TDX_ENUMERATION, sig);
+  cpuid_leaf (CPUID_INTEL_TDX_ENUMERATION, sig, true);
 
   if (memcmp (sig, CPUID_SIG_INTEL_TDX, sizeof(sig)) == 0)
 puts ("intel-tdx");
@@ -368,7 +371,7 @@ cpu_sig (void)
 return;
 
   memset (sig, 0, sizeof sig);
-  cpuid_leaf (0, sig);
+  cpuid_leaf (0, sig, true);
 
   if (memcmp (sig, CPUID_SIG_AMD, sizeof(sig)) == 0)
 cpu_sig_amd ();
-- 
2.40.1



[PATCH 3/5] virt-what-cvm: probe for SNP/HCL on HyperV/Azure via CPUID

2023-06-29 Thread Daniel P . Berrangé
When running a confidential VM on Azure (HyperV) we can probe
CPUID leaf 0x4003 to detect if VM isolation is present,
and 0x400c to detect what kind of isolation is used.

Signed-off-by: Daniel P. Berrangé 
---
 virt-what-cvm.c | 62 +++--
 1 file changed, 60 insertions(+), 2 deletions(-)

diff --git a/virt-what-cvm.c b/virt-what-cvm.c
index 1e7c50b..a7a224f 100644
--- a/virt-what-cvm.c
+++ b/virt-what-cvm.c
@@ -70,14 +70,33 @@ static bool dodebug = false;
 
 #define CPUID_INTEL_TDX_ENUMERATION 0x21
 
+/* Requirements for Implementing the Microsoft Hypervisor Interface
+ * 
https://learn.microsoft.com/en-us/virtualization/hyper-v-on-windows/tlfs/tlfs
+ */
+#define CPUID_HYPERV_VENDOR_AND_MAX_FUNCTIONS 0x4000
+
+#define CPUID_HYPERV_FEATURES 0x4003
+
+#define CPUID_HYPERV_ISOLATION_CONFIG 0x400C
+
+#define CPUID_HYPERV_MIN 0x4005
+#define CPUID_HYPERV_MAX 0x4000
 
 #define CPUID_SIG_AMD   "AuthenticAMD"
 #define CPUID_SIG_INTEL "GenuineIntel"
 #define CPUID_SIG_INTEL_TDX "IntelTDX"
+#define CPUID_SIG_HYPERV"Microsoft Hv"
 
 /* ecx bit 31: set => hyperpvisor, unset => bare metal */
 #define CPUID_FEATURE_HYPERVISOR (1 << 31)
 
+/* Linux include/asm-generic/hyperv-tlfs.h */
+#define CPUID_HYPERV_CPU_MANAGEMENT (1 << 12) /* root partition */
+#define CPUID_HYPERV_ISOLATION  (1 << 22) /* confidential VM partition */
+
+#define CPUID_HYPERV_ISOLATION_TYPE_MASK 0xf
+#define CPUID_HYPERV_ISOLATION_TYPE_SNP 2
+
 /*
  * This TPM NV data format is not explicitly documented anywhere,
  * but the header definition is present in code at:
@@ -272,6 +291,44 @@ cpu_sig_amd_azure (void)
   return ret;
 }
 
+static bool
+cpu_sig_amd_hyperv (void)
+{
+  uint32_t eax, ebx, ecx, edx;
+  char sig[13];
+  uint32_t feat;
+
+  feat = cpuid_leaf (CPUID_HYPERV_VENDOR_AND_MAX_FUNCTIONS, sig, false);
+
+  if (feat < CPUID_HYPERV_MIN ||
+  feat > CPUID_HYPERV_MAX)
+return false;
+
+  if (memcmp (sig, CPUID_SIG_HYPERV, sizeof(sig)) != 0)
+return false;
+
+  debug ("CPUID is on hyperv\n");
+  eax = CPUID_HYPERV_FEATURES;
+  ebx = ecx = edx = 0;
+
+  cpuid(, , , );
+
+  if (ebx & CPUID_HYPERV_ISOLATION &&
+  !(ebx & CPUID_HYPERV_CPU_MANAGEMENT)) {
+
+eax = CPUID_HYPERV_ISOLATION_CONFIG;
+ebx = ecx = edx = 0;
+cpuid(, , , );
+
+if ((ebx & CPUID_HYPERV_ISOLATION_TYPE_MASK) ==
+   CPUID_HYPERV_ISOLATION_TYPE_SNP) {
+  return true;
+}
+  }
+
+  return false;
+}
+
 static void
 cpu_sig_amd (void)
 {
@@ -298,9 +355,10 @@ cpu_sig_amd (void)
* exposes a SEV-SNP attestation report as evidence.
*/
   if (!(eax & (1 << 1))) {
-debug ("No sev in CPUID, try azure TPM NV\n");
+debug ("No sev in CPUID, try hyperv CPUID/azure TPM NV\n");
 
-if (cpu_sig_amd_azure()) {
+if (cpu_sig_amd_hyperv () ||
+cpu_sig_amd_azure()) {
   puts ("amd-sev-snp");
   puts ("azure-hcl");
 } else {
-- 
2.40.1



[PATCH virt-what] Introduce 'virt-what-cvm' program

2023-05-26 Thread Daniel P . Berrangé
The 'virt-what' program prints facts that reflect the hypervisor that
the guest is running under.

The new complementary 'virt-what-cvm' program prints facts that reflect
the confidential virtualization technology the guest is running under,
if any.

It is kept as a separate tool, rather than incorporating the facts into
'virt-what' output because it is considering a different aspect of the
virtualization. Furthermore there are specific security concerns around
the usage of facts reported by 'virt-what-cvm'.

The tool has been tested in a number of environments

 * Azure confidential guest with AMD SEV-SNP (GA)
 * Azure confidential guest with Intel TDX (technology preview)
 * Fedora 37 QEMU/KVM guest with AMD SEV (GA)
 * Fedora 37 QEMU/KVM guest with AMD SEV-ES (GA)
 * Fedora 38 QEMU/KVM guest with AMD SEV-SNP + SVSM (devel snapshot)

Signed-off-by: Daniel P. Berrangé 
---
 .gitignore|   3 +
 Makefile.am   |  12 +-
 configure.ac  |   3 +
 virt-what-cvm.c   | 404 ++
 virt-what-cvm.pod | 195 ++
 5 files changed, 613 insertions(+), 4 deletions(-)
 create mode 100644 virt-what-cvm.c
 create mode 100644 virt-what-cvm.pod

diff --git a/.gitignore b/.gitignore
index 4833fd6..ba897a1 100644
--- a/.gitignore
+++ b/.gitignore
@@ -26,5 +26,8 @@ Makefile.in
 /test-driver
 /virt-what
 /virt-what-cpuid-helper
+/virt-what-cvm
+/virt-what-cvm.1
+/virt-what-cvm.txt
 /virt-what.1
 /virt-what.txt
diff --git a/Makefile.am b/Makefile.am
index 5435132..2050bef 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -24,20 +24,24 @@ EXTRA_DIST = .gitignore virt-what.in virt-what.pod
 SUBDIRS = . tests
 
 sbin_SCRIPTS = virt-what
+sbin_PROGRAMS = virt-what-cvm
 libexec_PROGRAMS = virt-what-cpuid-helper
 if HOST_CPU_IA64
 libexec_PROGRAMS += virt-what-ia64-xen-rdtsc-test
 endif
 
+virt_what_cvm_LDADD = $(TPM2_TSS_LIBS)
+virt_what_cvm_CFLAGS = $(TPM2_TSS_CFLAGS)
+
 if HAVE_POD2MAN
 
-CLEANFILES += virt-what.1 virt-what.txt
-man_MANS = virt-what.1
+CLEANFILES += virt-what.1 virt-what-cvm.1 virt-what.txt virt-what-cvm.txt
+man_MANS = virt-what.1 virt-what-cvm.1
 
-virt-what.1: virt-what.pod
+%.1: %.pod
pod2man -c "Virtualization Support" --release "$(PACKAGE)-$(VERSION)" \
  $? > $@
-virt-what.txt: virt-what.pod
+%.txt: %.pod
pod2text $? > $@
 
 endif
diff --git a/configure.ac b/configure.ac
index a28a716..77b7665 100644
--- a/configure.ac
+++ b/configure.ac
@@ -32,6 +32,9 @@ dnl Architecture we are compiling for.
 AC_CANONICAL_HOST
 AM_CONDITIONAL([HOST_CPU_IA64], [ test "x$host_cpu" = "xia64" ])
 
+PKG_HAVE_DEFINE_WITH_MODULES(TPM2_TSS, tss2-esys, [tpm2-tss package])
+
+
 dnl List of tests.
 tests="\
alibaba-cloud-arm \
diff --git a/virt-what-cvm.c b/virt-what-cvm.c
new file mode 100644
index 000..407efb4
--- /dev/null
+++ b/virt-what-cvm.c
@@ -0,0 +1,404 @@
+/* virt-what-cvm-helper: Are we running inside confidential VM
+ * Copyright (C) 2023 Red Hat Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#include "config.h"
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#ifdef HAVE_TPM2_TSS
+#include 
+#include 
+#endif
+
+static bool dodebug = false;
+
+#define debug(...) do { if (dodebug) fprintf(stderr, __VA_ARGS__); } while(0)
+
+/*
+ * AMD64 Architecture Programmer’s Manual Volume 3:
+ * General-Purpose and System Instructions.
+ * Chapter: E4.1 - Maximum Extended Function Number and Vendor String
+ *  https://www.amd.com/system/files/TechDocs/24594.pdf
+ */
+#define CPUID_GET_HIGHEST_FUNCTION 0x8000
+
+/*
+ * AMD64 Architecture Programmer’s Manual Volume 3:
+ * General-Purpose and System Instructions.
+ * Chapter: E4.17 - Encrypted Memory Capabilities
+ *  https://www.amd.com/system/files/TechDocs/24594.pdf
+ */
+#define CPUID_AMD_GET_ENCRYPTED_MEMORY_CAPABILITIES 0x801f
+
+/*
+ * AMD64 Architecture Programmer’s Manual Volume 3:
+ * General-Purpose and System Instructions.
+ * Chapter: 15.34.10 - SEV_STATUS MSR
+ * https://www.amd.com/system/files/TechDocs/24593.pdf
+ */
+#define MSR_AMD64_SEV 0xc0010131
+
+/*
+ * Intel® TDX Module v1.5 Base Architecture Specification
+ * Chapter: 11.2
+ * 
https://www.intel.com/content/www/us/en/c

Re: virt-manager upgrade

2022-10-07 Thread Daniel P . Berrangé
On Fri, Oct 07, 2022 at 10:21:18AM +0100, Richard W.M. Jones wrote:
> On Thu, Oct 06, 2022 at 07:14:39PM +, woodcab wrote:
> > Hello,
> > 
> >  Still new to Linux and had a few questions regarding upgrading to 4.1.0
> >
> >  I'm using virt-manager 2.2.1 on Ubuntu MATE 20.4. There is an
> > upgrade available for Ubuntu Mate to MATE 20.4.1. I am told that
> > this is a minor upgrade and that it can be done with my files in
> > place. I have backed up my files and VMs along with the .xml
> > files. My question is how to best proceed with that upgrade? If
> > virt-manager is upgraded to 4.1.0 in the process, will my VMs that
> > were setup in 2.2.1 still work in 4.1.0? Or should I upgrade
> > virt-manager to an intermediate version before going to 4.1.0?
> 
> I don't think I've ever had to worry about existing VMs when upgrading
> libvirt or virt-manager (BTW I think *libvirt* is more relevant to
> this since that's what is actually managing the VMs).  I have upgraded
> libvirt many times on my system that runs VMs even while the VMs were
> running.

Libvirt goes to quite alot of trouble to ensure you can upgrade
the host software stack and not cause problems to your VMs. You
can even upgrade libvirt itself, while QEMU VMs are running,
and it should seemlessly reconnect.

Upgrading virt-manger itself will not have any impact of VMs,
since virt-manager is merely a GUI frontend.

Downgrading software, however, is totally unsupported, and you
get to keep any pieces & put it back together if it breaks :-)

> Since you've done the right thing by backing up the existing VMs and
> *.xml files, you should be good even in the unlikely case that
> anything goes wrong during the upgrade, so I guess go for it!

With regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: Support for IPv6 NAT Mode

2022-08-24 Thread Daniel P . Berrangé
On Wed, Aug 24, 2022 at 10:22:29AM +0530, vaishu venkat wrote:
> Hello Daniel,
> 
> Thanks for the quick response.
> 
> By changing the above lines in the xml file, when applying/saving the
> changes, it reverts back. Still I could notice the FORWARDING mode as
> routed, not as NAT for IPv6.  Please find the attached screenshot for your
> reference.
> 
> Please share your observations.

Note changes in the XML only apply once you stop and then start the
network again - it won't be visible immediately after saving/applying.

With regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: How can I disable secure boot using virt-install cli?

2022-08-03 Thread Daniel P . Berrangé
On Wed, Aug 03, 2022 at 11:00:30AM -0500, Andrea Bolognani wrote:
> On Wed, Aug 03, 2022 at 01:17:33PM +0800, Lucas Liu wrote:
> > Hello all:
> >
> > I am looking for a way to disable secure boot for UEFI guests:
> > In 3.2.0 I use the command blow to achieve it:
> >
> > # virt-install --name GuestOne --location #URL --machine q35 --vcpus=2
> > --memory 4096 --file-size=20 --boot uefi --boot
> > nvram.template=/usr/share/edk2/ovmf/OVMF_VARS.fd
> >
> > However, in 4.0.0 I cannot get the same result for this cmd
> >
> > Expect VM is booted with secureboot disabled. But the actual result is the
> > VM is booted with secureboot enabled.
> >
> > # mokutil --sb-state
> > SecureBoot enabled
> >
> > ...
> > 
> > hvm
> >  > type='pflash'>/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd
> >  > template='/usr/share/edk2/ovmf/OVMF_VARS.secboot.fd'>/var/lib/libvirt/qemu/nvram/rhel9_VARS.fd
> > 
> >   
> > ...
> >
> > It seems it still creates guests with
> > "/usr/share/edk2/ovmf/OVMF_VARS.secboot.fd" as the nvram template.
> 
> This should do what you want:
> 
>   --boot 
> uefi,firmware.feature0.name=enrolled-keys,firmware.feature0.enabled=no,firmware.feature1.name=secure-boot,firmware.feature1.enabled=yes
> 
> A bit of a mouthful, I know :) The equivalent XML snippet would be
> 
>   
> 
>   
>   
> 
>   

This seems to kas to leave secureboot enabled, but with no enrolled
keys.

To disable secureboot fully I use this

 --boot 
firmware=efi,firmware.feature0.enabled=no,firmware.feature0.name=secure-boot \



With regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: [PATCH] detect AWS arm virtual instance as kvm

2022-05-25 Thread Daniel P . Berrangé
On Wed, May 25, 2022 at 02:34:53PM +0800, Frank Liang wrote:
> Hello,
> 
> aws arm virtual instances are running on top of Nitro hypervisor(KVM-based
> hypervisor).
> I am proposing this patch to keep virt-what output consistent with x86
> virtual instances.
> Please review it. Thanks
> 
> With this patch:
> $ sudo virt-what
> kvm
> aws
> $ cat /sys/devices/virtual/dmi/id/product_name
> t4g.small
> 
> Without it:
> $ sudo virt-what
> aws
> 
> Here is the test pass log with this patch.
> # cat test-suite.log
> ==
>virt-what 1.22: tests/aws-kvm-arm/test-suite.log
> ==
> 
> # TOTAL: 1
> # PASS:  1
> # SKIP:  0
> # XFAIL: 0
> # FAIL:  0
> # XPASS: 0
> # ERROR: 0
> 
> .. contents:: :depth: 2
> 
> Rgs,
> Frank

> From 931e5f24b9e7e6e61cebe3213166bd691df80db0 Mon Sep 17 00:00:00 2001
> From: Xiao Liang 
> Date: Tue, 24 May 2022 17:34:52 +0800
> Subject: [PATCH] detect AWS arm virtual instance as kvm
> 
> AWS arm virtual instance is KVM based hypervisor.
> With this patch:
> $ sudo virt-what
> kvm
> aws
> $ cat /sys/devices/virtual/dmi/id/product_name
> t4g.small
> 
> Without it:
> $ sudo virt-what
> aws
> 
> Signed-off-by: Xiao Liang 
> ---
>  tests/aws-kvm-arm/test.sh | 3 ++-
>  virt-what.in  | 5 +
>  2 files changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/tests/aws-kvm-arm/test.sh b/tests/aws-kvm-arm/test.sh
> index 5d31995..d469a99 100755
> --- a/tests/aws-kvm-arm/test.sh
> +++ b/tests/aws-kvm-arm/test.sh
> @@ -18,7 +18,8 @@
>  output="$(PATH=../..:$PATH virt-what --test-root=. 2>&1)"
>  # XXX At the moment KVM cannot be detected on this platform.
>  # We hope to fix that, but for now the only fact printed is "aws".

This comment looks outdated given the change you made

> -expected="aws"
> +expected="kvm
> +aws"
>  
>  if [ "$output" != "$expected" ]; then
>  echo "$0: test failed because output did not match expected"
> diff --git a/virt-what.in b/virt-what.in
> index b59714e..9afbd40 100644
> --- a/virt-what.in
> +++ b/virt-what.in
> @@ -339,6 +339,11 @@ if ! "$skip_qemu_kvm"; then
>  echo qemu
>  skip_lkvm=true
>  fi
> +if echo "$dmi" | grep -q 'Amazon EC2' &&
> +echo "$dmi" | grep -q 'System is a virtual machine'; then
> +echo kvm
> +skip_lkvm=true
> +fi
>  elif [ -d ${root}/proc/device-tree/hypervisor ] &&
>   grep -q "linux,kvm" /proc/device-tree/hypervisor/compatible; then
>  # We are running as a spapr KVM guest on ppc64
> -- 
> 2.36.1
> 


With regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: no release tarball

2022-04-21 Thread Daniel P . Berrangé
On Thu, Apr 21, 2022 at 01:30:29PM +0200, Henrik Riomar wrote:
> On Thu, Apr 21, 2022 at 1:11 PM Richard W.M. Jones  wrote:
> >
> > On Thu, Apr 21, 2022 at 12:54:42PM +0200, Henrik Riomar wrote:
> 
> > > I now note that commit b7cc3d93a613ef6d0ac5ccd6e32cc3d66e057243, that
> > > is part of v1.22 introduced two bashisms:
> > >
> > > possible bashism in ./virt-what line 119 (echo -e):
> > >   │
> > > if ( { echo -e "GET /latest/meta-datainstance/instance-type
> > > HTTP/1.0\r\nHost: 100.100.100.200\r\n\r" │
> > > >&3; grep -sq 'ebm' <&3 ; } 3<> /dev/tcp/100.100.100.200/80 ) 2>/dev/null 
> > > >; then │
> > > possible bashism in ./virt-what line 119 (/dev/(tcp|udp)):
> > >   │
> > > if ( { echo -e "GET /latest/meta-datainstance/instance-type
> > > HTTP/1.0\r\nHost: 100.100.100.200\r\n\r" │
> > > >&3; grep -sq 'ebm' <&3 ; } 3<> /dev/tcp/100.100.100.200/80 ) 2>/dev/null 
> > > >; then
> > >
> > > Can this be fixed? as that is the only bashisms in virt-what.
> >
> > Can you send a patch to fix it?
> 
> The echo -e should be a simple replace with printf "%b\n", but the
> code using file descriptor 3 and the bash feature to open a tcp
> socket, needs some more work,
> it can be replaced with nc, but the problem is that I have no Alibaba
> Cloud VM to test a fix on.

Or just change the first line of the script to #!/bin/bash, so it
doesn't rely on the default shell impl.

With regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: Add support for enabling Secure Encrypted Virtualization in the GUI

2022-04-20 Thread Daniel P . Berrangé
On Wed, Apr 20, 2022 at 09:11:27AM -0400, Cole Robinson wrote:
> On 4/19/22 1:21 PM, Daniel P. Berrangé wrote:
> > On Tue, Apr 19, 2022 at 12:05:29PM -0400, Cole Robinson wrote:
> >> On 4/4/22 11:49 AM, Charles Arnold wrote:
> >>>
> >>>
> >>> On 4/4/22 6:50 AM, Daniel P. Berrangé wrote:
> >>>> On Fri, Apr 01, 2022 at 12:13:17PM -0600, Charles Arnold wrote:
> >>>>>  From d700e8cee7cd525c0022b5a9a440f64c4ab149f0 Mon Sep 17 00:00:00 2001
> >>>>> From: Charles Arnold 
> >>>>> Date: Fri, 1 Apr 2022 12:01:21 -0600
> >>>>> Subject: [PATCH 1/1] Add support for enabling Secure Encrypted
> >>>>> Virtualization
> >>>>>   in the GUI
> >>>>>
> >>>>> Add an "Enable Launch Security" checkbox on the Details memory tab.
> >>>>> Do the minimal configuration required for libvirt to enable this feature
> >>>>> on compatible hardware.
> >>>>>
> >>>> Don't we need to turn on the 'iommu' option for all virtio devices
> >>>> too, and disable PXE on any NICs ?
> >>>>
> >>>> https://libvirt.org/kbase/launch_security_sev.html#virtio
> >>>>
> >>>
> >>> I used to enumerate through the virtio devices in an old version of this
> >>> patch
> >>> for virt-manager and enable iommu but it really wasn't reasonable for
> >>> virt-manager to track which virtio devices needed iommu enabled.
> >>> Additionally,
> >>> libvirt will sometimes add a device when a VM is created. This patch
> >>> leans on libvirt to do the right thing when sev is enabled similar to what
> >>> happens when launch security is specified on the virt-install command 
> >>> line.
> >>>
> >>
> >> Yeah, I would still like to see libvirt do this unless there's a good
> >> reason why it can't. From my July 2020 mail
> >> https://listman.redhat.com/archives/virt-tools-list/2020-July/017087.html
> >>
> >>> if sev
> >>> launchSecurity _requires_ every virtio device to have iommu='on' then
> >>> libvirt should be setting that itself. It doesn't need to hardcode it
> >>> into the XML, it can set it at VM startup time. If the user set an
> >>> explicit value in the XML then honor that but otherwise fill it in at
> >>> runtime when it is required. Trying to deal with this in an app where we
> >>> want to advertise turning the config off is basically an impossible
> >>> problem to know if we are going to undo any explicit user config or not.
> >>
> >> danpb does this sound reasonable? If so I can work on this.
> > 
> > Something automagic sounds ok
> > 
> >> Also, anyone know if TDX and SNP will require virtio iommu setting as well?
> > 
> > I expect SNP will, but no idea about TDX.
> > 
> 
> So apparently qemu 6.0.0+ already does this for us:
> https://gitlab.com/qemu-project/qemu/-/commit/9f88a7a3df
> 
> Which is strange because some of my sev testing definitely failed with
> an iommu sounding error in the guest kernel until I enabled iommu on
> virtio-blk, but I can't reproduce now. Maybe I was using an older qemu,
> I think the host was fedora 34 at the time

Or you had new QEMU, but the guest was fixed on the older machine
type perhaps

With regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: Add support for enabling Secure Encrypted Virtualization in the GUI

2022-04-19 Thread Daniel P . Berrangé
On Tue, Apr 19, 2022 at 12:05:29PM -0400, Cole Robinson wrote:
> On 4/4/22 11:49 AM, Charles Arnold wrote:
> > 
> > 
> > On 4/4/22 6:50 AM, Daniel P. Berrangé wrote:
> >> On Fri, Apr 01, 2022 at 12:13:17PM -0600, Charles Arnold wrote:
> >>>  From d700e8cee7cd525c0022b5a9a440f64c4ab149f0 Mon Sep 17 00:00:00 2001
> >>> From: Charles Arnold 
> >>> Date: Fri, 1 Apr 2022 12:01:21 -0600
> >>> Subject: [PATCH 1/1] Add support for enabling Secure Encrypted
> >>> Virtualization
> >>>   in the GUI
> >>>
> >>> Add an "Enable Launch Security" checkbox on the Details memory tab.
> >>> Do the minimal configuration required for libvirt to enable this feature
> >>> on compatible hardware.
> >>>
> >> Don't we need to turn on the 'iommu' option for all virtio devices
> >> too, and disable PXE on any NICs ?
> >>
> >> https://libvirt.org/kbase/launch_security_sev.html#virtio
> >>
> > 
> > I used to enumerate through the virtio devices in an old version of this
> > patch
> > for virt-manager and enable iommu but it really wasn't reasonable for
> > virt-manager to track which virtio devices needed iommu enabled.
> > Additionally,
> > libvirt will sometimes add a device when a VM is created. This patch
> > leans on libvirt to do the right thing when sev is enabled similar to what
> > happens when launch security is specified on the virt-install command line.
> > 
> 
> Yeah, I would still like to see libvirt do this unless there's a good
> reason why it can't. From my July 2020 mail
> https://listman.redhat.com/archives/virt-tools-list/2020-July/017087.html
> 
> > if sev
> > launchSecurity _requires_ every virtio device to have iommu='on' then
> > libvirt should be setting that itself. It doesn't need to hardcode it
> > into the XML, it can set it at VM startup time. If the user set an
> > explicit value in the XML then honor that but otherwise fill it in at
> > runtime when it is required. Trying to deal with this in an app where we
> > want to advertise turning the config off is basically an impossible
> > problem to know if we are going to undo any explicit user config or not.
> 
> danpb does this sound reasonable? If so I can work on this.

Something automagic sounds ok

> Also, anyone know if TDX and SNP will require virtio iommu setting as well?

I expect SNP will, but no idea about TDX.

With regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: Add support for enabling Secure Encrypted Virtualization in the GUI

2022-04-04 Thread Daniel P . Berrangé
On Mon, Apr 04, 2022 at 06:21:42PM +0200, Boris Fiuczynski wrote:
> On 4/4/22 5:48 PM, Charles Arnold wrote:
> > On 4/4/22 8:37 AM, Boris Fiuczynski wrote:
> > > On 4/4/22 2:50 PM, Daniel P. Berrangé wrote:
> > > > On Fri, Apr 01, 2022 at 12:13:17PM -0600, Charles Arnold wrote:
> > > > >  From d700e8cee7cd525c0022b5a9a440f64c4ab149f0 Mon Sep 17 00:00:00 
> > > > > 2001
> > > > > From: Charles Arnold 
> > > > > Date: Fri, 1 Apr 2022 12:01:21 -0600
> > > > > Subject: [PATCH 1/1] Add support for enabling Secure Encrypted
> > > > > Virtualization
> > > > >   in the GUI
> > > > > 
> > > > > Add an "Enable Launch Security" checkbox on the Details memory tab.
> > > > > Do the minimal configuration required for libvirt to enable
> > > > > this feature
> > > > > on compatible hardware.
> > > > > 
> > > > 
> > > > Don't we need to turn on the 'iommu' option for all virtio devices
> > > > too, and disable PXE on any NICs ?
> > > > 
> > > > https://libvirt.org/kbase/launch_security_sev.html#virtio
> > > > 
> > > > With regards,
> > > > Daniel
> > > > 
> > > 
> > > Hi Arnold,
> > > your patch does not take into account that libvirt uses launch
> > > security for more types besides sev.
> > > 
> > > 
> > Good point. I haven't taken into account the s390 case which I can correct.
> > I'm not aware of other launch security types besides those two.
> > 
> > - Charles
> > 
> 
> There has been a patch series for TDX on the mailing list in July 2021 but I
> am not sure what finally happened to it.
> https://listman.redhat.com/archives/libvir-list/2021-July/221098.html

TDX support isn't merged in the Linux kernel/KVM, nor in QEMU, nor
OVMF AFAIK, so anything related to libvirt & above is on hold until
the lower TDX bits are ready.


With regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: Add support for enabling Secure Encrypted Virtualization in the GUI

2022-04-04 Thread Daniel P . Berrangé
On Fri, Apr 01, 2022 at 12:13:17PM -0600, Charles Arnold wrote:
> From d700e8cee7cd525c0022b5a9a440f64c4ab149f0 Mon Sep 17 00:00:00 2001
> From: Charles Arnold 
> Date: Fri, 1 Apr 2022 12:01:21 -0600
> Subject: [PATCH 1/1] Add support for enabling Secure Encrypted
> Virtualization
>  in the GUI
> 
> Add an "Enable Launch Security" checkbox on the Details memory tab.
> Do the minimal configuration required for libvirt to enable this feature
> on compatible hardware.
>

Don't we need to turn on the 'iommu' option for all virtio devices
too, and disable PXE on any NICs ?

https://libvirt.org/kbase/launch_security_sev.html#virtio

With regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: virt-viewer light theme?

2021-11-29 Thread Daniel P . Berrangé
On Fri, Nov 26, 2021 at 02:37:45PM -0800, ToddAndMargo wrote:
> Hi All,
> 
> FC35
> Xfce 4.16
> virt-viewer-10.0-4.fc35.x86_64
> 
> Is there a way to switch virt-viewer's decorations
> from its current dark theme to a light or system
> theme?

You can set an env variable if desired

  GTK_THEME=Adwaita:light

Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: [PATCH virt-manager v2 1/2] tests: add tests to support MDEV name change

2021-11-12 Thread Daniel P . Berrangé
On Fri, Nov 12, 2021 at 11:51:48AM +0100, Shalini Chellathurai Saroja wrote:
> Add tests to ensure that the UUID of mediated devices are parsed
> correctly in both newer and older(<7.3.0) versions of libvirt.
> 
> The node device names of mediated devices are changed from
> 'MDEV_$UUID'(eg: mdev_b204c698_6731_4f25_b5f4_894614a05ec0) to
> 'MDEV_$UUID_$PARENT'(eg:
> mdev_b204c698_6731_4f25_b5f4_894614a05ec0_0_0_0014) in libvirt version
> 7.8.0. This change impacted the UUID parsing of mediated devices,
> which is fixed with commit 0c146b250384cefd2cc0d76b9e808377ebe5.

Commit 0c146b250384cefd2cc0d76b9e808377ebe5 introduced
test coverage for this changed approach.  I'm not seeing
what scenarios this covers that aren't already covered ?

There are only two codepaths in get_mdev_uuid() and we
test both of them.

> 
> Signed-off-by: Shalini Chellathurai Saroja 
> ---
>  tests/data/testdriver/testdriver.xml | 28 
>  tests/test_nodedev.py| 27 +++
>  2 files changed, 55 insertions(+)
> 
> diff --git a/tests/data/testdriver/testdriver.xml 
> b/tests/data/testdriver/testdriver.xml
> index e4880936..13f40136 100644
> --- a/tests/data/testdriver/testdriver.xml
> +++ b/tests/data/testdriver/testdriver.xml
> @@ -3679,6 +3679,20 @@ ba
>
>  
>  
> +
> +  mdev_c6c57090_3490_428e_a7fb_8d557f985717_0_0_0023
> +  
> /sys/devices/css0/0.0.0023/c6c57090-3490-428e-a7fb-8d557f985717
> +  css_0_0_0023
> +  
> +vfio_mdev
> +  
> +  
> +
> +c6c57090-3490-428e-a7fb-8d557f985717
> +
> +  
> +
> +
>  
>ap_matrix
>/sys/devices/vfio_ap/matrix
> @@ -3712,6 +3726,20 @@ ba
>
>  
>  
> +
> +  mdev_45938d10_4d5d_4e8f_b5ce_27be07b5bab6_matrix
> +  
> /sys/devices/vfio_ap/matrix/45938d10-4d5d-4e8f-b5ce-27be07b5bab6
> +  ap_matrix
> +  
> +vfio_mdev
> +  
> +  
> +
> +45938d10-4d5d-4e8f-b5ce-27be07b5bab6
> +
> +  
> +
> +
>  
>mdev_4b20d080_1b54_4048_85b3_a6a62d165c01
>
> /sys/devices/pci:00/:00:02.0/4b20d080-1b54-4048-85b3-a6a62d165c01
> diff --git a/tests/test_nodedev.py b/tests/test_nodedev.py
> index 41435262..8de0facf 100644
> --- a/tests/test_nodedev.py
> +++ b/tests/test_nodedev.py
> @@ -135,6 +135,7 @@ def testDASDMdev():
>  assert dev.parent == "css_0_0_0023"
>  assert dev.device_type == "mdev"
>  assert dev.type_id == "vfio_ccw-io"
> +assert dev.get_mdev_uuid() == "8e37ee90-2b51-45e3-9b25-bf8283c03110"
>  
>  
>  def testAPQNMdev():
> @@ -145,6 +146,7 @@ def testAPQNMdev():
>  assert dev.parent == "ap_matrix"
>  assert dev.device_type == "mdev"
>  assert dev.type_id == "vfio_ap-passthrough"
> +assert dev.get_mdev_uuid() == "11f92c9d-b0b0-4016-b306-a8071277f8b9"
>  
>  
>  def testPCIMdev():
> @@ -157,7 +159,32 @@ def testPCIMdev():
>  assert dev.type_id == "nvidia-11"
>  assert dev.get_mdev_uuid() == "4b20d080-1b54-4048-85b3-a6a62d165c01"
>  
> +
>  # libvirt <7.3.0 doesn't support  in the mdev node device xml
> +@pytest.mark.skipif(libvirt.getVersion() < 7003000, reason="libvirt version 
> doesn't support new mdev format")
> +def testDASDMdevNewFormat():
> +conn = utils.URIs.open_testdriver_cached()
> +devname = "mdev_c6c57090_3490_428e_a7fb_8d557f985717_0_0_0023"
> +dev = _nodeDevFromName(conn, devname)
> +assert dev.name == devname
> +assert dev.parent == "css_0_0_0023"
> +assert dev.device_type == "mdev"
> +assert dev.type_id == "vfio_ccw-io"
> +assert dev.get_mdev_uuid() == "c6c57090-3490-428e-a7fb-8d557f985717"
> +
> +
> +@pytest.mark.skipif(libvirt.getVersion() < 7003000, reason="libvirt version 
> doesn't support new mdev format")
> +def testAPQNMdevNewFormat():
> +conn = utils.URIs.open_testdriver_cached()
> +devname = "mdev_45938d10_4d5d_4e8f_b5ce_27be07b5bab6_matrix"
> +dev = _nodeDevFromName(conn, devname)
> +assert dev.name == devname
> +assert dev.parent == "ap_matrix"
> +assert dev.device_type == "mdev"
> +assert dev.type_id == "vfio_ap-passthrough"
> +assert dev.get_mdev_uuid() == "45938d10-4d5d-4e8f-b5ce-27be07b5bab6"
> +
> +
>  @pytest.mark.skipif(libvirt.getVersion() < 7003000, reason="libvirt version 
> doesn't support new mdev format")
>  def testPCIMdevNewFormat():
>  conn = utils.URIs.open_testdriver_cached()
> -- 
> 2.30.2
> 

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: Triggering a checkpoint from inside the VM

2021-09-09 Thread Daniel P . Berrangé
On Thu, Sep 09, 2021 at 08:31:58AM +, Leek, Jim wrote:
> I’ll try snapshot create, I didn’t know about that.
> 
> I understand your security concerns, but they don’t apply in this case. The 
> target environment is air gapped and isolated. I can’t see any reason to fear 
> an attack on the VM before an attack on the host.
> 
> 2. Well, I was experimenting with getting libvirt out of the system to 
> simplify things, but I didn’t have much luck yet.

Taking libvirt out of the loop is likely to make your life harder
in the long term, especially as you'll no longer be insulated from
changes in QEMU's external management interfaces. QEMU may look
simple to manage directly at first glance, but it is actually quite
a complicated to do it well when you look closely.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: Triggering a checkpoint from inside the VM

2021-09-09 Thread Daniel P . Berrangé
On Thu, Sep 09, 2021 at 12:30:53AM +, Leek, Jim wrote:
> Here's the error I sometimes get on checkpoint/restore.  It just happened 
> again.  In this test I had at least a 40 second break between checkpoint and 
> restore, so extra sleep doesn't seem to help much.
> 
> Error unpausing domain: Timed out during operation: cannot acquire state 
> change lock (held by monitor=qemuDispatchDomainMonitorCommand)
> 
> Traceback (most recent call last):
>   File "/usr/share/virt-manager/virtManager/asyncjob.py", line 75, in 
> cb_wrapper
> callback(asyncjob, *args, **kwargs)
>   File "/usr/share/virt-manager/virtManager/asyncjob.py", line 111, in tmpcb
> callback(*args, **kwargs)
>   File "/usr/share/virt-manager/virtManager/object/libvirtobject.py", line 
> 66, in newfn
> ret = fn(self, *args, **kwargs)
>   File "/usr/share/virt-manager/virtManager/object/domain.py", line 1312, in 
> resume
> self._backend.resume()
>   File "/usr/lib64/python3.6/site-packages/libvirt.py", line 2174, in resume
> if ret == -1: raise libvirtError ('virDomainResume() failed', dom=self)
> libvirt.libvirtError: Timed out during operation: cannot acquire state change 
> lock (held by monitor=qemuDispatchDomainMonitorCommand)

This suggests that the previously executed 'qemu-montor-command' has not
finished running yet.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: [PATCH v2] virtmanager: Fixed the legacy firmware installation error

2021-01-12 Thread Daniel P . Berrangé
On Tue, Jan 12, 2021 at 05:38:48PM +0800, Yu-Chen, Cho wrote:
> From: "Cho, Yu-Chen" 
> 
> When we choose /usr/share/qemu/bios.bin or /usr/share/qemu/bios-256k.bin
> in Customize configuration before install, show this error message:
> 
> Unable to complete install: 'operation failed: unable to find any master var 
> store for loader: /usr/share/qemu/bios.bin'
> 
> Traceback (most recent call last):
>   File "/usr/share/virt-manager/virtManager/asyncjob.py", line 65, in 
> cb_wrapper
> callback(asyncjob, *args, **kwargs)
>   File "/usr/share/virt-manager/virtManager/createvm.py", line 2081, in 
> _do_async_install
> installer.start_install(guest, meter=meter)
>   File "/usr/share/virt-manager/virtinst/install/installer.py", line 721, in 
> start_install
> domain = self._create_guest(
>   File "/usr/share/virt-manager/virtinst/install/installer.py", line 669, in 
> _create_guest
> domain = self.conn.createXML(install_xml or final_xml, 0)
>   File "/usr/lib64/python3.8/site-packages/libvirt.py", line 4347, in 
> createXML
> raise libvirtError('virDomainCreateXML() failed')
> libvirt.libvirtError: operation failed: unable to find any master var store 
> for loader: /usr/share/qemu/bios.bin
> 
> This patch check the loader if it is include bios.bin and bios-256k.bin.
> 
> Signed-off-by: Yu-Chen, Cho 
> --
> v2:
> Fixed tabs vs spaces issues.
> 
> v1:
> Check the loader if it is include bios.bin and bios-256k.bin.
> ---
>  virtManager/object/domain.py | 18 +-
>  virtinst/guest.py| 13 +
>  2 files changed, 22 insertions(+), 9 deletions(-)
> 
> diff --git a/virtManager/object/domain.py b/virtManager/object/domain.py
> index cc2f506d..2fafe2b4 100644
> --- a/virtManager/object/domain.py
> +++ b/virtManager/object/domain.py
> @@ -671,16 +671,16 @@ class vmmDomain(vmmLibvirtObject):
>  
>  if loader != _SENTINEL:
>  if loader is None:
> -# Implies seabios, aka the default, so clear everything
> -guest.os.loader = None
> -guest.os.loader_ro = None
> -guest.os.loader_type = None
> -guest.os.nvram = None
> -guest.os.nvram_template = None
> +# Implies the default, so clear everything
> +guest.set_legacy_path(None)
>  else:
> -# Implies UEFI
> -guest.set_uefi_path(loader)
> -guest.disable_hyperv_for_uefi()
> +if ("bios.bin" or "bios-256k.bin" in loader):
> +# Implies Custom: seabios, not UEFI
> +guest.set_legacy_path(loader)
> +else:
> +# Implies UEFI
> +guest.set_uefi_path(loader)
> +guest.disable_hyperv_for_uefi()

This if/else condition is going to be wrong for all non-x86
architectures.

I wonder if the UI shuld have a "firmware is UEFI" checkbox
instead.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: [PATCH] Add -m/--machine option. Pick the default machine type depending on arch.

2020-09-10 Thread Daniel P . Berrangé
On Thu, Sep 10, 2020 at 01:40:09PM +0100, Richard W.M. Jones wrote:
> Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1875763
> ---
>  qemu-sanity-check.in | 20 +---
>  qemu-sanity-check.pod.in |  8 
>  2 files changed, 25 insertions(+), 3 deletions(-)
> 
> diff --git a/qemu-sanity-check.in b/qemu-sanity-check.in
> index 5536eda..e1abafd 100644
> --- a/qemu-sanity-check.in
> +++ b/qemu-sanity-check.in
> @@ -1,7 +1,7 @@
>  #!/bin/bash
>  # -*- shell-script -*-
>  # qemu-sanity-check
> -# Copyright (C) 2013 Red Hat Inc.
> +# Copyright (C) 2013-2020 Red Hat Inc.
>  #
>  # This program is free software; you can redistribute it and/or modify
>  # it under the terms of the GNU General Public License as published by
> @@ -27,6 +27,14 @@ canonical_arch="$(uname -m | sed 's/i[456]86/i386/')"
>  timeout=10m
>  accel=kvm:tcg
>  
> +# Default machine type depends on arch.  You can override this using
> +# -m|--machine option.
> +case "$canonical_arch" in
> +arm*|aarch*) machine=virt ;;
> +# On non-ARM let qemu pick the default.
> +*) machine= ;;
> +esac

avr, rx and tricore also lack a defalt but none of those matter for
this purpose

Reviewed-by: Daniel P. Berrangé 


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: [virt-viewer PATCH] about ui: year 2020 in Copyright

2020-08-28 Thread Daniel P . Berrangé
On Fri, Aug 28, 2020 at 10:40:07AM -0400, Frediano Ziglio wrote:
> > 
> > On Fri, Aug 28, 2020 at 10:22:49AM -0400, Frediano Ziglio wrote:
> > > > 
> > > > On Thu, Aug 27, 2020 at 09:39:37PM +0300, Uri Lublin wrote:
> > > > > rhbz#1848267
> > > > > 
> > > > > Signed-off-by: Uri Lublin 
> > > > 
> > > > Patches need to be subitted as pull requests to the gitlab repo
> > > > please.
> > > > 
> > > > > ---
> > > > > 
> > > > > Should po/* files be modified too ?
> > > > 
> > > > Not really. There's no need to keep changing copyright dates in source
> > > > files in general.
> > > > 
> > > 
> > > Yes, but the string this patch refers is marked as translatable and it's
> > > translated in some *.po files. It's not about source file dates.
> > 
> > We should make it non-translatable, as it doesn't make sense to change
> > copyright statements in translations.
> > 
> 
> I remember some cases where some people complained that in some countries
> the "(C)" was not correct from the legal point of view and needed the usage
> of ©.

I can't see that as a real issue since every single source file
uses "(C)" in license boilerplate and never gets translated.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: [virt-viewer PATCH] about ui: year 2020 in Copyright

2020-08-28 Thread Daniel P . Berrangé
On Fri, Aug 28, 2020 at 10:22:49AM -0400, Frediano Ziglio wrote:
> > 
> > On Thu, Aug 27, 2020 at 09:39:37PM +0300, Uri Lublin wrote:
> > > rhbz#1848267
> > > 
> > > Signed-off-by: Uri Lublin 
> > 
> > Patches need to be subitted as pull requests to the gitlab repo
> > please.
> > 
> > > ---
> > > 
> > > Should po/* files be modified too ?
> > 
> > Not really. There's no need to keep changing copyright dates in source
> > files in general.
> > 
> 
> Yes, but the string this patch refers is marked as translatable and it's
> translated in some *.po files. It's not about source file dates.

We should make it non-translatable, as it doesn't make sense to change
copyright statements in translations.

Regardless, po/*.po files must NEVER be touched by any commit.

Only weblate is permitted to touch the po files otherwise we
get conflicts.

The .pot file is periodically refreshed out of band.

> 
> > > 
> > > ---
> > >  src/resources/ui/virt-viewer-about.ui | 2 +-
> > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > 
> > > diff --git a/src/resources/ui/virt-viewer-about.ui
> > > b/src/resources/ui/virt-viewer-about.ui
> > > index 681590f..25b2165 100644
> > > --- a/src/resources/ui/virt-viewer-about.ui
> > > +++ b/src/resources/ui/virt-viewer-about.ui
> > > @@ -15,7 +15,7 @@
> > >  True
> > >  Virtual Machine Viewer
> > >  Copyright (C) 2007-2012
> > >  Daniel P. Berrange
> > > -Copyright (C) 2007-2014 Red Hat, Inc.
> > > +Copyright (C) 2007-2020 Red Hat, Inc.
> > >  A remote desktop client
> > >  built with GTK-VNC, SPICE-GTK and libvirt
> > >   > >  name="website">https://gitlab.com/virt-viewer/virt-viewer
> > >   > >  
> > > name="website_label">https://gitlab.com/virt-viewer/virt-viewer
> > > --
> > > 2.26.2
> > > 
> > 
> > Regards,
> > Daniel

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: [virt-viewer PATCH] about ui: year 2020 in Copyright

2020-08-28 Thread Daniel P . Berrangé
On Thu, Aug 27, 2020 at 09:39:37PM +0300, Uri Lublin wrote:
> rhbz#1848267
> 
> Signed-off-by: Uri Lublin 

Patches need to be subitted as pull requests to the gitlab repo
please.

> ---
> 
> Should po/* files be modified too ?

Not really. There's no need to keep changing copyright dates in source
files in general.

> 
> ---
>  src/resources/ui/virt-viewer-about.ui | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/src/resources/ui/virt-viewer-about.ui 
> b/src/resources/ui/virt-viewer-about.ui
> index 681590f..25b2165 100644
> --- a/src/resources/ui/virt-viewer-about.ui
> +++ b/src/resources/ui/virt-viewer-about.ui
> @@ -15,7 +15,7 @@
>  True
>  Virtual Machine Viewer
>  Copyright (C) 2007-2012 
> Daniel P. Berrange
> -Copyright (C) 2007-2014 Red Hat, Inc.
> +Copyright (C) 2007-2020 Red Hat, Inc.
>  A remote desktop client 
> built with GTK-VNC, SPICE-GTK and libvirt
>   name="website">https://gitlab.com/virt-viewer/virt-viewer
>   name="website_label">https://gitlab.com/virt-viewer/virt-viewer
> -- 
> 2.26.2
> 

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: [PATCH] Correct two translations for zh_CN

2020-08-05 Thread Daniel P . Berrangé
On Wed, Aug 05, 2020 at 10:42:22AM -0400, Cole Robinson wrote:
> On 8/5/20 4:32 AM, Pavel Hrdina wrote:
> > On Wed, Aug 05, 2020 at 12:44:24PM +0800, Huacai Chen wrote:
> >> Hi, Pavel,
> >>
> >> On Mon, Aug 3, 2020 at 8:06 PM Pavel Hrdina  wrote:
> >>>
> >>> On Mon, Aug 03, 2020 at 04:45:03PM +0800, Huacai Chen wrote:
>  There is no "Error" in "Refresh snapshot list" and "Restoring virtual
>  machine memory from disk", so remove "出错" in the Chinese translations.
> 
>  Signed-off-by: Huacai Chen 
>  ---
>   po/zh_CN.po | 5 ++---
>   1 file changed, 2 insertions(+), 3 deletions(-)
> >>>
> >>> Hi, thank you for your contribution to virt-manager. The translation
> >>> changes are not pushed directly to git repository. We use weblate
> >>> service hosted by fedora where you should make the changes:
> >>>
> >>> https://translate.fedoraproject.org/projects/virt-manager/
> >> I have registered, and my user-name is chenhuacai. It seems that I
> >> should be a member of cvsl10n, I have joined but it needs to be
> >> approved. Could you please help me?
> > 
> 
> Please keep us informed if you get approved or hit any more roadblocks.
> Weblate is new for us so I'm not entirely certain what the process is here
> 
> > I don't have the permissions to approve it but if you follow this guide
> > you should be able to get approved https://fedoraproject.org/wiki/L10N/Guide
> > 
> 
> FWIW Pavel I added you as a virt-manager weblate admin now

IMHO, in this kind of case it is better if one of the project maintainers
just copy the change into Weblate, instead of expecting the patch submitter
to create themselves an account. The account creation is only a good use
of time if the contributor intends to become a long term translator.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: How to update virt-viewer translations

2020-07-09 Thread Daniel P . Berrangé
On Thu, Jul 09, 2020 at 04:46:30AM -0400, Frediano Ziglio wrote:
> Hi,
>   I'm trying to fix https://bugzilla.redhat.com/show_bug.cgi?id=1503187,
> a translation bug in virt-viewer.
> It's not clear how to proceed. Looking at virt-viewer source the translations
> are synced from Zanata. I have an account on Zanata but the virt-viewer 
> project
> seems in a read-only state. How to update translations?

Zanata is dead, we're moving to Weblate. Needs this reviewed and merged:

  https://gitlab.com/virt-viewer/virt-viewer/-/merge_requests/14

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: Multiple video elements lead to erratic mouse behavior through spice

2020-06-10 Thread Daniel P . Berrangé
On Wed, Jun 10, 2020 at 01:58:13PM +0200, Christian Ehrhardt wrote:
> Hi,
> with a guest set up as in the following example mouse behavior breaks:
> 
> 
>  
>  
>
>
>   primary='yes'/>
>   function='0x0'/>
>
>
>  
>   function='0x0'/>
>
> 
> Please note that this is for illustration purposes as this everyone can
> easily retry.
> I originally had QXL + GPU as mediated device, where the behavior is the
> same.
> 
> If I know connect to that via spice, spice is smart and opens both displays:
>   $ virt-viewer --connect qemu+ssh://ubuntu@10.246.115.255/system guestname
> 
> But while with one display mouse integration (to move in/out smoothly) and
> positioning of the mouse pointer is fine.
> It is broken with this double display setup.
> What I see is:
> - no mouse integration anymore (falls back to capture the mouse)
> - often the mouse can not move upwards anymore
> - sometimes "left" also doesn't work
> 
> Let me know if someone wants  a video in case the description is still too
> odd.
> 
> I have tried spice-client-gtk and virt-viewer, but both behave the same way.
> 
> If I take away one of the displays it works well again.
> So I'm wondering how virtual-multi-monitor is supposed to work, are there
> known bugs or tweaks that would apply here?

Did you enable the SPICE agent ?  IIUC, the SPICE agent provides a
paravirtualized mouse device, and that may be what's required to have
this work properly with multi-monitor.  The agent is also what lets
you dynamically turn on/off each head and do display resizing.

BTW, there are two ways todo multi-monitor with SPICE. One as youve
shown here, and the oither using a single QXL card, but with heads=NNN
instead of heads=1.  One approach works best with Windows, and one
approach works best with Linux, but I can't remember which way around.

I expect they'll probably still have the same mouse behaviour though.

> P.S. I tried VNC but that just has "other" issues. It only shows 1 of 2
> displays and on the one I see over amplifies the mouse movement by the
> amount the other display adds to ther overall desktop size.

VNC simply doesn't support multi-monitor at all. In theory we could
block this in libvirt, but then if QEMU fixed it, libvirt would be
needlessly blocking it.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: Connection issues with virt-viewer 9.0 (win)

2020-05-20 Thread Daniel P . Berrangé
On Wed, May 20, 2020 at 12:19:59PM -0500, fletch b wrote:
> Hi,
> I just upgraded to virt-viever-9.0(win) and ran into issues I never saw
> with ver 7,0
> 
> When trying to connect to my vm I would get to the point where I would see
> this:
> 
> "connected to graphic server" and nothing after that other than a spinning
> circle.
> 
> It does not do this every time. Out of 20-25 connection attempts, it has
> connected just fine 3 times.
> 
> Looking for a pattern and only thing I can come up with is time. If I wait
> a few minutes it sometimes will work again.
> 
> Also no sound in 9.
> 
> Gone back to 7 for now..
> any ideas?

The lack of sound is reported here and we know the problem and will have
a fixed MSI available next week:

  https://gitlab.com/virt-viewer/virt-viewer/-/issues/2


I've not heard the connectin problem reported before, so please file
a  bug for that. Also collect the debug logs from spice-gtk or gtk-vnc

  virt-viewer --debug --spice-debug --gtk-vnc-debug

and attach to the new issue

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: How to install an OS with 2 installation iso file

2020-05-05 Thread Daniel P . Berrangé
On Tue, May 05, 2020 at 03:14:25PM +0100, Richard W.M. Jones wrote:
> On Sun, May 03, 2020 at 03:31:38PM +0530, Harsh chopra wrote:
> > Hi everyone,
> > I was trying to install Centos 2.1 on the virtual machine,
> 
> Ambitious!  Note that older operating systems often don't work in
> qemu, not for any particular reason but just because they don't get
> much attention and things regress, drivers in particular.  Or the
> drivers such as IDE don't emulate corners which are needed by the old
> OS.  For example I tried running some really ancient Linux versions in
> qemu a while back with no success at all.
> 
> Having said that I have run RHEL 3 recently on qemu and that's fairly
> similar to 2.1.
> 
> > the installation
> > was complete 50% but then it asked to insert the second installation iso
> > file.
> > i.e. "Please insert disc 2 in cdrom"
> > I tried to change the XML file and insert the line
> > 
> > under device name  but nothing helped.
> 
> When you say you "tried to change the XML file" - how did you do that?
> You should use ‘virsh edit ’ only, and not attempt to edit any
> XML file directly.  Or of course you can use virt-manager as the other
> reply suggested - it essentially does the same thing as virsh edit.

Actually "virsh edit" is not what you need in this case.

That will update the persistent XML config only.

For a multi-stage installer, you need to perform a CD media change
in the live VM. IIRC the following should work:

  $ virsh change-media $GUESTNAME /dev/hdc --update /path/to/iso

This is equivalent to using  "--eject" followed by "--insert /path/to/iso"


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



ANNOUCE: virt-viewer version 9.0 released

2020-05-01 Thread Daniel P . Berrangé
I am happy to announce a new bugfix release of virt-viewer 9.0:

https://virt-manager.org/download/sources/virt-viewer/virt-viewer-9.0.tar.gz
https://virt-manager.org/download/sources/virt-viewer/virt-viewer-9.0.tar.gz.asc

including experimental Windows installers for Win x86:

https://virt-manager.org/download/sources/virt-viewer/virt-viewer-x86-9.0.msi
https://virt-manager.org/download/sources/virt-viewer/virt-viewer-x86-9.0.msi.asc

and Win x64:

https://virt-manager.org/download/sources/virt-viewer/virt-viewer-x64-9.0.msi
https://virt-manager.org/download/sources/virt-viewer/virt-viewer-x64-9.0.msi.asc

Signatures are created with key

  DAF3 A6FD B26B 6291 2D0E 8E3F BE86 EBB4 1510 4FDF (4096R)

With this release the project has moved over to use GitLab for its
hosting needs instead of Pagure. Instead of sending patches to the
old mailing list, we have adopted modern best practices and now
welcome contributions as merge requests, from where they undergo
automated CI testing of the build. Bug reports, directed towards
upstream maintainers, should also be filed at the GitLab project
now instead of the Red Hat Bugzilla.

All historical releases are available from:

http://virt-manager.org/download/

Changes in this release include:

  - Project moved to https://gitlab.com/virt-viewer/virt-viewer
  - Allow toggling shared clipboard in remote-viewer
  - Fix handling when initial spice connection fails
  - Fix check for govirt library
  - Add bash completion of cli args
  - Improve errors in file transfer dialog
  - Fix ovirt foreign menu storage domains query
  - Prefer TLS certs from oVirt instead of CLI
  - Improve USB device cleanup when Ctrl-C is used
  - Remember monitor mappings across restarts
  - Add a default file extension to screenshots
  - Updated translations
  - Fix misc memory leaks

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: [PATCH] virtinst: use 'host-passthrough' as default 'host' on AArch64

2020-04-07 Thread Daniel P . Berrangé
On Tue, Apr 07, 2020 at 08:01:50AM -0400, Liang Yan wrote:
> Thanks for the input.
> 
> On 4/7/20 3:54 AM, Andrea Bolognani wrote:
> > On Mon, 2020-04-06 at 16:51 -0400, Liang Yan wrote:
> >> virt-install already uses 'host-passthrough' as default when no setup
> >> for cmline '--cpu'. However, it will still use 'host-model' when comes
> >> with '-cpu host'. This will be a problem for aarch64 platfrom as
> >> 'host-model' for aarch64 kvm domain on aarch64 host is not supported yet.
> > 
> > virt-install's --cpu host maps to libvirt's
> > 
> >   
> > 
> > while virt-install's --cpu host-passthrough maps to libvirt's
> > 
> >   
> > 
> > These semantics are not architecture dependent and, while the choice
> > of name is a bit unfortunate considering that QEMU's -cpu host and
> > virt-install's --cpu host have different meanings, I think they're
> > completely unambiguous and reasonably documented, and we should not
> > mess with them.
> > 
> 
> 
> virt-install supports both "host","host-mdoel" and "host-passthrough",
> "--cpu host" is mapped to "host-model" based on "cli:det_model_b",
> however if you check the default setup in "cpu"set_defaults", it has
> different configuration. Btw, it has different setup for different
> architecture already.  Nova also has different setup for aarch64 in my
> defense.
> 
> https://opendev.org/openstack/nova/commit/8bc7b950b7c0a3c80cdd120fe4df97c14848c344
> 
> 
> 
> I agree we could document this situation at least. It does block our
> aarch64 tests while it is ok for x86_64.

That is a bug in the tests - they shouldn't be trying to use a feature
which doesn't make sense for that arch,


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



virt-viewer move to GitLab, with merge requests

2020-04-07 Thread Daniel P . Berrangé
Hi All,

Historically the virt-viewer project has accepted contributions via patches
submitted to this mailing list, with the master git repository originally
being on fedorahosted.org and then on pagure.io.

With libvirt moving to GitLab and merge requests, and spice-gtk/gtk-vnc
also both already being on GitLab, it is time to move virt-viewer too.

Henceforth the master repository is now:

 https://gitlab.com/virt-viewer/virt-viewer/

We now expect patches to be submitted as merge requests to this repo, and
so use of this mailing list should be discontinued for work related to
virt-viewer. Similar use of bugzilla.redhat.com for *upstream* issues
should also stop, in favour of the gitlab issue tracker.

I also intend to stop using virt-manager.org for release news, downloads,
etc, since virt-viewer has nothing in common with virt-manager except for
the fact that both projects were originally started by me. It is increasingly
confusing / unhelpful to use the same website for both projects. I'll probably
just setup a simple static site on gitlab pages.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: [PATCH remote-viewer] Fix potential file descriptor leaks found by Coverity.

2020-04-01 Thread Daniel P . Berrangé
On Wed, Apr 01, 2020 at 09:25:18AM +0200, Julien Ropé wrote:
> The error code returned by virt_viewer_session_open_fd() and 
> virt_viewer_session_channel_open_fd() were not checked. The file descriptor 
> passed to them could then be left opened even if the function failed, causing 
> a leak of resources.
> 
> This was reported by a Coverity scan, logged under 
> https://bugzilla.redhat.com/show_bug.cgi?id=1655792
> 
> Signed-off-by: Julien Ropé 
> ---
>  src/virt-viewer-app.c | 11 +--
>  1 file changed, 9 insertions(+), 2 deletions(-)

Reviewed-by: Daniel P. Berrangé 


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: Adding an "Enable Launch Security" checkbox to the Memory Details dialog

2020-03-27 Thread Daniel P . Berrangé
On Fri, Mar 27, 2020 at 12:13:09PM -0400, Cole Robinson wrote:
> CCing Erik who knows more about that launchSecurity/sev than I do
> 
> On 3/27/20 11:44 AM, Charles Arnold wrote:
> > What is the opinion of adding a checkbox called "Enable Launch
> > Security" under the 'Current allocation' and 'Maximum allocation' boxes
> > on the Details->Memory dialog? It would only be enabled if libvirt
> > detected support for it.
> > 
> 
> Provided libvirt capabilities report everything we need to know to
> whether it's really supported on the host and will actually work, and
> there's a sensible noncontroversial set of defaults we can fill in, then
> a single checkbox is worth considering. It's certainly an advanced
> feature but it's also getting more and more mention these days so maybe
> it's good to get out ahead of any future RFEs.

Two issues right now.  There is a ridiculously low limit of 15 VMs
on first generation CPUs, perhaps not a huge problem for typical
scenarios using virt-manager though.  Second though is that while
libvirt reports whether the feature exists & is supported in QEMU,
QEMU is lieing to us, because it isn't checking whether kvm-amd
actually allows the feature to be used.

https://bugzilla.redhat.com/show_bug.cgi?id=1689202
https://bugzilla.redhat.com/show_bug.cgi?id=1731439

As long as the checkbox isn't enabled by default, its probably ok
to ignore those two issues


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: shm as domain to virt-viewer protocol?

2020-03-26 Thread Daniel P . Berrangé
On Wed, Mar 25, 2020 at 06:16:47PM +, David Geise wrote:
> Hi,
> 
> I've been using an open-source project that's currently in beta called
> Looking Glass (https://looking-glass.hostfission.com/) to achieve high-
> performance VM graphics in a standalone workstation scenario where the
> guest VM and viewer are both on the host. I use it to virtualize my
> windows development environment.  I need it to do work in the Unreal
> Engine, which requires full GPU acceleration.  My understanding is that
> currently GPU isn't supported directly in the virt-manager/virt-viewer
> environment.  I know the cloud is the big thing these days but this
> windows vm-on-host scenario is an increasingly common use-case and it
> seems like there's a great opportunity to improve virt-viewer/virt-manager
> here.  I expect if Linux were to better support a high-performance windows+
> gpu on linux scenario, this should improve consumer pc uptake of linux.

This isn't my area of expertise, but my understanding was/is that the
virtio-gpu device and spice are intended to fill the GPU accelerated
rendering scenario. IIUC this will include shared memory buffers between
guest and host QEMU and host virt-viewer (well spice-gtk), to avoid memory
copies in the local scenario.

I think the main difference is probably the guest focus - virtio-gpu has
targetted OpenGL accleration IIUC with plans for Vulkan too, which are
the common technologies for Linux, where as looking glass is presuambly
focusing on Windows rendering, though honestly this is mostly guesswork
on my part.

> With all that being said, I'd like to ask the virt-viewer developers:
> 
> 1. Is there any interest in developing a low-latency 'standalone' transport
> for virt-viewer?  Is any such work contemplated or underway?  Or is some
> alternative (virtio-gpu comes to mind) being worked on that is an even
> better alternative?

Yeah, my hope/expectation was that virtio-gpu / spice-gtk are going to solve
this problem. From the virt-viewer side, we really leave all the real work
to the spice-gtk client widget. virt-viewer just adds the window management
on top, so we don't really get involved in low level stuff that's performance
critical ourselves.

> 2. Is there truly an intrinsic opposition to, say, a proposed Gtk-shm
> widget & corresponding driver being added to the virt viewer project?
> Would it be possible / non-problematic to develop a platform-neutral
> solution to optimize latency for this local vm scenario?

As you know virt-viewer/virt-manager currently have VNC and SPICE support,
with majority of important work taking place in context of SPICE. It is
certainly possible to add further backends, but that will obviously have a
cost associated with it. Assuming the development work taking place in QEMU
and spice to support accelerated graphics is going to deliver a viable
solution, then I'd not be too enthusiastic about adding an alternative
backend that tries to solve the same problem. I think is more compelling to
users to have a single solution that can deliver everything they need rather
than having to decide between many choices.

> I'm contemplating whether to try to fork virt-viewer and do a shm transport
> integration myself or if I should contribute to the looking-glass project
> instead.  Would the virt-developers like to participate or take the lead
> in this?  Would the repository owners be open to a pull request to such a
> contribution?  Would RedHat sign such a driver?  I expect there might be
> strategic / political implications at play here.

I think perhaps it might be worth taking your message to the spice mailing
list for feedback, as they can probably give a better viewpoint on the real
technical details for how looking-glass project compares to the work that
is being done for virtio-gpu / spice than I can, and thus whether there's
a need for both solutions or better to focus on one only.

> For what it's worth, I've got 25 years of multi-language experience, mostly
> heavy C++, mostly on windows.  My C / DDK experience is rusty (like 15 years
> since I've coded in raw C, even longer since unix!) but I might be able to
> contribute something worthwhile.

One thing that's true for all open source projects is that there is not
enough developer time todo everything they want, so whether its working
on looking-glass vs spice/virtio-gpu, I'm sure there's useful work that
can be contributed. 

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: [PATCH remote-viewer v2 1/1] Remember monitor mapping on close.

2020-02-07 Thread Daniel P . Berrangé
On Fri, Feb 07, 2020 at 10:14:57AM +0100, Julien Ropé wrote:
> When the application is stopped, if the windows are in fullscreen, their
> position on the client will be remembered.
> 
> This change uses the existing option 'monitor-mapping' in the settings
> file to save the position and reuse it on next launch.
> 
> This implements part of the requirement from
> https://bugzilla.redhat.com/show_bug.cgi?id=1179070
> 
> NOTE: this feature is effective only with GTK >= 3.22
> 
> Signed-off-by: Julien Ropé 
> ---
>  src/virt-viewer-app.c | 120 ++
>  1 file changed, 120 insertions(+)

Reviewed-by: Daniel P. Berrangé 

and pushed to git master

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: [PATCH remote-viewer 1/1] Remember monitor mapping on close.

2020-02-05 Thread Daniel P . Berrangé
On Wed, Feb 05, 2020 at 10:02:25AM +0100, Julien ROPE wrote:
> 
> Hi Daniel,
> 
> I looked into this, and I'm not sure how to fix it.
> 
> I'm new to GTK. But from what I can tell, there was no way of accessing 
> Monitor information before GTK 3.22.
> 
> All those APIs that I'm using were introduced in this version, and in 
> previous documentation on GTK, I find no equivalent.
> 
> If any of you know differently, I'd be happy to be corrected.
> 
> 
> If that's the case, it means we can't add this feature on platforms with GTK 
> < 3.22.
> 
> Now I find code here and there in virt-viewer that are protected with "#if 
> GTK_CHECK_VERSION(...)" statements (cf. virt_viewer_window_enter_fullscreen() 
> for instance).
> 
> Should I use something like that to make that code available only with GTK 
> 3.22 and above?

Yes, you can use conditionals like that. You'll need to squash deprecation
warnings as that code does too.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: [PATCH remote-viewer 1/1] Remember monitor mapping on close.

2020-01-28 Thread Daniel P . Berrangé
On Tue, Jan 21, 2020 at 08:48:35AM +0100, Julien Ropé wrote:
> When the application is stopped, if the windows are in fullscreen, their
> position on the client will be remembered.
> 
> This change uses the existing option 'monitor-mapping' in the settings
> file to save the position and reuse it on next launch.
> 
> This implements part of the requirement from
> https://bugzilla.redhat.com/show_bug.cgi?id=1179070
> 
> Signed-off-by: Julien Ropé 
> ---
>  src/virt-viewer-app.c | 114 ++
>  1 file changed, 114 insertions(+)
> 
> diff --git a/src/virt-viewer-app.c b/src/virt-viewer-app.c
> index da8cfa9..f955882 100644
> --- a/src/virt-viewer-app.c
> +++ b/src/virt-viewer-app.c
> @@ -106,6 +106,7 @@ static void virt_viewer_app_set_fullscreen(VirtViewerApp 
> *self, gboolean fullscr
>  static void virt_viewer_app_update_menu_displays(VirtViewerApp *self);
>  static void virt_viewer_update_smartcard_accels(VirtViewerApp *self);
>  static void virt_viewer_app_add_option_entries(VirtViewerApp *self, 
> GOptionContext *context, GOptionGroup *group);
> +static VirtViewerWindow *virt_viewer_app_get_nth_window(VirtViewerApp *self, 
> gint nth);
>  
>  
>  struct _VirtViewerAppPrivate {
> @@ -400,6 +401,116 @@ 
> virt_viewer_app_get_monitor_mapping_for_section(VirtViewerApp *self, const 
> gchar
>  return mapping;
>  }
>  
> +/*
> + *  save the association display/monitor in the config for reuse on next 
> connection
> + */
> +static void virt_viewer_app_set_monitor_mapping_for_display(VirtViewerApp 
> *self, VirtViewerDisplay *display)
> +{
> +GError *error = NULL;
> +gsize nmappings = 0;
> +gchar **mappings = NULL;
> +gchar **tokens = NULL;
> +
> +int i;
> +
> +gint virt_viewer_display = virt_viewer_display_get_nth(display);
> +gint virt_viewer_monitor = virt_viewer_display_get_monitor(display);
> +
> +if (virt_viewer_monitor == -1) {
> +// find which monitor the window is on
> +GdkDisplay *gdk_dpy = gdk_display_get_default();
> +VirtViewerWindow *vvWindow = virt_viewer_app_get_nth_window(self, 
> virt_viewer_display) ;
> +GdkWindow *window = 
> gtk_widget_get_window(GTK_WIDGET(virt_viewer_window_get_window(vvWindow)));
> +GdkMonitor *pMonitor = gdk_display_get_monitor_at_window(gdk_dpy, 
> window);

This breaks the build due to using APIs that are newer than our min
reqiured GTK version:

virt-viewer-app.c: In function 
'virt_viewer_app_set_monitor_mapping_for_display':
virt-viewer-app.c:428:9: warning: 'gdk_display_get_monitor_at_window' is 
deprecated: Not available before 3.22 [-Wdeprecated-declarations]
  428 | GdkMonitor *pMonitor = 
gdk_display_get_monitor_at_window(gdk_dpy, window);
  | ^~
In file included from /usr/include/gtk-3.0/gdk/gdkscreen.h:32,
 from /usr/include/gtk-3.0/gdk/gdkapplaunchcontext.h:31,
 from /usr/include/gtk-3.0/gdk/gdk.h:32,
 from /usr/include/gtk-3.0/gtk/gtk.h:30,
 from virt-viewer-app.c:28:
/usr/include/gtk-3.0/gdk/gdkdisplay.h:194:14: note: declared here
  194 | GdkMonitor * gdk_display_get_monitor_at_window (GdkDisplay *display,
  |  ^


We currently request

  GTK_REQUIRED="3.12"
  GTK_ENCODED_VERSION="GDK_VERSION_3_12"

based on our CI platforms, Ubuntu 16.04 is the oldest GTK
at version 3.18.9

So we can bump our min GTK to 3.18, but we can't use
APIs from 3.22 yet

Once we drop Ubuntu 16.04 - most likely around the April 2020 timefram
when Ubuntu 20.04 is released, then Debian 9 & CentOS 7 will be the
oldest at version 3.22.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: virt-manager version question

2020-01-28 Thread Daniel P . Berrangé
On Mon, Jan 27, 2020 at 02:15:28PM -0500, Scott Reeve wrote:
> Virtual Machine Manager version I have is 0.9.5.
> Says: Copyright (C) 2006-2011 Red Hat Inc.
> 
> I see this version on a ubuntu machine running 14.04 (yes old).
> See the same version on a Centos machine running 6.6.
> 
> On a Centos 7 machine, I get virt-manager 1.5.
> 
> Is it possible to run the latest virt-manager on ubuntu 14.04 or Centos 6.6

I don't know what your constraints are wrt the operating systems you
have to use, but if at all possible, I'd recommend installing a much
more modern OS. The virtualization world has massively advanced in
the time since Ubuntu 14.04 / Centos 6.6.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: [virt-viewer PATCH v4 1/1] remote-viewer: add handler for SIGINT signal

2020-01-17 Thread Daniel P . Berrangé
On Fri, Jan 17, 2020 at 04:06:13PM +0100, Francesco Giudici wrote:
> When remote-viewer is started from terminal, CTRL-C sends a SIGINT
> signal to the program causing immediate termination. On linux clients
> usb redirected devices are left without any kernel driver attached,
> causing them to appear as no more available to the host.
> Add a SIGINT handler to allow a clean exit, in particular to allow
> the kernel to attach back the host driver.
> The issue is present on linux only.
> 
> To perform usb device redirection, virt-viewer leverages spice-gtk
> library, which in turn relies on usbredir library, which uses libusb.
> In order to take control of the usb device the auto-loaded kernel
> driver must be detached. This is achieved (in the very end) with
> libusb_detach_kernel_driver(). Then the device interfaces can be claimed
> with libusb_claim_interface() and get in control to the application.
> During normal application termination, the usb channel is teared down,
> performing a reset of the usb device and giving back the control of the
> device to the kernel (libusb_attach_kernel_driver()).
> If the application quits without doing this, the device interfaces will
> end up with no driver attached, making them not usable in the host
> system.
> 
> Note that enabling libusb_set_auto_detach_kernel_driver() does not solve
> the issue, as this is just a convenient API from libusb: libusb will
> take care of detaching/attaching the driver to the interfaces of the usb
> device each time a call to libusb_release_interface() and
> libusb_claim_interface() is performed. An unexpected quit of the
> application will skip the libusb_release_interface() call too, leaving
> the interfaces without any driver attached.
> 
> Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1713311
> 
> Signed-off-by: Francesco Giudici 
> ---
>  src/virt-viewer-app.c | 25 +
>  1 file changed, 25 insertions(+)

Thanks, this is merged

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|



Re: [virt-viewer PATCH v2] remote-viewer: add handler for SIGINT signal

2020-01-17 Thread Daniel P . Berrangé
On Fri, Jan 17, 2020 at 12:04:08PM +0100, Francesco Giudici wrote:
> When remote-viewer is started from terminal, CTRL-C sends a SIGINT
> signal to the program causing immediate termination. On linux clients
> usb redirected devices are left without any kernel driver attached,
> causing them to appear as no more available to the host.
> Add a SIGINT handler to allow a clean exit, in particular to allow
> the kernel to attach back the host driver.
> The issue is present on linux only.
> 
> To perform usb device redirection, virt-viewer leverages spice-gtk
> library, which in turn relies on usbredir library, which uses libusb.
> In order to take control of the usb device the auto-loaded kernel
> driver must be detached. This is achieved (in the very end) with
> libusb_detach_kernel_driver(). Then the device interfaces can be claimed
> with libusb_claim_interface() and get in control to the application.
> During normal application termination, the usb channel is teared down,
> performing a reset of the usb device and giving back the control of the
> device to the kernel (libusb_attach_kernel_driver()).
> If the application quits without doing this, the device interfaces will
> end up with no driver attached, making them not usable in the host
> system.
> 
> Note that enabling libusb_set_auto_detach_kernel_driver() does not solve
> the issue, as this is just a convenient API from libusb: libusb will
> take care of detaching/attaching the driver to the interfaces of the usb
> device each time a call to libusb_release_interface() and
> libusb_claim_interface() is performed. An unexpected quit of the
> application will skip the libusb_release_interface() call too, leaving
> the interfaces without any driver attached.
> 
> Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1713311
> 
> Signed-off-by: Francesco Giudici 
> ---
>  src/virt-viewer-app.c | 79 +++
>  1 file changed, 79 insertions(+)
> 
> diff --git a/src/virt-viewer-app.c b/src/virt-viewer-app.c
> index da8cfa9..06e237b 100644
> --- a/src/virt-viewer-app.c
> +++ b/src/virt-viewer-app.c
> @@ -36,6 +36,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>  
>  #ifdef HAVE_SYS_SOCKET_H
>  #include 
> @@ -1756,6 +1757,74 @@ static gboolean opt_fullscreen = FALSE;
>  static gboolean opt_kiosk = FALSE;
>  static gboolean opt_kiosk_quit = FALSE;
>  
> +#ifndef G_OS_WIN32
> +static int sigint_pipe[2];
> +
> +static void
> +sigint_handler(int signum)
> +{
> +int savedErrno;
> +
> +g_return_if_fail(signum == SIGINT);
> +
> +savedErrno = errno;
> +if (write(sigint_pipe[1], "x", 1) == -1 && errno != EAGAIN)
> +g_debug("SIGINT handler failure\n");
> +errno = savedErrno;
> +}
> +
> +static void
> +register_sigint_handler()
> +{
> +int flags, i;
> +struct sigaction sa;
> +
> +if (pipe(sigint_pipe) == -1)
> +goto err;
> +
> +for (i = 0; i < 2; i++) {
> +flags = fcntl(sigint_pipe[i], F_GETFL);
> +if (flags == -1)
> +goto err;
> +flags |= O_NONBLOCK;
> +if (fcntl(sigint_pipe[i], F_SETFL, flags) == -1)
> +goto err;
> +}
> +
> +sigemptyset(_mask);
> +sa.sa_flags = SA_RESTART;
> +sa.sa_handler = sigint_handler;
> +if (sigaction(SIGINT, , NULL) == -1)
> +goto err;
> +
> +return;
> +
> +err:
> +g_debug("Cannot register SIGINT handler\n");
> +}
> +
> +static gboolean
> +sigint_cb(GIOChannel *source,
> +  GIOCondition condition,
> +  gpointer data)
> +{
> +VirtViewerApp *self = VIRT_VIEWER_APP(data);
> +VirtViewerAppPrivate *priv = self->priv;
> +gchar sbuf;
> +
> +g_assert(condition == G_IO_IN);
> +
> +g_debug("got SIGINT, quitting\n");
> +if (priv->started)
> +virt_viewer_app_quit(self);
> +else
> +exit(0);
> +
> +g_io_channel_read_chars (source, , 1, NULL, NULL);
> +return TRUE;
> +}
> +#endif
> +
>  static void
>  title_maybe_changed(VirtViewerApp *self, GParamSpec* pspec G_GNUC_UNUSED, 
> gpointer user_data G_GNUC_UNUSED)
>  {
> @@ -1766,10 +1835,20 @@ static void
>  virt_viewer_app_init(VirtViewerApp *self)
>  {
>  GError *error = NULL;
> +#ifndef G_OS_WIN32
> +GIOChannel *sigint_channel = NULL;
> +#endif
> +
>  self->priv = virt_viewer_app_get_instance_private(self);
>  
>  gtk_window_set_default_icon_name("virt-viewer");
>  
> +#ifndef G_OS_WIN32
> +register_sigint_handler();
> +sigint_channel = g_io_channel_unix_new(sigint_pipe[0]);
> +g_io_add_watch(sigint_channel, G_IO_IN, sigint_cb, self);
> +#endif

Just yesterday I learnt that GLib has native support for signal handling
in its event loop which allows this to be simplified:

   g_unix_signal_add (SIGINT, sigint_cb, self);

>  self->priv->displays = g_hash_table_new_full(g_direct_hash, 
> g_direct_equal, NULL, g_object_unref);
>  self->priv->config = g_key_file_new();
>  self->priv->config_file = 

Re: [virt-tools-list] FYI: intention to remove mail subject prefix & footer text

2020-01-15 Thread Daniel P . Berrangé
FYI, I'm intending to make this change tomorrow (Thurs Jan 16th)

On Fri, Jan 10, 2020 at 11:44:04AM +, Daniel P. Berrangé wrote:
> Hi List Subscribers,
> 
> In recent months we have been seeing an increasing number of bounced
> deliveries from the mailing list due to DMARC policies on list
> subscriber's mail servers. IOW, many subscribers are only receiving
> a subset of mails sent to this list.
> 
> We believe the root cause of many of the problems is that mailman is
> modifying the mail subject to add the "[virt-tools-list]" prefix, and
> modifying the mail body to add the footer with links to the listinfo
> page.
> 
> These modifications invalidate the DKIM signatures on mails sent to
> the list by some of our subscribers. This in turn causes DMARC policy
> rejections by the destination SMTP servers when mailman delivers
> messages.
> 
> The solution is to disable any feature in mailman which modifies
> parts of the mail validated by the DKIM signature. This means removing
> the subject prefix and the mail body header. Further information on
> this approach can be seen here:
> 
>https://begriffs.com/posts/2018-09-18-dmarc-mailing-list.html
> 
> QEMU has made the same change on their mailing list last year:
> 
>https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg00416.html
> 
> If you are currently doing mail filtering / sorting based on the subject
> prefix, you will need to change to use the List-Id header instead.
> 
> I will wait until the latter part of next week before making this change
> to allow people time to adapt any filters.
> 
> Regards,
> Daniel
> -- 
> |: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org -o-https://fstop138.berrange.com :|
> |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
> 
> ___
> virt-tools-list mailing list
> virt-tools-list@redhat.com
> https://www.redhat.com/mailman/listinfo/virt-tools-list

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

[virt-tools-list] FYI: intention to remove mail subject prefix & footer text

2020-01-10 Thread Daniel P . Berrangé
Hi List Subscribers,

In recent months we have been seeing an increasing number of bounced
deliveries from the mailing list due to DMARC policies on list
subscriber's mail servers. IOW, many subscribers are only receiving
a subset of mails sent to this list.

We believe the root cause of many of the problems is that mailman is
modifying the mail subject to add the "[virt-tools-list]" prefix, and
modifying the mail body to add the footer with links to the listinfo
page.

These modifications invalidate the DKIM signatures on mails sent to
the list by some of our subscribers. This in turn causes DMARC policy
rejections by the destination SMTP servers when mailman delivers
messages.

The solution is to disable any feature in mailman which modifies
parts of the mail validated by the DKIM signature. This means removing
the subject prefix and the mail body header. Further information on
this approach can be seen here:

   https://begriffs.com/posts/2018-09-18-dmarc-mailing-list.html

QEMU has made the same change on their mailing list last year:

   https://lists.gnu.org/archive/html/qemu-devel/2019-09/msg00416.html

If you are currently doing mail filtering / sorting based on the subject
prefix, you will need to change to use the List-Id header instead.

I will wait until the latter part of next week before making this change
to allow people time to adapt any filters.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list



Re: [virt-tools-list] virt-manager: snapshot uefi vm

2019-12-24 Thread Daniel P . Berrangé
On Mon, Dec 23, 2019 at 03:27:05PM -0800, femi wrote:
> When will snapshots of UEFI virtual machines become a reality?

It is blocked by limitations in QEMU. The savevm command doens't have any
way to be told a list of disks to snapshot & this breaks with UEFI as it
considers the NVRAM file a disk to snapshot. So savevm needs rewriting in
QEMU, and then wiring up in libvirt before it can be done in virt-manager.
Unfortunately no one in QEMU seems to care about internal snapshots, only
external snapshots, so there's no ETA :-(


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list



Re: [virt-tools-list] virt-manager test suite hangs on FreeBSD

2019-12-12 Thread Daniel P . Berrangé
On Thu, Dec 12, 2019 at 09:47:42AM +0100, Andrea Bolognani wrote:
> On Wed, 2019-12-11 at 18:01 -0500, Cole Robinson wrote:
> > On 12/11/19 7:40 AM, Andrea Bolognani wrote:
> > > On Wed, 2019-12-11 at 07:28 -0500, Cole Robinson wrote:
> > > > On 12/11/19 5:22 AM, Andrea Bolognani wrote:
> > > > > I have no idea how to debug this thing. Can an actual virt-manager
> > > > > developer jump in?
> > 
> > I reproduced. Part of the test suite was forking off /bin/true to hit a
> > code path. But when /bin/true wasn't available it would leave a stranded
> > process which caused the hang. Fixed both now upstream
> > 
> > > > You can
> > > > use './setup.py test --debug' which may give more info where it is
> > > > hanging. Possibly somewhere in the cli tests where we try to handle mock
> > > > stdin or try to fake --wait timeouts
> > > 
> > > Doing so doesn't really result in additional information being
> > > printed out: it still just quietly hangs there.
> > 
> > This was a separate bug, calling into the cli tools for clitest.py would
> > reset the test suite debugging. That's fixed too
> 
> That's awesome, thank you so much! I already posted a patch[1] that
> enables the test suite on FreeBSD in the CentOS CI environment, so
> if a regression manages to sneak in we'll catch it right away.
> 
> One more thing. Right now virt-manager is the only project we have on
> CentOS CI that uses the python3-libxml2 bindings in the first place,
> with all others using python3-lxml instead, and that leads to a
> reduced test matrix for it because the former are not available on
> Ubuntu 16.04 and CentOS 7.
> 
> So that makes me wonder, does python3-lxml provide a better API or
> something? Would it be a massive amount of work to port virt-manager
> to it, and would it even make sense to do so? In due time we'll stop
> supporting those two targets anyway...

They both use libxml native library under the hood. The lxml bindings
are exposing libxml in a way that is "more pythonic" and thus easier
and safer to use.  So in long term I expect virt-manager would likely
benefit from lxml, if anyone wants to do the grunt work.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list



Re: [virt-tools-list] virt-install and cloud-init, feedback wanted

2019-11-25 Thread Daniel P . Berrangé
On Mon, Nov 25, 2019 at 11:29:38AM +, Richard W.M. Jones wrote:
> On Mon, Nov 25, 2019 at 11:23:14AM +0000, Daniel P. Berrangé wrote:
> > On Mon, Nov 25, 2019 at 11:13:00AM +, Richard W.M. Jones wrote:
> > > On Fri, Nov 22, 2019 at 09:56:17AM +, Daniel P. Berrangé wrote:
> > > > On Thu, Nov 21, 2019 at 09:39:46PM -0500, Dusty Mabe wrote:
> > > > > 
> > > > > 
> > > > > On 11/21/19 6:20 AM, Daniel P. Berrangé wrote:
> > > > > > On Thu, Nov 21, 2019 at 11:07:24AM +0000, Richard W.M. Jones wrote:
> > > > > >> On Thu, Nov 21, 2019 at 10:34:14AM +, Daniel P. Berrangé wrote:
> > > > > >>> On Wed, Nov 20, 2019 at 08:18:01PM -0500, Dusty Mabe wrote:
> > > > > >>>> Basically in Fedora CoreOS we need a generic user data mechanism 
> > > > > >>>> that works across
> > > > > >>>> platforms (x86_64, aarch64, ppc64le, s390x) and doesn't have 
> > > > > >>>> possible race conditions.
> > > > > >>>> Right now we're using `-fw_cfg` but it's not cross platform. We 
> > > > > >>>> don't have an answer
> > > > > >>>> yet: 
> > > > > >>>> https://github.com/coreos/ignition/issues/666#issuecomment-452835654
> > > > > >>>
> > > > > >>> For platform portability you need to find some device that is 
> > > > > >>> common
> > > > > >>> across all platforms, and either disk or network are the only two
> > > > > >>> good options that exist today or for the forseeable future.  If 
> > > > > >>> those
> > > > > >>> aren't acceptable then all we have left are platform specific 
> > > > > >>> options.
> > > > > >>
> > > > > >> While it's not a "good option that exists today", AF_VSOCK would 
> > > > > >> be a
> > > > > >> good choice to settle on in the future.  It's completely cross
> > > > > >> platform, available for Windows, and doesn't interfere with 
> > > > > >> existing
> > > > > >> network or disk devices.
> > > > > >>
> > > > > >> Would needing virtio be a barrier?  Our impl of AF_VSOCK runs over
> > > > > >> virtio, but there are other transports.
> > > > > > 
> > > > > > From a cloud-init POV, I don't see virtio as a barrier. Defining an
> > > > > > AF_VSOCK data source for it should be quite straightforward really
> > > > > > and they already have so many data sources, it seems reasonable
> > > > > > that they'd accept one more.
> > > > > > 
> > > > > > On the host side there's still the task of providing a metdata
> > > > > > service to expose the data, which is outside scope of virt-install.
> > > > > > 
> > > > > 
> > > > > Thanks for the fruitful conversation here regarding a cross platform 
> > > > > data source
> > > > > that we could use. Is this worth us writing up into a request for an 
> > > > > issue tracker
> > > > > somewhere where it could be discussed further?
> > > > 
> > > > Possibly - depends if AF_VSOCK is viable or not for Ignition.
> > > > 
> > > > Previously when we suggested use of a plain disk for Ignition, the issue
> > > > raised was that the /dev/sdXXX device nodes take a non-deterministic
> > > > amount of time to appear, and ignition doesn't know how long to wait
> > > > for them, because the disks might not exist at all in the first place.
> > > > 
> > > > Using AF_VSOCK is going to have exactly this same problem, so personally
> > > > I can't see it can satisfy Ignition, where virtio-blk failed.
> > > 
> > > Be interesting what exactly their concerns were.  Obviously both PCI
> > > initialization and (especially) kernel disk enumeration can take a
> > > long time with guests that have a lot of PCI slots or disks.  But
> > > that's what udev is for, right :-)  Couldn't they just hook up their
> > > initialization to a udev rule?
> > 
> > I mention it above - they don't know whether the user has even
> > provided the metadata in the first place. They don't want to wait
> > for a dev

Re: [virt-tools-list] virt-install and cloud-init, feedback wanted

2019-11-25 Thread Daniel P . Berrangé
On Mon, Nov 25, 2019 at 11:13:00AM +, Richard W.M. Jones wrote:
> On Fri, Nov 22, 2019 at 09:56:17AM +0000, Daniel P. Berrangé wrote:
> > On Thu, Nov 21, 2019 at 09:39:46PM -0500, Dusty Mabe wrote:
> > > 
> > > 
> > > On 11/21/19 6:20 AM, Daniel P. Berrangé wrote:
> > > > On Thu, Nov 21, 2019 at 11:07:24AM +, Richard W.M. Jones wrote:
> > > >> On Thu, Nov 21, 2019 at 10:34:14AM +, Daniel P. Berrangé wrote:
> > > >>> On Wed, Nov 20, 2019 at 08:18:01PM -0500, Dusty Mabe wrote:
> > > >>>> Basically in Fedora CoreOS we need a generic user data mechanism 
> > > >>>> that works across
> > > >>>> platforms (x86_64, aarch64, ppc64le, s390x) and doesn't have 
> > > >>>> possible race conditions.
> > > >>>> Right now we're using `-fw_cfg` but it's not cross platform. We 
> > > >>>> don't have an answer
> > > >>>> yet: 
> > > >>>> https://github.com/coreos/ignition/issues/666#issuecomment-452835654
> > > >>>
> > > >>> For platform portability you need to find some device that is common
> > > >>> across all platforms, and either disk or network are the only two
> > > >>> good options that exist today or for the forseeable future.  If those
> > > >>> aren't acceptable then all we have left are platform specific options.
> > > >>
> > > >> While it's not a "good option that exists today", AF_VSOCK would be a
> > > >> good choice to settle on in the future.  It's completely cross
> > > >> platform, available for Windows, and doesn't interfere with existing
> > > >> network or disk devices.
> > > >>
> > > >> Would needing virtio be a barrier?  Our impl of AF_VSOCK runs over
> > > >> virtio, but there are other transports.
> > > > 
> > > > From a cloud-init POV, I don't see virtio as a barrier. Defining an
> > > > AF_VSOCK data source for it should be quite straightforward really
> > > > and they already have so many data sources, it seems reasonable
> > > > that they'd accept one more.
> > > > 
> > > > On the host side there's still the task of providing a metdata
> > > > service to expose the data, which is outside scope of virt-install.
> > > > 
> > > 
> > > Thanks for the fruitful conversation here regarding a cross platform data 
> > > source
> > > that we could use. Is this worth us writing up into a request for an 
> > > issue tracker
> > > somewhere where it could be discussed further?
> > 
> > Possibly - depends if AF_VSOCK is viable or not for Ignition.
> > 
> > Previously when we suggested use of a plain disk for Ignition, the issue
> > raised was that the /dev/sdXXX device nodes take a non-deterministic
> > amount of time to appear, and ignition doesn't know how long to wait
> > for them, because the disks might not exist at all in the first place.
> > 
> > Using AF_VSOCK is going to have exactly this same problem, so personally
> > I can't see it can satisfy Ignition, where virtio-blk failed.
> 
> Be interesting what exactly their concerns were.  Obviously both PCI
> initialization and (especially) kernel disk enumeration can take a
> long time with guests that have a lot of PCI slots or disks.  But
> that's what udev is for, right :-)  Couldn't they just hook up their
> initialization to a udev rule?

I mention it above - they don't know whether the user has even
provided the metadata in the first place. They don't want to wait
for a device that doesn't exist or they'll be waiting forever. If
you wait with a timeout, then it delays boot, and creates failure
conditions under high load.

> > The main benefit I see of AF_VSOCK over virtio-blk disk / cdrom, is that
> > it is a live data channel so you are not restricted to providing data that
> > is fixed at boot time, it can be live updated on the fly.
> > 
> > The only mechanism that can avoid the problem of waiting for some
> > device driver to initialize asynchonrously is to use a directly
> > mapped memory region that is immediately exposed by the core kenrel
> > functionality. This is what fw_cfg, SMBIOS, and/or kernel command
> > line args all achieve, and the other options not based on mapped
> > memory will all fail to achieve.
> > 
> > Regards,
> > Daniel
> 
> I added Stefan to CC because he might be interested in this use case too.
> 
> Rich.
> 
> -- 
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> Fedora Windows cross-compiler. Compile Windows programs, test, and
> build Windows installers. Over 100 libraries supported.
> http://fedoraproject.org/wiki/MinGW

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

Re: [virt-tools-list] virt-install and cloud-init, feedback wanted

2019-11-22 Thread Daniel P . Berrangé
On Thu, Nov 21, 2019 at 07:44:38PM -0500, Cole Robinson wrote:
> On 11/21/19 5:28 AM, Daniel P. Berrangé wrote:
> > One option is to take the lazy approach and always enable cloud init
> > if we are given a pre-built disk image.
> > 
> >  * If the disk image doesn't support cloud init, then its fairly
> >harmless to provide cloud-init data.
> > 
> >  * If the disk image does support cloud init but has already been
> >booted, and the user has marked cloud init as disable, then it
> >is again harmless to provide cloud-init data
> > 
> >  * If the disk image does support cloud init but has already been
> >booted, and the user left cloud init enabled so that it runs on
> >every boot, then passing cloud init data is probably good thing.
> > 
> >  * If the disk image does support cloud init and has never been
> >booted, then obviously providing cloud init data is good
> >
> 
> If we had a SMBIOS or -fw_cfg option, then maybe this is an option. But
> I don't think I could stomach doing this by default with the current
> 'cdrom' approach, surely there would be complications, and at minimum
> complaints about a needless attached CDROM device.

Yeah, I can understand this concern.

> I think the 'cdrom' case is good enough for a first crack at this and
> anything else can build on that initial support. I think the command
> line is flexible enough that in the future we could have a --cloud-init
> method=SMBIOS/... for users to explicitly choose what injection method
> they want, but otherwise virt-install will choose a default, which may
> be different depending on the arch.

That sounds reasonable.

> >> * Do you have an opinion of what behavior bare '--cloud-init' should give?
> > 
> > My feeling is that a bare --cloud-init should inject the current user's
> > SSH identity and/or authorized keys list.
> > 
> > ie either we grab $HOME/.ssh/id_rsa.pub, or we grab all the keys from
> > the SSH agent connection (ie ssh-add -l)
> > 
> > Given that virt-intsall is run as root, there probably isn't a
> > /root/.ssh/id_rsa.pub file for most people. Grabbing the ssh-agent
> > authorized keys is probably best default approach.
> 
> How to handle ssh keys kinda makes my head spin. I've shrunk the CC list
> on this mail to just the folks that mentioned ssh in some way in their
> replies
> 
> If I'm thinking of the ideal interactive virt-install UI, it would be:
> 
> $ virt-install --name MYVM --disk Fedora-Cloud-...qcow2 --cloud-init
> INFO: SSH keys added to MYVM
> INFO: Connecting to MYVM with: ssh fed...@192.168.122.xxx
> ...
> [fedora@localhost:~] $
> 
> 
> If all the pieces come together, that's pretty nice. But there's a lot
> of missing pieces needed for that, and some unclear bits:
> 
> * What is the cloud-init user name? Passing in an ssh key gives access
> to the default cloud-init configured username, which is different
> depending on the distro (might be fedora, ubuntu, etc). To properly
> handle this, we either need libosinfo distro detection, or use
> libguestfs to fish the default user out of /etc/cloud/cloud.cfg in the
> VM. And root account is typically locked, and ssh access disabled, and
> even if we could hack things to give root ssh access it seems like going
> against the grain WRT cloud images.

IIRC, you can pass an SSH key for root separately, but not sure how
widely supported that is. Sticking to just providing 1 SSH key for
the default unprivileged user likely suffices.

> So best I can think of for an immediate solution for SSH is to print
> something explanatory:
> 
> INFO: you can ssh to the VM using the distro's default username. This is
> usually one of: fedora, ubuntu, ...
> 
> Or something along those lines. Maybe we can tie that into --os-variant
> as well so the user can inform us what distro it is. Still it will take
> some work.

This is a bit of a tangent, but this makes me wonder if qcow2 format
could benefit from having support for arbitrary user metadata fields

People building cloud images could then tag their images with useful
things including vendor, product name, product version, whether it
supports cloud-init or ignition, what the default user account name
is, suggested min ram, and so on.

Of course people might say this is all stuff that's within scope
of external metadata like OVF, but I've rarely, if ever, seen anyone
say anything nice about OVF. Also there's a certain convenience in
being able to directly pass around a bootable disk image, rather than
a disk image + separate metadata file, or worse a ZIP file containing
both which you have to unpack.

Having some simple metadata in the qcow2 file would simplify life I
think. If someone 

Re: [virt-tools-list] virt-install and cloud-init, feedback wanted

2019-11-22 Thread Daniel P . Berrangé
On Thu, Nov 21, 2019 at 09:39:46PM -0500, Dusty Mabe wrote:
> 
> 
> On 11/21/19 6:20 AM, Daniel P. Berrangé wrote:
> > On Thu, Nov 21, 2019 at 11:07:24AM +, Richard W.M. Jones wrote:
> >> On Thu, Nov 21, 2019 at 10:34:14AM +, Daniel P. Berrangé wrote:
> >>> On Wed, Nov 20, 2019 at 08:18:01PM -0500, Dusty Mabe wrote:
> >>>> Basically in Fedora CoreOS we need a generic user data mechanism that 
> >>>> works across
> >>>> platforms (x86_64, aarch64, ppc64le, s390x) and doesn't have possible 
> >>>> race conditions.
> >>>> Right now we're using `-fw_cfg` but it's not cross platform. We don't 
> >>>> have an answer
> >>>> yet: https://github.com/coreos/ignition/issues/666#issuecomment-452835654
> >>>
> >>> For platform portability you need to find some device that is common
> >>> across all platforms, and either disk or network are the only two
> >>> good options that exist today or for the forseeable future.  If those
> >>> aren't acceptable then all we have left are platform specific options.
> >>
> >> While it's not a "good option that exists today", AF_VSOCK would be a
> >> good choice to settle on in the future.  It's completely cross
> >> platform, available for Windows, and doesn't interfere with existing
> >> network or disk devices.
> >>
> >> Would needing virtio be a barrier?  Our impl of AF_VSOCK runs over
> >> virtio, but there are other transports.
> > 
> > From a cloud-init POV, I don't see virtio as a barrier. Defining an
> > AF_VSOCK data source for it should be quite straightforward really
> > and they already have so many data sources, it seems reasonable
> > that they'd accept one more.
> > 
> > On the host side there's still the task of providing a metdata
> > service to expose the data, which is outside scope of virt-install.
> > 
> 
> Thanks for the fruitful conversation here regarding a cross platform data 
> source
> that we could use. Is this worth us writing up into a request for an issue 
> tracker
> somewhere where it could be discussed further?

Possibly - depends if AF_VSOCK is viable or not for Ignition.

Previously when we suggested use of a plain disk for Ignition, the issue
raised was that the /dev/sdXXX device nodes take a non-deterministic
amount of time to appear, and ignition doesn't know how long to wait
for them, because the disks might not exist at all in the first place.

Using AF_VSOCK is going to have exactly this same problem, so personally
I can't see it can satisfy Ignition, where virtio-blk failed.

The main benefit I see of AF_VSOCK over virtio-blk disk / cdrom, is that
it is a live data channel so you are not restricted to providing data that
is fixed at boot time, it can be live updated on the fly.

The only mechanism that can avoid the problem of waiting for some
device driver to initialize asynchonrously is to use a directly
mapped memory region that is immediately exposed by the core kenrel
functionality. This is what fw_cfg, SMBIOS, and/or kernel command
line args all achieve, and the other options not based on mapped
memory will all fail to achieve.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

Re: [virt-tools-list] virt-install and cloud-init, feedback wanted

2019-11-21 Thread Daniel P . Berrangé
On Thu, Nov 21, 2019 at 12:04:11PM +0100, Christian Ehrhardt wrote:
> On Thu, Nov 21, 2019 at 11:52 AM Florian Weimer  wrote:
> >
> > * Daniel P. Berrangé:
> >
> > >> This goes probably in a different direction of what has been implement
> > >> so far, but would it actually harm to enable the network-based
> > >> instance-data injection by default?  The advantage would be that it also
> > >> blocks these requests from leaking to untrusted parties, which could
> > >> then serve bogus data to compromise the virtual machine.
> > >
> > > I don't understand what you mean by leaking data to untrusted parties
> > > here in contetx of config drive ? I've considerd the config drive to
> > > be more secure / less risky than network service.
> >
> > I'm assuming that cloud-init will try all sources in parallel, given
> > that there's a delay for both the network coming about and hardware
> > being detected.
> 
> Hi,
> there are many controls to that. By default it is most configurable,
> but you can set it to your needs of e.g. only local data sources.
> 
> As outlined by Daniel already this is pretty safe, but if still
> concerned about it, you can control it [1]:
> - image builders can disable things by a drop in file that controls
> which sources are queried
> - local users can control it via kernel-commandline (which most tools
> provide an option to append things to)

With pre-built disks images, virt-install can't directly control the
kernel command line without using a tool like guestfish to get inside
the image & modify grub config.

Cloud-init can, however, look at SMBIOS to extract the information
for the specific data source to use. Currently it is abusing the
system-serial-number field for this purpose. I proposed a patch
to make it use the SMBIOS OEM strings field instead

  https://bugs.launchpad.net/cloud-init/+bug/1753558

Either way though, virt-install can set the SMBIOS data in the
guest to explicitly tell cloud-init to only use the configdrive
ISO, and thus prevent it ever talking to the network metdata
service.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

Re: [virt-tools-list] virt-install and cloud-init, feedback wanted

2019-11-21 Thread Daniel P . Berrangé
On Thu, Nov 21, 2019 at 12:06:49PM +0100, Florian Weimer wrote:
> * Daniel P. Berrangé:
> 
> > On Thu, Nov 21, 2019 at 11:52:26AM +0100, Florian Weimer wrote:
> >> * Daniel P. Berrangé:
> >> 
> >> >> This goes probably in a different direction of what has been implement
> >> >> so far, but would it actually harm to enable the network-based
> >> >> instance-data injection by default?  The advantage would be that it also
> >> >> blocks these requests from leaking to untrusted parties, which could
> >> >> then serve bogus data to compromise the virtual machine.
> >> >
> >> > I don't understand what you mean by leaking data to untrusted parties
> >> > here in contetx of config drive ? I've considerd the config drive to
> >> > be more secure / less risky than network service.
> >> 
> >> I'm assuming that cloud-init will try all sources in parallel, given
> >> that there's a delay for both the network coming about and hardware
> >> being detected.
> >
> > IIRC, the network sources all use link-local addresses, so by default
> > you would need to have configured the 169.254.169.254 on one of the
> > NICs on the host that the guest can reach. It connects to port 80 on
> > this address.
> 
> Too many IPv4 deployment treat 169.254.0.0/16 as global unicast
> addresses and forward them via the default route.  Only once they reach
> the DFZ, these packets get dropped, but only if no one has announced a
> route for it.

Ah, I see what you mean now.

> The instance-data DNS lookup is typically forwarded to the DNS root
> servers.  Local resolvers will only filter it if they are
> DNSSEC-enabled.
> 
> I have argued for a long time that separate cloud and local KVM images
> are needed because the cloud images are dangerous in a non-cloud
> environment, but so far without success.

Libvirt has support for per-guest NIC network filters and ships with
a "clean-traffic" filter that blocks ARP, IP & MAC spoofing. We could
use this feature as a way to block access to the cloud-init metadata
service IP address if desired.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

Re: [virt-tools-list] virt-install and cloud-init, feedback wanted

2019-11-21 Thread Daniel P . Berrangé
On Thu, Nov 21, 2019 at 11:07:24AM +, Richard W.M. Jones wrote:
> On Thu, Nov 21, 2019 at 10:34:14AM +0000, Daniel P. Berrangé wrote:
> > On Wed, Nov 20, 2019 at 08:18:01PM -0500, Dusty Mabe wrote:
> > > Basically in Fedora CoreOS we need a generic user data mechanism that 
> > > works across
> > > platforms (x86_64, aarch64, ppc64le, s390x) and doesn't have possible 
> > > race conditions.
> > > Right now we're using `-fw_cfg` but it's not cross platform. We don't 
> > > have an answer
> > > yet: https://github.com/coreos/ignition/issues/666#issuecomment-452835654
> > 
> > For platform portability you need to find some device that is common
> > across all platforms, and either disk or network are the only two
> > good options that exist today or for the forseeable future.  If those
> > aren't acceptable then all we have left are platform specific options.
> 
> While it's not a "good option that exists today", AF_VSOCK would be a
> good choice to settle on in the future.  It's completely cross
> platform, available for Windows, and doesn't interfere with existing
> network or disk devices.
> 
> Would needing virtio be a barrier?  Our impl of AF_VSOCK runs over
> virtio, but there are other transports.

>From a cloud-init POV, I don't see virtio as a barrier. Defining an
AF_VSOCK data source for it should be quite straightforward really
and they already have so many data sources, it seems reasonable
that they'd accept one more.

On the host side there's still the task of providing a metdata
service to expose the data, which is outside scope of virt-install.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

Re: [virt-tools-list] virt-install and cloud-init, feedback wanted

2019-11-21 Thread Daniel P . Berrangé
On Thu, Nov 21, 2019 at 11:52:26AM +0100, Florian Weimer wrote:
> * Daniel P. Berrangé:
> 
> >> This goes probably in a different direction of what has been implement
> >> so far, but would it actually harm to enable the network-based
> >> instance-data injection by default?  The advantage would be that it also
> >> blocks these requests from leaking to untrusted parties, which could
> >> then serve bogus data to compromise the virtual machine.
> >
> > I don't understand what you mean by leaking data to untrusted parties
> > here in contetx of config drive ? I've considerd the config drive to
> > be more secure / less risky than network service.
> 
> I'm assuming that cloud-init will try all sources in parallel, given
> that there's a delay for both the network coming about and hardware
> being detected.

IIRC, the network sources all use link-local addresses, so by default
you would need to have configured the 169.254.169.254 on one of the
NICs on the host that the guest can reach. It connects to port 80 on
this address.

Thus to be able to serve malicious data by spoofing the network
metadata service, the host would first need this IP address to be
configured on a NIC, an second something needs to bind to port 80.
Both of these steps require that you already have root on the host
system, so I think the risk here is negligible.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

Re: [virt-tools-list] virt-install and cloud-init, feedback wanted

2019-11-21 Thread Daniel P . Berrangé
On Thu, Nov 21, 2019 at 08:40:12AM +0100, Florian Weimer wrote:
> * Cole Robinson:
> 
> > One more point: my main interaction with cloud-init has historically
> > been by grabbing a Fedora/RHEL cloud image, passing it to
> > virt-install/virt-manager, and watching the boot hang, because there's
> > no data provider and cloud-init times out talking to the network, and
> > then I can't log in. I expect many people have hit this issue before.
> > I've always worked around this by using 'virt-customize' to disable
> > cloud-init and reset the root password. That's about the extent of my
> > usage here, which is broadly why the bare `--cloud-init` is the way it was.
> 
> This is also my use case. 8-/
> 
> > I'm also thinking to the future, if one day virt-install can detect that
> > it was passed a distro cloud-init image, perhaps we can invoke some
> > default behavior that gives the user a better chance of this config
> > being usable out of the box. I figure that will match whatever we choose
> > for the bare '--cloud-init' behavior
> 
> This goes probably in a different direction of what has been implement
> so far, but would it actually harm to enable the network-based
> instance-data injection by default?  The advantage would be that it also
> blocks these requests from leaking to untrusted parties, which could
> then serve bogus data to compromise the virtual machine.

I don't understand what you mean by leaking data to untrusted parties
here in contetx of config drive ? I've considerd the config drive to
be more secure / less risky than network service.

Use of the ISO image should really minimize the risk of unexected data
leakage, because the ISO image is restricted access on the host and
is only granted to the specific QEMU process, with SELinux enforcement
between each VM & other processes/users on the host.

With a network based service there are many greater risks of data leakage
I think. There is no SELinux separation between data for each VM. The
metadata service holds all the data for all VMs. There's also the risk
of the metdata service being accessed from processes on the host, or even
on the WAN.

If the network service is placed on a separate subnet, then you've got
the mgmt problem of creating an extra NIC for the guest and extra device
on the host to attach it to, and managing firewall rules to correctly
isolate the metadta service from unintended access.

Many OSP deployments refuse to enable the metadata service because they
consider it exposes more attack vectors than configdrive does.

Having said all that metadata service can be useful from a managability
POV because it is a live data channel, and thus enables you to change the
data being served while the VM is running. This can be useful for the
userdata blob. Config drive is fixed at time the VM boots.

Network metadata service though is out of scope for virt-install. If
someone ones to deploy one they can do so already today without needing
any more features in virt-install.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list



Re: [virt-tools-list] virt-install and cloud-init, feedback wanted

2019-11-21 Thread Daniel P . Berrangé
On Wed, Nov 20, 2019 at 08:18:01PM -0500, Dusty Mabe wrote:
> 
> 
> On 11/20/19 5:49 PM, Cole Robinson wrote:
> > Hi all. The purpose of this mail is to get some feedback on pending
> > cloud-init support in virt-install. If you're on the CC list here, I
> > either pulled your email from a cloud-init discussion on the the
> > virt-tools-list mailing list, or from the CC list of this virt-install bug:
> > 
> > RFE: Provide cloud-init integration for VMs
> > https://bugzilla.redhat.com/show_bug.cgi?id=981693
> > 
> > For GSOC 2019 Athina Plaskasoviti completed some cloud-init integration
> > work in virt-install. You can read her wrap up here:
> > https://athinapl.home.blog/2019/08/25/gsoc-2019-cloud-init-configuration-for-virt-manager-virt-install/
> 
> First off, thanks Cole and Athina for working on this. This is something I 
> have
> wanted for a long time! In my dayjob I've moved away from cloud-init a bit 
> because
> our new platforms are running Ignition: https://github.com/coreos/ignition
> 
> From the Ignition side I've added Andrew Jeddeloh and Benjamin Gilbert to CC.
> 
> > 
> > Right now the code is sitting in a virt-manager.git branch, not yet
> > pushed to master:
> > https://github.com/virt-manager/virt-manager/tree/cloudinit
> > 
> > I'll summarize the current behavior, and then ask some questions.
> > 
> > 
> > The branch adds a new 'virt-install --cloud-init' argument with several
> > sub options. When specified, virt-install generates an empty meta-data
> > file, a user-data file with the requested changes, stuffs them both into
> > a cidata iso, which is used for the first VM boot and then deleted.
> 
> Is there any chance we could go a different route than an ISO image? The 
> developers
> of ignition have experienced problems with relying on a device here (i.e. is
> there no device coming or has the device just not shown up yet?). Some 
> discussion over
> in: https://github.com/coreos/ignition/issues/656
> 
> Basically in Fedora CoreOS we need a generic user data mechanism that works 
> across
> platforms (x86_64, aarch64, ppc64le, s390x) and doesn't have possible race 
> conditions.
> Right now we're using `-fw_cfg` but it's not cross platform. We don't have an 
> answer
> yet: https://github.com/coreos/ignition/issues/666#issuecomment-452835654

For platform portability you need to find some device that is common
across all platforms, and either disk or network are the only two
good options that exist today or for the forseeable future.  If those
aren't acceptable then all we have left are platform specific options.

I think relying on platform specific options *is* acceptable, provided
we can identify a viable option for each platform.  We have SMBIOS
OEM strings as recommended solution for x86 and aarch64. We are yet
to identify any good options for ppc or s390 though. Perhaps we can
define a mapping for SMBIOS into these arches, as SMBIOS is ultimately
just a spec for a set of tables in a region of memory. It possible,
probably even likely, that we will have to choose something entirely
different though for these arches.

> > This behavior is only triggered when --cloud-init is specified in some
> > form, there's no automagic invocation of this support.
> > 
> > The command sub options are:
> > 
> > $ ./virt-install --cloud-init=help
> > --cloud-init options:
> >   disable
> >   root-password-file
> >   root-password-generate
> >   ssh-key
> >   user-data
> > 
> > Their behavior:
> > 
> > * disable=yes: boolean option to disable cloud-init in the VM for
> > subsequent boots. Adds this block to user-data:
> >   runcmd:
> >   - [ sudo, touch, /etc/cloud/cloud-init.disabled ]
> > 
> > * root-password-file=/MY/PATH: set the desired root password from the
> > content of /MY/PATH on the host
> > 
> > * root-password-generate=yes: boolean option, generate a random root
> > password, set it in user-data, print it to the host text console and
> > pause for a bit for the user to see and copy it. sorta inspired by
> > virt-builder
> > 
> > * ssh-key=/MY/KEY.pub: inject the ssh key from /MY/KEY.pub on the host
> > into the cloud-init user-data
> > 
> > * user-data=/PATH/TO/user-data: ignore all other options and copy this
> > file to the .iso as user-data
> > 
> > When bare '--cloud-init' is specified, we default to generating a random
> > root password and disabling cloud-init for subsequent boots:
> > --cloud-init root-password-generate=yes,disable=yes
> > 
> > We've explicitly rejected something like root-password=MY-PASSWORD
> > because of the security implications of encouraging a password to end up
> > in command line history. We've already had a CVE for something similar
> > in virt-install.
> > 
> > Also, I don't want virt-install to be in the business of specifying
> > every cloud-init option under the sun, there's gotta be a better tool
> > for that already. So I'd like to keep --cloud-init suboptions
> > specifically targeted to expected virt use cases, and anything else can
> > 

Re: [virt-tools-list] virt-install and cloud-init, feedback wanted

2019-11-21 Thread Daniel P . Berrangé
On Wed, Nov 20, 2019 at 05:49:45PM -0500, Cole Robinson wrote:
> Hi all. The purpose of this mail is to get some feedback on pending
> cloud-init support in virt-install. If you're on the CC list here, I
> either pulled your email from a cloud-init discussion on the the
> virt-tools-list mailing list, or from the CC list of this virt-install bug:
> 
> RFE: Provide cloud-init integration for VMs
> https://bugzilla.redhat.com/show_bug.cgi?id=981693
> 
> For GSOC 2019 Athina Plaskasoviti completed some cloud-init integration
> work in virt-install. You can read her wrap up here:
> https://athinapl.home.blog/2019/08/25/gsoc-2019-cloud-init-configuration-for-virt-manager-virt-install/
> 
> Right now the code is sitting in a virt-manager.git branch, not yet
> pushed to master:
> https://github.com/virt-manager/virt-manager/tree/cloudinit
> 
> I'll summarize the current behavior, and then ask some questions.
> 
> 
> The branch adds a new 'virt-install --cloud-init' argument with several
> sub options. When specified, virt-install generates an empty meta-data
> file, a user-data file with the requested changes, stuffs them both into
> a cidata iso, which is used for the first VM boot and then deleted.
> 
> This behavior is only triggered when --cloud-init is specified in some
> form, there's no automagic invocation of this support.
> 
> The command sub options are:
> 
> $ ./virt-install --cloud-init=help
> --cloud-init options:
>   disable
>   root-password-file
>   root-password-generate
>   ssh-key
>   user-data
> 
> Their behavior:
> 
> * disable=yes: boolean option to disable cloud-init in the VM for
> subsequent boots. Adds this block to user-data:
>   runcmd:
>   - [ sudo, touch, /etc/cloud/cloud-init.disabled ]
> 
> * root-password-file=/MY/PATH: set the desired root password from the
> content of /MY/PATH on the host
> 
> * root-password-generate=yes: boolean option, generate a random root
> password, set it in user-data, print it to the host text console and
> pause for a bit for the user to see and copy it. sorta inspired by
> virt-builder
> 
> * ssh-key=/MY/KEY.pub: inject the ssh key from /MY/KEY.pub on the host
> into the cloud-init user-data
> 
> * user-data=/PATH/TO/user-data: ignore all other options and copy this
> file to the .iso as user-data
> 
> When bare '--cloud-init' is specified, we default to generating a random
> root password and disabling cloud-init for subsequent boots:
> --cloud-init root-password-generate=yes,disable=yes
> 
> We've explicitly rejected something like root-password=MY-PASSWORD
> because of the security implications of encouraging a password to end up
> in command line history. We've already had a CVE for something similar
> in virt-install.
> 
> Also, I don't want virt-install to be in the business of specifying
> every cloud-init option under the sun, there's gotta be a better tool
> for that already. So I'd like to keep --cloud-init suboptions
> specifically targeted to expected virt use cases, and anything else can
> be served with custom user-data=
> 
> One more point: my main interaction with cloud-init has historically
> been by grabbing a Fedora/RHEL cloud image, passing it to
> virt-install/virt-manager, and watching the boot hang, because there's
> no data provider and cloud-init times out talking to the network, and
> then I can't log in. I expect many people have hit this issue before.
> I've always worked around this by using 'virt-customize' to disable
> cloud-init and reset the root password. That's about the extent of my
> usage here, which is broadly why the bare `--cloud-init` is the way it was.
> 
> I'm also thinking to the future, if one day virt-install can detect that
> it was passed a distro cloud-init image, perhaps we can invoke some
> default behavior that gives the user a better chance of this config
> being usable out of the box. I figure that will match whatever we choose
> for the bare '--cloud-init' behavior

One option is to take the lazy approach and always enable cloud init
if we are given a pre-built disk image.

 * If the disk image doesn't support cloud init, then its fairly
   harmless to provide cloud-init data.

 * If the disk image does support cloud init but has already been
   booted, and the user has marked cloud init as disable, then it
   is again harmless to provide cloud-init data

 * If the disk image does support cloud init but has already been
   booted, and the user left cloud init enabled so that it runs on
   every boot, then passing cloud init data is probably good thing.

 * If the disk image does support cloud init and has never been
   booted, then obviously providing cloud init data is good
   
> * What are the usecases you see for virt-install cloud-init support?

To me the critical use case is simply providing a way for the user to
login to the guest. This is covered by password and/or SSH key inject.
Once they have that working, anything else is just nice to have.

> * Does the above meet your expectations?

Re: [virt-tools-list] [PATCH 0/3] virt-viewer: add SIGINT handler

2019-11-12 Thread Daniel P . Berrangé
On Tue, Nov 12, 2019 at 12:29:09PM +0100, Francesco Giudici wrote:
> When remote-viewer or virt-viewer are terminated by a signal, they quit
> without explicitly releasing resources. A bug[1] has been filed against
> virt-viewer for not releasing redirected usb devices when it's terminated
> by CTRL-C. Clearly this could be solved by adding a signal handler doing
> proper shutdown of the application.

All resources used by a process are automatically released by the
kernel when the process exits.

IOW, doing a "graceful exit" to manually release USB devices should
not be required, not least because we want everything to behave
well even in the face of an abnormal application crash where we have
no way to manually releasing devices.

Can you explain what is preventing the USN device release from the
kernel POV ?


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list



Re: [virt-tools-list] [PATCH 3/3] remote-viewer: allow to build with Microsoft Visual Studio

2019-11-12 Thread Daniel P . Berrangé
On Tue, Nov 12, 2019 at 12:29:12PM +0100, Francesco Giudici wrote:
> In order to meaningfully use the g_io_channel_win32_new_fd() function
> the C runtime used by GLib is required, which is msvcrt.dll. In current
> Microsoft compilers it is near impossible to convince it to build code
> that would use msvcrt.dll. The last Microsoft compiler version that
> supported using msvcrt.dll as the C runtime was version 6. The GNU
> compiler and toolchain for Windows, also known as Mingw, fully supports
> msvcrt.dll.
> For this reason, drop the SIGINT handler when building with Microsoft
> Visual Studio.

IMHO we should explicitly not support the microsoft compilers at
all. Just mandate use of GCC or CLang, as they are available for
every OS platform that matters and support a much better feature
set and would allow us to simplify code by making use of C
extensions such as "g_autoptr".


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list



Re: [virt-tools-list] [RFC virt-viewer] virt-viewer-app: force quit if quit is requested twice

2019-11-01 Thread Daniel P . Berrangé
On Thu, Oct 31, 2019 at 12:19:17PM +0100, Kevin Pouget wrote:
> The investigation of BZ#1713548 shows that the viewer cannot be closed
> if Spice-GTK does not send the DISCONNECTED signal [1].
> 
> This patch allows a 'force-close' if the user requests twice to close
> the app:
> 
> 1. The first time, 
> `virt_viewer_session_close(VIRT_VIEWER_SESSION(priv->session))` is called, 
> which should trigger the DISCONNECTED signal and close the app. If this fails,
> 2. The second time, the application is closed, no matter the internal state.
> 
> See [2] for the patch that introduced the two-step disconnection.
> 
> 1: 
> https://gitlab.freedesktop.org/spice/spice-gtk/blob/0c52ce8937c849d8ae32ade1f22ce3a48c56c732/src/spice-session.c#L2322
> 2: https://pagure.io/virt-viewer/c/8ec03e50
> ---
>  src/virt-viewer-app.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

If there is a problem causing spice-gtk to not reliably send the DISCONNECTED
signal, we should be fixing spice, rather than hacking around the problem
in apps. If fixing spice-gtk is not possible, then we should not wait for
DISCONNECTED at all, just quit immediately. Exiting the app will close all
the connections anyway and the spice server needs to support cleanup in
this way regardless.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list



Re: [virt-tools-list] [virt-manager PATCH 2/2] unattended: Don't log user & admin passwords

2019-07-03 Thread Daniel P . Berrangé
On Wed, Jul 03, 2019 at 02:42:30PM +0200, Fabiano Fidêncio wrote:
> On Wed, Jul 3, 2019 at 1:54 PM Pavel Hrdina  wrote:
> >
> > On Tue, Jul 02, 2019 at 09:21:45PM +0200, Fabiano Fidêncio wrote:
> > > Logging user & admin passwords in the command-line is a security issue,
> > > let's avoid doing so by:
> > > - Not printing the values set by the user when setting up the
> > > install-script config file;
> > > - Removing the values used in the install-scripts, when printing their
> > > content;
> > >
> > > 'CVE-2019-10183' has been assigned to the virt-install --unattended
> > > admin-password=xxx disclosure issue.
> > >
> > > Signed-off-by: Fabiano Fidêncio 
> > > ---
> > >  virtinst/install/unattended.py | 10 --
> > >  1 file changed, 8 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/virtinst/install/unattended.py 
> > > b/virtinst/install/unattended.py
> > > index 4f311296..04758563 100644
> > > --- a/virtinst/install/unattended.py
> > > +++ b/virtinst/install/unattended.py
> > > @@ -97,8 +97,6 @@ def _make_installconfig(script, osobj, unattended_data, 
> > > arch, hostname, url):
> > >  log.debug("InstallScriptConfig created with the following params:")
> > >  log.debug("username: %s", config.get_user_login())
> > >  log.debug("realname: %s", config.get_user_realname())
> > > -log.debug("user password: %s", config.get_user_password())
> > > -log.debug("admin password: %s", config.get_admin_password())
> > >  log.debug("target disk: %s", config.get_target_disk())
> > >  log.debug("hardware arch: %s", config.get_hardware_arch())
> > >  log.debug("hostname: %s", config.get_hostname())
> > > @@ -195,6 +193,14 @@ class OSInstallScript:
> > >  content = self.generate()
> > >  open(scriptpath, "w").write(content)
> > >
> > > +user_password = self._config.get_user_password()
> > > +if user_password:
> > > +content = content.replace(user_password, "[SCRUBBED]")
> > > +
> > > +admin_password = self._config.get_admin_password()
> > > +if admin_password:
> > > +content = content.replace(admin_password, "[SCRUBBED]")
> >
> > There is a small issue with this approach, if you for testing purposes
> > or for any other reason use password that matches anything else in the
> > script file (kickstart for example) it will be replaced as well:
> >
> > ""
> > %post --erroronfail
> >
> > useradd -G wheel phrdina # Add user
> > if [SCRUBBED] -z ''; then
> > passwd -d phrdina # Make user account passwordless
> > else
> > echo '' |passwd --stdin phrdina
> > fi
> >
> > if [SCRUBBED] -z '[SCRUBBED]'; then
> > passwd -d root # Make root account passwordless
> > else
> > echo '[SCRUBBED]' |passwd --stdin root
> > fi
> > "
> >
> > Here I used as a password 'test' and you can see the test command was
> > replaced as well.  Probably not a big deal is it modifies only the log
> > output, not the actual file but I thought that I should at least point
> > to this corner case issue.  Do we care about it or not?
> 
> I thought about that and I sincerely don't care much about that.
> Otherwise we'll have to learn exactly what to match in all the install
> scripts supported (as in, kickstarts, autoyast, jeos, ...).

You can avoid this kind of matching at all but just using a different
Libosinfo.InstallConfig. eg just copy the master config and set the
password params to a sanitized  value & generate a new script.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

Re: [virt-tools-list] [virt-manager PATCH v3] Introduction of cloud-init configuration in virt-install

2019-07-02 Thread Daniel P . Berrangé
On Fri, Jun 28, 2019 at 07:05:18PM +0300, Athina Plaskasoviti wrote:
> Usage:
> --cloud-init
> 
> Signed-off-by: Athina Plaskasoviti 
> ---
>  virt-install|  5 +++
>  virtinst/cli.py | 25 +
>  virtinst/install/cloudinit.py   | 57 +
>  virtinst/install/installer.py   | 17 +
>  virtinst/install/installerinject.py | 20 +-
>  5 files changed, 115 insertions(+), 9 deletions(-)
>  create mode 100644 virtinst/install/cloudinit.py
> 
> diff --git a/virt-install b/virt-install
> index ee2b9006..1ccf7a31 100755
> --- a/virt-install
> +++ b/virt-install
> @@ -444,6 +444,9 @@ def build_installer(options, guest, installdata):
>  installer.set_initrd_injections(options.initrd_inject)
>  if options.autostart:
>  installer.autostart = True
> +if options.cloud_init:
> +cloudinit_data = cli.parse_cloud_init(options.cloud_init)
> +installer.set_cloudinit_data(cloudinit_data)
>  
>  return installer
>  
> @@ -821,6 +824,8 @@ def parse_args():
>  help=_("Perform an unattended installation"))
>  insg.add_argument("--install",
>  help=_("Specify fine grained install options"))
> +insg.add_argument("--cloud-init", nargs="?", const=1,
> +help=_("Perform a cloud image installation, configuring 
> cloud-init"))
>  
>  # Takes a URL and just prints to stdout the detected distro name
>  insg.add_argument("--test-media-detection", help=argparse.SUPPRESS)
> diff --git a/virtinst/cli.py b/virtinst/cli.py
> index 9a1fe2f6..9a6badcd 100644
> --- a/virtinst/cli.py
> +++ b/virtinst/cli.py
> @@ -28,6 +28,7 @@ from .nodedev import NodeDevice
>  from .osdict import OSDB
>  from .storage import StoragePool, StorageVolume
>  from .install.unattended import UnattendedData
> +from .install.cloudinit import CloudInitData
>  
>  
>  ##
> @@ -1602,6 +1603,30 @@ def parse_install(optstr):
>  return installdata
>  
>  
> +
> +# --cloud-init parsing #
> +
> +
> +class ParserCloudInit(VirtCLIParser):
> +cli_arg_name = "cloud-init"
> +
> +@classmethod
> +def _init_class(cls, **kwargs):
> +VirtCLIParser._init_class(**kwargs)
> +cls.add_arg("root-password", "root_password")
> +
> +
> +def parse_cloud_init(optstr):
> +ret = CloudInitData()
> +if optstr == 1:
> +# This means bare --cloud-init, so there's nothing to parse
> +return ret
> +
> +parser = ParserCloudInit(optstr)
> +parser.parse(ret)
> +return ret
> +
> +
>  ##
>  # --location parsing #
>  ##
> diff --git a/virtinst/install/cloudinit.py b/virtinst/install/cloudinit.py
> new file mode 100644
> index ..41667f4b
> --- /dev/null
> +++ b/virtinst/install/cloudinit.py
> @@ -0,0 +1,57 @@
> +import tempfile
> +import random
> +import string
> +import time
> +from ..logger import log
> +
> +
> +class CloudInitData():
> +root_password = None
> +
> +
> +def create_metadata(scratchdir, hostname=None):
> +if hostname:
> +instance = hostname
> +else:
> +hostname = instance = "localhost"
> +content = 'instance-id: %s\n' % instance
> +content += 'hostname: %s\n' % hostname
> +log.debug("Generated cloud-init metadata:\n%s", content)
> +
> +fileobj = tempfile.NamedTemporaryFile(
> +prefix="virtinst-", suffix="-metadata",
> +dir=scratchdir, delete=False)
> +filename = fileobj.name
> +
> +with open(filename, "w") as f:
> +f.write(content)
> +return filename
> +
> +
> +def create_userdata(scratchdir, cloudinit_data, username=None, 
> password=None):
> +if not password:
> +password = ""
> +for dummy in range(16):
> +password += random.choice(string.ascii_letters + string.digits)
> +content = "#cloud-config\n"
> +if username:
> +content += "name: %s\n" % username
> +if cloudinit_data.root_password == "generate":
> +pass
> +else:
> +content += "password: %s\n" % password
> +log.debug("Generated password for first boot: \n%s", password)
> +time.sleep(20)
> +content += "runcmd:\n"
> +content += "- [ sudo, touch, /etc/cloud/cloud-init.disabled ]\n"
> +log.debug("Generated cloud-init userdata:\n%s", content)

This message is going to leak the root password too. It is nice to
log the cloud init data though, so we need a fix to scrub the actual
root password from it before logging.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com

Re: [virt-tools-list] [virt-manager PATCH v3] Introduction of cloud-init configuration in virt-install

2019-07-02 Thread Daniel P . Berrangé
On Fri, Jun 28, 2019 at 07:05:18PM +0300, Athina Plaskasoviti wrote:
> Usage:
> --cloud-init
> 
> Signed-off-by: Athina Plaskasoviti 
> ---
>  virt-install|  5 +++
>  virtinst/cli.py | 25 +
>  virtinst/install/cloudinit.py   | 57 +
>  virtinst/install/installer.py   | 17 +
>  virtinst/install/installerinject.py | 20 +-
>  5 files changed, 115 insertions(+), 9 deletions(-)
>  create mode 100644 virtinst/install/cloudinit.py


> +def create_userdata(scratchdir, cloudinit_data, username=None, 
> password=None):
> +if not password:
> +password = ""
> +for dummy in range(16):
> +password += random.choice(string.ascii_letters + string.digits)
> +content = "#cloud-config\n"
> +if username:
> +content += "name: %s\n" % username
> +if cloudinit_data.root_password == "generate":
> +pass
> +else:
> +content += "password: %s\n" % password
> +log.debug("Generated password for first boot: \n%s", password)
> +time.sleep(20)

Sleeping to let the user see it is nice, but we should allow the user
to press enter to continue the operation instead of forcing them to
wait the full 20 seconds.

Also, never put passwds into log messages - log files are frequently
uploaded to public bug trackers, so having passwds in there will
violate users' privacy / security.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list


Re: [virt-tools-list] [virt-manager PATCH][v2][RFC] Introduction of cloud-init configuration in virt-install

2019-06-25 Thread Daniel P . Berrangé
On Mon, Jun 24, 2019 at 01:53:38PM -0400, Cole Robinson wrote:
> On 6/24/19 12:40 PM, Daniel P. Berrangé wrote:
> > On Mon, Jun 24, 2019 at 12:12:16PM -0400, Cole Robinson wrote:
> >> On 6/21/19 1:18 PM, Athina Plaskasoviti wrote:
> >>> I am sending a v2 of the patch for cloud init basic configuration during 
> >>> cloud image installation, applying previous suggestions.
> >>>
> >>> Fixed: 
> >>>
> >>> -the way the installation is triggered (no_install is no longer required):
> >>>   --install is_cloud=yes ... --import
> >>>
> >>> -added tmpfile for the files that are being generated in the installation 
> >>> process 
> >>> (meta-data, user-data, config.iso)
> >>>
> >>> -changed condition in installerinject.py from label='cloud' to 
> >>> cloudinit=True
> >>>
> >>
> >>
> >> I did some digging into cloud-init behavior to try and get a better idea
> >> of what the minimal viable config is, and get ideas for future work.
> >>
> >>
> >> cloud-init behavior
> >> ===
> >>
> >> The patch generates an .iso with volume label 'cidata' which is the
> >> magic to make cloud-init search it for config data. cloud-init must find
> >> a 'meta-data' and 'user-data' file in order to consider the iso a valid
> >> data source. There isn't any requirements on the contents though. They
> >> can both basically be empty AFAICT.
> >>
> >> This patch is setting meta-data 'instance-id' and 'hostname' fields but
> >> neither are required: instance-id seems to only be needed on real cloud
> >> providers (and cloud-init fills in a 'nocloud' value for us by default).
> >> hostname will predictably set the VM hostname but that's not something
> >> we need to set by default. So in fact meta-data is fine to leave empty
> >> and I think that should be our default behavior.
> >>
> >> Every distro cloud image has some default config in
> >> /etc/cloud/cloud.cfg. This is generated from cloud-init.git
> >> tools/render-cloudcfg which generates cloud.cfg from
> >> config/cloud.cfg.tmpl. Basically the only input to the template is the
> >> distro name. For example fedora generates in in the RPM spec with simply:
> >>
> >> python3 tools/render-cloudcfg --variant fedora >
> >> $RPM_BUILD_ROOT/%{_sysconfdir}/cloud/cloud.cfg
> >>
> >> cloud images seem to have this default behavior:
> >> - ssh is enabled and running
> >> - root account is locked
> >> - cloud.cfg will explicitly disallow ssh to the root account, via
> >> disable_root. Attempts to ssh to root will print an error message
> >> pointing to the default account
> >> - cloud.cfg will create a default account, locked, with full sudo access.
> >>
> >> The name of the default account depends on the value passed to the
> >> tools/render-cloudcfg script. For Fedora it's 'fedora', Ubuntu is
> >> 'ubuntu'. The openstack page here documents the common ones:
> >> https://docs.openstack.org/image-guide/obtain-images.html
> >>
> >> An extra note: I was wrong when I said in other threads that fedora
> >> images have a default password, they just have a default account name
> >> which is locked by default.
> > 
> > That good, so I was in fact right in remembering that.
> > 
> 
> Yeah, sorry for the confusion.
> 
> >> virt-install cli
> >> 
> >>
> >> I think the virt-install entry point for requesting cloud-init stuff
> >> should actually be its own top level option, like --unattended, because
> >> we may want to add suboptions and I don't want to mix those in with
> >> --install. So I'm thinking:
> >>
> >> # does whatever we choose for default behavior, like --install is_cloud
> >> virt-install --cloud-init ...
> >>
> >> # User provides their own user-data file. Maybe for meta-data too?
> >> virt-install --cloud-init user-data=/path/to/user-data
> >>
> >> # Specify password for the default user. Yeah I know this isn't
> >> # a good pattern to put a password on the cli but we could combine it
> >> # with immediate password expiration so the user has to change it
> >> # on first log in
> >> virt-install --cloud-init password=FOOBAR
> >>
> >> # Something like this to inject an ssh key? But honestly I'm not
> >> # convinced we should go 

Re: [virt-tools-list] [virt-manager PATCH][v2][RFC] Introduction of cloud-init configuration in virt-install

2019-06-24 Thread Daniel P . Berrangé
On Mon, Jun 24, 2019 at 12:12:16PM -0400, Cole Robinson wrote:
> On 6/21/19 1:18 PM, Athina Plaskasoviti wrote:
> > I am sending a v2 of the patch for cloud init basic configuration during 
> > cloud image installation, applying previous suggestions.
> > 
> > Fixed: 
> > 
> > -the way the installation is triggered (no_install is no longer required):
> > --install is_cloud=yes ... --import
> > 
> > -added tmpfile for the files that are being generated in the installation 
> > process 
> > (meta-data, user-data, config.iso)
> > 
> > -changed condition in installerinject.py from label='cloud' to 
> > cloudinit=True
> > 
> 
> 
> I did some digging into cloud-init behavior to try and get a better idea
> of what the minimal viable config is, and get ideas for future work.
> 
> 
> cloud-init behavior
> ===
> 
> The patch generates an .iso with volume label 'cidata' which is the
> magic to make cloud-init search it for config data. cloud-init must find
> a 'meta-data' and 'user-data' file in order to consider the iso a valid
> data source. There isn't any requirements on the contents though. They
> can both basically be empty AFAICT.
> 
> This patch is setting meta-data 'instance-id' and 'hostname' fields but
> neither are required: instance-id seems to only be needed on real cloud
> providers (and cloud-init fills in a 'nocloud' value for us by default).
> hostname will predictably set the VM hostname but that's not something
> we need to set by default. So in fact meta-data is fine to leave empty
> and I think that should be our default behavior.
> 
> Every distro cloud image has some default config in
> /etc/cloud/cloud.cfg. This is generated from cloud-init.git
> tools/render-cloudcfg which generates cloud.cfg from
> config/cloud.cfg.tmpl. Basically the only input to the template is the
> distro name. For example fedora generates in in the RPM spec with simply:
> 
> python3 tools/render-cloudcfg --variant fedora >
> $RPM_BUILD_ROOT/%{_sysconfdir}/cloud/cloud.cfg
> 
> cloud images seem to have this default behavior:
> - ssh is enabled and running
> - root account is locked
> - cloud.cfg will explicitly disallow ssh to the root account, via
> disable_root. Attempts to ssh to root will print an error message
> pointing to the default account
> - cloud.cfg will create a default account, locked, with full sudo access.
> 
> The name of the default account depends on the value passed to the
> tools/render-cloudcfg script. For Fedora it's 'fedora', Ubuntu is
> 'ubuntu'. The openstack page here documents the common ones:
> https://docs.openstack.org/image-guide/obtain-images.html
> 
> An extra note: I was wrong when I said in other threads that fedora
> images have a default password, they just have a default account name
> which is locked by default.

That good, so I was in fact right in remembering that.

> virt-install cli
> 
> 
> I think the virt-install entry point for requesting cloud-init stuff
> should actually be its own top level option, like --unattended, because
> we may want to add suboptions and I don't want to mix those in with
> --install. So I'm thinking:
> 
> # does whatever we choose for default behavior, like --install is_cloud
> virt-install --cloud-init ...
> 
> # User provides their own user-data file. Maybe for meta-data too?
> virt-install --cloud-init user-data=/path/to/user-data
> 
> # Specify password for the default user. Yeah I know this isn't
> # a good pattern to put a password on the cli but we could combine it
> # with immediate password expiration so the user has to change it
> # on first log in
> virt-install --cloud-init password=FOOBAR
> 
> # Something like this to inject an ssh key? But honestly I'm not
> # convinced we should go down that route and instead just make users
> # manually set this themselves like they would for any other VM
> virt-install --cloud-init sshkey=/path/to/key

I think ssh keys is the preferred way to setup cloud images in many
cases as no one sane should ever be using passwords for ssh auth these
days given the continual brute force attacks they face.

Saying users must set a password and then manually copy their ssh key
in afterwards is crazy when cloud init supports getting the ssh key
in straight away without needing passwords.

I would not be surprised to even find cloud images which have
explicitly disabled SSH passwords entirely leaving SSH keys as the
only option. Some organizations will have a complete ban on use of
passwords for login purposes too.

 
> --cloud-init default behavior
> =
> 
> This is the bit I'm not really sure about. We need to manipulate some
> password in some way for the user to be able to log in.
> 
> For starters I think no matter what option we choose we should set the
> password to expire on first log in, thus forcing the user to choose a
> password of their own.
> 
> The safest: generate a long random password for the default user. Print
> it to the 

Re: [virt-tools-list] RFC: virt-manager UI philosophy draft

2019-06-24 Thread Daniel P . Berrangé
On Wed, Jun 19, 2019 at 06:34:38PM -0400, Cole Robinson wrote:
> Hi all,
> 
> I've drafted a wiki page about virt-manager UI philosophy, for lack of a
> better term, suggestions welcome. The intention here is to provide some
> guidance about what types of things we want to expose in the UI, both to
> devs, potential contributors, and users requesting RFEs.
> 
> https://github.com/virt-manager/virt-manager/wiki/WIP-UI-philosophy
> 
> I don't think there's much new here, it's just formalized. There's a
> section at the end of the wiki page for previously rejected features as
> well. If people are cool with this document I'll use it to close a bunch
> of upstream bugs as well which will grow that list.
> 
> Another part of my plan for this is to apply the standards to the
> current UI which would result in feature removals. I'll put my thoughts
> in a follow up mail.
> 
> 
> ##  virt-manager UI philosophy
> 
> virt-manager is a UI toolbox-style frontend for libvirt. It provides UI
> access to common virt management tasks and operations. virt-manager aims
> to provide a simple UI, but not too simple that it excludes valid
> usecases. virt-manager prioritizes stability over features.
> 
> * _Basic virt users_ should be able to meet their needs with virt-manager.
> * _Intermediate virt users_ should generally find virt-manager
> sufficiently flexible for their needs.
> * _Advanced virt users_ will not find explicit UI support for their
> advanced use cases, but virt-manager should still function correctly in
> the face of their manually configured advanced virt usage. virt-manager
> should not get in their way.
> 
> See the user definitions at the end of this doc for more details on those.
> 
> Here are some things that virt-manager explicitly is not aiming to
> reimplement:
> * gnome-boxes: a heavily desktop integrated VM manager with an emphasis
> on UI design and simplifying virt management. They prioritize a seamless
> designed experience over flexibility, our goals are different.
> * virt-viewer/remote-viewer, vncviewer: our graphical VM window should
> 'just work' for most needs but any advanced console configuration is
> left up to these other better suited tools.
> * VirtualBox, VMWare Workstation: It's a nice idea to aim to be the
> equivalent of those apps for the QEMU+KVM+Libvirt stack. But to get
> there would require a level of resource investment that is unlikely to
> ever happen.
> * oVirt, Openstack: virt-manager does not aim to support management of
> many hosts with many VMs. virt-manager won't reject this case and we try
> within reason to keep it working. But the UI is not designed for it and
> we will not change the UI to facilitate these style of usecases.
> 
> 
> ## How do we evaluate UI changes
> 
> When is it worth it to expose something in the UI? Here's some criteria
> we will use:
> * How many users do we expect will use it: This is handwavy of course
> but some things come up in email/IRC discussion regularly, and some are
> mentioned once in 5 years.
> * How critical is it for users who need/want it: if it's an absolute
> blocker just to get a working config for some people, that can influence
> the discussion
> * How self explanatory is the feature: 'Enable 3D acceleration' is
> fairly self explanatory. Disk io native vs threads, not so much.
> * How dangerous or difficult to use is the feature: If it works in only
> specific cases or renders the VM unbootable for certain scenarios, this
> matters.
> * How much work is it to maintain, test
> * How much work is it to implement: If something requires significant
> app specific logic on top of libvirt, libosinfo, or spice-gtk that would
> also be useful to other virt apps, it is suspect. We should be aiming to
> share common functionality
> 
> 
> ## User definitions
> 
> ### Basic virt user
> They know little or nothing about libvirt, qemu, and kvm, but they
> understand the high level concept of a virtual machine. They have a
> Windows or Linux distro ISO and they want to create a VM and interact
> with it graphically. They should be able to figure out how to do that by
> running virt-manager and following the UI. The defaults we provide for
> new VMs should be sufficient for their needs.
> 
> After the VM is installed, the UI should facilitate intuitive UI tasks like:
> 
> * lifecycle operations: start/stop/pause the VM; save, snapshot the VM;
> delete, clone the VM
> * rename the VM; change the VM title or description
> * eject/insert CDROM media; change VM boot order
> * increase VM memory
> * attach a host USB device to the VM; possibly add an additional disk to
> the VM
> * graphical operations like send a keycombo, screenshot

I would note that this is essentially the use case / scenario that
GNOME Boxes aims to fill.

Boxes will inevitably do a better job here since its simplified
view will enable it to be more friendly than virt-manager will
be able to do.

IMHO virt-manager should position itself as the natural choice of

Re: [virt-tools-list] RFC: virt-manager feature removals

2019-06-24 Thread Daniel P . Berrangé
On Fri, Jun 21, 2019 at 07:14:56PM -0400, Cole Robinson wrote:
> On 6/19/19 6:34 PM, Cole Robinson wrote:
> > Hi all,
> > 
> > I've drafted a wiki page about virt-manager UI philosophy, for lack of a
> > better term, suggestions welcome. The intention here is to provide some
> > guidance about what types of things we want to expose in the UI, both to
> > devs, potential contributors, and users requesting RFEs.
> > 
> > https://github.com/virt-manager/virt-manager/wiki/WIP-UI-philosophy
> 
> Following on from the UI philosophy document, here's a list of features
> in current virt-manager/virt-install features that I think are
> candidates for removal.

> * the migration wizard: migration is IMO an advanced use case. It
> requires host config outside of virt-manager to have a chance of
> working. the only place migration is really critical for users is
> serious multihost production scenarios where virt-manager isn't really
> appropriate per the UI philosophy. And besides some UI friendliness
> virt-manager doesn't really add much over virsh IMO. Nearly every bug I
> can recall about its usage was from one person who happens to be a
> co-worker developing qemu migration, and redhat QE, which means it's
> either perfect or no one uses it :) I think it should go

I'm not really convinced by this, as I think it dumbs down virt-manager
too much. Telling GUI users to use virsh instead is not a very compelling
story IMHO. virt-manager has multi-host as a feature, and so I think it
is reasonable to expect it to support migration. It isn't only large
openstack scale deployments that can find it useful. 


> * console scaling options (always, never, fullscreen): we've had this in
> the UI forever, but I don't think it has much usage. virt-viewer doesn't
> even expose it how we do, instead it just has a zoom option which I
> don't think we need to implement either. just today there was a bug
> reported saying that scale->always is fairly obviously broken and has
> been for a few releases, which makes me think no one is using it.
> dropping this could be part of a larger think of how console sizing
> works, if we should default to spice auto-resize, or if there's some
> more modern config we should be taking care of.
> https://bugzilla.redhat.com/show_bug.cgi?id=1722088

The right answer is different for VNC vs SPICE.

For VNC you just want to resize the main window to fit, if possible,
otherwise scale it down. No compelling reason to want to be scaling up.

For SPICE though if the guest agent is present then you want to guest
display to auto-resize to match whatever the host window allows to
display. That way you never need scaling down.

> * graphics keymap stuff. in virtinst/hostkeymap.py we have some logic
> to parse host keymap from various locations like /etc/vconsole.conf and
> try to map it to a qemu keymap name. You can trigger this via the UI in
> the keymap dropdown 'Copy local keymap' and via virt-install --graphics
> keymap=local, though the latter isn't even documented. This all was
> the default behavior we used and essentially needed years ago, but
> qemu and spice-gtk/gtk-vnc have for a decade been able to route
> around the need by sending raw scancodes back and forth. This code
> could in theory still be valuable for someone to set the keymap to
> work with a vncviewer for example which doesn't have the scancode
> support IIRC, but I think at that point we can just ask them to specify
> the keymap themselves. So I propose killing hostkeymap entirely, making
> keymap=local print a cli warning, and drop the keymap dropdown from the
> UI. Users can set it in the XML editor if they need it

Yes, there's no compelling reason to set the keymap - even non-gtk-vnc
based clients like tigervnc support the scancode mode now.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list


Re: [virt-tools-list] [virt-manager PATCH][v2][RFC] Introduction of cloud-init configuration in virt-install

2019-06-24 Thread Daniel P . Berrangé
On Fri, Jun 21, 2019 at 01:50:51PM -0400, Cole Robinson wrote:
> On 6/21/19 1:34 PM, Daniel P. Berrangé wrote:
> > On Fri, Jun 21, 2019 at 08:18:45PM +0300, Athina Plaskasoviti wrote:
> >> Triggered by:
> >> --install is_cloud=yes ... --import
> >>
> >> Signed-off-by: Athina Plaskasoviti 
> >> ---
> >>  virt-install|  9 ---
> >>  virtinst/cli.py |  2 ++
> >>  virtinst/install/cloudinit.py   | 41 +
> >>  virtinst/install/installer.py   | 16 ++-
> >>  virtinst/install/installerinject.py | 20 +++---
> >>  5 files changed, 75 insertions(+), 13 deletions(-)
> >>  create mode 100644 virtinst/install/cloudinit.py
> >>
> >> diff --git a/virt-install b/virt-install
> >> index ee2b9006..b3608662 100755
> >> --- a/virt-install
> >> +++ b/virt-install
> >> @@ -399,6 +399,7 @@ def build_installer(options, guest, installdata):
> >>  install_kernel_args = installdata.kernel_args
> >>  install_os = installdata.os
> >>  no_install = installdata.no_install
> >> +is_cloud = installdata.is_cloud
> >>  if installdata.kernel_args:
> >>  if installdata.kernel_args_overwrite:
> >>  install_kernel_args = installdata.kernel_args
> >> @@ -417,10 +418,11 @@ def build_installer(options, guest, installdata):
> >>  no_install = True
> >>  elif options.pxe:
> >>  install_bootdev = "network"
> >> +elif options.import_install:
> >> +no_install = True
> >>  elif installdata.is_set:
> >>  pass
> >> -elif (options.import_install or
> >> -  options.xmlonly or
> >> +elif (options.xmlonly or
> >>options.boot):
> >>  no_install = True
> >>  
> >> @@ -433,7 +435,8 @@ def build_installer(options, guest, installdata):
> >>  install_kernel=install_kernel,
> >>  install_initrd=install_initrd,
> >>  install_kernel_args=install_kernel_args,
> >> -no_install=no_install)
> >> +no_install=no_install,
> >> +is_cloud=is_cloud)
> >>  
> >>  if options.unattended:
> >>  unattended_data = cli.parse_unattended(options.unattended)
> >> diff --git a/virtinst/cli.py b/virtinst/cli.py
> >> index 9a1fe2f6..a2a501a5 100644
> >> --- a/virtinst/cli.py
> >> +++ b/virtinst/cli.py
> >> @@ -1580,6 +1580,7 @@ class ParserInstall(VirtCLIParser):
> >>  is_onoff=True)
> >>  cls.add_arg("os", "os")
> >>  cls.add_arg("no_install", "no_install", is_onoff=True)
> >> +cls.add_arg("is_cloud", "is_cloud", is_onoff=True)
> >>  
> >>  
> >>  class InstallData:
> >> @@ -1592,6 +1593,7 @@ class InstallData:
> >>  self.os = None
> >>  self.is_set = False
> >>  self.no_install = None
> >> +self.is_cloud = None
> >>  
> >>  
> >>  def parse_install(optstr):
> >> diff --git a/virtinst/install/cloudinit.py b/virtinst/install/cloudinit.py
> >> new file mode 100644
> >> index ..25b2a79b
> >> --- /dev/null
> >> +++ b/virtinst/install/cloudinit.py
> >> @@ -0,0 +1,41 @@
> >> +import tempfile
> >> +from ..logger import log
> >> +
> >> +
> >> +def create_metadata(scratchdir, hostname=None):
> >> +if hostname:
> >> +instance = hostname
> >> +else:
> >> +hostname = instance = "localhost"
> >> +
> >> +fileobj = tempfile.NamedTemporaryFile(
> >> +prefix="virtinst-", suffix="-metadata",
> >> +dir=scratchdir, delete=False)
> >> +filename = fileobj.name
> >> +
> >> +with open(filename, "w") as f:
> >> +log.debug("Writing instance-id and hostname to file meta-data")
> >> +f.writelines(['instance-id: %s\n' % instance,
> >> +'hostname: %s\n' % hostname])
> >> +return filename
> > 
> 
> Just a general note, for now we are just trying to straighten out the
> plumbing and get something testable. I'm not planning on pushing
> anything without a broader 

Re: [virt-tools-list] [virt-manager PATCH][v2][RFC] Introduction of cloud-init configuration in virt-install

2019-06-21 Thread Daniel P . Berrangé
On Fri, Jun 21, 2019 at 08:18:45PM +0300, Athina Plaskasoviti wrote:
> Triggered by:
> --install is_cloud=yes ... --import
> 
> Signed-off-by: Athina Plaskasoviti 
> ---
>  virt-install|  9 ---
>  virtinst/cli.py |  2 ++
>  virtinst/install/cloudinit.py   | 41 +
>  virtinst/install/installer.py   | 16 ++-
>  virtinst/install/installerinject.py | 20 +++---
>  5 files changed, 75 insertions(+), 13 deletions(-)
>  create mode 100644 virtinst/install/cloudinit.py
> 
> diff --git a/virt-install b/virt-install
> index ee2b9006..b3608662 100755
> --- a/virt-install
> +++ b/virt-install
> @@ -399,6 +399,7 @@ def build_installer(options, guest, installdata):
>  install_kernel_args = installdata.kernel_args
>  install_os = installdata.os
>  no_install = installdata.no_install
> +is_cloud = installdata.is_cloud
>  if installdata.kernel_args:
>  if installdata.kernel_args_overwrite:
>  install_kernel_args = installdata.kernel_args
> @@ -417,10 +418,11 @@ def build_installer(options, guest, installdata):
>  no_install = True
>  elif options.pxe:
>  install_bootdev = "network"
> +elif options.import_install:
> +no_install = True
>  elif installdata.is_set:
>  pass
> -elif (options.import_install or
> -  options.xmlonly or
> +elif (options.xmlonly or
>options.boot):
>  no_install = True
>  
> @@ -433,7 +435,8 @@ def build_installer(options, guest, installdata):
>  install_kernel=install_kernel,
>  install_initrd=install_initrd,
>  install_kernel_args=install_kernel_args,
> -no_install=no_install)
> +no_install=no_install,
> +is_cloud=is_cloud)
>  
>  if options.unattended:
>  unattended_data = cli.parse_unattended(options.unattended)
> diff --git a/virtinst/cli.py b/virtinst/cli.py
> index 9a1fe2f6..a2a501a5 100644
> --- a/virtinst/cli.py
> +++ b/virtinst/cli.py
> @@ -1580,6 +1580,7 @@ class ParserInstall(VirtCLIParser):
>  is_onoff=True)
>  cls.add_arg("os", "os")
>  cls.add_arg("no_install", "no_install", is_onoff=True)
> +cls.add_arg("is_cloud", "is_cloud", is_onoff=True)
>  
>  
>  class InstallData:
> @@ -1592,6 +1593,7 @@ class InstallData:
>  self.os = None
>  self.is_set = False
>  self.no_install = None
> +self.is_cloud = None
>  
>  
>  def parse_install(optstr):
> diff --git a/virtinst/install/cloudinit.py b/virtinst/install/cloudinit.py
> new file mode 100644
> index ..25b2a79b
> --- /dev/null
> +++ b/virtinst/install/cloudinit.py
> @@ -0,0 +1,41 @@
> +import tempfile
> +from ..logger import log
> +
> +
> +def create_metadata(scratchdir, hostname=None):
> +if hostname:
> +instance = hostname
> +else:
> +hostname = instance = "localhost"
> +
> +fileobj = tempfile.NamedTemporaryFile(
> +prefix="virtinst-", suffix="-metadata",
> +dir=scratchdir, delete=False)
> +filename = fileobj.name
> +
> +with open(filename, "w") as f:
> +log.debug("Writing instance-id and hostname to file meta-data")
> +f.writelines(['instance-id: %s\n' % instance,
> +'hostname: %s\n' % hostname])
> +return filename

We should probably use the UUID for the newly to-be-created VM as
the instance-id value, and probably not set hostname at all unless
we know what it will end up being.


> +
> +
> +def create_userdata(scratchdir, username=None, password=None):
> +if not password:
> +password = "password"

We must not do this - it is a secret flaw to have a hardcoded
password.

If the user doesn't spply a password, we should simply not
set one.

As default behaviour it is probably more useful to 
take $HOME/.ssh/authorized_keys and inject it into
the guest. We should also allow an alterantive file
in case they want different keys in some cases.


> +fileobj = tempfile.NamedTemporaryFile(
> +prefix="virtinst-", suffix="-userdata",
> +dir=scratchdir, delete=False)
> +filename = fileobj.name
> +
> +with open(filename, "w+") as f:
> +f.write("#cloud-config\n")
> +if username:
> +log.debug("Writing username to file user-data")
> +f.write("name: %s\n" % username)
> +log.debug("Writing password and cmd for disabling of cloud-init to 
> file user-data")
> +f.writelines(["password: %s\n" % password,
> +"chpasswd: { expire: False }\n", "runcmd:\n",
> +"- [ sudo, touch, /etc/cloud/cloud-init.disabled ]"])
> +return filename

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-

Re: [virt-tools-list] [virt-manager PATCH] installer: Prefer "cdrom" over "floppy"

2019-06-07 Thread Daniel P . Berrangé
On Fri, Jun 07, 2019 at 01:32:55PM +0200, Fabiano Fidêncio wrote:
> On Fri, Jun 7, 2019 at 12:49 PM Peter Crowther
>  wrote:
> >
> > On Fri, 7 Jun 2019 at 11:34, Daniel P. Berrangé  wrote:
> >>
> >> On Fri, Jun 07, 2019 at 11:27:36AM +0100, Peter Crowther wrote:
> >> > On Fri, 7 Jun 2019 at 11:18, Daniel P. Berrangé  
> >> > wrote:
> >> >
> >> > > On Fri, Jun 07, 2019 at 12:00:56PM +0200, Fabiano Fidêncio wrote:
> >> > > > Instead of using "floppy" as the way to perform unattended 
> >> > > > installations
> >> > > > for Windoes, let's prefer using "cdrom" instead.
> >> > >
> >> > > Aren't there versions of windows which /only/ support "floppy", which
> >> > > would require us to preferentially use "cdrom" but fallback to "floppy"
> >> > > for older versions ?
> 
> Daniel,
> 
> Older versions of Windows would require floppy. However:
> - There are no floppy support for VMs using q35 in some OSes;

This doesn't bother me. If some OS vendors want to intentionally cripple
floppy support they simply don't get to use OS which need floppy install.

> - mtools has been orphaned on Fedora;

This is more of a problem. We need devs to be able to test the code and
so if Fedora has dropped mtools, the main dev platform of virt-manager is
unable to test it.

> 
> With those two things in mind, I'd happily not support those systems
> as part of a new feature for virt-install.
> Still, if you prefer (and Cole agrees), I'd be okay providing a
> fallback code to use floppy as the injection method for Windows and
> keep supporting Windows XP.

Unless there's a viable alternative to mtools that isn't a huge amount
of work, then I think we probably have to drop it.

> >> > Not for many, many years.  Even Windows 2000 enabled you to burn a CD 
> >> > with
> >> > a winnt.sif answer file on it; I've not checked further back.  How far 
> >> > back
> >> > do we wish to support?
> >>
> >> libosinfo provides install scripts back to XP vintage
> >>
> > All fine - CD-only unattended installation was well established by that 
> > point.
> >
> 
> Peter,
> 
> Are you sure about this? I couldn't start a unattended installation
> using a second cdrom containing the winnit.sif file on it.

I think you probably needed to modify the primary cdrom media to
contain the sif file.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

Re: [virt-tools-list] [virt-manager PATCH] installer: Prefer "cdrom" over "floppy"

2019-06-07 Thread Daniel P . Berrangé
On Fri, Jun 07, 2019 at 11:27:36AM +0100, Peter Crowther wrote:
> On Fri, 7 Jun 2019 at 11:18, Daniel P. Berrangé  wrote:
> 
> > On Fri, Jun 07, 2019 at 12:00:56PM +0200, Fabiano Fidêncio wrote:
> > > Instead of using "floppy" as the way to perform unattended installations
> > > for Windoes, let's prefer using "cdrom" instead.
> >
> > Aren't there versions of windows which /only/ support "floppy", which
> > would require us to preferentially use "cdrom" but fallback to "floppy"
> > for older versions ?
> >
> 
> Not for many, many years.  Even Windows 2000 enabled you to burn a CD with
> a winnt.sif answer file on it; I've not checked further back.  How far back
> do we wish to support?

libosinfo provides install scripts back to XP vintage

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

Re: [virt-tools-list] [virt-manager PATCH] installer: Prefer "cdrom" over "floppy"

2019-06-07 Thread Daniel P . Berrangé
On Fri, Jun 07, 2019 at 12:00:56PM +0200, Fabiano Fidêncio wrote:
> Instead of using "floppy" as the way to perform unattended installations
> for Windoes, let's prefer using "cdrom" instead.

Aren't there versions of windows which /only/ support "floppy", which
would require us to preferentially use "cdrom" but fallback to "floppy"
for older versions ?

> The main reasons behind this change are:
> - VMs using q35 may have floppy device disabled in some systems;
> - We can drop mstools dependency;
> 
> Generating the ISO depends on genisofs, tho. However, it's not a big
> deal as genisofs is already a virt-manager dependency.
> 
> Signed-off-by: Fabiano Fidêncio 
> ---
>  virt-manager.spec.in |  5 
>  virtinst/{floppyinject.py => cdrominject.py} | 26 +++--
>  virtinst/installer.py| 30 ++--
>  virtinst/unattended.py   |  2 +-
>  4 files changed, 33 insertions(+), 30 deletions(-)
>  rename virtinst/{floppyinject.py => cdrominject.py} (55%)
> 
> diff --git a/virt-manager.spec.in b/virt-manager.spec.in
> index b5f6979b..43e0c10b 100644
> --- a/virt-manager.spec.in
> +++ b/virt-manager.spec.in
> @@ -48,11 +48,6 @@ Requires: dconf
>  # no ambiguity.
>  Requires: vte291
>  
> -# Those two dependencies are needed in order to support unattended
> -# installation for Windows guests.
> -Requires: dosfstools
> -Requires: mtools
> -
>  # Weak dependencies for the common virt-manager usecase
>  Recommends: (libvirt-daemon-kvm or libvirt-daemon-qemu)
>  Recommends: libvirt-daemon-config-network
> diff --git a/virtinst/floppyinject.py b/virtinst/cdrominject.py
> similarity index 55%
> rename from virtinst/floppyinject.py
> rename to virtinst/cdrominject.py
> index 05f91ee0..9dc6e20a 100644
> --- a/virtinst/floppyinject.py
> +++ b/virtinst/cdrominject.py
> @@ -6,11 +6,12 @@
>  
>  import logging
>  import os
> +import shutil
>  import subprocess
>  import tempfile
>  
>  
> -def perform_floppy_injections(injections, scratchdir):
> +def perform_cdrom_injections(injections, scratchdir):
>  """
>  Insert files into the root directory of a floppy
>  """
> @@ -20,17 +21,24 @@ def perform_floppy_injections(injections, scratchdir):
>  tempdir = tempfile.mkdtemp(dir=scratchdir)
>  os.chmod(tempdir, 0o775)
>  
> -img = os.path.join(tempdir, "unattended.img")
> +tempfiles = []
> +iso = os.path.join(tempdir, "unattended.iso")
> +for filename in injections:
> +shutil.copy(filename, tempdir)
> +
> +tempfiles = os.listdir(tempdir)
>  
> -cmd = ["mkfs.msdos", "-C", img, "1440"]
> +cmd = ["mkisofs",
> +   "-o", iso,
> +   "-J",
> +   "-input-charset", "utf8",
> +   "-rational-rock",
> +   tempdir]
>  logging.debug("Running mkisofs: %s", cmd)
>  output = subprocess.check_output(cmd)
>  logging.debug("cmd output: %s", output)
>  
> -for filename in injections:
> -logging.debug("Copying %s to the floppy.", filename)
> -cmd = ["mcopy", "-i", img, filename, "::"]
> -output = subprocess.check_output(cmd)
> -logging.debug("cmd output: %s", output)
> +for f in tempfiles:
> +os.unlink(os.path.join(tempdir, f))
>  
> -return img
> +return iso
> diff --git a/virtinst/installer.py b/virtinst/installer.py
> index 45caf930..e6524708 100644
> --- a/virtinst/installer.py
> +++ b/virtinst/installer.py
> @@ -15,7 +15,7 @@ from .devices import DeviceDisk
>  from .domain import DomainOs
>  from .osdict import OSDB, OsMedia
>  from .installertreemedia import InstallerTreeMedia
> -from .floppyinject import perform_floppy_injections
> +from .cdrominject import perform_cdrom_injections
>  from . import unattended
>  from . import util
>  
> @@ -51,7 +51,7 @@ class Installer(object):
>  self._install_kernel = None
>  self._install_initrd = None
>  self._install_cdrom_device_added = False
> -self._install_floppy_device = False
> +self._unattended_install_cdrom_device = None
>  self._unattended_files = []
>  self._defaults_are_set = False
>  self._unattended_data = None
> @@ -119,25 +119,25 @@ class Installer(object):
>  disk.sync_path_props()
>  break
>  
> -def _add_install_floppy_device(self, guest, location):
> -if self._install_floppy_device:
> +def _add_unattended_install_cdrom_device(self, guest, location):
> +if self._unattended_install_cdrom_device:
>  return
>  dev = DeviceDisk(self.conn)
> -dev.device = dev.DEVICE_FLOPPY
> +dev.device = dev.DEVICE_CDROM
>  dev.path = location
>  dev.sync_path_props()
>  dev.validate()
>  dev.set_defaults(guest)
> -self._install_floppy_device = dev
> -guest.add_device(self._install_floppy_device)
> +self._unattended_install_cdrom_device = dev
> +

[virt-tools-list] [virt-manager PATCH] domcapabilities: add md-clear to automatically enabled security features

2019-05-14 Thread Daniel P . Berrangé
The bit is set when microcode provides the mechanism to invoke a flush
of various exploitable CPU buffers by invoking the VERW instruction.

CVE-2018-12126, CVE-2018-12127, CVE-2018-12130, CVE-2019-11091

Signed-off-by: Daniel P. Berrangé 
---
 virtinst/domcapabilities.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/virtinst/domcapabilities.py b/virtinst/domcapabilities.py
index 8993822e..acc91f81 100644
--- a/virtinst/domcapabilities.py
+++ b/virtinst/domcapabilities.py
@@ -281,7 +281,8 @@ class DomainCapabilities(XMLBuilder):
 'spec-ctrl',
 'ssbd',
 'ibpb',
-'virt-ssbd']
+'virt-ssbd',
+'md-clear']
 
 if self._features:
 return self._features
-- 
2.21.0

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

Re: [virt-tools-list] [virt-manager PATCH 5/5] virt-manager: add new checkbox to control CPU security features

2019-04-04 Thread Daniel P . Berrangé
On Wed, Apr 03, 2019 at 03:52:51PM +0200, Pavel Hrdina wrote:
> By default we copy CPU security features to the guest if specific CPU
> model is selected.  However, this may break migration and will affect
> performance of the guest.  This adds an option to disable this default
> behavior.
> 
> The checkbox is clickable only on x86 and only on host where we can
> detect any CPU security features, otherwise a tooltip is set to notify
> users that there is nothing to copy.
> 
> Signed-off-by: Pavel Hrdina 
> ---
>  ui/details.ui  | 25 +
>  virtManager/details.py | 21 +
>  virtManager/domain.py  |  5 +++--
>  virtinst/domain/cpu.py | 30 ++
>  4 files changed, 79 insertions(+), 2 deletions(-)
> 
> diff --git a/ui/details.ui b/ui/details.ui
> index 8b3f939e..2ae1c104 100644
> --- a/ui/details.ui
> +++ b/ui/details.ui
> @@ -2122,6 +2122,31 @@
>   name="top_attach">1
>
>  
> +
> +   id="cpu-secure-label">
> + name="visible">True
> + name="can_focus">False
> + translatable="yes">Copy host security features:

I'd probably change the label to

  "Enable known CPU security flaw mitigations"


Reviewed-by: Daniel P. Berrangé 


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

Re: [virt-tools-list] [virt-manager PATCH 4/5] domcapabilities: add caching of CPU security features

2019-04-04 Thread Daniel P . Berrangé
On Wed, Apr 03, 2019 at 03:52:50PM +0200, Pavel Hrdina wrote:
> We will call this function multiple times so it makes sense to cache the
> result so we don't have to call libvirt APIs every time we will check
> what security features are available on the host.
> 
> Signed-off-by: Pavel Hrdina 
> ---
>  virtinst/domcapabilities.py | 11 ---
>  1 file changed, 8 insertions(+), 3 deletions(-)

Arguably you can cache the entire capabilities API call for virt-install
since it won't change in the lifetime if a short virt-install
invokation and even if it does change you likely want virt-install to
have a consistent view of capabilities throughout its execution.

For virt-manager though being more dynamic is possibly more desirable
given its long life.

Reviewed-by: Daniel P. Berrangé 


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

Re: [virt-tools-list] [virt-manager PATCH 3/5] cli: introduce CPU secure parameter

2019-04-04 Thread Daniel P . Berrangé
On Thu, Apr 04, 2019 at 10:14:21AM +0100, Daniel P. Berrangé wrote:
> On Wed, Apr 03, 2019 at 03:52:49PM +0200, Pavel Hrdina wrote:
> > This will allow users to override the default behavior of virt-install
> > which copies CPU security features available on the host to the guest
> > XML if specific CPU model is configured.
> > 
> > Signed-off-by: Pavel Hrdina 
> > ---
> >  man/virt-install.pod  |  8 +-
> >  .../compare/virt-install-cpu-disable-sec.xml  | 93 +++
> >  tests/clitest.py  |  1 +
> >  virtinst/cli.py   |  1 +
> >  virtinst/domain/cpu.py|  7 +-
> >  5 files changed, 108 insertions(+), 2 deletions(-)
> >  create mode 100644 
> > tests/cli-test-xml/compare/virt-install-cpu-disable-sec.xml
> > 
> > diff --git a/man/virt-install.pod b/man/virt-install.pod
> > index 8407e795..18d44808 100644
> > --- a/man/virt-install.pod
> > +++ b/man/virt-install.pod
> > @@ -216,7 +216,13 @@ required value is MODEL, which is a valid CPU model as 
> > known to libvirt.
> >  
> >  Libvirt's feature policy values force, require, optional, disable, or 
> > forbid,
> >  or with the shorthand '+feature' and '-feature', which equal 
> > 'force=feature'
> > -and 'disable=feature' respectively
> > +and 'disable=feature' respectively.
> > +
> > +If exact CPU model is specified virt-install will automatically copy CPU
> > +security features available on the host to mitigate recent CPU CVEs.
> 
> I'd tweak it slightly to
> 
> s/security features/features/
> 
> s/CPU CVEs/CPU speculative execution side channel security vulnerabilities./
> 
> > +This however will have some impact on performance and will break migration
> > +to hosts without security patches.  In order to turn off this default 
> > behavior
> > +there is a B parameter.  Possible values are I and I.
> 
> At the end, add
> 
>  , with I as the default. It is highly recommended to leave this
>  enabled and ensure all virtualization hosts have fully up to date
>  microcode, kernel & virtualization software installed.

With those changes applied

 Reviewed-by: Daniel P. Berrangé 
 

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

Re: [virt-tools-list] [virt-manager PATCH 3/5] cli: introduce CPU secure parameter

2019-04-04 Thread Daniel P . Berrangé
On Wed, Apr 03, 2019 at 03:52:49PM +0200, Pavel Hrdina wrote:
> This will allow users to override the default behavior of virt-install
> which copies CPU security features available on the host to the guest
> XML if specific CPU model is configured.
> 
> Signed-off-by: Pavel Hrdina 
> ---
>  man/virt-install.pod  |  8 +-
>  .../compare/virt-install-cpu-disable-sec.xml  | 93 +++
>  tests/clitest.py  |  1 +
>  virtinst/cli.py   |  1 +
>  virtinst/domain/cpu.py|  7 +-
>  5 files changed, 108 insertions(+), 2 deletions(-)
>  create mode 100644 
> tests/cli-test-xml/compare/virt-install-cpu-disable-sec.xml
> 
> diff --git a/man/virt-install.pod b/man/virt-install.pod
> index 8407e795..18d44808 100644
> --- a/man/virt-install.pod
> +++ b/man/virt-install.pod
> @@ -216,7 +216,13 @@ required value is MODEL, which is a valid CPU model as 
> known to libvirt.
>  
>  Libvirt's feature policy values force, require, optional, disable, or forbid,
>  or with the shorthand '+feature' and '-feature', which equal 'force=feature'
> -and 'disable=feature' respectively
> +and 'disable=feature' respectively.
> +
> +If exact CPU model is specified virt-install will automatically copy CPU
> +security features available on the host to mitigate recent CPU CVEs.

I'd tweak it slightly to

s/security features/features/

s/CPU CVEs/CPU speculative execution side channel security vulnerabilities./

> +This however will have some impact on performance and will break migration
> +to hosts without security patches.  In order to turn off this default 
> behavior
> +there is a B parameter.  Possible values are I and I.

At the end, add

 , with I as the default. It is highly recommended to leave this
 enabled and ensure all virtualization hosts have fully up to date
 microcode, kernel & virtualization software installed.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list


Re: [virt-tools-list] [virt-manager PATCH 1/5] domcapabilities: remove recommended CPU features from security features

2019-04-04 Thread Daniel P . Berrangé
On Wed, Apr 03, 2019 at 03:52:47PM +0200, Pavel Hrdina wrote:
> These features are only recommended to be enabled since they improve
> performance of the VMs if security features are enabled.
> 
> Signed-off-by: Pavel Hrdina 
> ---
>  tests/cli-test-xml/compare/virt-install-qemu-plain.xml  | 2 --
>  .../compare/virt-install-singleton-config-2.xml | 4 
>  virtinst/domcapabilities.py | 6 +-
>  3 files changed, 1 insertion(+), 11 deletions(-)

> diff --git a/virtinst/domcapabilities.py b/virtinst/domcapabilities.py
> index d1b0f4ed..72844512 100644
> --- a/virtinst/domcapabilities.py
> +++ b/virtinst/domcapabilities.py
> @@ -274,14 +274,10 @@ class DomainCapabilities(XMLBuilder):
>  
>  def get_cpu_security_features(self):
>  sec_features = [
> -'pcid',
>  'spec-ctrl',
>  'ssbd',
> -'pdpe1gb',
>  'ibpb',
> -'virt-ssbd',
> -'amd-ssbd',
> -'amd-no-ssb']
> +'virt-ssbd']

This all makes sense - rationale for each removed one is:

pcid is a very useful perf feature, but missing in some silicon
so not portable.

pdpe1gb lets the guest use 1 GB pages which is good for perf
but again not all silicon can do it

amd-ssbd is a security feature which fixes the same SSBD flaws as the
virt-ssbd feature does. virt-ssbd is usable across all CPU models
affected by SSBD, while amd-ssbd is only available in very new silicon.
So virt-ssbd is the bette rchoice.

amd-no-ssb just indicates that the CPU is not affected by SSBD, so not
critical to expose. I expect a future named CPU model will include that
where appropriate.

Reviewed-by: Daniel P. Berrangé 


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

Re: [virt-tools-list] [virt-manager PATCH 2/5] domcapabilities: fix typo in function name

2019-04-04 Thread Daniel P . Berrangé
On Wed, Apr 03, 2019 at 03:52:48PM +0200, Pavel Hrdina wrote:
> Signed-off-by: Pavel Hrdina 
> ---
>  virtinst/domcapabilities.py | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)

Reviewed-by: Daniel P. Berrangé 


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

Re: [virt-tools-list] [virt-manager PATCH 0/5] CPU security features improvements

2019-04-04 Thread Daniel P . Berrangé
On Thu, Apr 04, 2019 at 10:00:21AM +0100, Peter Crowther wrote:
> On Thu, 4 Apr 2019 at 09:15, Daniel P. Berrangé  wrote:
> 
> > On Wed, Apr 03, 2019 at 07:49:48PM -0400, Cole Robinson wrote:
> > I
> > think it is reasonable to assume that if the user has upgraded the
> > microcode on some hosts they will have done it on all hosts.
> 
> 
> Don't rely on it - certainly in the cases of the smaller organisations I
> work with / audit, patching can be politely described as "haphazard".  Even
> in the large public-sector organisations I work with, there's almost never
> money for test rigs that have the same hardware as the live hosts.  It's
> not uncommon for a live host to be the "guinea pig" receiving new microcode
> before others, then returned to live service.
> 
> The real world is not tidy :-(.

Sure, that's why this series proposes a command line option to let the
user disable the security features if they are in a messy situations.

> 
> If they
> > have not upgraded on all hosts, I still think it is sensible to
> > apply the security mitigations on the hosts which have been upgraded
> > unless the user explicitly says to use the insecure mode.
> >
> 
> Agree.
> 
> We should do the right thing out of the box to enable the
> > security mitigations.  The fact that virt-intsall doesn't do this for
> > named CPU models is arguably worthy of filing a CVE against virt-install
> > itself.
> >
> 
> Agree.
> 
> >
> > Regards,
> > Daniel
> >
> 
> - Peter

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

Re: [virt-tools-list] [virt-manager PATCH 0/5] CPU security features improvements

2019-04-04 Thread Daniel P . Berrangé
On Wed, Apr 03, 2019 at 07:49:48PM -0400, Cole Robinson wrote:
> ccing danpb
> 
> On 4/3/19 9:52 AM, Pavel Hrdina wrote:
> > Pavel Hrdina (5):
> >   domcapabilities: remove recommended CPU features from security
> > features
> >   domcapabilities: fix typo in function name
> >   cli: introduce CPU secure parameter
> >   domcapabilities: add caching of CPU security features
> >   virt-manager: add new checkbox to control CPU security features
> > 
> 
> So the bug spawning the original work is:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1582667
> 
> Some CPU model names lack security critical CPU flags. Example
> Opteron_G5 lacks ibpb. Even if the user is running on an Opteron_G5 host
> CPU which has ibpb, if they select Opteron_G5 the guest doesn't see it.
> 
> In that case, the existing patches in git will check to see if the host
> supports ibpb, and if so, manually specify that  in the 
> config. So now use of --cpu Opteron_G5 is secure by default.
> 
> But the problem is this creates a migration compat issue: selecting
> --cpu Opteron_G5 means different things for different hosts. So these
> patches basically make the 'secure'  additions the default
> behavior, but gives users the option to turn it off on the cli and
> virt-manager UI.

You are correct about migration compat in the general case, but I
think it is reasonable to assume that if the user has upgraded the
microcode on some hosts they will have done it on all hosts. If they
have not upgraded on all hosts, I still think it is sensible to
apply the security mitigations on the hosts which have been upgraded
unless the user explicitly says to use the insecure mode.

> Taking a step back, I don't really get what the usecase is for users to
> be selecting manual cpu model names with virt-manager or virt-install. I
> understand the migration compat thing, if you need to generate a
> baseline CPU that works for migrating a machine across multiple hosts,
> but in practice that seems like pretty advanced usage for people using
> virt-manager or even virt-install, and if they are using virt-install
> they have all the tools at their disposal to generate a full + secure
>  config if they want.

I think using a explicit CPU model is quite a reasonable thing todo.
It isn't solely live migration it helps. Even save/restore where you
restore the image on 2nd machine wants a portable CPU model. It may
not be the out of the box common case, but I don't think we should
write it off as "advanced" usage and require people to take extra
steps to figure out which security flags they need to set themselves.


> Anyways, since the current code requires the user to explicitly opt in
> to choosing a manual CPU model name, what if instead we just warn them?

For these hardware security flaws I don't think it is acceptable to merely
warn the user. We should do the right thing out of the box to enable the
security mitigations.  The fact that virt-intsall doesn't do this for
named CPU models is arguably worthy of filing a CVE against virt-install
itself.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list


Re: [virt-tools-list] virt-viewer 8.0?

2019-03-29 Thread Daniel P . Berrangé
On Fri, Mar 29, 2019 at 06:43:34PM +0300, Hetz Ben Hamo wrote:
> Oh thanks.
> Any chance for a Fedora 29 build also please? :)

I generally avoid sending rebases into existing Fedora releases. The
time between Fedora releases is short enough that users don't have to
wait long to get the new versions & I prefer to avoid risk of creating
regressions in existing releases.


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list


Re: [virt-tools-list] virt-viewer 8.0?

2019-03-29 Thread Daniel P . Berrangé
On Fri, Mar 29, 2019 at 05:38:29PM +0300, Hetz Ben Hamo wrote:
> Hi,
> 
> I see virt-viewer is available as version 8.0 on the home page. It's
> available for Windows as well as the source.
> 
> When will it be available as a binary package for linux distributions?

I forgot to update Fedora, so I've submitted it as an update for rawhide
and Fedora 30 now.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list


Re: [virt-tools-list] [virt-viewer PATCH 01/14] po: provide custom make rules for po file management

2019-03-11 Thread Daniel P . Berrangé
On Mon, Mar 11, 2019 at 12:05:25PM +0100, Christophe Fergeau wrote:
> Hey,
> 
> On Thu, Mar 07, 2019 at 02:42:45PM +0000, Daniel P. Berrangé wrote:
> > After investigating in more detail it seems you can provide custom rules
> > for XML files by setting GETTEXTDATADIR, so I can support mime files with
> > this extra diff:
> 
> Ah indeed, nice finding. Seeing what is shipped in
> https://git.savannah.gnu.org/cgit/gettext.git/tree/gettext-tools/its
> (gsettings, glade, ... rules), this mime rule could probably go there as
> well.
> 
> > 
> > diff --git a/data/Makefile.am b/data/Makefile.am
> > index 0e50f3d..d4089be 100644
> > --- a/data/Makefile.am
> > +++ b/data/Makefile.am
> > @@ -4,6 +4,8 @@ MANUFACTURER = Virt Manager Project
> >  
> >  EXTRA_DIST =   \
> > virt-viewer.wxs.in  \
> > +   gettext/its/mime.its\
> > +   gettext/its/mime.loc\
> > $(NULL)
> >  
> >  # this make sure those files are regenerated when they change
> > @@ -71,8 +73,11 @@ desktop_DATA = $(DESKTOPFILES:.desktop.in=.desktop)
> >  %.desktop: %.desktop.in
> > $(AM_V_GEN)$(MSGFMT) --desktop --template $< -d $(top_srcdir)/po -o 
> > $@
> >  
> > +MIMEFILES = virt-viewer-mime.xml.in
> >  mimedir = $(datadir)/mime/packages
> >  mime_DATA = virt-viewer-mime.xml
> > +%-mime.xml: %-mime.xml.in
> > +   $(AM_V_GEN)GETTEXTDATADIR=$(srcdir)/gettext $(MSGFMT) --xml 
> > --template $< -d $(top_srcdir)/po -o $@
> 
> Something I just noticed (but which is related to the initial patch
> rather than this one) is that the .xml.in -> .xml rule is no longer
> triggered when one of the .po file changes (same for the .desktop file).
> I tried adding $(wildcard $(top_srcdir)/po/*.po) as a dependency, and as
> far as I can tell, this is giving the behaviour I want.

Yes, I can see the original rule had such a dep

$ grep XML_RULE /usr/share/aclocal/intltool.m4
  INTLTOOL_XML_RULE='%.xml:   %.xml.in   $(INTLTOOL_MERGE) 
$(wildcard $(top_srcdir)/po/*.po)

I'll add this dep before pushing.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

Re: [virt-tools-list] [virt-viewer PATCH 01/14] po: provide custom make rules for po file management

2019-03-07 Thread Daniel P . Berrangé
On Tue, Mar 05, 2019 at 11:51:01AM +0100, Christophe Fergeau wrote:
> On Tue, Mar 05, 2019 at 10:27:07AM +0000, Daniel P. Berrangé wrote:
> > On Tue, Mar 05, 2019 at 11:23:34AM +0100, Christophe Fergeau wrote:
> > > On Wed, Feb 20, 2019 at 05:50:52PM +, Daniel P. Berrangé wrote:
> > > > Historically we have relied on intltool to install a standard
> > > > po/Makefile.in.in which has very limited scope for customization.
> > > > intltool is deprecated in favour of standard gettextize tools,
> > > > but these share the same disadvantages.
> > > > 
> > > > Writing make rules for po file management is no more difficult
> > > > than any other rules libvirt-glib has, so stop using intltool
> > > 
> > > s/libvirt-glib/virt-viewer
> > > 
> > > > and don't use gettextize ether.
> > > 
> > > > Signed-off-by: Daniel P. Berrangé 
> > > 
> > > > diff --git a/data/Makefile.am b/data/Makefile.am
> > > > index 1f6c8bf..0e50f3d 100644
> > > > --- a/data/Makefile.am
> > > > +++ b/data/Makefile.am
> > > > @@ -68,17 +68,17 @@ else #!WIN32
> > > >  desktopdir   = $(datadir)/applications
> > > >  DESKTOPFILES = remote-viewer.desktop.in
> > > >  desktop_DATA = $(DESKTOPFILES:.desktop.in=.desktop)
> > > > -@INTLTOOL_DESKTOP_RULE@
> > > > +%.desktop: %.desktop.in
> > > > +   $(AM_V_GEN)$(MSGFMT) --desktop --template $< -d 
> > > > $(top_srcdir)/po -o $@
> > > >  
> > > > -MIMEFILES = virt-viewer-mime.xml.in
> > > >  mimedir = $(datadir)/mime/packages
> > > >  mime_DATA = virt-viewer-mime.xml
> > > > -@INTLTOOL_XML_RULE@
> > > 
> > > Why drop translations from the mime file?
> > 
> > I don't see any other existing mime files translated, and the gettext
> > tools don't recognise the file format AFAICT, so I didn't look like
> > this was really needed in the first place. 
> 
> On my system, grep -l xml:lang /usr/share/mime/packages/* gives a few
> hits, including libreoffice, scribus and the main shared-mime-info file.

After investigating in more detail it seems you can provide custom rules
for XML files by setting GETTEXTDATADIR, so I can support mime files with
this extra diff:

diff --git a/data/Makefile.am b/data/Makefile.am
index 0e50f3d..d4089be 100644
--- a/data/Makefile.am
+++ b/data/Makefile.am
@@ -4,6 +4,8 @@ MANUFACTURER = Virt Manager Project
 
 EXTRA_DIST =   \
virt-viewer.wxs.in  \
+   gettext/its/mime.its\
+   gettext/its/mime.loc\
$(NULL)
 
 # this make sure those files are regenerated when they change
@@ -71,8 +73,11 @@ desktop_DATA = $(DESKTOPFILES:.desktop.in=.desktop)
 %.desktop: %.desktop.in
$(AM_V_GEN)$(MSGFMT) --desktop --template $< -d $(top_srcdir)/po -o $@
 
+MIMEFILES = virt-viewer-mime.xml.in
 mimedir = $(datadir)/mime/packages
 mime_DATA = virt-viewer-mime.xml
+%-mime.xml: %-mime.xml.in
+   $(AM_V_GEN)GETTEXTDATADIR=$(srcdir)/gettext $(MSGFMT) --xml --template 
$< -d $(top_srcdir)/po -o $@
 
 appdatadir   = $(datadir)/appdata
 APPDATAFILES = remote-viewer.appdata.xml.in
@@ -92,8 +97,8 @@ if ENABLE_UPDATE_MIMEDB
$(UPDATE_MIME_DATABASE) "$(DESTDIR)$(datadir)/mime";
 endif
 
-CLEANFILES += $(desktop_DATA) $(appdata_DATA)
-EXTRA_DIST += $(mime_DATA) $(DESKTOPFILES) $(APPDATAFILES)
+CLEANFILES += $(mime_DATA) $(desktop_DATA) $(appdata_DATA)
+EXTRA_DIST += $(MIMEFILES) $(DESKTOPFILES) $(APPDATAFILES)
 
 endif
 
diff --git a/data/gettext/its/mime.its b/data/gettext/its/mime.its
new file mode 100644
index 000..dec8d13
--- /dev/null
+++ b/data/gettext/its/mime.its
@@ -0,0 +1,6 @@
+
+http://www.w3.org/2005/11/its;
+   version="2.0">
+  
+  
+
diff --git a/data/gettext/its/mime.loc b/data/gettext/its/mime.loc
new file mode 100644
index 000..a34fe8d
--- /dev/null
+++ b/data/gettext/its/mime.loc
@@ -0,0 +1,6 @@
+
+
+  
+
+  
+
diff --git a/data/virt-viewer-mime.xml b/data/virt-viewer-mime.xml.in
similarity index 100%
rename from data/virt-viewer-mime.xml
rename to data/virt-viewer-mime.xml.in



Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

Re: [virt-tools-list] [virt-viewer PATCH 00/14] po: improve translation handling

2019-03-05 Thread Daniel P . Berrangé
On Tue, Mar 05, 2019 at 11:24:49AM +0100, Christophe Fergeau wrote:
> A few comments on the first patch, for the rest of the series,
> 
> Reviewed-by: Christophe Fergeau 
> 
> Did you get feedback from people involved with translations regarding
> these changes?

No, but in general case it will have no impact on them. We did this in
libvirt a while back & have seen no complaints either.

We outsourced translation to Zanata, and the people doing translations
in Zanata see zero changes in content presented to them, as zanata
unpacks .po files into its own DB & reconciles with the .pot. So all
the metadata we discard from the .po files was never used by Zanata
in the first place, as it gets it all from the pot.

In the unlikely event anyone was using the .po files in git, the
metadata they were relying on was always outdated & thus misleading
and/or useless. They would have to re-generate the .po file to get
accurate source locations.

With the new system we discarded all the inaccurate metadata, and
people can fully up2date metadata by running 'make update-po'.

IOW, this is better for any one using .po files directly, as the
info they see is now actually accurate.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list


Re: [virt-tools-list] [virt-viewer PATCH] Add a git-publish configuration file

2019-03-04 Thread Daniel P . Berrangé
On Tue, Feb 19, 2019 at 03:10:33PM +, Daniel P. Berrangé wrote:
> On Tue, Feb 19, 2019 at 04:05:00PM +0100, Marc-André Lureau wrote:
> > Hi
> > On Tue, Feb 19, 2019 at 4:03 PM Daniel P. Berrangé  
> > wrote:
> > >
> > > Signed-off-by: Daniel P. Berrangé 
> > > ---
> > >  .gitpublish | 4 
> > >  1 file changed, 4 insertions(+)
> > >  create mode 100644 .gitpublish
> > >
> > > diff --git a/.gitpublish b/.gitpublish
> > > new file mode 100644
> > > index 000..bf82571
> > > --- /dev/null
> > > +++ b/.gitpublish
> > > @@ -0,0 +1,4 @@
> > > +[gitpublishprofile "default"]
> > > +base = master
> > > +to = virt-tools-list@redhat.com
> > > +prefix = virt-viewer PATCH
> > 
> > In general, I believe "PATCH virt-viewer" order is more common.
> 
> When we did this for libvirt, the exact opposite was asserted, hence
> we used "$module PATCH".  I think it does make sense to have the
> "PATCH" word next to the patch sequence numbers too. I'd like to be
> consistent with libvirt in this area in any case.

virt-manager patches which also go to this list have the same format
as I proposed here, so I've pushed this as is, for consistency between
projects

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

Re: [virt-tools-list] [virt-viewer PATCH 00/14] po: improve translation handling

2019-03-04 Thread Daniel P . Berrangé
ping...

On Wed, Feb 20, 2019 at 05:50:51PM +, Daniel P. Berrangé wrote:
> This applies the same improvements previously done in libvirt:
> 
>   https://www.redhat.com/archives/libvir-list/2018-April/msg01004.html
>   
> https://www.berrange.com/posts/2018/11/29/improved-translation-po-file-handling-by-ditching-gettext-autotools-integration/
> 
> The key problems with our current approach are:
> 
>  - The pot & po files stored in GIT contain huge set of
>annotations about source file names & line numbers.
>These are out of date as soon as a change is commited
>to git following a translation refresh. This makes
>diffs impossible to meaningfully review, as they are
>98% noise, 2% signal.
> 
>  - The po file messages are sorted by source location,
>so when we move code between files, or rename files,
>the po file message order changes for no good reason.
>This makes diffs even more impossible to review.
> 
>  - The po files contain entries for all messages even
>if most have no translation, bloating size of po/
>data stored in git
> 
>  - Whenever 'make dist' is run, it alters all the pot
>and po files, so developers need to then reset their
>content to match git HEAD manually. This is caused
>by having auto-generated content (source file locations)
>mixed in with the static content (the actual translated
>strings)
> 
> After this series, we only minimized po files in git, with
> the redundated & outdated source locations info stripped.
> This stripped info is re-added automatically during build
> to create the real .po files, that we distribute, and/or
> upload to translators in Zanata.
> 
> As a result the po directory is smaller in size, and
> when refreshing from Zanata, we have git commits that
> clearly show *only* the altered translations, nothing
> else. The importance of this cannot be overstated - by
> having these clear diffs when doing this change in libvirt,
> I discovered a serious bug in the Zanata client that has
> been screwing up translations in every project that uses
> Zanata by adding bogus "fuzzy" annotations.
> 
> The particularly attractive statistic:
> 
>  193 files changed, 11904 insertions(+), 80334 deletions(-)
> 
> Daniel P. Berrangé (14):
>   po: provide custom make rules for po file management
>   po: remove language list from zanata configuration
>   po: add rules for integration with zanata
>   po: minimize & canonicalize translations stored in git
>   po: minimize af am anp ar as ast bal be bg
>   po: minimize bn_IN bn bo br brx bs ca cs cy da
>   po: minimize de_CH de el en_GB eo es et eu fa fi
>   po: minimize fr gl gu he hi hr hu ia
>   po: minimize id ilo is it ja ka kk km kn ko
>   po: minimize kw_GB kw@kkcor kw kw@uccor ky lt lv mai mk ml
>   po: minimize mn mr ms nb nds ne nl nn nso
>   po: minimize or pa pl pt_BR pt ro ru si sk sl
>   po: minimize sq sr@latin sr sv ta te tg th tr tw
>   po: minimize uk ur vi wba yo zh_CN zh_HK zh_TW zu
> 
>  Makefile.am   |7 -
>  autogen.sh|1 -
>  build-aux/minimize-po.pl  |   37 +
>  configure.ac  |6 +-
>  data/Makefile.am  |   12 +-
>  data/remote-viewer.appdata.xml.in |   12 +-
>  data/remote-viewer.desktop.in |4 +-
>  ...iewer-mime.xml.in => virt-viewer-mime.xml} |0
>  m4/virt-nls.m4|   45 +
>  mingw-virt-viewer.spec.in |2 +-
>  po/Makefile.am|  115 ++
>  po/POTFILES   |   26 +
>  po/POTFILES.in|   27 -
>  po/README.md  |   76 ++
>  po/af.mini.po |   19 +
>  po/af.po  |  864 -
>  po/am.mini.po |   19 +
>  po/am.po  |  864 -
>  po/anp.mini.po|   19 +
>  po/anp.po |  864 -
>  po/ar.mini.po |   20 +
>  po/ar.po  |  869 -
>  po/as.mini.po |  327 +
>  po/as.po  |  884 -
>  po/ast.mini.po|   19 +
>  po/ast.po |  864 -
>  po/bal.mini.po|   19 +
>  po/bal.po   

Re: [virt-tools-list] virt manager downloads not working

2019-02-27 Thread Daniel P . Berrangé
On Tue, Feb 26, 2019 at 01:07:15PM -0800, Dusty Humphries wrote:
> Hi,
> 
> I am attempting to download virt-viewer and virt-manager from
> virtualize-manager.org but the downloads are not working for both.  Is
> someone on this list able to correct the issue please?

That website address is wrong. You need this:

  https://virt-manager.org/download/

I've confirmed that downloads work from there


Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list


[virt-tools-list] [virt-viewer PATCH 04/14] po: minimize & canonicalize translations stored in git

2019-02-22 Thread Daniel P . Berrangé
Similar to the virt-viewer.pot, .po files contain line numbers and file
names identifying where in the source a translatable string comes from.
The source locations in the .po files are thrown away and replaced with
content from the virt-viewer.pot whenever msgmerge is run, so this is not
precious information that needs to be stored in git.

When msgmerge processes a .po file, it will add in any msgids from the
virt-viewer.pot that were not already present. Thus, if a particular msgid
currently has no translation, it can be considered redundant and again
does not need storing in git.

When msgmerge processes a .po file and can't find an exact existing
translation match, it will try todo fuzzy matching instead, marking such
entries with a "# fuzzy" comment to alert the translator to take a
look and either discard, edit or accept the match. Looking at the
existing fuzzy matches in .po files shows that the quality is awful,
with many having a completely different set of printf format specifiers
between the msgid and fuzzy msgstr entry. Fortunately when msgfmt
generates the .gmo, the fuzzy entries are all ignored anyway. The fuzzy
entries could be useful to translators if they were working on the .po
files directly from git, but Virt-Viewer outsourced translation to the
Fedora Zanata system, so keeping fuzzy matches in git is not much help.

Finally, by default msgids are sorted based on source location. Thus, if
a bit of code with translatable text is moved from one file to another,
it may shift around in the .po file, despite the msgid not itself changing.
If the msgids were sorted alphabetically, the .po files would have
stable ordering when code is refactored.

This patch takes advantage of the above observations to canonicalize
and minimize the content stored for .po files in git. Instead of storing
the real .po files, we now store .mini.po files.

The .mini.po files are the same file format as .po files, but have no
source location comments, are sorted alphabetically, and all fuzzy
msgstrs and msgids with no translation are discarded. This cuts the size
of content in the po directory.

Users working from a virt-viewer git checkout who need the full .po files
can run "make update-po", which merges the virt-viewer.pot and .mini.po
file to create a .po file containing all the content previously stored
in git.

Conversely if a full .po file has been modified, for example, by
downloading new content from Zanata, the .mini.po files can be updated
by running "make update-mini-po". The resulting diffs of the .mini.po
file will clearly show the changed translations without any of the noise
that previously obscured content. Being able to see content changes
clearly actually identified a bug in the zanata python client where it
was adding bogus "fuzzy" annotations to many messages:

  https://bugzilla.redhat.com/show_bug.cgi?id=1564497

Users working from virt-viewer releases should not see any difference in
behaviour, since the tarballs only contain the full .po files, not the
.mini.po files.

As an added benefit, generating tarballs with "make dist", will no
longer cause creation of dirty files in git, since it won't touch the
.mini.po files, only the .po files which are no longer kept in git.

The languages are minimized in the following commit since it is a
large mechanical process.

Signed-off-by: Daniel P. Berrangé 
---
 build-aux/minimize-po.pl | 37 +
 po/Makefile.am   | 30 +++--
 po/README.md | 58 +---
 3 files changed, 101 insertions(+), 24 deletions(-)
 create mode 100755 build-aux/minimize-po.pl

diff --git a/build-aux/minimize-po.pl b/build-aux/minimize-po.pl
new file mode 100755
index 000..497533a
--- /dev/null
+++ b/build-aux/minimize-po.pl
@@ -0,0 +1,37 @@
+#!/usr/bin/perl
+
+my @block;
+my $msgstr = 0;
+my $empty = 0;
+my $unused = 0;
+my $fuzzy = 0;
+while (<>) {
+if (/^$/) {
+if (!$empty && !$unused && !$fuzzy) {
+print @block;
+}
+@block = ();
+$msgstr = 0;
+$fuzzy = 0;
+push @block, $_;
+} else {
+if (/^msgstr/) {
+$msgstr = 1;
+$empty = 1;
+}
+if (/^#.*fuzzy/) {
+$fuzzy = 1;
+}
+if (/^#~ msgstr/) {
+$unused = 1;
+}
+if ($msgstr && /".+"/) {
+$empty = 0;
+}
+push @block, $_;
+}
+}
+
+if (@block && !$empty && !$unused) {
+print @block;
+}
diff --git a/po/Makefile.am b/po/Makefile.am
index f32537d..4950cbe 100644
--- a/po/Makefile.am
+++ b/po/Makefile.am
@@ -2,7 +2,7 @@ DOMAIN = $(PACKAGE_NAME)
 COPYRIGHT_HOLDER = The Libvirt authors
 MSGID_BUGS_ADDRESS = https://libvirt.org/bugs.html
 
-MAINTAINERCLEANFILES = $(GMOFILES) $(POTFILE)
+MAINTAINERCLEANFILES = $(GMOFILES) $(POTFILE) $(POFILES)

[virt-tools-list] [virt-viewer PATCH 03/14] po: add rules for integration with zanata

2019-02-22 Thread Daniel P . Berrangé
Add rules to handle pushing virt-viewer.pot to zanata, and refreshing .po
files with new content from zanata.

Signed-off-by: Daniel P. Berrangé 
---
 po/Makefile.am | 8 
 1 file changed, 8 insertions(+)

diff --git a/po/Makefile.am b/po/Makefile.am
index c5f7c36..f32537d 100644
--- a/po/Makefile.am
+++ b/po/Makefile.am
@@ -63,6 +63,14 @@ update-po: $(POFILES)
 
 update-gmo: $(GMOFILES)
 
+push-pot: $(POTFILE)
+   zanata push --push-type=source
+
+pull-po: $(POTFILE)
+   zanata pull --create-skeletons
+   $(MAKE) update-po
+   $(MAKE) update-gmo
+
 $(POTFILE): POTFILES $(POTFILE_DEPS)
$(XGETTEXT) -o $@-t $(XGETTEXT_ARGS) \
  --files-from=$(abs_srcdir)/POTFILES
-- 
2.20.1

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

[virt-tools-list] [virt-viewer PATCH 02/14] po: remove language list from zanata configuration

2019-02-22 Thread Daniel P . Berrangé
The  element in zanata.xml is no longer relevant as this info
is recorded server side.

Signed-off-by: Daniel P. Berrangé 
---
 po/zanata.xml | 99 ---
 1 file changed, 99 deletions(-)

diff --git a/po/zanata.xml b/po/zanata.xml
index 63c682e..09396d3 100644
--- a/po/zanata.xml
+++ b/po/zanata.xml
@@ -4,103 +4,4 @@
   virt-viewer
   master
   gettext
-
-  
-sq
-ar
-as
-ast
-bal
-eu
-bn
-bn-IN
-brx
-bs
-br
-bg
-ca
-zh-CN
-zh-HK
-zh-TW
-kw
-kw-GB
-cs
-da
-nl
-en-GB
-eo
-et
-fi
-fr
-gl
-ka
-de
-el
-gu
-he
-hi
-hu
-is
-id
-ia
-it
-ja
-kn
-kk
-km
-ky
-ko
-lt
-nds
-mk
-mai
-ms
-ml
-mr
-mn
-ne
-nb
-nn
-or
-pa
-fa
-pl
-pt
-pt-BR
-ro
-ru
-sr
-sr@latin
-si
-sk
-sl
-es
-sv
-tg
-ta
-te
-bo
-tr
-uk
-ur
-wba
-cy
-lv
-kw@uccor
-kw@kkcor
-af
-am
-be
-hr
-de-CH
-th
-vi
-zu
-ilo
-nso
-tw
-yo
-anp
-  
-
 
-- 
2.20.1

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

[virt-tools-list] [virt-viewer PATCH v2 3/3] Switch to gnulib's compiler warning flags

2019-02-22 Thread Daniel P . Berrangé
This enables many more compiler warnings than the current code. It also
ensures that -Werror is enabled by default when building from GIT so
that maintainers see regressions as hard failures.

Reviewed-by: Marc-André Lureau 
Signed-off-by: Daniel P. Berrangé 
---
 acinclude.m4   |  98 ---
 configure.ac   |   2 +-
 m4/manywarnings.m4 | 334 +
 m4/virt-viewer-warnings.m4 | 159 ++
 m4/warnings.m4 | 115 +
 prepare-release.sh |   2 +-
 6 files changed, 610 insertions(+), 100 deletions(-)
 delete mode 100644 acinclude.m4
 create mode 100644 m4/manywarnings.m4
 create mode 100644 m4/virt-viewer-warnings.m4
 create mode 100644 m4/warnings.m4

diff --git a/acinclude.m4 b/acinclude.m4
deleted file mode 100644
index 68398a6..000
--- a/acinclude.m4
+++ /dev/null
@@ -1,98 +0,0 @@
-dnl
-dnl Taken from gnome-common/macros2/gnome-compiler-flags.m4
-dnl
-dnl We've added:
-dnl   -Wextra -Wshadow -Wcast-align -Wwrite-strings -Waggregate-return 
-Wstrict-prototypes -Winline -Wredundant-decls
-dnl We've removed
-dnl   CFLAGS="$realsave_CFLAGS"
-dnl   to avoid clobbering user-specified CFLAGS
-dnl
-AC_DEFUN([VIRT_VIEWER_COMPILE_WARNINGS],[
-dnl **
-dnl More compiler warnings
-dnl **
-
-AC_ARG_ENABLE(compile-warnings,
-  
AC_HELP_STRING([--enable-compile-warnings=@<:@no/minimum/yes/maximum/error@:>@],
- [Turn on compiler warnings]),,
-  [enable_compile_warnings="m4_default([$1],[maximum])"])
-
-warnCFLAGS=
-
-try_compiler_flags="-fexceptions -fstack-protector 
--param=ssp-buffer-size=4 -fasynchronous-unwind-tables"
-
-case "$enable_compile_warnings" in
-no)
-   ;;
-minimum)
-   try_compiler_flags="$try_compiler_flags -Wall"
-   ;;
-yes)
-   try_compiler_flags="$try_compiler_flags -Wall -Wmissing-prototypes 
-std=c99"
-   ;;
-maximum|error)
-   try_compiler_flags="$try_compiler_flags -Wall -Wmissing-prototypes 
-std=c99 -Wnested-externs -Wpointer-arith"
-try_compiler_flags="$try_compiler_flags -Wextra -Wshadow -Wcast-align 
-Wwrite-strings -Waggregate-return"
-# Removed -Wstrict-prototypes to avoid GTK bug
-   try_compiler_flags="$try_compiler_flags -Winline -Wredundant-decls 
-Wdeprecated-declarations -Wno-sign-compare"
-   # Remove as glib function casts hit
-   try_compiler_flags="$try_compiler_flags -Wno-cast-function-type"
-   if test "$enable_compile_warnings" = "error" ; then
-   try_compiler_flags="$try_compiler_flags -Werror"
-   fi
-   ;;
-*)
-   AC_MSG_ERROR(Unknown argument '$enable_compile_warnings' to 
--enable-compile-warnings)
-   ;;
-esac
-
-AH_VERBATIM([FORTIFY_SOURCE],
-[/* Enable compile-time and run-time bounds-checking, and some 
warnings. */
- #if !defined _FORTIFY_SOURCE && defined __OPTIMIZE__ && 
__OPTIMIZE__
- # define _FORTIFY_SOURCE 2
- #endif
-])
-
-compiler_flags=
-for option in $try_compiler_flags; do
-   SAVE_CFLAGS="$CFLAGS"
-   CFLAGS="$CFLAGS $option"
-   AC_MSG_CHECKING([whether gcc understands $option])
-   AC_TRY_COMPILE([], [],
-   has_option=yes,
-   has_option=no,)
-   CFLAGS="$SAVE_CFLAGS"
-   AC_MSG_RESULT($has_option)
-   if test $has_option = yes; then
- compiler_flags="$compiler_flags $option"
-   fi
-   unset has_option
-   unset SAVE_CFLAGS
-done
-unset option
-unset try_compiler_flags
-
-AC_ARG_ENABLE(iso-c,
-  AC_HELP_STRING([--enable-iso-c],
- [Try to warn if code is not ISO C ]),,
-  [enable_iso_c=no])
-
-AC_MSG_CHECKING(what language compliance flags to pass to the C compiler)
-complCFLAGS=
-if test "x$enable_iso_c" != "xno"; then
-   if test "x$GCC" = "xyes"; then
-   case " $CFLAGS " in
-   *[\ \   ]-ansi[\ \  ]*) ;;
-   *) complCFLAGS="$complCFLAGS -ansi" ;;
-   esac
-   case " $CFLAGS " in
-   *[\ \   ]-pedantic[\ \  ]*) ;;
-   *) complCFLAGS="$complCFLAGS -pedantic" ;;
-   esac
-   fi
-fi
-AC_MSG_RESULT($complCFLAGS)
-
-WARN_CFLAGS="$compiler_flags $complCFLAGS"
-AC_SUBST(WARN_CFLAGS)
-])
diff --git a/configure.ac b/configure.ac
index 5598c61..8ff69ac 100644
--- a/configure.ac
+++ b/configure.ac
@@ -90,7 +90,7 @@ m4_if(m4_version_compare([2.61a.100],
   [AC_CONFIG_LINKS([$GNUmak

[virt-tools-list] [virt-viewer PATCH v2 0/3] Enable many more compiler warnings

2019-02-22 Thread Daniel P . Berrangé
This replaces the old GNOME code for compiler warnings with the more
advanced GNULIB code, which lets us enable many more warnings. It also
ensures that -Werror is enabled for anyone building from GIT to help
prevent regressions appearing.

Changed in v2:

 - Pushed already ack'd patches
 - Use G_GNUC_PRINTF
 - Fix win32 warnings about strncpy

Daniel P. Berrangé (3):
  Annotate message dialog helpers with format(gnu_printf)
  Fix warning about missing nul terminator with strncpy
  Switch to gnulib's compiler warning flags

 acinclude.m4  |  98 --
 configure.ac  |   2 +-
 m4/manywarnings.m4| 334 ++
 m4/virt-viewer-warnings.m4| 159 
 m4/warnings.m4| 115 
 prepare-release.sh|   2 +-
 src/virt-viewer-app.c |  38 ++--
 src/virt-viewer-notebook.c|   4 +-
 src/windows-cmdline-wrapper.c |   2 +-
 9 files changed, 636 insertions(+), 118 deletions(-)
 delete mode 100644 acinclude.m4
 create mode 100644 m4/manywarnings.m4
 create mode 100644 m4/virt-viewer-warnings.m4
 create mode 100644 m4/warnings.m4

-- 
2.20.1

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

[virt-tools-list] [virt-viewer PATCH v2 1/3] Annotate message dialog helpers with format(gnu_printf)

2019-02-22 Thread Daniel P . Berrangé
This allows the compiler to validate the format string and args passed
to the function.

Signed-off-by: Daniel P. Berrangé 
---
 src/virt-viewer-app.c  | 38 +++---
 src/virt-viewer-notebook.c |  4 ++--
 2 files changed, 25 insertions(+), 17 deletions(-)

diff --git a/src/virt-viewer-app.c b/src/virt-viewer-app.c
index 2592484..15e7b3d 100644
--- a/src/virt-viewer-app.c
+++ b/src/virt-viewer-app.c
@@ -197,19 +197,16 @@ virt_viewer_app_set_debug(gboolean debug)
 doDebug = debug;
 }
 
-static GtkWidget*
-virt_viewer_app_make_message_dialog(VirtViewerApp *self,
-const char *fmt, ...)
+G_GNUC_PRINTF(2, 0) static GtkWidget*
+virt_viewer_app_make_message_dialogv(VirtViewerApp *self,
+ const char *fmt, va_list vargs)
 {
 g_return_val_if_fail(VIRT_VIEWER_IS_APP(self), NULL);
 GtkWindow *window = 
GTK_WINDOW(virt_viewer_window_get_window(self->priv->main_window));
 GtkWidget *dialog;
 char *msg;
-va_list vargs;
 
-va_start(vargs, fmt);
 msg = g_strdup_vprintf(fmt, vargs);
-va_end(vargs);
 
 dialog = gtk_message_dialog_new(window,
 GTK_DIALOG_MODAL |
@@ -224,23 +221,34 @@ virt_viewer_app_make_message_dialog(VirtViewerApp *self,
 return dialog;
 }
 
-void
+G_GNUC_PRINTF(2, 3) static GtkWidget*
+virt_viewer_app_make_message_dialog(VirtViewerApp *self,
+const char *fmt, ...)
+{
+g_return_val_if_fail(VIRT_VIEWER_IS_APP(self), NULL);
+GtkWidget *dialog;
+va_list vargs;
+
+va_start(vargs, fmt);
+dialog = virt_viewer_app_make_message_dialogv(self, fmt, vargs);
+va_end(vargs);
+
+return dialog;
+}
+
+G_GNUC_PRINTF(2, 3) void
 virt_viewer_app_simple_message_dialog(VirtViewerApp *self,
   const char *fmt, ...)
 {
 GtkWidget *dialog;
-char *msg;
 va_list vargs;
 
 va_start(vargs, fmt);
-msg = g_strdup_vprintf(fmt, vargs);
+dialog = virt_viewer_app_make_message_dialogv(self, fmt, vargs);
 va_end(vargs);
 
-dialog = virt_viewer_app_make_message_dialog(self, msg);
 gtk_dialog_run(GTK_DIALOG(dialog));
 gtk_widget_destroy(dialog);
-
-g_free(msg);
 }
 
 static void
@@ -668,7 +676,7 @@ virt_viewer_app_open_unix_sock(const char *unixsock, GError 
**error)
 
 #endif /* defined(HAVE_SOCKETPAIR) && defined(HAVE_FORK) */
 
-void
+G_GNUC_PRINTF(2, 3) void
 virt_viewer_app_trace(VirtViewerApp *self,
   const char *fmt, ...)
 {
@@ -1871,7 +1879,7 @@ virt_viewer_app_on_application_startup(GApplication *app)
 
 if (!virt_viewer_app_start(self, )) {
 if (error && !g_error_matches(error, VIRT_VIEWER_ERROR, 
VIRT_VIEWER_ERROR_CANCELLED))
-virt_viewer_app_simple_message_dialog(self, error->message);
+virt_viewer_app_simple_message_dialog(self, "%s", error->message);
 
 g_clear_error();
 g_application_quit(app);
@@ -2477,7 +2485,7 @@ show_status_cb(gpointer value,
 virt_viewer_notebook_show_status(nb, text);
 }
 
-void
+G_GNUC_PRINTF(2, 3) void
 virt_viewer_app_show_status(VirtViewerApp *self, const gchar *fmt, ...)
 {
 va_list args;
diff --git a/src/virt-viewer-notebook.c b/src/virt-viewer-notebook.c
index 3a74e9f..310ea12 100644
--- a/src/virt-viewer-notebook.c
+++ b/src/virt-viewer-notebook.c
@@ -82,7 +82,7 @@ virt_viewer_notebook_init (VirtViewerNotebook *self)
 gtk_notebook_append_page(GTK_NOTEBOOK(self), priv->status, NULL);
 }
 
-void
+G_GNUC_PRINTF(2, 0) void
 virt_viewer_notebook_show_status_va(VirtViewerNotebook *self, const gchar 
*fmt, va_list args)
 {
 VirtViewerNotebookPrivate *priv;
@@ -99,7 +99,7 @@ virt_viewer_notebook_show_status_va(VirtViewerNotebook *self, 
const gchar *fmt,
 g_free(text);
 }
 
-void
+G_GNUC_PRINTF(2, 3) void
 virt_viewer_notebook_show_status(VirtViewerNotebook *self, const gchar *fmt, 
...)
 {
 va_list args;
-- 
2.20.1

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

[virt-tools-list] [virt-viewer PATCH 01/14] po: provide custom make rules for po file management

2019-02-20 Thread Daniel P . Berrangé
Historically we have relied on intltool to install a standard
po/Makefile.in.in which has very limited scope for customization.
intltool is deprecated in favour of standard gettextize tools,
but these share the same disadvantages.

Writing make rules for po file management is no more difficult
than any other rules libvirt-glib has, so stop using intltool
and don't use gettextize ether.

Signed-off-by: Daniel P. Berrangé 
---
 Makefile.am   |   7 --
 autogen.sh|   1 -
 configure.ac  |   6 +-
 data/Makefile.am  |  12 +-
 data/remote-viewer.appdata.xml.in |  12 +-
 data/remote-viewer.desktop.in |   4 +-
 ...iewer-mime.xml.in => virt-viewer-mime.xml} |   0
 m4/virt-nls.m4|  45 
 mingw-virt-viewer.spec.in |   2 +-
 po/Makefile.am| 105 ++
 po/POTFILES   |  26 +
 po/POTFILES.in|  27 -
 po/README.md  |  38 +++
 po/lt.po  |   2 +-
 virt-viewer.spec.in   |   2 +-
 15 files changed, 234 insertions(+), 55 deletions(-)
 rename data/{virt-viewer-mime.xml.in => virt-viewer-mime.xml} (100%)
 create mode 100644 m4/virt-nls.m4
 create mode 100644 po/Makefile.am
 create mode 100644 po/POTFILES
 delete mode 100644 po/POTFILES.in
 create mode 100644 po/README.md

diff --git a/Makefile.am b/Makefile.am
index d33367a..4cfdd59 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -8,9 +8,6 @@ AM_DISTCHECK_CONFIGURE_FLAGS = --disable-update-mimedb
 EXTRA_DIST =   \
$(PACKAGE).spec $(PACKAGE).spec.in  \
mingw-$(PACKAGE).spec.in\
-   intltool-extract.in \
-   intltool-merge.in   \
-   intltool-update.in  \
GNUmakefile \
maint.mk\
cfg.mk  \
@@ -24,9 +21,6 @@ EXTRA_DIST =  \
 DISTCLEAN_FILES =  \
$(PACKAGE).spec \
mingw-$(PACKAGE).spec   \
-   intltool-extract\
-   intltool-merge  \
-   intltool-update \
$(NULL)
 
 MAINTAINERCLEANFILES = \
@@ -34,7 +28,6 @@ MAINTAINERCLEANFILES =\
$(srcdir)/INSTALL   \
$(GITIGNORE_MAINTAINERCLEANFILES_TOPLEVEL)  \
$(GITIGNORE_MAINTAINERCLEANFILES_M4_LIBTOOL)\
-   $(srcdir)/m4/intltool.m4\
$(NULL)
 
 dist-hook: gen-ChangeLog gen-AUTHORS
diff --git a/autogen.sh b/autogen.sh
index b77ac33..9c8968f 100755
--- a/autogen.sh
+++ b/autogen.sh
@@ -29,7 +29,6 @@ touch ChangeLog AUTHORS
 
 mkdir -p m4
 autoreconf -vfi
-intltoolize --force
 
 cd $THEDIR
 
diff --git a/configure.ac b/configure.ac
index 5598c61..a03d846 100644
--- a/configure.ac
+++ b/configure.ac
@@ -96,8 +96,8 @@ GETTEXT_PACKAGE=virt-viewer
 AC_SUBST(GETTEXT_PACKAGE)
 AC_DEFINE_UNQUOTED([GETTEXT_PACKAGE],"$GETTEXT_PACKAGE", [GETTEXT package 
name])
 
-AM_GLIB_GNU_GETTEXT
-IT_PROG_INTLTOOL([0.35.0])
+VIRT_VIEWER_ARG_NLS
+VIRT_VIEWER_CHECK_NLS
 
 PKG_PROG_PKG_CONFIG
 GLIB_MKENUMS=`$PKG_CONFIG --variable=glib_mkenums glib-2.0`
@@ -261,7 +261,7 @@ AC_CONFIG_FILES([
 icons/256x256/Makefile
 man/Makefile
 mingw-virt-viewer.spec
-po/Makefile.in
+po/Makefile
 src/Makefile
 src/virt-viewer.rc
 tests/Makefile
diff --git a/data/Makefile.am b/data/Makefile.am
index 1f6c8bf..0e50f3d 100644
--- a/data/Makefile.am
+++ b/data/Makefile.am
@@ -68,17 +68,17 @@ else #!WIN32
 desktopdir   = $(datadir)/applications
 DESKTOPFILES = remote-viewer.desktop.in
 desktop_DATA = $(DESKTOPFILES:.desktop.in=.desktop)
-@INTLTOOL_DESKTOP_RULE@
+%.desktop: %.desktop.in
+   $(AM_V_GEN)$(MSGFMT) --desktop --template $< -d $(top_srcdir)/po -o $@
 
-MIMEFILES = virt-viewer-mime.xml.in
 mimedir = $(datadir)/mime/packages
 mime_DATA = virt-viewer-mime.xml
-@INTLTOOL_XML_RULE@
 
 appdatadir   = $(datadir)/appdata
 APPDATAFILES = remote-viewer.appdata.xml.in
 appdata_DATA = $(APPDATAFILES:.xml.in=.xml)
-@INTLTOOL_XML_RULE@
+%.appdata.xml: %.appdata.xml.in
+   $(AM_V_GEN)$(MSGFMT) --xml --template $< -d $(top_srcdir)/po -o $@
 
 install-data-hook:
 if ENABLE_UPDATE_MIMEDB
@@ -92,8 +92,8 @@ if ENABLE_UPDATE_MIMEDB
$(UPDATE_MIME_DATABASE) "$(DESTDIR)$(datadir)/mime";
 endif
 
-CLEANFILES += $(mime_DATA) $(desktop_DATA) $(appdata_DATA)
-EXTRA_DIST += $(MIMEFILES) $(DESKTOPFI

[virt-tools-list] [virt-viewer PATCH 00/14] po: improve translation handling

2019-02-20 Thread Daniel P . Berrangé
This applies the same improvements previously done in libvirt:

  https://www.redhat.com/archives/libvir-list/2018-April/msg01004.html
  
https://www.berrange.com/posts/2018/11/29/improved-translation-po-file-handling-by-ditching-gettext-autotools-integration/

The key problems with our current approach are:

 - The pot & po files stored in GIT contain huge set of
   annotations about source file names & line numbers.
   These are out of date as soon as a change is commited
   to git following a translation refresh. This makes
   diffs impossible to meaningfully review, as they are
   98% noise, 2% signal.

 - The po file messages are sorted by source location,
   so when we move code between files, or rename files,
   the po file message order changes for no good reason.
   This makes diffs even more impossible to review.

 - The po files contain entries for all messages even
   if most have no translation, bloating size of po/
   data stored in git

 - Whenever 'make dist' is run, it alters all the pot
   and po files, so developers need to then reset their
   content to match git HEAD manually. This is caused
   by having auto-generated content (source file locations)
   mixed in with the static content (the actual translated
   strings)

After this series, we only minimized po files in git, with
the redundated & outdated source locations info stripped.
This stripped info is re-added automatically during build
to create the real .po files, that we distribute, and/or
upload to translators in Zanata.

As a result the po directory is smaller in size, and
when refreshing from Zanata, we have git commits that
clearly show *only* the altered translations, nothing
else. The importance of this cannot be overstated - by
having these clear diffs when doing this change in libvirt,
I discovered a serious bug in the Zanata client that has
been screwing up translations in every project that uses
Zanata by adding bogus "fuzzy" annotations.

The particularly attractive statistic:

 193 files changed, 11904 insertions(+), 80334 deletions(-)

Daniel P. Berrangé (14):
  po: provide custom make rules for po file management
  po: remove language list from zanata configuration
  po: add rules for integration with zanata
  po: minimize & canonicalize translations stored in git
  po: minimize af am anp ar as ast bal be bg
  po: minimize bn_IN bn bo br brx bs ca cs cy da
  po: minimize de_CH de el en_GB eo es et eu fa fi
  po: minimize fr gl gu he hi hr hu ia
  po: minimize id ilo is it ja ka kk km kn ko
  po: minimize kw_GB kw@kkcor kw kw@uccor ky lt lv mai mk ml
  po: minimize mn mr ms nb nds ne nl nn nso
  po: minimize or pa pl pt_BR pt ro ru si sk sl
  po: minimize sq sr@latin sr sv ta te tg th tr tw
  po: minimize uk ur vi wba yo zh_CN zh_HK zh_TW zu

 Makefile.am   |7 -
 autogen.sh|1 -
 build-aux/minimize-po.pl  |   37 +
 configure.ac  |6 +-
 data/Makefile.am  |   12 +-
 data/remote-viewer.appdata.xml.in |   12 +-
 data/remote-viewer.desktop.in |4 +-
 ...iewer-mime.xml.in => virt-viewer-mime.xml} |0
 m4/virt-nls.m4|   45 +
 mingw-virt-viewer.spec.in |2 +-
 po/Makefile.am|  115 ++
 po/POTFILES   |   26 +
 po/POTFILES.in|   27 -
 po/README.md  |   76 ++
 po/af.mini.po |   19 +
 po/af.po  |  864 -
 po/am.mini.po |   19 +
 po/am.po  |  864 -
 po/anp.mini.po|   19 +
 po/anp.po |  864 -
 po/ar.mini.po |   20 +
 po/ar.po  |  869 -
 po/as.mini.po |  327 +
 po/as.po  |  884 -
 po/ast.mini.po|   19 +
 po/ast.po |  864 -
 po/bal.mini.po|   19 +
 po/bal.po |  863 -
 po/be.mini.po |   20 +
 po/be.po  |  866 -
 po/bg.mini.po |  307 +
 po/bg.po  |  881 -
 po/bn.mini.po |  256 
 po/bn.po  |  885 -
 po/bn_IN.mini.po  |  329 +
 po/bn_IN.po   |  886

Re: [virt-tools-list] [virt-viewer PATCH] Add a git-publish configuration file

2019-02-19 Thread Daniel P . Berrangé
On Tue, Feb 19, 2019 at 04:05:00PM +0100, Marc-André Lureau wrote:
> Hi
> On Tue, Feb 19, 2019 at 4:03 PM Daniel P. Berrangé  
> wrote:
> >
> > Signed-off-by: Daniel P. Berrangé 
> > ---
> >  .gitpublish | 4 
> >  1 file changed, 4 insertions(+)
> >  create mode 100644 .gitpublish
> >
> > diff --git a/.gitpublish b/.gitpublish
> > new file mode 100644
> > index 000..bf82571
> > --- /dev/null
> > +++ b/.gitpublish
> > @@ -0,0 +1,4 @@
> > +[gitpublishprofile "default"]
> > +base = master
> > +to = virt-tools-list@redhat.com
> > +prefix = virt-viewer PATCH
> 
> In general, I believe "PATCH virt-viewer" order is more common.

When we did this for libvirt, the exact opposite was asserted, hence
we used "$module PATCH".  I think it does make sense to have the
"PATCH" word next to the patch sequence numbers too. I'd like to be
consistent with libvirt in this area in any case.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

___
virt-tools-list mailing list
virt-tools-list@redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

  1   2   >