Re: [pve-devel] [PATCH] Qemu Guest IOPS Graphing

[James Watson]

Is the suggested method something that will be accepted into proxmox? If 
this is the case I can go ahead and get a patch formed.


Kind Regards,
James Watson

On 13/11/15 05:00, Dietmar Maurer wrote:

I am still trying to come up with an alternative method that results in
no blip on the statistics, but possibly someone has on the list might
have a suggestion? I ask as how else are changes going to be made to the
RRD tables if slight data loss and some broken graphs (due to missing
tables on mixed version clusters) isn't an option?

This is really not easy, and I thought about the following:

- do not touch existing message format
- do not touch existing rrd images

instead, send an additional message in pvestatd:

PVE::Cluster::broadcast_rrd("pve2.3-vm/$vmid", $data); # current code
PVE::Cluster::broadcast_rrd("pve4.0-vm/$vmid", $modified_data); # new 
message
 
Inside pmxcfs, we simply write pve4.0-vm data into another rrd file (and

ignore pve2.3-vm messages.

I guess that would avoid data loss (but old data is in other file).

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] Qemu Guest IOPS Graphing

[Dietmar Maurer]

> Is the suggested method something that will be accepted into proxmox? 

Yes, that is the plan.

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] backup : adding a storage option like "max concurrent backup" ?

[Dietmar Maurer]

> My idea was to add some kind of job queue for the backup storage

You could use the pmxcfs locking feature (instead of local lock)...

That way only one task can be started cluster wide.

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] add ovmf uefi roms support

[Dietmar Maurer]

> I have sent another patch series with correct rom.
> 
> Tested with windows 2012 install, uefi is correctly deployed and bootable.

Thanks. I added the binaries to the pve-qemu-kvm package.

But it is still not working for me. I tested with a normal PVE install CD,
and the OVMF Bios is unable to boot that disk, although we create an EFI
boot partition?? 

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] add qemu agent gui option

[Dietmar Maurer]

applied, thanks!

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] reenable steal time

[Dietmar Maurer]

applied

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] influxdb : use multi values measurement

[Dietmar Maurer]

applied.

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH] add ovmf uefi roms support

[Alexandre DERUMIER]

>>But it is still not working for me. I tested with a normal PVE install CD, 
>>and the OVMF Bios is unable to boot that disk, although we create an EFI 
>>boot partition?? 

I'll try with PVE install CD as guest. (windows works 100%)

- Mail original -
De: "dietmar" 
À: "aderumier" , "pve-devel" 
Envoyé: Jeudi 26 Novembre 2015 09:07:14
Objet: Re: [pve-devel] [PATCH] add ovmf uefi roms support

> I have sent another patch series with correct rom. 
> 
> Tested with windows 2012 install, uefi is correctly deployed and bootable. 

Thanks. I added the binaries to the pve-qemu-kvm package. 

But it is still not working for me. I tested with a normal PVE install CD, 
and the OVMF Bios is unable to boot that disk, although we create an EFI 
boot partition?? 
___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] backup : adding a storage option like "max concurrent backup" ?

[Thomas Lamprecht]

On 11/26/2015 09:04 AM, Dietmar Maurer wrote:

My idea was to add some kind of job queue for the backup storage

You could use the pmxcfs locking feature (instead of local lock)...

That way only one task can be started cluster wide.


The cfs_lock_domain method could be helpful here:


add function to lock a domain

This can be used to execute code on an 'action domain' basis.
E.g.: if there are actions that cannot be run simultaneously even if
they, for example, don't access a common file and maybe also spread
across different packages we can now secure the consistence of said
actions on an 'action domain' basis.




___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH V2 pve-common] add function file_copy

[Dietmar Maurer]

applied

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH V2] Add LXC firewall conifg to backup

[Dietmar Maurer]

applied with some cleanups (s/firewall.fw/pct.fw/)

> This patch series will include the firewall setting in the vzdump backup.
> The firewall settings will always restored from the dumpfile,
> if there is a firewall config include the dumpfile.
> 
> Changes in V2 naming of file_atomic_copy to file_copy 
> 
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [[PATCH V3] pve-qemu-kvm] add firewall config to qmp_backup

[Wolfgang Link]

now it is possible to backup the firewall setting with qemu live backup
---
 debian/patches/add_firewall_to_vma.patch | 90 
 debian/patches/series|  1 +
 2 files changed, 91 insertions(+)
 create mode 100644 debian/patches/add_firewall_to_vma.patch

diff --git a/debian/patches/add_firewall_to_vma.patch 
b/debian/patches/add_firewall_to_vma.patch
new file mode 100644
index 000..5df800d
--- /dev/null
+++ b/debian/patches/add_firewall_to_vma.patch
@@ -0,0 +1,90 @@
+Index: qemu-kvm-dev/qapi-schema.json
+===
+--- qemu-kvm-dev.orig/qapi-schema.json
 qemu-kvm-dev/qapi-schema.json
+@@ -416,6 +416,7 @@
+ { 'command': 'backup', 'data': { 'backup-file': 'str',
+ '*format': 'BackupFormat',
+ '*config-file': 'str',
++'*firewall-file': 'str',
+ '*devlist': 'str', '*speed': 'int' },
+   'returns': 'UuidInfo' }
+ 
+Index: qemu-kvm-dev/qmp-commands.hx
+===
+--- qemu-kvm-dev.orig/qmp-commands.hx
 qemu-kvm-dev/qmp-commands.hx
+@@ -1204,7 +1204,7 @@ EQMP
+ 
+ {
+ .name   = "backup",
+-.args_type  = 
"backup-file:s,format:s?,config-file:F?,speed:o?,devlist:s?",
++.args_type  = 
"backup-file:s,format:s?,config-file:F?,firewall-file:F?,speed:o?,devlist:s?",
+ .mhandler.cmd_new = qmp_marshal_input_backup,
+ },
+ 
+Index: qemu-kvm-dev/blockdev.c
+===
+--- qemu-kvm-dev.orig/blockdev.c
 qemu-kvm-dev/blockdev.c
+@@ -2483,6 +2483,7 @@ static void pvebackup_run_next_job(void)
+ UuidInfo *qmp_backup(const char *backup_file, bool has_format,
+ BackupFormat format,
+ bool has_config_file, const char *config_file,
++  bool has_firewall_file, const char *firewall_file,
+ bool has_devlist, const char *devlist,
+ bool has_speed, int64_t speed, Error **errp)
+ {
+@@ -2667,6 +2668,39 @@ UuidInfo *qmp_backup(const char *backup_
+ g_free(cdata);
+ }
+ 
++/* add firewall file to archive */
++if (has_firewall_file) {
++char *cdata = NULL;
++gsize clen = 0;
++GError *err = NULL;
++if (!g_file_get_contents(firewall_file, , , )) {
++error_setg(errp, "unable to read file '%s'", firewall_file);
++goto err;
++}
++
++char *basename = g_path_get_basename(firewall_file);
++
++if (format == BACKUP_FORMAT_VMA) {
++if (vma_writer_add_config(vmaw, basename, cdata, clen) != 0) {
++error_setg(errp, "unable to add firewall config data to vma 
archive");
++g_free(cdata);
++g_free(basename);
++goto err;
++}
++} else if (format == BACKUP_FORMAT_DIR) {
++char config_path[PATH_MAX];
++snprintf(config_path, PATH_MAX, "%s/%s", backup_dir, basename);
++if (!g_file_set_contents(config_path, cdata, clen, )) {
++error_setg(errp, "unable to write firewall config file '%s'", 
config_path);
++g_free(cdata);
++g_free(basename);
++goto err;
++}
++}
++
++g_free(basename);
++g_free(cdata);
++}
+ /* initialize global backup_state now */
+ 
+ backup_state.cancel = false;
+Index: qemu-kvm-dev/hmp.c
+===
+--- qemu-kvm-dev.orig/hmp.c
 qemu-kvm-dev/hmp.c
+@@ -1465,7 +1465,7 @@ void hmp_backup(Monitor *mon, const QDic
+ int64_t speed = qdict_get_try_int(qdict, "speed", 0);
+ 
+ qmp_backup(backup_file, true, dir ? BACKUP_FORMAT_DIR : BACKUP_FORMAT_VMA,
+-   false, NULL, !!devlist,
++   false, NULL, false, NULL, !!devlist,
+devlist, qdict_haskey(qdict, "speed"), speed, );
+ 
+ hmp_handle_error(mon, );
diff --git a/debian/patches/series b/debian/patches/series
index 7dfd2e8..60adfb8 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -34,3 +34,4 @@ jemalloc.patch
 0001-smm_available-false.patch
 mirror-fix-zero-init.patch
 fix-emulator-version-string.patch
+add_firewall_to_vma.patch
-- 
2.1.4


___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [[PATCH V3] pve-qemu-kvm] add firewall config to qmp_backup

[Dietmar Maurer]

applied, thanks!

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [[PATCH] pve-qemu-kvm] refactor qmp_backup

[Wolfgang Link]

---
 debian/patches/add_firewall_to_vma.patch | 93 +---
 1 file changed, 73 insertions(+), 20 deletions(-)

diff --git a/debian/patches/add_firewall_to_vma.patch 
b/debian/patches/add_firewall_to_vma.patch
index 5df800d..0ab20a0 100644
--- a/debian/patches/add_firewall_to_vma.patch
+++ b/debian/patches/add_firewall_to_vma.patch
@@ -27,50 +27,103 @@ Index: qemu-kvm-dev/blockdev.c
 ===
 --- qemu-kvm-dev.orig/blockdev.c
 +++ qemu-kvm-dev/blockdev.c
-@@ -2483,6 +2483,7 @@ static void pvebackup_run_next_job(void)
- UuidInfo *qmp_backup(const char *backup_file, bool has_format,
- BackupFormat format,
- bool has_config_file, const char *config_file,
-+  bool has_firewall_file, const char *firewall_file,
- bool has_devlist, const char *devlist,
- bool has_speed, int64_t speed, Error **errp)
- {
-@@ -2667,6 +2668,39 @@ UuidInfo *qmp_backup(const char *backup_
- g_free(cdata);
+@@ -2457,6 +2457,44 @@ void qmp_backup_cancel(Error **errp)
  }
+ }
  
-+/* add firewall file to archive */
-+if (has_firewall_file) {
++static int config_to_vma(const char *file, BackupFormat format,
++   const char *backup_dir, VmaWriter *vmaw,
++   Error **errp)
++{
 +char *cdata = NULL;
 +gsize clen = 0;
 +GError *err = NULL;
-+if (!g_file_get_contents(firewall_file, , , )) {
-+error_setg(errp, "unable to read file '%s'", firewall_file);
-+goto err;
++if (!g_file_get_contents(file, , , )) {
++error_setg(errp, "unable to read file '%s'", file);
++return 1;
 +}
 +
-+char *basename = g_path_get_basename(firewall_file);
++char *basename = g_path_get_basename(file);
 +
 +if (format == BACKUP_FORMAT_VMA) {
 +if (vma_writer_add_config(vmaw, basename, cdata, clen) != 0) {
-+error_setg(errp, "unable to add firewall config data to vma 
archive");
++error_setg(errp, "unable to add %s config data to vma archive", 
file);
 +g_free(cdata);
 +g_free(basename);
-+goto err;
++return 1;
 +}
 +} else if (format == BACKUP_FORMAT_DIR) {
 +char config_path[PATH_MAX];
 +snprintf(config_path, PATH_MAX, "%s/%s", backup_dir, basename);
 +if (!g_file_set_contents(config_path, cdata, clen, )) {
-+error_setg(errp, "unable to write firewall config file '%s'", 
config_path);
++error_setg(errp, "unable to write config file '%s'", config_path);
 +g_free(cdata);
 +g_free(basename);
-+goto err;
++return 1;
 +}
 +}
 +
 +g_free(basename);
 +g_free(cdata);
++
++  return 0;
++}
++
+ static void pvebackup_run_next_job(void)
+ {
+ GList *l = backup_state.di_list;
+@@ -2483,6 +2521,7 @@ static void pvebackup_run_next_job(void)
+ UuidInfo *qmp_backup(const char *backup_file, bool has_format,
+ BackupFormat format,
+ bool has_config_file, const char *config_file,
++  bool has_firewall_file, const char *firewall_file,
+ bool has_devlist, const char *devlist,
+ bool has_speed, int64_t speed, Error **errp)
+ {
+@@ -2635,38 +2674,17 @@ UuidInfo *qmp_backup(const char *backup_
+ 
+ /* add configuration file to archive */
+ if (has_config_file) {
+-char *cdata = NULL;
+-gsize clen = 0;
+-GError *err = NULL;
+-if (!g_file_get_contents(config_file, , , )) {
+-error_setg(errp, "unable to read file '%s'", config_file);
+-goto err;
+-}
+-
+-char *basename = g_path_get_basename(config_file);
+-
+-if (format == BACKUP_FORMAT_VMA) {
+-if (vma_writer_add_config(vmaw, basename, cdata, clen) != 0) {
+-error_setg(errp, "unable to add config data to vma archive");
+-g_free(cdata);
+-g_free(basename);
+-goto err;
+-}
+-} else if (format == BACKUP_FORMAT_DIR) {
+-char config_path[PATH_MAX];
+-snprintf(config_path, PATH_MAX, "%s/%s", backup_dir, basename);
+-if (!g_file_set_contents(config_path, cdata, clen, )) {
+-error_setg(errp, "unable to write config file '%s'", 
config_path);
+-g_free(cdata);
+-g_free(basename);
+-goto err;
+-}
+-}
+-
+-g_free(basename);
+-g_free(cdata);
++  if(config_to_vma(config_file, format, backup_dir, vmaw, errp) != 0) {
++  goto err;
++  }
+ }
+ 
++/* add firewall file to archive */
++if 

Re: [pve-devel] backup : adding a storage option like "max concurrent backup" ?

[Alexandre DERUMIER]

>> My idea was to add some kind of job queue for the backup storage 
> You could use the pmxcfs locking feature (instead of local lock)... 
> 
> That way only one task can be started cluster wide. 

>>The cfs_lock_domain method could be helpful here: 

Sound great! I'll try to look at this. 

- Mail original -
De: "Thomas Lamprecht" 
À: "pve-devel" 
Envoyé: Jeudi 26 Novembre 2015 09:42:12
Objet: Re: [pve-devel] backup : adding a storage option like "max concurrent 
backup" ?

On 11/26/2015 09:04 AM, Dietmar Maurer wrote: 
>> My idea was to add some kind of job queue for the backup storage 
> You could use the pmxcfs locking feature (instead of local lock)... 
> 
> That way only one task can be started cluster wide. 

The cfs_lock_domain method could be helpful here: 

> add function to lock a domain 
> 
> This can be used to execute code on an 'action domain' basis. 
> E.g.: if there are actions that cannot be run simultaneously even if 
> they, for example, don't access a common file and maybe also spread 
> across different packages we can now secure the consistence of said 
> actions on an 'action domain' basis. 



___ 
pve-devel mailing list 
pve-devel@pve.proxmox.com 
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel 

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH V2] Add KVM firewall conifg to backup

[Dietmar Maurer]

applied with some cleanups (s/firewall.fw/qemu-server.fw/)

> On November 25, 2015 at 10:20 AM Wolfgang Link  wrote:
> 
> 
> This patch series will include the firewall setting form KVM in the vzdump
> backup.
> The firewall settings will always restored from the dumpfile,
> if there is a firewall config include the dumpfile.
> 
> 
> Changes in V2 naming of file_atomic_copy to file_copy
> 
> 
> ___
> pve-devel mailing list
> pve-devel@pve.proxmox.com
> http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-container] Redhat: fix ipv4 dhcp only setup

[Dietmar Maurer]

applied

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-container] Redhat: handle /etc/sysconfig/network better

[Dietmar Maurer]

applied

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-storage] allow rx permissions for group/world on .subvol dirs

[Dietmar Maurer]

applied

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-zsync] safer disk matching

[Dietmar Maurer]

applied

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-common] print_property_string: skip keys early

[Dietmar Maurer]

applied

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-container 1/7] introduce internal mountpont 'type' property

[Dietmar Maurer]

applied whole series

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-container] added Setup::Fedora

[Dietmar Maurer]

applied

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [[PATCH] pve-qemu-kvm] refactor qmp_backup

[Dietmar Maurer]

applied

___
pve-devel mailing list
pve-devel@pve.proxmox.com
http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel