Re: [pve-devel] Integration of FreeNAS iSCSI target initiator in Proxmox Enterprise repo
On Mon, 8 Jun 2020 15:00:11 +0200 Thomas Lamprecht wrote: > > You can effectively provide a full custom plugin, so it has not more > limitations than any existing one. What extra functionality regarding > interface ABI would the FreeNAS Plugin require? > I seem to have read somewhere that a custom plugin could not get access to individual volumes. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc https://pgp.key-server.io/pks/lookup?search=0xD3C9A00E mir datanom net https://pgp.key-server.io/pks/lookup?search=0xE501F51C mir miras org https://pgp.key-server.io/pks/lookup?search=0xE3E80917 -- /usr/games/fortune -es says: I never cheated an honest man, only rascals. They wanted something for nothing. I gave them nothing for something. -- Joseph "Yellow Kid" Weil pgp69aDat5anp.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Integration of FreeNAS iSCSI target initiator in Proxmox Enterprise repo
On Mon, 8 Jun 2020 14:25:16 +0200 Thomas Lamprecht wrote: > > FYI, there's the source code of the plugin loading here: > https://git.proxmox.com/?p=pve-storage.git;a=blob;f=PVE/Storage.pm;h=07a4f5300f5a7e21b635fb3927dd26f799d89439;hb=HEAD#l64 > > An example plugin could be the one from linbit > https://github.com/LINBIT/linstor-proxmox/blob/master/LINSTORPlugin.pm > > But one can effectively copy any existing and add an api method > providing the version to get started. > As I understand the limitations of the custom plugin framework it seems not to be adequate for the FreeNAS plugin, or have I misunderstand something? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc https://pgp.key-server.io/pks/lookup?search=0xD3C9A00E mir datanom net https://pgp.key-server.io/pks/lookup?search=0xE501F51C mir miras org https://pgp.key-server.io/pks/lookup?search=0xE3E80917 -- /usr/games/fortune -es says: Hodie natus est radici frater. [ Unto the root is born a brother ] pgpmdFJdFVSyO.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Integration of FreeNAS iSCSI target initiator in Proxmox Enterprise repo
On Mon, 8 Jun 2020 11:16:29 +0200 Andreas Steinel wrote: > [2] > https://forum.proxmox.com/threads/frage-zfs-over-nfs.70734/post-318459 > And for those who does not understand German? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc https://pgp.key-server.io/pks/lookup?search=0xD3C9A00E mir datanom net https://pgp.key-server.io/pks/lookup?search=0xE501F51C mir miras org https://pgp.key-server.io/pks/lookup?search=0xE3E80917 -- /usr/games/fortune -es says: But maybe we don't really need that... -- Larry Wall in <199709011851.laa07...@wall.org> pgpOxxB_cN11Q.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Integration of FreeNAS iSCSI target initiator in Proxmox Enterprise repo
On Wed, 3 Jun 2020 18:16:11 +0200 "b...@todoo.biz" wrote: > > I'll try to see what can be done… > But I am more involved in firewalling hardcore dev than Proxmox at > this stage ! > This is the file you should copy an adapt to ctld: https://git.proxmox.com/?p=pve-storage.git;a=blob;f=PVE/Storage/LunCmd/Istgt.pm;h=2f758f908aafa7fa4e19b5a82b7244d77d949fb6;hb=HEAD Very little adaption needs to be done in this file: https://git.proxmox.com/?p=pve-storage.git;a=blob;f=PVE/Storage/ZFSPlugin.pm;h=383f0a0cde932da5ce34d792aafb7206517b74ee;hb=HEAD Apart from the above mentioned a few places in the GUI needs to be aware of the new plugin such as in this file: https://git.proxmox.com/?p=pve-storage.git;a=blob;f=PVE/Storage/ZFSPlugin.pm;h=383f0a0cde932da5ce34d792aafb7206517b74ee;hb=HEAD -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc https://pgp.key-server.io/pks/lookup?search=0xD3C9A00E mir datanom net https://pgp.key-server.io/pks/lookup?search=0xE501F51C mir miras org https://pgp.key-server.io/pks/lookup?search=0xE3E80917 -- /usr/games/fortune -es says: The idea is to die young as late as possible. -- Ashley Montague pgppShUIxKPzT.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Integration of FreeNAS iSCSI target initiator in Proxmox Enterprise repo
On Wed, 3 Jun 2020 18:16:11 +0200 "b...@todoo.biz" wrote: > > What is your upper limit for this ? > I prefer a world without limitations;-) BTW. Some years ago I asked phk (Poul Henning Kamp) about this and according to him there was no kernel reason behind the limitation and to the best of his knowledge the number of LUNs, within reasons, should be nearly endless. Of course the number of CPU cores and the amount of memory should be equipped accordingly. > I have a large project with Proxmox which might be popping up, if > this is the case, I'll try so sponsor this ;-) I am convinced that the Proxmox team will welcome such a plugin since supported more storage solution can only benefit Proxmox as a product. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc https://pgp.key-server.io/pks/lookup?search=0xD3C9A00E mir datanom net https://pgp.key-server.io/pks/lookup?search=0xE501F51C mir miras org https://pgp.key-server.io/pks/lookup?search=0xE3E80917 -- /usr/games/fortune -es says: Delta: We never make the same mistake three times. -- David Letterman pgp9okdTFGsV0.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Integration of FreeNAS iSCSI target initiator in Proxmox Enterprise repo
On Wed, 3 Jun 2020 16:54:17 +0200 "b...@todoo.biz" wrote: > > A lot of time has passed since version 9 of FreeBSD / FreeNAS. Six > years to be precise. > The version numbers was for explanation purposes only. BTW. When FreeNAS soon will vanish an be consumed be TrueNAS what happens then? Will it be a closed and commercial product only? > iSCSI on FreeBSD is used by very large corporation (Gandi not to > name them https://www.gandi.net/en <https://www.gandi.net/en> is > hosting all it's VM using FreeBSD + Linux on the initiator side) with > an excellent level of stability. > I have nothing against FreeBSD on the contrary I have very much fun using FreeBSD myself (For one usecase I am addicted to pfSense). Only shortcoming in Proxmox realm in the past was the very non enterprise iSCSI implementation (istgt) which was replaced by cltd, a very fine iSCSI implementation, but limited to 1024 LUNs per target in the past but according to your links has since increased. (By the time I was looking into to this I had numerous discussions with the developer about this issue and I promised to inform me personally when he removed the limitation - I haven't heard from him) > Solution derivated from Illumos are either overpriced, closed source > and scarcely used or abandoned (if not all at the same time). > This is where you I misinformed ;-) Illumos is truly FOSS an lives well. Commercially with SmartOS, but not closed source since all patches are sent upstream. The non-commercial branch of Illumos also lives well as OmniosCE and are well maintained by Andy and Tobias to mention a few (I have a small part as well ;-). I think you are thinking of Solaris as in Oracle Solaris which is definitely on the usual way to desintegration as all other stuff Larry gets his hands on. You should also remember that all ZFS development, apart from the black caves near Redwood City, is done in OpenZFS and all current implementations apart from Oracle uses the same source tree. > I think that It might be time to re-considering FreeBSD as a > potential stable solution to host iSCSI targets. Considering the very > important efforts FreeBSD has made to implement ZFS and stick to the > OpenZFS standards. > This has never been my attitude. FreeBSD is rock solid an has been for decades. > > This is not true. > > The limit that you are mentioning here has been overridden by a > tunable parameter. > https://www.freebsd.org/cgi/man.cgi?query=ctl&sektion=4 > <https://www.freebsd.org/cgi/man.cgi?query=ctl&sektion=4> > > It turns out that this patch has been developed by one of my fellow > developer couple of years ago to bypass the limitations of 1024 LUNs > that you mentioned. > Do you have any personal experiences with raising the number of LUNs? Eg. increase it by a factor of 10? > So It should qualify to be selected as an enterprise grade solution > to host iSCSI target. > > The fact that some developers are patching Proxmox code to allow *BSD > / FreeNAS to perform is a sign that there might have a need for such > tool. > > So beside this max luns problem, what else seems to be causing > problem with the BSD target ? > The LUN limitation was my only objection when I stopped to develop the storage plugin. It should be rather easy to write such a plugin using the istgt code an replace the iSCSI part with cltd. For a motivated developer this should not take more than a weekends work :-) -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc https://pgp.key-server.io/pks/lookup?search=0xD3C9A00E mir datanom net https://pgp.key-server.io/pks/lookup?search=0xE501F51C mir miras org https://pgp.key-server.io/pks/lookup?search=0xE3E80917 -- /usr/games/fortune -es says: Quid me anxius sum? [ What? Me, worry? ] pgptxPvZQth9T.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Integration of FreeNAS iSCSI target initiator in Proxmox Enterprise repo
On Wed, 3 Jun 2020 12:03:05 +0200 Michael Rasmussen wrote: > so therefore qualifies Should have read 'so therefore not qualifies' ;-) -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc https://pgp.key-server.io/pks/lookup?search=0xD3C9A00E mir datanom net https://pgp.key-server.io/pks/lookup?search=0xE501F51C mir miras org https://pgp.key-server.io/pks/lookup?search=0xE3E80917 -- /usr/games/fortune -es says: Toto, I don't think we're in Kansas anymore. -- Judy Garland, "Wizard of Oz" pgpJpAVcokaO9.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Integration of FreeNAS iSCSI target initiator in Proxmox Enterprise repo
On Wed, 3 Jun 2020 11:34:35 +0200 Andreas Steinel wrote: > > If I remember correctly, the problem with the ZFS-over-iSCSI stuff is > that the backend provider in FreeNAS/FreeBSD has changed numerous > times and at least one version did not support online > reconfiguration. That may be solved right now, but I haven't look at > the plugin for a long time. It just works for Debian as Mario > (@fireon) described in [2]. > A few years ago the FreeNAS API was fluctuating between every new minor release (eg, 9.3 -> 9.4) and was as such called 'not production ready' by IXSystems and therefore labelled beta, experimental etc (I know that no product developed by Alfabet (google) never has transitioned from beta to production state ;-) This means that maintaining such a plugin is very time consuming and since this is bade in peoples free time there likely will be the same resources required to do that. On the other hands those plugin already present in Proxmox uses production stamped solutions which rarely/ever makes breaking changes is more likely to be maintainable in peoples free time. As a side note: The current iscsi initiator (ctld) in FreeBSD still has an upper limit of 1024 LUNs per target so therefore qualifies as an plugin alternative to proxmox #define MAX_LUNS 1024 [0] [0] https://github.com/freebsd/freebsd/blob/master/usr.sbin/ctld/ctld.h -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc https://pgp.key-server.io/pks/lookup?search=0xD3C9A00E mir datanom net https://pgp.key-server.io/pks/lookup?search=0xE501F51C mir miras org https://pgp.key-server.io/pks/lookup?search=0xE3E80917 -- /usr/games/fortune -es says: Youth doesn't excuse everything. -- Dr. Janice Lester (in Kirk's body), "Turnabout Intruder", stardate 5928.5. pgp1Zh2H0ur0T.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] scsi-hd vs scsi-generic with iSCSI
On Wed, 9 Oct 2019 17:39:56 +0200 Thomas Lamprecht wrote: > > I would actually really like to change this to scsi-hd, but we need > to be sure it's OK for all possible supported setups.. > > So I tried to investigate a bit how it came to the use of > scsi-generic, I came to a commit[0] from Alexandre (CCd) which adds > support for access with libiscsi. > > Maybe he knows why the -generic was used and not the -hd one? > > [0]: > https://git.proxmox.com/?p=qemu-server.git;a=commitdiff;h=d454d040338a6216c8d3e5cc9623d6223476cb5a > For my part I can say that I have several times pointed out that using 'generic' also disables any kind of IO stats so therefore I have for the last 3 years replaced: $devicetype = 'generic' with $devicetype = 'block' to have IO stats when using zfs_over_iscsi. This have caused no problems what so +1 for me to have it removed PS. The issue was discussed on this list 2016-09-29 to 2016-09-30 between Alexandre, Dietmar, and me. This was a patch by Alexandre witch was related to an issue related to cpu iothread bottleneck with Ceph and scsi-block vs scsi-generic. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc https://pgp.key-server.io/pks/lookup?search=0xD3C9A00E mir datanom net https://pgp.key-server.io/pks/lookup?search=0xE501F51C mir miras org https://pgp.key-server.io/pks/lookup?search=0xE3E80917 -- /usr/games/fortune -es says: lp1 on fire (One of the more obfuscated kernel messages) pgpSlPHyv_be4.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH storage V2 0/3] Fixes in ZFS over iSCSI
On Tue, 24 Sep 2019 07:13:21 +0200 Thomas Lamprecht wrote: > On 9/23/19 7:22 PM, Michael Rasmussen wrote: > > On Mon, 23 Sep 2019 16:35:00 +0200 > > Thomas Lamprecht wrote: > > > >>> PVE/Storage/ZFSPlugin.pm | 10 +- > > Could this be a fix to > > https://bugzilla.proxmox.com/show_bug.cgi?id=2176 ? > > > > I guess not, as this affects only the LIO sub-plugin, > and you mention that you use Comstar which does not share > any code - so there cannot be any semantic changes there.. > This is not true. If you study the quoted part in my reply you will see a reference to a patch to ZFSPlugin.pm which in a number of functions are added code for parsing volname correct. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc https://pgp.key-server.io/pks/lookup?search=0xD3C9A00E mir datanom net https://pgp.key-server.io/pks/lookup?search=0xE501F51C mir miras org https://pgp.key-server.io/pks/lookup?search=0xE3E80917 -- /usr/games/fortune -es says: Happiness is good health and a bad memory. -- Ingrid Bergman pgpxdSVuLUPaD.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH storage V2 0/3] Fixes in ZFS over iSCSI
On Mon, 23 Sep 2019 16:35:00 +0200 Thomas Lamprecht wrote: > > PVE/Storage/ZFSPlugin.pm | 10 +- Could this be a fix to https://bugzilla.proxmox.com/show_bug.cgi?id=2176 ? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc https://pgp.key-server.io/pks/lookup?search=0xD3C9A00E mir datanom net https://pgp.key-server.io/pks/lookup?search=0xE501F51C mir miras org https://pgp.key-server.io/pks/lookup?search=0xE3E80917 -- /usr/games/fortune -es says: He hated being thought of as one of those people that wore stupid ornamental armour. It was gilt by association. -- Terry Pratchett, "Night Watch" pgp1a77bs6xop.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] HA Migration on shutdown/reboot
On Tue, 28 May 2019 08:09:30 +0200 (CEST) Alexandre DERUMIER wrote: > > with a real problem, when we have 2 hosts in same ha group, with > differents priority, > > and user want to do maintenance/reboot, he would like to migrate all > vms on other node, before reboot. But currently manually migrating > vms (with migrate_all for example), the HA is migrated them back > before the migrate all is finished, so it's super difficult to have > the node empty for reboot. > Will checking 'nofailback' for the group not solve this? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Lady Nancy Astor: "Winston, if you were my husband, I'd put poison in your coffee." Winston Churchill: "Nancy, if you were my wife, I'd drink it." pgpYOfFISk0q3.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH storage 0/1] Fix for ZFS over iSCSI on Solaris 11
On Thu, 16 May 2019 14:02:30 +0200 Dominik Csapak wrote: > > first i looked at omnios (current release) and there > 'stmfadm list-view' has no '-v' argument and fails > This is true and I guess it will never have that option. > as i know that at least the original author uses omnios with this, > i do not want to break it, or what is your stand on this @mir ? > As this is an Oracle Solaris thing and since Oracle deliberately does not want to share with the community (Oracle Solaris is published under a proprietary closed source licens) it is impossible to know the direction. As is know adding the patch will break all other Solaris offspring's for ever besides Oracle Solaris so the patch will need to be able to handle both Solaris and Oracle Solaris. > it would probably ok to try and detect the solaris version (how?) > and only do this if it supports those options > To be able to detect this you would require access to an Oracle Solaris installation. Maybe using uname could be a way? > as for get_initiator_name, we could probably put this into pve-common, > it seems general enough, then we could access it from qemu-server and > pve-storage > > This sounds like a good idea since initiator name is related to the consumer and not the provider. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: "If you don't want your dog to have bad breath, do what I do: Pour a little Lavoris in the toilet." -- Jay Leno pgphUpyra1sXk.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] HA: vm shutdown/stop behaviour and other HA questions
On Wed, 6 Mar 2019 07:59:14 +0100 (CET) Alexandre DERUMIER wrote: > > In the futur, I would like to add some kind of balancing/scheduling > of vm (memory/cpu balancing), I think it's the right place to do it ? > > I have looked in /usr/share/perl5/PVE/HA/Manager.pm, sub > select_service_node, seem pretty basic for now. (try node by > priority, and by nodeid, try_next if it's failing). I think they are > lot of improvment possible here. One serious HA feature really missing is a way to configure that two or more VMs most never run on the same node. Eg. there is really no HA for a cluster if every VM member dies with the same node, -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: There is more simplicity in the man who eats caviar on impulse than in the man who eats Grape-Nuts on principle. -- G. K. Chesterton pgphC6GuyRTx3.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] cpuflag: pcid needed in guest for good performance after meltdown
On Tue, 9 Jan 2018 19:21:15 +0100 Fabian Grünbichler wrote: > > yes. it's an Intel CPU feature, and only needed to reduce the > performance impact of KPTI, which is a mitigation for a bug affecting > Intel CPUs (and some ARM). since KPTI is disabled unless manually forced > on AMD CPUs, the fact that there is no PCID does not matter there ;) > As shown in my last post (except for the mail server censuring images) was, that I have pki=on and this seems to have little or no effect on Opeteron processors what so ever. This could imply that this is an Intel problem only (NSA conspiracy hat on) -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: I got the bill for my surgery. Now I know what those doctors were wearing masks for. -- James Boren pgpFufyfQjDsK.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] cpuflag: pcid needed in guest for good performance after meltdown
On Tue, 9 Jan 2018 19:04:29 +0100 (CET) Alexandre DERUMIER wrote: > >>Is this an Intel only cpu flag? > > Yes, I think that it's only an intel flag. > > I'm not sure that amd will be impacted by the kernel patch anyway. (related > to pcid) > > I have some opterons to test, I'll try next week. All my Opterons show either no effect or only marginally performance hits attached a week average for my mail server. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: The most popular labor-saving device today is still a husband with money. -- Joey Adams, "Cindy and I" pgpF598jfgfbO.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] cpuflag: pcid needed in guest for good performance after meltdown
On Tue, 9 Jan 2018 16:31:40 +0100 Stefan Priebe - Profihost AG wrote: > > Thanks! I would love to see it as a default for the specific qemu cpu > models. > > I mean we know exactly that sandybridge, ivybridge, ... support it or not? > Is this an Intel only cpu flag? lscpu |grep Model Model: 2 Model name:AMD Opteron(tm) Processor 3350 HE grep pcid /proc/cpuinfo -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: You can't judge a book by the way it wears its hair. pgplQZJkkEGDZ.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] cpuflag: pcid needed in guest for good performance after meltdown
On Mon, 8 Jan 2018 21:34:57 +0100 Stefan Priebe - Profihost AG wrote: > Hello, > > for meltdown mitigation and performance it's important to have the pcid > flag passed down to the guest (f.e. > https://groups.google.com/forum/m/#!topic/mechanical-sympathy/L9mHTbeQLNU). > As I understand it it is only required given the guest also uses a kernel that take advantage of this feature and which is, for Linux, a kernel 4.14 post Meltdown fix? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Life is like a tin of sardines. We're, all of us, looking for the key. -- Beyond the Fringe pgpDN3i0eAXqD.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Request for improvement of Network handling regarding LXC
On Thu, 20 Jul 2017 13:22:58 +0200
Tom Weber wrote:
> + if (defined($d->{ip}) and ($d->{ip} ne "dhcp")) {
> + $raw .= "lxc.network.ipv4 = $d->{ip}\n";
> + $raw .= "lxc.network.ipv4.gateway = $d->{gw}\n" if
> defined($d->{gw});
> + $raw .= "lxc.network.flags = up\n" if defined($d->{ip});
The if defined($d->{ip}) in this line is not needed since this is
already true when entering this condition.
--
Hilsen/Regards
Michael Rasmussen
Get my public GnuPG keys:
michael rasmussen cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir datanom net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir miras org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--
/usr/games/fortune -es says:
Cats, no less liquid than their shadows, offer no angles to the wind.
pgpdpaWNTS9kJ.pgp
Description: OpenPGP digital signature
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH v7 pve-storage 00/10] FreeNAS storage plugin
Hi all, Is Fabian away for vacation? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: You are a very redundant person, that's what kind of person you are. pgp_GDFkAnriG.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH v7 pve-storage 02/10] Basic FreeNAS API interaction code
On Wed, 21 Jun 2017 15:58:13 +0200
Fabian Grünbichler wrote:
> > +die $res->code."\n" unless $res->is_success;
>
> this deviates from v5 - now you die with the HTTP error code, and not
> with the full status message (except for in freenas_create_target_group
> and freenas_create_target) - is this intentional?
>
This is intentional since a need sometimes to get the status code when
it dies.
> +foreach my $zvol (@$zvols) {
> +next unless $zvol->{name} =~ /^(base|vm)-(\d+)-disk-\d+$/;
>
I cannot use parse_volname here since a pool could easily have zvols
not related to proxmox in any shape or fashion. Using parse_volname
here would make the plugin die if such zvols were found.
> custom volume names still missing here? but maybe you'll include that
> when switching to parse_volname..
>
For the moment custom volume names is not supported. Frankly, I hardly
see the usecase?
>
> the hack with encoding the linked clone origin relationship in a snapshot name
> still seems really really bad, but I guess there is no other way given the
> current FreeNAS API limitations. I do wonder whether it would not be better to
> skip linked clones until the API exposes this information? otherwise we have
> to
> keep this workaround supported forever..
>
I intend to keep it since linked clones is a huge bonus.
--
Hilsen/Regards
Michael Rasmussen
Get my public GnuPG keys:
michael rasmussen cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir datanom net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir miras org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--
/usr/games/fortune -es says:
***
***
*
** Confucious say: "Is stuffy inside fortune cookie."
***
***
pgpNBUPLieyCO.pgp
Description: OpenPGP digital signature
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH v7 pve-storage 02/10] Basic FreeNAS API interaction code
Hi Fabian, On Wed, 21 Jun 2017 15:58:13 +0200 Fabian Grünbichler wrote: > > high level question #1: how do you guarantee that inbetween pagination > requests, the underlying lists/.. on the freenas side don't change > because of concurrent actions? > > getting outdated information altogether is probably fine (as in, you > can't expect not to sometimes get it). but since the result is > potentially a list, it might contain duplicates unless there is some > mechanism on the freenas side preventing the following: > > - request for first page, with last element "foo" returned > - concurrent request adding element "bar", which is sorted before > element "foo" according to whatever criteria is used to generate the > pagination > - request for second page, with first element "foo" returned > > end result: we don't know about "bar", but we get "foo" twice > Obviously, I can give no guarantee but I assume that the API behaves properly. No evidence so far profes me wrong. > > high level question #2: does the FreeNAS API return non-JSON apart from > error cases? I am not sure whether the mix and match between lists and > strings below is really necessary.. > The API only returns JSON. Error cases are covered by status codes. The reason for dealing with strings is to be prepared to deal with errors not coming from the API - eg. nginx (the webserver), bugs etc. > > high level question #3: why not introduce wrappers for get, post, put and > delete? e.g., > A good idea. Why did I not think about that ;-) PS. You keep complaining about using this stile: do unless something and do if something and want this stile instead: if not something do and if something do In my book your stile is bad Perl which is more C or Java stile instead. This is something that old Perl programmers will agree to (I consider myself an old Perl programmer since I have been writing Perl for more than 25 years). Since you are the gate keeper to the Proxmox code base I will of course change it - but with objections! ;-) -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: The root of all superstition is that men observe when a thing hits, but not when it misses. -- Francis Bacon pgppMCO82OG7H.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH v7 pve-storage 00/10] FreeNAS storage plugin
On Tue, 20 Jun 2017 22:39:52 +0200 m...@datanom.net wrote: A hole bunch of words. I forgot to mention that in response to previous comments from Fabian the following is on the todo list so no need to raise those again: TODO * Make target prefix configurable in gui * Configurable in gui whether to validate SSL certificate or not * Password and username in /etc/pve/priv * - /^(base|vm)-(\d+)-disk-\d+$/ in freenas_list_zvols, only $2 is used - /^(vm|base)-(\d+)-/ in freenas_get_target_name, only $2 is used -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Use statement labels that mean something. - The Elements of Programming Style (Kernighan & Plaugher) pgpv5x7XntPd6.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH v6 pve-storage 0/6] FreeNAS storage plugin
On Tue, 20 Jun 2017 09:22:52 +0200 Fabian Grünbichler wrote: > once you are happy with your commits: > "git format-patch -v7 --signoff --cover-letter HEAD~N" (where N is the > number of commits) > Just realized that FreeNAS 11 was released last week ;-) So when this first version is acceptet and the possible errors has been sorted out I will dig into whether 11 has something useful added which could improve the current version of the plugin in regards to API access. I am going to build a new storage server in a couple of days so I have the chance of real live tests instead of running FreeNAS virtualized to gather some trustworthy performance numbers. If by any chance somebody reading this has a FreeNAS with a CPU with hardware support for AESni I would be very interested in hearing what performance loss if any there is if the pool are encrypted using the GELI encryption extension which is provided in the FreeNAS gui. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Life is a yo-yo, and mankind ties knots in the string. pgpRSOVHglSxO.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH pve-storage 0/5] v5 fix regression and indentation
On Mon, 19 Jun 2017 20:54:18 +0200 Michael Rasmussen wrote: > Found it ;-) No need to change any code in LXC.pm > Cause: The code I added to circumvent the FreeNAS bug was doing a > rescan of the session which deactivated the volume before the command > unshare -m -- sh -c 'mount --make-rprivate / && mount > /dev/disk/by-path/ip-10.0.1.32:3260-iscsi-iqn.2005-10.org.freenas.ctl:vm-113-lun-0 > /tmp && resize2fs > /dev/disk/by-path/ip-10.0.1.32:3260-iscsi-iqn.2005-10.org.freenas.ctl:vm-113-lun-0' > was running resulting in a removing of the LUN from the scsi subsystem. > Resolving this bug also resolved not been able to do online resize of VM's so it seems my work-around was working anyway:-) -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Some rise by sin and some by virtue fall. pgpi0HtVgV0m_.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH pve-storage 0/5] v5 fix regression and indentation
On Mon, 19 Jun 2017 12:57:24 +0200 Fabian Grünbichler wrote: > > yes, that or debugging what mount is actually doing seems like the way > to go. if it fails as well with "-t nonfs", then I think you need to > check your plugin code for further timing problems. > Found it ;-) No need to change any code in LXC.pm Cause: The code I added to circumvent the FreeNAS bug was doing a rescan of the session which deactivated the volume before the command unshare -m -- sh -c 'mount --make-rprivate / && mount /dev/disk/by-path/ip-10.0.1.32:3260-iscsi-iqn.2005-10.org.freenas.ctl:vm-113-lun-0 /tmp && resize2fs /dev/disk/by-path/ip-10.0.1.32:3260-iscsi-iqn.2005-10.org.freenas.ctl:vm-113-lun-0' was running resulting in a removing of the LUN from the scsi subsystem. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Sometimes you get an almost irresistible urge to go on living. pgpEjoX3O2qxg.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH pve-storage 0/5] v5 fix regression and indentation
On Mon, 19 Jun 2017 13:06:48 +0200 "datanom.net" wrote: > > > I will try to add an udevadm trigger/settle before returning from > > > activate_volume in the code I see whether this solves the problem. > Forgot I already do this: +$os_request->(['iscsiadm', '-m', 'session', '-r', $sid, '-R'], 0, 60); +$os_request->( +['udevadm', 'trigger', '--type=devices', '--subsystem-match=scsi_disk'] , 0, 60); +$os_request->(['udevadm', 'settle', '-t', $api_timeout], 0, 60); -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: You never hesitate to tackle the most difficult problems. pgpABPqKQz1BK.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH v6 pve-storage 0/6] FreeNAS storage plugin
On Mon, 19 Jun 2017 16:50:43 +0200 Michael Rasmussen wrote: > Hi all, > > I do not know why this '[PATCH v6 pve-storage 1/1] A lot of bug fixes > and clean-ups' is part of the series!!?? It is identical to '[PATCH v6 > pve-storage 6/6] Bug fixes and clean-ups in response to review from Fabian.' > I now I found out, forgot to clean my_patches before sending :-( Should I resend without the wrong file part of the series? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: I love you more than anything in this world. I don't expect that will last. -- Elvis Costello pgpWg_mwX_PCO.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH v6 pve-storage 0/6] FreeNAS storage plugin
Hi all, I do not know why this '[PATCH v6 pve-storage 1/1] A lot of bug fixes and clean-ups' is part of the series!!?? It is identical to '[PATCH v6 pve-storage 6/6] Bug fixes and clean-ups in response to review from Fabian.' -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: QOTD: "Lack of planning on your part doesn't consitute an emergency on my part." pgpPHibUrLMWH.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH v6 pve-storage 0/1] FreeNAS storage plugin
Hi Fabian, If I understand you correctly you want a separate patch for every fix comment? On June 19, 2017 8:24:36 AM GMT+02:00, "Fabian Grünbichler" wrote: >On Sat, Jun 17, 2017 at 11:52:41PM +0200, m...@datanom.net wrote: >> From: Michael Rasmussen >> >> This is a patch to be added to v5 of the patch series. >> > >sorry - but like I said this is not how patches are supposed to be >sent. > >please split up your patch series into sensible patches based on what >they do, not when you have written them. then add/update/remove patches >between versions as makes sense. reviewing 1000 line diffs for every >iteration when the whole code is just a little over that size is crazy. > >the bigger the changes, the more important it is to keep the whole >thing >managable ;) > >> fixes >> # if ! to unless >> # all helper methods now private >> # remove unused includes >> # remove unused code >> # more descriptive variable names >> # change api timeout to be in sync with the PVE API >> # add loop over limit until empty resultset is returned >> # fix API version check >> # fix handling 409 code in create_target and create_target_group >> # Remove unnecessary error handling >> # Replace hardcoded max luns number with variable >> # Fix error handling in freenas_create_lun method >> # Write directly to file instead of using a shell to echo to file >> # Fix error handling in deactivate_lun and get_active_luns >> # Improve code readability and error handling in rescan_session >> # Declaring variables where used in create_base >> # Check if base already exists when creating base >> # Improve error handling in create_base >> # Fix clone_image error handling and remove unnecessary check for >running base >> # Remove unused variable in alloc_image >> # More robust error handling in free_image >> # Remove HTML code from volume_resize >> # Fix error handling in shapshot rollback >> # Improve code readability and error handling in activate_lun >> # Improve code readability and error handling in deactivate_lun >> # Replace sleep 1 with a combination of udevadm trigger udevadm >settle >> # Remove check of running VM and CT. relaying on parsed option >running for >> Qemu and handle running LXC in PVE::API2::LXC now sends running >status >> as part of call to volume_resize when storage id is freenas. >> >> TODO >> # Make target prefix configurable in gui >> # Configurable in gui whether to validate SSL certificate or not >> >> >> Michael Rasmussen (1): >> A lot of bug fixes and clean-ups >> >> PVE/Storage/FreeNASPlugin.pm | 1011 >++ >> 1 file changed, 533 insertions(+), 478 deletions(-) -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH pve-storage 0/5] v5 fix regression and indentation
Hi Fabian,
On Fri, 16 Jun 2017 14:21:34 +0200
Fabian Grünbichler wrote:
> > sub volume_resize {
> > my ($class, $scfg, $storeid, $volname, $size, $running) = @_;
> >
> > my ($vtype, $name, $vmid) = $class->parse_volname($volname);
> >
> > my $run = PVE::QemuServer::check_running($vmid);
> > if (!$run) {
> > $run = PVE::LXC::check_running($vmid);
> > }
>
> not allowed here - see comment on the very top.
>
> alternative solutions:
> - expand storage API here to split $running into two parameters, modify
> PVE::LXC and PVE::QemuServer to pass $running and the new parameter
> accordingly, adapt other plugins to handle $running && $new_parameter
> like they used to handle $running (if they did), adapt your plugin to
> die if $running to forbid online resizing for now.
> - special case your plugin in PVE::LXC to handle running containers
> there (Qemu does pass $running correctly).
>
I have studied this phenomenon in greater depth and more and more I
begin to think that this is a bug burruniied deep down somewhere in
PVE::Tools::run_command, it is not related to FreeNAS or my plugin.
When you try to resize a running container you get this error message:
mount.nfs: Failed to resolve server /dev/disk/by-path/ip-10.0.1.32: Name or
service not known
Failed to update the container's filesystem: command 'unshare -m -- sh -c
'mount --make-rprivate / && mount
/dev/disk/by-path/ip-10.0.1.32:3260-iscsi-iqn.2005-10.org.freenas.ctl:vm-113-lun-0
/tmp && resize2fs
/dev/disk/by-path/ip-10.0.1.32:3260-iscsi-iqn.2005-10.org.freenas.ctl:vm-113-lun-0''
failed: exit code 32
If you copy/paste the command which pve-manager tries to run to the
command line everything works as expected, it is only when runned from
pve-manager the command fails. So resizing running CT's is disabled due
to a bug in pve-manager and not in the FreeNAS API or the plugin.
As a side note I can tell that path now returns the path with : escaped
as you sugggested but to no avail.
A small test case included:
#!/usr/bin/perl
use PVE::Tools qw(run_command);
$timeout = 5;
$noerr = 1;
my $text = '';
my $output = sub {
my $line = shift;
$text .= "$line\n";
};
my $cmd = ['unshare', '-m', '--', 'sh', '-c', 'mount --make-rprivate / && mount
/dev/disk/by-path/ip-10.0.1.32:3260-iscsi-iqn.2005-10.org.freenas.ctl:vm-113-lun-0
/tmp && resize2fs
/dev/disk/by-path/ip-10.0.1.32:3260-iscsi-iqn.2005-10.org.freenas.ctl:vm-113-lun-0'];
my $exit_code = run_command($cmd, noerr => $noerr, errfunc => $output, outfunc
=> $output, timeout => $timeout);
print "$exit_code: $text";
--
Hilsen/Regards
Michael Rasmussen
Get my public GnuPG keys:
michael rasmussen cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir datanom net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir miras org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--
/usr/games/fortune -es says:
"A power so great, it can only be used for Good or Evil!"
-- Firesign Theatre, "The Giant Rat of Summatra"
pgp_77e8Aotjl.pgp
Description: OpenPGP digital signature
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH pve-storage 0/5] v5 fix regression and indentation
On Sat, 17 Jun 2017 13:33:38 +0200 Andreas Steinel wrote: > > You said that you do not have the required hardware, however, you do > not need additional hardware for that. > > I can provide you with a multipath configuration if you like. iSCSI is > not different to FC-based multipath, so configuration files are > interchangeable. My storage is connected through Infiniband and Infiniband does not support either bonding nor having to devices in the same network so there is no way for me to test this. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: But what can you do with it? -- ubiquitous cry from Linux-user partner. (Submitted by Andy Pearce, a...@hpopd.pwd.hp.com) pgp4yE4cEALFa.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH pve-storage 0/5] v5 fix regression and indentation
On Sat, 17 Jun 2017 10:15:25 +0200 Andreas Steinel wrote: > Mi Michael, > > On Fri, Jun 16, 2017 at 3:10 AM, Michael Rasmussen wrote: > > MPIO is not tested since I don't have the required hardware to do so. > > Is there more than adding another IP to the FreeNAS machine? I've done > that on a Debian-based target and it automatically logged into each > portal, so multipathing worked out of the box. > It requires multipath tools on the proxmox node: https://pve.proxmox.com/wiki/ISCSI_Multipath -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Oh, get ahold of yourself. Nobody's proposing that we parse English. -- Larry Wall in <199709032332.qaa21...@wall.org> pgpNFN_bvXJEt.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH pve-storage 0/5] v5 fix regression and indentation
On Fri, 16 Jun 2017 14:21:34 +0200 Fabian Grünbichler wrote: > > my $limit = 1; # limit for get requests > > this seems kind of arbitrary? if you really need such limit, you also > need to implement pagination or something like it, otherwise you are in > trouble. > The reason for this is that the API has a default limit which is configured to 20 and there is no way of knowing the number of rows in the full dataset. Setting the number to this high level should prevent having to loop. This was only chosen for the beta so the next patch will have a loop requesting a batch at the time until an empty resultset is returned. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Don't comment bad code - rewrite it. - The Elements of Programming Style (Kernighan & Plaugher) pgpJD5utmYzar.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH pve-storage 0/5] v5 fix regression and indentation
On Fri, 16 Jun 2017 14:21:34 +0200 Fabian Grünbichler wrote: > > one big point that I haven't really looked at is whether you need to > guard some operations with locks. especially the calls to iscsiadm are > triggered in a lot of code paths, and I don't know whether those are > problematic when executed concurrently or not. some of the higher level I have not found any evidence that locks should be needed since, by nature, the scsi subsystem is thread safe. This is not a problem I have seen so far. > > also haven't checked for how it would handle an already existing local > (open-)iscsi setup.. did you? > My proof of concept (POC) was tested on an existing iscsi setup added libiscsi to the pool. There were no interfering with existing setup what so ever. The clever thing in my setup is that all interaction with the scsi subsystem is closely tied to a specific session which is invisible to other sessions. You can compared it to a database transaction. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Come quickly, I am tasting stars! -- Dom Perignon, upon discovering champagne. pgplms2CWxo9u.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-storage 3/5] Fix missing $vmid
Signed-off-by: Michael Rasmussen
---
PVE/Storage/FreeNASPlugin.pm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/PVE/Storage/FreeNASPlugin.pm b/PVE/Storage/FreeNASPlugin.pm
index c438519..0f2a56d 100644
--- a/PVE/Storage/FreeNASPlugin.pm
+++ b/PVE/Storage/FreeNASPlugin.pm
@@ -1072,6 +1072,8 @@ sub free_image {
sub volume_resize {
my ($class, $scfg, $storeid, $volname, $size, $running) = @_;
+my ($vtype, $name, $vmid) = $class->parse_volname($volname);
+
my $run = PVE::QemuServer::check_running($vmid);
if (!$run) {
$run = PVE::LXC::check_running($vmid);
@@ -1080,8 +1082,6 @@ sub volume_resize {
die 'mode failure - unable to resize disk(s) on a running system due to
FreeNAS bug.
See bug report: https://bugs.freenas.org/issues/24432"; target="_blank">#24432'
if $run;
-my ($vtype, $name, $vmid) = $class->parse_volname($volname);
-
my $data = {
volsize => $size,
};
--
2.11.0
This mail was virus scanned and spam checked before delivery.
This mail is also DKIM signed. See header dkim-signature.
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-storage 2/5] Proper check if VM/CT is running
Signed-off-by: Michael Rasmussen
---
PVE/Storage/FreeNASPlugin.pm | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/PVE/Storage/FreeNASPlugin.pm b/PVE/Storage/FreeNASPlugin.pm
index 771061e..c438519 100644
--- a/PVE/Storage/FreeNASPlugin.pm
+++ b/PVE/Storage/FreeNASPlugin.pm
@@ -1072,8 +1072,13 @@ sub free_image {
sub volume_resize {
my ($class, $scfg, $storeid, $volname, $size, $running) = @_;
+my $run = PVE::QemuServer::check_running($vmid);
+if (!$run) {
+$run = PVE::LXC::check_running($vmid);
+}
+
die 'mode failure - unable to resize disk(s) on a running system due to
FreeNAS bug.
-See bug report: https://bugs.freenas.org/issues/24432"; target="_blank">#24432'
if $running;
+See bug report: https://bugs.freenas.org/issues/24432"; target="_blank">#24432'
if $run;
my ($vtype, $name, $vmid) = $class->parse_volname($volname);
--
2.11.0
This mail was virus scanned and spam checked before delivery.
This mail is also DKIM signed. See header dkim-signature.
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-storage 1/5] First beta of FreeNAS storage plugin. Missing snapshot backup for LXC
Signed-off-by: Michael Rasmussen
---
PVE/Storage.pm |2 +
PVE/Storage/FreeNASPlugin.pm | 1289 ++
PVE/Storage/Makefile |2 +-
PVE/Storage/Plugin.pm|2 +-
4 files changed, 1293 insertions(+), 2 deletions(-)
create mode 100644 PVE/Storage/FreeNASPlugin.pm
diff --git a/PVE/Storage.pm b/PVE/Storage.pm
index ee2295a..f33249f 100755
--- a/PVE/Storage.pm
+++ b/PVE/Storage.pm
@@ -32,6 +32,7 @@ use PVE::Storage::GlusterfsPlugin;
use PVE::Storage::ZFSPoolPlugin;
use PVE::Storage::ZFSPlugin;
use PVE::Storage::DRBDPlugin;
+use PVE::Storage::FreeNASPlugin;
# Storage API version. Icrement it on changes in storage API interface.
use constant APIVER => 1;
@@ -49,6 +50,7 @@ PVE::Storage::GlusterfsPlugin->register();
PVE::Storage::ZFSPoolPlugin->register();
PVE::Storage::ZFSPlugin->register();
PVE::Storage::DRBDPlugin->register();
+PVE::Storage::FreeNASPlugin->register();
# load third-party plugins
if ( -d '/usr/share/perl5/PVE/Storage/Custom' ) {
diff --git a/PVE/Storage/FreeNASPlugin.pm b/PVE/Storage/FreeNASPlugin.pm
new file mode 100644
index 000..771061e
--- /dev/null
+++ b/PVE/Storage/FreeNASPlugin.pm
@@ -0,0 +1,1289 @@
+package PVE::Storage::FreeNASPlugin;
+
+use strict;
+use warnings;
+use IO::File;
+use POSIX;
+use PVE::Tools qw(run_command);
+use PVE::Storage::Plugin;
+use PVE::RPCEnvironment;
+use PVE::QemuServer;
+use PVE::LXC;
+use LWP::UserAgent;
+use HTTP::Status;
+use JSON;
+use Data::Dumper;
+
+use base qw(PVE::Storage::Plugin);
+
+my $api = '/api/v1.0';
+my $timeout = 60; # seconds
+my $max_luns = 20; # maximum supported luns per target group in freenas is 25
+ # but reserve 5 for temporary LUNs (snapshots with RAM and
+ # snapshot backup)
+my $active_snaps = 4;
+my $limit = 1; # limit for get requests
+my $version;
+my $fullversion;
+my $target_prefix = 'iqn.2005-10.org.freenas.ctl';
+
+sub freenas_request {
+my ($scfg, $request, $section, $data, $valid_code) = @_;
+my $ua = LWP::UserAgent->new;
+$ua->agent("ProxmoxUA/0.1");
+$ua->ssl_opts( verify_hostname => 0 );
+$ua->timeout($timeout);
+push @{ $ua->requests_redirectable }, 'POST';
+push @{ $ua->requests_redirectable }, 'PUT';
+push @{ $ua->requests_redirectable }, 'DELETE';
+my $req;
+
+my $url = "https://$scfg->{portal}$api/$section";
+
+if ($request eq 'GET') {
+$req = HTTP::Request->new(GET => $url);
+} elsif ($request eq 'POST') {
+$req = HTTP::Request->new(POST => $url);
+$req->content($data);
+} elsif ($request eq 'PUT') {
+$req = HTTP::Request->new(PUT => $url);
+$req->content($data);
+} elsif ($request eq 'DELETE') {
+$req = HTTP::Request->new(DELETE => $url);
+} else {
+die "$request: Unknown request";
+}
+
+$req->content_type('application/json');
+$req->authorization_basic($scfg->{username}, $scfg->{password});
+my $res = $ua->request($req);
+
+ #print Dumper($res);
+return $res->code unless $res->is_success;
+
+return $res->content;
+}
+
+sub freenas_get_version {
+my ($scfg) = @_;
+
+ my $response = freenas_request($scfg, 'GET', "system/version");
+ die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+ my $info = decode_json($response);
+ $fullversion = $info->{fullversion};
+ if ($fullversion =~ /^\w+-(\d+)\.(\d*)\.(\d*)/) {
+ my $minor = $2;
+ my $micro = $3;
+
+ if ($minor) {
+ $minor = "0$minor" unless $minor > 9;
+ } else {
+ $minor = '00';
+ }
+
+ if ($micro) {
+ $micro = "0$micro" unless $micro > 9;
+ } else {
+ $micro = '00';
+ }
+
+ $version = "$1$minor$micro";
+ } else {
+ $version = '90200';
+ }
+}
+
+sub freenas_list_zvol {
+my ($scfg) = @_;
+
+ freenas_get_version($scfg) unless $version;
+
+ my $response = freenas_request($scfg, 'GET',
"storage/volume/$scfg->{pool}/zvols?limit=$limit");
+ die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+my $zvols = decode_json($response);
+ $response = freenas_request($scfg, 'GET',
"storage/snapshot?limit=$limit");
+ die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+my $snapshots = decode_json($response);
+
+my $list
[pve-devel] [PATCH pve-storage 4/5] Add support for creating LXC snapshot backup
Signed-off-by: Michael Rasmussen
---
PVE/Storage/FreeNASPlugin.pm | 129 +--
1 file changed, 74 insertions(+), 55 deletions(-)
diff --git a/PVE/Storage/FreeNASPlugin.pm b/PVE/Storage/FreeNASPlugin.pm
index 0f2a56d..fee3d27 100644
--- a/PVE/Storage/FreeNASPlugin.pm
+++ b/PVE/Storage/FreeNASPlugin.pm
@@ -427,40 +427,38 @@ sub freenas_find_free_diskname {
}
sub freenas_get_lun_number {
-my ($scfg, $volname, $snap) = @_;
+my ($scfg, $volname) = @_;
my $lunid = undef;
-if ($volname =~ /^(vm|base)-\d+-disk-(\d+)$/ && ! defined $snap) {
+if ($volname =~ /^(vm|base)-\d+-disk-(\d+)$/) {
$lunid = $2 - 1;
} elsif ($volname =~ /^vm-(\d+)-state/) {
# Find id for temporary LUN
- if ($snap) {
- # TODO
- # Required to be able to exposed read-only LUNs for snapshot
backup CT
- } else {
- my $target = freenas_get_target($scfg, $1);
- my $id = $max_luns;
- my $response = freenas_request($scfg, 'GET',
"services/iscsi/targettoextent?limit=$limit");
- die HTTP::Status::status_message($response) if
$response =~ /^\d+$/;
- my $t2extents = decode_json($response);
-
- foreach my $t2extent (@$t2extents) {
- next unless $t2extent->{iscsi_target} ==
$target &&
-
$t2extent->{iscsi_lunid} + 1 > $max_luns &&
-
$t2extent->{iscsi_lunid} < $max_luns + $active_snaps;
- my $eid = freenas_get_extent($scfg, $volname);
- if ($eid) {
- $response = freenas_request($scfg,
'GET', "services/iscsi/extent/$eid");
- die
HTTP::Status::status_message($response) if $response =~ /^\d+$/;
- my $extent = decode_json($response);
- # Request to get lunid for an existing
lun
- last if $t2extent->{iscsi_extent} eq
$eid;
- }
- $id++;
+ my $target = freenas_get_target($scfg, $1);
+ my $id = $max_luns;
+ my $response = freenas_request($scfg, 'GET',
"services/iscsi/targettoextent?limit=$limit");
+ die HTTP::Status::status_message($response) if $response =~
/^\d+$/;
+ my $t2extents = decode_json($response);
+
+ foreach my $t2extent (@$t2extents) {
+ next unless $t2extent->{iscsi_target} == $target &&
+ $t2extent->{iscsi_lunid} + 1 >
$max_luns &&
+ $t2extent->{iscsi_lunid} <
$max_luns + $active_snaps;
+ my $eid = freenas_get_extent($scfg, $volname);
+ if ($eid) {
+ $response = freenas_request($scfg, 'GET',
"services/iscsi/extent/$eid");
+ die HTTP::Status::status_message($response) if
$response =~ /^\d+$/;
+ my $extent = decode_json($response);
+ # Request to get lunid for an existing lun
+ last if $t2extent->{iscsi_extent} eq $eid;
}
- die "Max snapshots (4) is reached" unless ($id -
$max_luns) < $active_snaps;
- $lunid = $id;
- }
+ $id++;
+ }
+ die "Max snapshots (4) is reached" unless ($id - $max_luns) <
$active_snaps;
+ $lunid = $id;
+} elsif ($volname =~ /^(vm|base)-\d+-disk-\d+\@vzdump$/) {
+# Required to be able to exposed read-only LUNs for snapshot backup CT
+$lunid = $max_luns + $active_snaps;
}
return $lunid;
@@ -891,12 +889,13 @@ sub list_images {
sub path {
my ($class, $scfg, $volname, $storeid, $snapname) = @_;
-
-die "direct access to snapshots not implemented"
- if defined($snapname);
+#die "direct access to snapshots not implemented"
+ #if defined($snapname);
my ($vtype, $vname, $vmid) = $class->parse_volname($volname);
+ $vname = "$vname\@$snapname" if $snapname;
+
my $luns = get_active_luns($class, $storeid, $scfg, $vname);
my $path = disk_by_path($scfg, $vname);
@@ -1116,24 +1115,54 @@ sub volume_resize {
sub volume_snapshot {
my ($class, $scfg, $storeid, $volname, $snap) = @_;
-
-my
[pve-devel] [PATCH pve-storage 5/5] Fix regression and indentation
snapshot backup of CT broke general snapshot handling.
Handle snapshot creation and deletion properly.
Signed-off-by: Michael Rasmussen
---
PVE/Storage/FreeNASPlugin.pm | 1570 +-
1 file changed, 782 insertions(+), 788 deletions(-)
diff --git a/PVE/Storage/FreeNASPlugin.pm b/PVE/Storage/FreeNASPlugin.pm
index fee3d27..9fe64d6 100644
--- a/PVE/Storage/FreeNASPlugin.pm
+++ b/PVE/Storage/FreeNASPlugin.pm
@@ -58,7 +58,6 @@ sub freenas_request {
$req->authorization_basic($scfg->{username}, $scfg->{password});
my $res = $ua->request($req);
- #print Dumper($res);
return $res->code unless $res->is_success;
return $res->content;
@@ -66,43 +65,43 @@ sub freenas_request {
sub freenas_get_version {
my ($scfg) = @_;
-
- my $response = freenas_request($scfg, 'GET', "system/version");
- die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
- my $info = decode_json($response);
- $fullversion = $info->{fullversion};
- if ($fullversion =~ /^\w+-(\d+)\.(\d*)\.(\d*)/) {
- my $minor = $2;
- my $micro = $3;
-
- if ($minor) {
- $minor = "0$minor" unless $minor > 9;
- } else {
- $minor = '00';
- }
-
- if ($micro) {
- $micro = "0$micro" unless $micro > 9;
- } else {
- $micro = '00';
- }
-
- $version = "$1$minor$micro";
- } else {
- $version = '90200';
- }
+
+my $response = freenas_request($scfg, 'GET', "system/version");
+die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+my $info = decode_json($response);
+$fullversion = $info->{fullversion};
+if ($fullversion =~ /^\w+-(\d+)\.(\d*)\.(\d*)/) {
+my $minor = $2;
+my $micro = $3;
+
+if ($minor) {
+$minor = "0$minor" unless $minor > 9;
+} else {
+$minor = '00';
+}
+
+if ($micro) {
+$micro = "0$micro" unless $micro > 9;
+} else {
+$micro = '00';
+}
+
+$version = "$1$minor$micro";
+} else {
+$version = '90200';
+}
}
sub freenas_list_zvol {
my ($scfg) = @_;
- freenas_get_version($scfg) unless $version;
-
- my $response = freenas_request($scfg, 'GET',
"storage/volume/$scfg->{pool}/zvols?limit=$limit");
- die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+freenas_get_version($scfg) unless $version;
+
+my $response = freenas_request($scfg, 'GET',
"storage/volume/$scfg->{pool}/zvols?limit=$limit");
+die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
my $zvols = decode_json($response);
- $response = freenas_request($scfg, 'GET',
"storage/snapshot?limit=$limit");
- die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+$response = freenas_request($scfg, 'GET', "storage/snapshot?limit=$limit");
+die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
my $snapshots = decode_json($response);
my $list = ();
@@ -110,13 +109,13 @@ sub freenas_list_zvol {
my $vmid;
my $parent;
foreach my $zvol (@$zvols) {
- next unless $zvol->{name} =~ /^(base|vm)-(\d+)-disk-\d+$/;
- $vmid = $2;
- $parent = undef;
- foreach my $snap (@$snapshots) {
- next unless $snap->{name} eq "__base__$vmid";
- $parent = $snap->{filesystem} =~ /^$scfg->{pool}\/(.+)$/ ? $1 :
undef;
- }
+next unless $zvol->{name} =~ /^(base|vm)-(\d+)-disk-\d+$/;
+$vmid = $2;
+$parent = undef;
+foreach my $snap (@$snapshots) {
+next unless $snap->{name} eq "__base__$vmid";
+$parent = $snap->{filesystem} =~ /^$scfg->{pool}\/(.+)$/ ? $1 :
undef;
+}
$list->{$scfg->{pool}}->{$zvol->{name}} = {
name => $zvol->{name},
size => $zvol->{volsize},
@@ -125,280 +124,280 @@ sub freenas_list_zvol {
format => 'raw',
};
if ($zvol->{name} =~ /^base-(.*)/) {
- $hide->{"vm-$1"} = 1;
+$hide->{"vm-$1"} = 1;
}
}
delete @{$list->{$scfg->{pool}}}{keys %$hide};
-
+
return $list;
}
sub freenas_no_more_extents {
- my ($scfg, $target) = @
[pve-devel] [PATCH pve-storage 0/5] v5 fix regression and indentation
v5 regression and indentation fix V4 Adds support for creating snapshot backups of LXC containers. This more or less makes the plugin feature complete. Only outstanding feature depends on a bug fix in the FreeNAS API which is first scheduled for next stable release: https://bugs.freenas.org/issues/23881 (Targeted for 11.1) This patch series adds a storage plugin for FreeNAS using the FreeNAS API. The plugin supports both Qemu and LXC VM. Features supported, see below: VM CT create YES YES delete YES YES resize (YES) (YES) Note 1 snapshot offline YES YES live YES "YES" (state is not saved) backup snapshot YES YES standby YES YES offline YES YES clone full YES YES (Unavailable in GUI) linked YES YES (Unavailable in GUI) Note 1: Due to a bug in the FreeNAS API live resizing has been disabled in the plugin. See https://bugs.freenas.org/issues/24432 Since the plugin attaches disks through the local scsi subsystem via openiscsi and therefore not uses libiscsi there should be support for MPIO and authentication. MPIO is not tested since I don't have the required hardware to do so. Authentication is not enabled but should be easy to do if the requirements exists. Michael Rasmussen (5): First beta of FreeNAS storage plugin. Missing snapshot backup for LXC Proper check if VM/CT is running Fix missing $vmid Add support for creating LXC snapshot backup Fix regression and indentation PVE/Storage.pm |2 + PVE/Storage/FreeNASPlugin.pm | 1307 ++ PVE/Storage/Makefile |2 +- PVE/Storage/Plugin.pm|2 +- 4 files changed, 1311 insertions(+), 2 deletions(-) create mode 100644 PVE/Storage/FreeNASPlugin.pm -- 2.11.0 This mail was virus scanned and spam checked before delivery. This mail is also DKIM signed. See header dkim-signature. ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-storage 2/5] Proper check if VM/CT is running
Signed-off-by: Michael Rasmussen
---
PVE/Storage/FreeNASPlugin.pm | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/PVE/Storage/FreeNASPlugin.pm b/PVE/Storage/FreeNASPlugin.pm
index 771061e..c438519 100644
--- a/PVE/Storage/FreeNASPlugin.pm
+++ b/PVE/Storage/FreeNASPlugin.pm
@@ -1072,8 +1072,13 @@ sub free_image {
sub volume_resize {
my ($class, $scfg, $storeid, $volname, $size, $running) = @_;
+my $run = PVE::QemuServer::check_running($vmid);
+if (!$run) {
+$run = PVE::LXC::check_running($vmid);
+}
+
die 'mode failure - unable to resize disk(s) on a running system due to
FreeNAS bug.
-See bug report: https://bugs.freenas.org/issues/24432"; target="_blank">#24432'
if $running;
+See bug report: https://bugs.freenas.org/issues/24432"; target="_blank">#24432'
if $run;
my ($vtype, $name, $vmid) = $class->parse_volname($volname);
--
2.11.0
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-storage 0/5] v5 fix regression and indentation
v5 regression and indentation fix V4 Adds support for creating snapshot backups of LXC containers. This more or less makes the plugin feature complete. Only outstanding feature depends on a bug fix in the FreeNAS API which is first scheduled for next stable release: https://bugs.freenas.org/issues/23881 (Targeted for 11.1) This patch series adds a storage plugin for FreeNAS using the FreeNAS API. The plugin supports both Qemu and LXC VM. Features supported, see below: VM CT create YES YES delete YES YES resize (YES) (YES) Note 1 snapshot offline YES YES live YES "YES" (state is not saved) backup snapshot YES YES standby YES YES offline YES YES clone full YES YES (Unavailable in GUI) linked YES YES (Unavailable in GUI) Note 1: Due to a bug in the FreeNAS API live resizing has been disabled in the plugin. See https://bugs.freenas.org/issues/24432 Since the plugin attaches disks through the local scsi subsystem via openiscsi and therefore not uses libiscsi there should be support for MPIO and authentication. MPIO is not tested since I don't have the required hardware to do so. Authentication is not enabled but should be easy to do if the requirements exists. Michael Rasmussen (5): First beta of FreeNAS storage plugin. Missing snapshot backup for LXC Proper check if VM/CT is running Fix missing $vmid Add support for creating LXC snapshot backup Fix regression and indentation PVE/Storage.pm |2 + PVE/Storage/FreeNASPlugin.pm | 1307 ++ PVE/Storage/Makefile |2 +- PVE/Storage/Plugin.pm|2 +- 4 files changed, 1311 insertions(+), 2 deletions(-) create mode 100644 PVE/Storage/FreeNASPlugin.pm -- 2.11.0 ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-storage 5/5] Fix regression and indentation
snapshot backup of CT broke general snapshot handling.
Handle snapshot creation and deletion properly.
Signed-off-by: Michael Rasmussen
---
PVE/Storage/FreeNASPlugin.pm | 1570 +-
1 file changed, 782 insertions(+), 788 deletions(-)
diff --git a/PVE/Storage/FreeNASPlugin.pm b/PVE/Storage/FreeNASPlugin.pm
index fee3d27..9fe64d6 100644
--- a/PVE/Storage/FreeNASPlugin.pm
+++ b/PVE/Storage/FreeNASPlugin.pm
@@ -58,7 +58,6 @@ sub freenas_request {
$req->authorization_basic($scfg->{username}, $scfg->{password});
my $res = $ua->request($req);
- #print Dumper($res);
return $res->code unless $res->is_success;
return $res->content;
@@ -66,43 +65,43 @@ sub freenas_request {
sub freenas_get_version {
my ($scfg) = @_;
-
- my $response = freenas_request($scfg, 'GET', "system/version");
- die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
- my $info = decode_json($response);
- $fullversion = $info->{fullversion};
- if ($fullversion =~ /^\w+-(\d+)\.(\d*)\.(\d*)/) {
- my $minor = $2;
- my $micro = $3;
-
- if ($minor) {
- $minor = "0$minor" unless $minor > 9;
- } else {
- $minor = '00';
- }
-
- if ($micro) {
- $micro = "0$micro" unless $micro > 9;
- } else {
- $micro = '00';
- }
-
- $version = "$1$minor$micro";
- } else {
- $version = '90200';
- }
+
+my $response = freenas_request($scfg, 'GET', "system/version");
+die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+my $info = decode_json($response);
+$fullversion = $info->{fullversion};
+if ($fullversion =~ /^\w+-(\d+)\.(\d*)\.(\d*)/) {
+my $minor = $2;
+my $micro = $3;
+
+if ($minor) {
+$minor = "0$minor" unless $minor > 9;
+} else {
+$minor = '00';
+}
+
+if ($micro) {
+$micro = "0$micro" unless $micro > 9;
+} else {
+$micro = '00';
+}
+
+$version = "$1$minor$micro";
+} else {
+$version = '90200';
+}
}
sub freenas_list_zvol {
my ($scfg) = @_;
- freenas_get_version($scfg) unless $version;
-
- my $response = freenas_request($scfg, 'GET',
"storage/volume/$scfg->{pool}/zvols?limit=$limit");
- die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+freenas_get_version($scfg) unless $version;
+
+my $response = freenas_request($scfg, 'GET',
"storage/volume/$scfg->{pool}/zvols?limit=$limit");
+die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
my $zvols = decode_json($response);
- $response = freenas_request($scfg, 'GET',
"storage/snapshot?limit=$limit");
- die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+$response = freenas_request($scfg, 'GET', "storage/snapshot?limit=$limit");
+die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
my $snapshots = decode_json($response);
my $list = ();
@@ -110,13 +109,13 @@ sub freenas_list_zvol {
my $vmid;
my $parent;
foreach my $zvol (@$zvols) {
- next unless $zvol->{name} =~ /^(base|vm)-(\d+)-disk-\d+$/;
- $vmid = $2;
- $parent = undef;
- foreach my $snap (@$snapshots) {
- next unless $snap->{name} eq "__base__$vmid";
- $parent = $snap->{filesystem} =~ /^$scfg->{pool}\/(.+)$/ ? $1 :
undef;
- }
+next unless $zvol->{name} =~ /^(base|vm)-(\d+)-disk-\d+$/;
+$vmid = $2;
+$parent = undef;
+foreach my $snap (@$snapshots) {
+next unless $snap->{name} eq "__base__$vmid";
+$parent = $snap->{filesystem} =~ /^$scfg->{pool}\/(.+)$/ ? $1 :
undef;
+}
$list->{$scfg->{pool}}->{$zvol->{name}} = {
name => $zvol->{name},
size => $zvol->{volsize},
@@ -125,280 +124,280 @@ sub freenas_list_zvol {
format => 'raw',
};
if ($zvol->{name} =~ /^base-(.*)/) {
- $hide->{"vm-$1"} = 1;
+$hide->{"vm-$1"} = 1;
}
}
delete @{$list->{$scfg->{pool}}}{keys %$hide};
-
+
return $list;
}
sub freenas_no_more_extents {
- my ($scfg, $target) = @
[pve-devel] [PATCH pve-storage 1/5] First beta of FreeNAS storage plugin. Missing snapshot backup for LXC
Signed-off-by: Michael Rasmussen
---
PVE/Storage.pm |2 +
PVE/Storage/FreeNASPlugin.pm | 1289 ++
PVE/Storage/Makefile |2 +-
PVE/Storage/Plugin.pm|2 +-
4 files changed, 1293 insertions(+), 2 deletions(-)
create mode 100644 PVE/Storage/FreeNASPlugin.pm
diff --git a/PVE/Storage.pm b/PVE/Storage.pm
index ee2295a..f33249f 100755
--- a/PVE/Storage.pm
+++ b/PVE/Storage.pm
@@ -32,6 +32,7 @@ use PVE::Storage::GlusterfsPlugin;
use PVE::Storage::ZFSPoolPlugin;
use PVE::Storage::ZFSPlugin;
use PVE::Storage::DRBDPlugin;
+use PVE::Storage::FreeNASPlugin;
# Storage API version. Icrement it on changes in storage API interface.
use constant APIVER => 1;
@@ -49,6 +50,7 @@ PVE::Storage::GlusterfsPlugin->register();
PVE::Storage::ZFSPoolPlugin->register();
PVE::Storage::ZFSPlugin->register();
PVE::Storage::DRBDPlugin->register();
+PVE::Storage::FreeNASPlugin->register();
# load third-party plugins
if ( -d '/usr/share/perl5/PVE/Storage/Custom' ) {
diff --git a/PVE/Storage/FreeNASPlugin.pm b/PVE/Storage/FreeNASPlugin.pm
new file mode 100644
index 000..771061e
--- /dev/null
+++ b/PVE/Storage/FreeNASPlugin.pm
@@ -0,0 +1,1289 @@
+package PVE::Storage::FreeNASPlugin;
+
+use strict;
+use warnings;
+use IO::File;
+use POSIX;
+use PVE::Tools qw(run_command);
+use PVE::Storage::Plugin;
+use PVE::RPCEnvironment;
+use PVE::QemuServer;
+use PVE::LXC;
+use LWP::UserAgent;
+use HTTP::Status;
+use JSON;
+use Data::Dumper;
+
+use base qw(PVE::Storage::Plugin);
+
+my $api = '/api/v1.0';
+my $timeout = 60; # seconds
+my $max_luns = 20; # maximum supported luns per target group in freenas is 25
+ # but reserve 5 for temporary LUNs (snapshots with RAM and
+ # snapshot backup)
+my $active_snaps = 4;
+my $limit = 1; # limit for get requests
+my $version;
+my $fullversion;
+my $target_prefix = 'iqn.2005-10.org.freenas.ctl';
+
+sub freenas_request {
+my ($scfg, $request, $section, $data, $valid_code) = @_;
+my $ua = LWP::UserAgent->new;
+$ua->agent("ProxmoxUA/0.1");
+$ua->ssl_opts( verify_hostname => 0 );
+$ua->timeout($timeout);
+push @{ $ua->requests_redirectable }, 'POST';
+push @{ $ua->requests_redirectable }, 'PUT';
+push @{ $ua->requests_redirectable }, 'DELETE';
+my $req;
+
+my $url = "https://$scfg->{portal}$api/$section";
+
+if ($request eq 'GET') {
+$req = HTTP::Request->new(GET => $url);
+} elsif ($request eq 'POST') {
+$req = HTTP::Request->new(POST => $url);
+$req->content($data);
+} elsif ($request eq 'PUT') {
+$req = HTTP::Request->new(PUT => $url);
+$req->content($data);
+} elsif ($request eq 'DELETE') {
+$req = HTTP::Request->new(DELETE => $url);
+} else {
+die "$request: Unknown request";
+}
+
+$req->content_type('application/json');
+$req->authorization_basic($scfg->{username}, $scfg->{password});
+my $res = $ua->request($req);
+
+ #print Dumper($res);
+return $res->code unless $res->is_success;
+
+return $res->content;
+}
+
+sub freenas_get_version {
+my ($scfg) = @_;
+
+ my $response = freenas_request($scfg, 'GET', "system/version");
+ die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+ my $info = decode_json($response);
+ $fullversion = $info->{fullversion};
+ if ($fullversion =~ /^\w+-(\d+)\.(\d*)\.(\d*)/) {
+ my $minor = $2;
+ my $micro = $3;
+
+ if ($minor) {
+ $minor = "0$minor" unless $minor > 9;
+ } else {
+ $minor = '00';
+ }
+
+ if ($micro) {
+ $micro = "0$micro" unless $micro > 9;
+ } else {
+ $micro = '00';
+ }
+
+ $version = "$1$minor$micro";
+ } else {
+ $version = '90200';
+ }
+}
+
+sub freenas_list_zvol {
+my ($scfg) = @_;
+
+ freenas_get_version($scfg) unless $version;
+
+ my $response = freenas_request($scfg, 'GET',
"storage/volume/$scfg->{pool}/zvols?limit=$limit");
+ die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+my $zvols = decode_json($response);
+ $response = freenas_request($scfg, 'GET',
"storage/snapshot?limit=$limit");
+ die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+my $snapshots = decode_json($response);
+
+my $list
[pve-devel] [PATCH pve-storage 4/5] Add support for creating LXC snapshot backup
Signed-off-by: Michael Rasmussen
---
PVE/Storage/FreeNASPlugin.pm | 129 +--
1 file changed, 74 insertions(+), 55 deletions(-)
diff --git a/PVE/Storage/FreeNASPlugin.pm b/PVE/Storage/FreeNASPlugin.pm
index 0f2a56d..fee3d27 100644
--- a/PVE/Storage/FreeNASPlugin.pm
+++ b/PVE/Storage/FreeNASPlugin.pm
@@ -427,40 +427,38 @@ sub freenas_find_free_diskname {
}
sub freenas_get_lun_number {
-my ($scfg, $volname, $snap) = @_;
+my ($scfg, $volname) = @_;
my $lunid = undef;
-if ($volname =~ /^(vm|base)-\d+-disk-(\d+)$/ && ! defined $snap) {
+if ($volname =~ /^(vm|base)-\d+-disk-(\d+)$/) {
$lunid = $2 - 1;
} elsif ($volname =~ /^vm-(\d+)-state/) {
# Find id for temporary LUN
- if ($snap) {
- # TODO
- # Required to be able to exposed read-only LUNs for snapshot
backup CT
- } else {
- my $target = freenas_get_target($scfg, $1);
- my $id = $max_luns;
- my $response = freenas_request($scfg, 'GET',
"services/iscsi/targettoextent?limit=$limit");
- die HTTP::Status::status_message($response) if
$response =~ /^\d+$/;
- my $t2extents = decode_json($response);
-
- foreach my $t2extent (@$t2extents) {
- next unless $t2extent->{iscsi_target} ==
$target &&
-
$t2extent->{iscsi_lunid} + 1 > $max_luns &&
-
$t2extent->{iscsi_lunid} < $max_luns + $active_snaps;
- my $eid = freenas_get_extent($scfg, $volname);
- if ($eid) {
- $response = freenas_request($scfg,
'GET', "services/iscsi/extent/$eid");
- die
HTTP::Status::status_message($response) if $response =~ /^\d+$/;
- my $extent = decode_json($response);
- # Request to get lunid for an existing
lun
- last if $t2extent->{iscsi_extent} eq
$eid;
- }
- $id++;
+ my $target = freenas_get_target($scfg, $1);
+ my $id = $max_luns;
+ my $response = freenas_request($scfg, 'GET',
"services/iscsi/targettoextent?limit=$limit");
+ die HTTP::Status::status_message($response) if $response =~
/^\d+$/;
+ my $t2extents = decode_json($response);
+
+ foreach my $t2extent (@$t2extents) {
+ next unless $t2extent->{iscsi_target} == $target &&
+ $t2extent->{iscsi_lunid} + 1 >
$max_luns &&
+ $t2extent->{iscsi_lunid} <
$max_luns + $active_snaps;
+ my $eid = freenas_get_extent($scfg, $volname);
+ if ($eid) {
+ $response = freenas_request($scfg, 'GET',
"services/iscsi/extent/$eid");
+ die HTTP::Status::status_message($response) if
$response =~ /^\d+$/;
+ my $extent = decode_json($response);
+ # Request to get lunid for an existing lun
+ last if $t2extent->{iscsi_extent} eq $eid;
}
- die "Max snapshots (4) is reached" unless ($id -
$max_luns) < $active_snaps;
- $lunid = $id;
- }
+ $id++;
+ }
+ die "Max snapshots (4) is reached" unless ($id - $max_luns) <
$active_snaps;
+ $lunid = $id;
+} elsif ($volname =~ /^(vm|base)-\d+-disk-\d+\@vzdump$/) {
+# Required to be able to exposed read-only LUNs for snapshot backup CT
+$lunid = $max_luns + $active_snaps;
}
return $lunid;
@@ -891,12 +889,13 @@ sub list_images {
sub path {
my ($class, $scfg, $volname, $storeid, $snapname) = @_;
-
-die "direct access to snapshots not implemented"
- if defined($snapname);
+#die "direct access to snapshots not implemented"
+ #if defined($snapname);
my ($vtype, $vname, $vmid) = $class->parse_volname($volname);
+ $vname = "$vname\@$snapname" if $snapname;
+
my $luns = get_active_luns($class, $storeid, $scfg, $vname);
my $path = disk_by_path($scfg, $vname);
@@ -1116,24 +1115,54 @@ sub volume_resize {
sub volume_snapshot {
my ($class, $scfg, $storeid, $volname, $snap) = @_;
-
-my
[pve-devel] [PATCH pve-storage 3/5] Fix missing $vmid
Signed-off-by: Michael Rasmussen
---
PVE/Storage/FreeNASPlugin.pm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/PVE/Storage/FreeNASPlugin.pm b/PVE/Storage/FreeNASPlugin.pm
index c438519..0f2a56d 100644
--- a/PVE/Storage/FreeNASPlugin.pm
+++ b/PVE/Storage/FreeNASPlugin.pm
@@ -1072,6 +1072,8 @@ sub free_image {
sub volume_resize {
my ($class, $scfg, $storeid, $volname, $size, $running) = @_;
+my ($vtype, $name, $vmid) = $class->parse_volname($volname);
+
my $run = PVE::QemuServer::check_running($vmid);
if (!$run) {
$run = PVE::LXC::check_running($vmid);
@@ -1080,8 +1082,6 @@ sub volume_resize {
die 'mode failure - unable to resize disk(s) on a running system due to
FreeNAS bug.
See bug report: https://bugs.freenas.org/issues/24432"; target="_blank">#24432'
if $run;
-my ($vtype, $name, $vmid) = $class->parse_volname($volname);
-
my $data = {
volsize => $size,
};
--
2.11.0
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH pve-storage 4/4] Add support for creating LXC snapshot backup
On Thu, 15 Jun 2017 23:03:56 +0200 Andreas Steinel wrote: > > Compared to what? Of course not local ZFS, because everything applies > also to local ZFS. TRIM support is the only point I understand in > comparison to "ordinary" iSCSI backed on "fat" files. > Compared to ZFS over NFS. NFS put a lot of work to the CPU for a reduced performance compared to ZFS over iSCSI and I also forgot to mention that ZFS over iSCSI also supports MPIO which is also lacking in ZFS over NFS. > I thought, that ZFS-over-iSCSI is superior to iSCSI because of the ZFS > features, therefore ZFS-over-NFS has the same ZFS features, but as a > file storage for container. No ordinary NFS has support for snapshots > or quota directed by PVE, so this would be a real benefit. Also, you > have a clusterwide-shared ZFS for KVM and LXC. In my opinion: it's the > dream, isn't it? My new FreeNAS plugin also supports containers as you can read from the release notes. This is why I see no usecase for ZFS over NFS. Why use ZFS over NFS when the FreeNAS plugin supports all the features supported by Proxmox? VM CT create YES YES delete YES YES resize (YES) (YES) Note 1 snapshot offline YES YES live YES "YES" (state is not saved) backup snapshot YES YES standby YES YES offline YES YES clone full YES YES (Unavailable in GUI) linked YES YES (Unavailable in GUI) -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: He who knows, does not speak. He who speaks, does not know. -- Lao Tsu pgpOLyKDkRnHa.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] Possible bug in pve-manager and/or qemu-kvm
Hi all, I have noticed a possible bug in pve-manager and/or qemu-kvm. Scenario. You have a VM running and now you want to should it down. To do this you can choose either to use the shutdown function from the gui or the console or use the client native shutdown method. This all works as expected except for one, IMHO, important situation: Using gui shutdown or console shutdown will generate this in syslog Jun 15 21:37:19 pve pvedaemon[4842]: end task UPID:pve:1FEE:0008B C24:5942E1EB:qmstart:110:root@pam: OK Jun 15 21:37:43 pve pvedaemon[5746]: starting task UPID:pve:203E: 0008C709:5942E207:qmshutdown:110:root@pam: Jun 15 21:37:43 pve pvedaemon[8254]: shutdown VM 110: UPID:pve:203E:0008C709 :5942E207:qmshutdown:110:root@pam: Jun 15 21:37:45 pve kernel: [ 5754.731169] sdb: sdb1 sdb2 < sdb5 > Jun 15 21:37:45 pve kernel: [ 5755.119112] vmbr0: port 2(tap110i0) entered disabled state Jun 15 21:37:45 pve pvedaemon[5746]: end task UPID:pve:1FFB:0008BCA6:5942E1ED:vncproxy:110:root@pam: OK Jun 15 21:37:45 pve kernel: [ 5755.333909] sd 33:0:0:0: [sdb] Synchronizing SCSI cache Jun 15 21:37:45 pve pvedaemon[5746]: end task UPID:pve:203E:0008C709:5942E207:qmshutdown:110:root@pam: OK Jun 15 21:37:45 pve iscsid: Connection1:0 to [target: iqn.2005-10.org.freenas.ctl:vm-110, portal: 10.0.1.32,3260] through [iface: default] is shutdown. As above shows the deactivate_volume is called for every disk belonging to the VM. Using native client to shutdown will generate this is syslog Jun 15 21:37:46 pve qm[8308]: VM 110 qmp command failed - VM 110 not running As above shows the deactivate_volume is not called for any disk belonging to the VM so every disk is left activated. I should mention that for CT's there is no differens whether gui shutdown or native shutdown is used. I wonder if the process login this 'pve qm[8308]: VM 110 qmp command failed - VM 110 not running' should not call deactivate_volume for every disk belonging to the VM? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Raising pet electric eels is gaining a lot of current popularity. pgpWTAN2jygAr.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH pve-storage 4/4] Add support for creating LXC snapshot backup
On Thu, 15 Jun 2017 18:54:29 +0200 Andreas Steinel wrote: > > And I can't see how ZFS-over-NFS would be any different than > ZFS-over-iSCSI? Am I see this so wrong? What are the usecases for > ZFS-over-iSCSI exactly? Three obvious advantages: 1) Better performance 2) Lower CPU usage 3) Support for TRIM/Discard via SCSI UNMAP -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Make sure every module hides something. - The Elements of Programming Style (Kernighan & Plaugher) pgpuLAMUhTdg0.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH pve-storage 4/4] Add support for creating LXC snapshot backup
I still dont see a usecase for this? On June 15, 2017 11:09:21 AM GMT+02:00, Andreas Steinel wrote: >On Thu, Jun 15, 2017 at 10:50 AM, Michael Rasmussen >wrote: >> Hi Andreas, >> Even better, I can sent you the deb packages? > >That would also work, I then can create a diff to see what you changed >and try to incorporate my idea with ZFS-over-NFS. >___ >pve-devel mailing list >pve-devel@pve.proxmox.com >https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH pve-storage 4/4] Add support for creating LXC snapshot backup
Hi Andreas,
Even better, I can sent you the deb packages?
On June 15, 2017 10:20:32 AM GMT+02:00, Andreas Steinel
wrote:
>Hi mir,
>
>have you a public git with all the changes? I'd like to try but I
>don't want to extract all the patches from this list, try to figure
>out which ones are already taken care of and manually apply them.
>
>Best,
>LnxBil
>
>On Wed, Jun 14, 2017 at 9:55 PM, Michael Rasmussen
>wrote:
>> Signed-off-by: Michael Rasmussen
>> ---
>> PVE/Storage/FreeNASPlugin.pm | 129
>+--
>> 1 file changed, 74 insertions(+), 55 deletions(-)
>>
>> diff --git a/PVE/Storage/FreeNASPlugin.pm
>b/PVE/Storage/FreeNASPlugin.pm
>> index 0f2a56d..fee3d27 100644
>> --- a/PVE/Storage/FreeNASPlugin.pm
>> +++ b/PVE/Storage/FreeNASPlugin.pm
>> @@ -427,40 +427,38 @@ sub freenas_find_free_diskname {
>> }
>>
>> sub freenas_get_lun_number {
>> -my ($scfg, $volname, $snap) = @_;
>> +my ($scfg, $volname) = @_;
>> my $lunid = undef;
>>
>> -if ($volname =~ /^(vm|base)-\d+-disk-(\d+)$/ && ! defined $snap)
>{
>> +if ($volname =~ /^(vm|base)-\d+-disk-(\d+)$/) {
>> $lunid = $2 - 1;
>> } elsif ($volname =~ /^vm-(\d+)-state/) {
>> # Find id for temporary LUN
>> - if ($snap) {
>> - # TODO
>> - # Required to be able to exposed read-only LUNs for
>snapshot backup CT
>> - } else {
>> - my $target = freenas_get_target($scfg, $1);
>> - my $id = $max_luns;
>> - my $response = freenas_request($scfg, 'GET',
>"services/iscsi/targettoextent?limit=$limit");
>> - die HTTP::Status::status_message($response)
>if $response =~ /^\d+$/;
>> - my $t2extents = decode_json($response);
>> -
>> - foreach my $t2extent (@$t2extents) {
>> - next unless $t2extent->{iscsi_target}
>== $target &&
>> -
>$t2extent->{iscsi_lunid} + 1 > $max_luns &&
>> -
>$t2extent->{iscsi_lunid} < $max_luns + $active_snaps;
>> - my $eid = freenas_get_extent($scfg,
>$volname);
>> - if ($eid) {
>> - $response =
>freenas_request($scfg, 'GET', "services/iscsi/extent/$eid");
>> - die
>HTTP::Status::status_message($response) if $response =~ /^\d+$/;
>> - my $extent =
>decode_json($response);
>> - # Request to get lunid for an
>existing lun
>> - last if
>$t2extent->{iscsi_extent} eq $eid;
>> - }
>> - $id++;
>> + my $target = freenas_get_target($scfg, $1);
>> + my $id = $max_luns;
>> + my $response = freenas_request($scfg, 'GET',
>"services/iscsi/targettoextent?limit=$limit");
>> + die HTTP::Status::status_message($response) if
>$response =~ /^\d+$/;
>> + my $t2extents = decode_json($response);
>> +
>> + foreach my $t2extent (@$t2extents) {
>> + next unless $t2extent->{iscsi_target} ==
>$target &&
>> +
>$t2extent->{iscsi_lunid} + 1 > $max_luns &&
>> +
>$t2extent->{iscsi_lunid} < $max_luns + $active_snaps;
>> + my $eid = freenas_get_extent($scfg,
>$volname);
>> + if ($eid) {
>> + $response = freenas_request($scfg,
>'GET', "services/iscsi/extent/$eid");
>> + die
>HTTP::Status::status_message($response) if $response =~ /^\d+$/;
>> + my $extent = decode_json($response);
>> + # Request to get lunid for an
>existing lun
>> + last if $t2extent->{iscsi_extent} eq
>$eid;
>> }
>> - die "Max snapshots (4) is reached" unless
>($id - $max_luns) < $active_snaps;
>> - $lunid = $id;
>> - }
[pve-devel] [PATCH pve-storage 0/4] v4 Support for snapshot LXC backup
V4 Adds support for creating snapshot backups of LXC containers. This more or less makes the plugin feature complete. Only outstanding feature depends on a bug fix in the FreeNAS API which is first scheduled for next stable release: https://bugs.freenas.org/issues/23881 (Targeted for 11.1) This patch series adds a storage plugin for FreeNAS using the FreeNAS API. The plugin supports both Qemu and LXC VM. Features supported, see below: VM CT create YES YES delete YES YES resize (YES) (YES) Note 1 snapshot offline YES YES live YES "YES" (state is not saved) backup snapshot YES YES standby YES YES offline YES YES clone full YES YES (Unavailable in GUI) linked YES YES (Unavailable in GUI) Note 1: Due to a bug in the FreeNAS API live resizing has been disabled in the plugin. See https://bugs.freenas.org/issues/24432 Since the plugin attaches disks through the local scsi subsystem via openiscsi and therefore not uses libiscsi there should be support for MPIO and authentication. MPIO is not tested since I don't have the required hardware to do so. Authentication is not enabled but should be easy to do if the requirements exists. Michael Rasmussen (4): First beta of FreeNAS storage plugin. Missing snapshot backup for LXC Proper check if VM/CT is running Fix missing $vmid Add support for creating LXC snapshot backup PVE/Storage.pm |2 + PVE/Storage/FreeNASPlugin.pm | 1313 ++ PVE/Storage/Makefile |2 +- PVE/Storage/Plugin.pm|2 +- 4 files changed, 1317 insertions(+), 2 deletions(-) create mode 100644 PVE/Storage/FreeNASPlugin.pm -- 2.1.4 This mail was virus scanned and spam checked before delivery. This mail is also DKIM signed. See header dkim-signature. ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-storage 3/4] Fix missing $vmid
Signed-off-by: Michael Rasmussen
---
PVE/Storage/FreeNASPlugin.pm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/PVE/Storage/FreeNASPlugin.pm b/PVE/Storage/FreeNASPlugin.pm
index c438519..0f2a56d 100644
--- a/PVE/Storage/FreeNASPlugin.pm
+++ b/PVE/Storage/FreeNASPlugin.pm
@@ -1072,6 +1072,8 @@ sub free_image {
sub volume_resize {
my ($class, $scfg, $storeid, $volname, $size, $running) = @_;
+my ($vtype, $name, $vmid) = $class->parse_volname($volname);
+
my $run = PVE::QemuServer::check_running($vmid);
if (!$run) {
$run = PVE::LXC::check_running($vmid);
@@ -1080,8 +1082,6 @@ sub volume_resize {
die 'mode failure - unable to resize disk(s) on a running system due to
FreeNAS bug.
See bug report: https://bugs.freenas.org/issues/24432"; target="_blank">#24432'
if $run;
-my ($vtype, $name, $vmid) = $class->parse_volname($volname);
-
my $data = {
volsize => $size,
};
--
2.1.4
This mail was virus scanned and spam checked before delivery.
This mail is also DKIM signed. See header dkim-signature.
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-storage 4/4] Add support for creating LXC snapshot backup
Signed-off-by: Michael Rasmussen
---
PVE/Storage/FreeNASPlugin.pm | 129 +--
1 file changed, 74 insertions(+), 55 deletions(-)
diff --git a/PVE/Storage/FreeNASPlugin.pm b/PVE/Storage/FreeNASPlugin.pm
index 0f2a56d..fee3d27 100644
--- a/PVE/Storage/FreeNASPlugin.pm
+++ b/PVE/Storage/FreeNASPlugin.pm
@@ -427,40 +427,38 @@ sub freenas_find_free_diskname {
}
sub freenas_get_lun_number {
-my ($scfg, $volname, $snap) = @_;
+my ($scfg, $volname) = @_;
my $lunid = undef;
-if ($volname =~ /^(vm|base)-\d+-disk-(\d+)$/ && ! defined $snap) {
+if ($volname =~ /^(vm|base)-\d+-disk-(\d+)$/) {
$lunid = $2 - 1;
} elsif ($volname =~ /^vm-(\d+)-state/) {
# Find id for temporary LUN
- if ($snap) {
- # TODO
- # Required to be able to exposed read-only LUNs for snapshot
backup CT
- } else {
- my $target = freenas_get_target($scfg, $1);
- my $id = $max_luns;
- my $response = freenas_request($scfg, 'GET',
"services/iscsi/targettoextent?limit=$limit");
- die HTTP::Status::status_message($response) if
$response =~ /^\d+$/;
- my $t2extents = decode_json($response);
-
- foreach my $t2extent (@$t2extents) {
- next unless $t2extent->{iscsi_target} ==
$target &&
-
$t2extent->{iscsi_lunid} + 1 > $max_luns &&
-
$t2extent->{iscsi_lunid} < $max_luns + $active_snaps;
- my $eid = freenas_get_extent($scfg, $volname);
- if ($eid) {
- $response = freenas_request($scfg,
'GET', "services/iscsi/extent/$eid");
- die
HTTP::Status::status_message($response) if $response =~ /^\d+$/;
- my $extent = decode_json($response);
- # Request to get lunid for an existing
lun
- last if $t2extent->{iscsi_extent} eq
$eid;
- }
- $id++;
+ my $target = freenas_get_target($scfg, $1);
+ my $id = $max_luns;
+ my $response = freenas_request($scfg, 'GET',
"services/iscsi/targettoextent?limit=$limit");
+ die HTTP::Status::status_message($response) if $response =~
/^\d+$/;
+ my $t2extents = decode_json($response);
+
+ foreach my $t2extent (@$t2extents) {
+ next unless $t2extent->{iscsi_target} == $target &&
+ $t2extent->{iscsi_lunid} + 1 >
$max_luns &&
+ $t2extent->{iscsi_lunid} <
$max_luns + $active_snaps;
+ my $eid = freenas_get_extent($scfg, $volname);
+ if ($eid) {
+ $response = freenas_request($scfg, 'GET',
"services/iscsi/extent/$eid");
+ die HTTP::Status::status_message($response) if
$response =~ /^\d+$/;
+ my $extent = decode_json($response);
+ # Request to get lunid for an existing lun
+ last if $t2extent->{iscsi_extent} eq $eid;
}
- die "Max snapshots (4) is reached" unless ($id -
$max_luns) < $active_snaps;
- $lunid = $id;
- }
+ $id++;
+ }
+ die "Max snapshots (4) is reached" unless ($id - $max_luns) <
$active_snaps;
+ $lunid = $id;
+} elsif ($volname =~ /^(vm|base)-\d+-disk-\d+\@vzdump$/) {
+# Required to be able to exposed read-only LUNs for snapshot backup CT
+$lunid = $max_luns + $active_snaps;
}
return $lunid;
@@ -891,12 +889,13 @@ sub list_images {
sub path {
my ($class, $scfg, $volname, $storeid, $snapname) = @_;
-
-die "direct access to snapshots not implemented"
- if defined($snapname);
+#die "direct access to snapshots not implemented"
+ #if defined($snapname);
my ($vtype, $vname, $vmid) = $class->parse_volname($volname);
+ $vname = "$vname\@$snapname" if $snapname;
+
my $luns = get_active_luns($class, $storeid, $scfg, $vname);
my $path = disk_by_path($scfg, $vname);
@@ -1116,24 +1115,54 @@ sub volume_resize {
sub volume_snapshot {
my ($class, $scfg, $storeid, $volname, $snap) = @_;
-
-my
[pve-devel] [PATCH pve-storage 2/4] Proper check if VM/CT is running
Signed-off-by: Michael Rasmussen
---
PVE/Storage/FreeNASPlugin.pm | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/PVE/Storage/FreeNASPlugin.pm b/PVE/Storage/FreeNASPlugin.pm
index 771061e..c438519 100644
--- a/PVE/Storage/FreeNASPlugin.pm
+++ b/PVE/Storage/FreeNASPlugin.pm
@@ -1072,8 +1072,13 @@ sub free_image {
sub volume_resize {
my ($class, $scfg, $storeid, $volname, $size, $running) = @_;
+my $run = PVE::QemuServer::check_running($vmid);
+if (!$run) {
+$run = PVE::LXC::check_running($vmid);
+}
+
die 'mode failure - unable to resize disk(s) on a running system due to
FreeNAS bug.
-See bug report: https://bugs.freenas.org/issues/24432"; target="_blank">#24432'
if $running;
+See bug report: https://bugs.freenas.org/issues/24432"; target="_blank">#24432'
if $run;
my ($vtype, $name, $vmid) = $class->parse_volname($volname);
--
2.1.4
This mail was virus scanned and spam checked before delivery.
This mail is also DKIM signed. See header dkim-signature.
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-storage 1/4] First beta of FreeNAS storage plugin. Missing snapshot backup for LXC
Signed-off-by: Michael Rasmussen
---
PVE/Storage.pm |2 +
PVE/Storage/FreeNASPlugin.pm | 1289 ++
PVE/Storage/Makefile |2 +-
PVE/Storage/Plugin.pm|2 +-
4 files changed, 1293 insertions(+), 2 deletions(-)
create mode 100644 PVE/Storage/FreeNASPlugin.pm
diff --git a/PVE/Storage.pm b/PVE/Storage.pm
index ee2295a..f33249f 100755
--- a/PVE/Storage.pm
+++ b/PVE/Storage.pm
@@ -32,6 +32,7 @@ use PVE::Storage::GlusterfsPlugin;
use PVE::Storage::ZFSPoolPlugin;
use PVE::Storage::ZFSPlugin;
use PVE::Storage::DRBDPlugin;
+use PVE::Storage::FreeNASPlugin;
# Storage API version. Icrement it on changes in storage API interface.
use constant APIVER => 1;
@@ -49,6 +50,7 @@ PVE::Storage::GlusterfsPlugin->register();
PVE::Storage::ZFSPoolPlugin->register();
PVE::Storage::ZFSPlugin->register();
PVE::Storage::DRBDPlugin->register();
+PVE::Storage::FreeNASPlugin->register();
# load third-party plugins
if ( -d '/usr/share/perl5/PVE/Storage/Custom' ) {
diff --git a/PVE/Storage/FreeNASPlugin.pm b/PVE/Storage/FreeNASPlugin.pm
new file mode 100644
index 000..771061e
--- /dev/null
+++ b/PVE/Storage/FreeNASPlugin.pm
@@ -0,0 +1,1289 @@
+package PVE::Storage::FreeNASPlugin;
+
+use strict;
+use warnings;
+use IO::File;
+use POSIX;
+use PVE::Tools qw(run_command);
+use PVE::Storage::Plugin;
+use PVE::RPCEnvironment;
+use PVE::QemuServer;
+use PVE::LXC;
+use LWP::UserAgent;
+use HTTP::Status;
+use JSON;
+use Data::Dumper;
+
+use base qw(PVE::Storage::Plugin);
+
+my $api = '/api/v1.0';
+my $timeout = 60; # seconds
+my $max_luns = 20; # maximum supported luns per target group in freenas is 25
+ # but reserve 5 for temporary LUNs (snapshots with RAM and
+ # snapshot backup)
+my $active_snaps = 4;
+my $limit = 1; # limit for get requests
+my $version;
+my $fullversion;
+my $target_prefix = 'iqn.2005-10.org.freenas.ctl';
+
+sub freenas_request {
+my ($scfg, $request, $section, $data, $valid_code) = @_;
+my $ua = LWP::UserAgent->new;
+$ua->agent("ProxmoxUA/0.1");
+$ua->ssl_opts( verify_hostname => 0 );
+$ua->timeout($timeout);
+push @{ $ua->requests_redirectable }, 'POST';
+push @{ $ua->requests_redirectable }, 'PUT';
+push @{ $ua->requests_redirectable }, 'DELETE';
+my $req;
+
+my $url = "https://$scfg->{portal}$api/$section";
+
+if ($request eq 'GET') {
+$req = HTTP::Request->new(GET => $url);
+} elsif ($request eq 'POST') {
+$req = HTTP::Request->new(POST => $url);
+$req->content($data);
+} elsif ($request eq 'PUT') {
+$req = HTTP::Request->new(PUT => $url);
+$req->content($data);
+} elsif ($request eq 'DELETE') {
+$req = HTTP::Request->new(DELETE => $url);
+} else {
+die "$request: Unknown request";
+}
+
+$req->content_type('application/json');
+$req->authorization_basic($scfg->{username}, $scfg->{password});
+my $res = $ua->request($req);
+
+ #print Dumper($res);
+return $res->code unless $res->is_success;
+
+return $res->content;
+}
+
+sub freenas_get_version {
+my ($scfg) = @_;
+
+ my $response = freenas_request($scfg, 'GET', "system/version");
+ die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+ my $info = decode_json($response);
+ $fullversion = $info->{fullversion};
+ if ($fullversion =~ /^\w+-(\d+)\.(\d*)\.(\d*)/) {
+ my $minor = $2;
+ my $micro = $3;
+
+ if ($minor) {
+ $minor = "0$minor" unless $minor > 9;
+ } else {
+ $minor = '00';
+ }
+
+ if ($micro) {
+ $micro = "0$micro" unless $micro > 9;
+ } else {
+ $micro = '00';
+ }
+
+ $version = "$1$minor$micro";
+ } else {
+ $version = '90200';
+ }
+}
+
+sub freenas_list_zvol {
+my ($scfg) = @_;
+
+ freenas_get_version($scfg) unless $version;
+
+ my $response = freenas_request($scfg, 'GET',
"storage/volume/$scfg->{pool}/zvols?limit=$limit");
+ die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+my $zvols = decode_json($response);
+ $response = freenas_request($scfg, 'GET',
"storage/snapshot?limit=$limit");
+ die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+my $snapshots = decode_json($response);
+
+my $list
[pve-devel] [PATCH pve-storage 2/3] Proper check if VM/CT is running
Signed-off-by: Michael Rasmussen
---
PVE/Storage/FreeNASPlugin.pm | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/PVE/Storage/FreeNASPlugin.pm b/PVE/Storage/FreeNASPlugin.pm
index 771061e..c438519 100644
--- a/PVE/Storage/FreeNASPlugin.pm
+++ b/PVE/Storage/FreeNASPlugin.pm
@@ -1072,8 +1072,13 @@ sub free_image {
sub volume_resize {
my ($class, $scfg, $storeid, $volname, $size, $running) = @_;
+my $run = PVE::QemuServer::check_running($vmid);
+if (!$run) {
+$run = PVE::LXC::check_running($vmid);
+}
+
die 'mode failure - unable to resize disk(s) on a running system due to
FreeNAS bug.
-See bug report: https://bugs.freenas.org/issues/24432"; target="_blank">#24432'
if $running;
+See bug report: https://bugs.freenas.org/issues/24432"; target="_blank">#24432'
if $run;
my ($vtype, $name, $vmid) = $class->parse_volname($volname);
--
2.1.4
This mail was virus scanned and spam checked before delivery.
This mail is also DKIM signed. See header dkim-signature.
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-storage 3/3] Fix missing $vmid
Signed-off-by: Michael Rasmussen
---
PVE/Storage/FreeNASPlugin.pm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/PVE/Storage/FreeNASPlugin.pm b/PVE/Storage/FreeNASPlugin.pm
index c438519..0f2a56d 100644
--- a/PVE/Storage/FreeNASPlugin.pm
+++ b/PVE/Storage/FreeNASPlugin.pm
@@ -1072,6 +1072,8 @@ sub free_image {
sub volume_resize {
my ($class, $scfg, $storeid, $volname, $size, $running) = @_;
+my ($vtype, $name, $vmid) = $class->parse_volname($volname);
+
my $run = PVE::QemuServer::check_running($vmid);
if (!$run) {
$run = PVE::LXC::check_running($vmid);
@@ -1080,8 +1082,6 @@ sub volume_resize {
die 'mode failure - unable to resize disk(s) on a running system due to
FreeNAS bug.
See bug report: https://bugs.freenas.org/issues/24432"; target="_blank">#24432'
if $run;
-my ($vtype, $name, $vmid) = $class->parse_volname($volname);
-
my $data = {
volsize => $size,
};
--
2.1.4
This mail was virus scanned and spam checked before delivery.
This mail is also DKIM signed. See header dkim-signature.
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-storage 1/3] First beta of FreeNAS storage plugin. Missing snapshot backup for LXC
Signed-off-by: Michael Rasmussen
---
PVE/Storage.pm |2 +
PVE/Storage/FreeNASPlugin.pm | 1289 ++
PVE/Storage/Makefile |2 +-
PVE/Storage/Plugin.pm|2 +-
4 files changed, 1293 insertions(+), 2 deletions(-)
create mode 100644 PVE/Storage/FreeNASPlugin.pm
diff --git a/PVE/Storage.pm b/PVE/Storage.pm
index ee2295a..f33249f 100755
--- a/PVE/Storage.pm
+++ b/PVE/Storage.pm
@@ -32,6 +32,7 @@ use PVE::Storage::GlusterfsPlugin;
use PVE::Storage::ZFSPoolPlugin;
use PVE::Storage::ZFSPlugin;
use PVE::Storage::DRBDPlugin;
+use PVE::Storage::FreeNASPlugin;
# Storage API version. Icrement it on changes in storage API interface.
use constant APIVER => 1;
@@ -49,6 +50,7 @@ PVE::Storage::GlusterfsPlugin->register();
PVE::Storage::ZFSPoolPlugin->register();
PVE::Storage::ZFSPlugin->register();
PVE::Storage::DRBDPlugin->register();
+PVE::Storage::FreeNASPlugin->register();
# load third-party plugins
if ( -d '/usr/share/perl5/PVE/Storage/Custom' ) {
diff --git a/PVE/Storage/FreeNASPlugin.pm b/PVE/Storage/FreeNASPlugin.pm
new file mode 100644
index 000..771061e
--- /dev/null
+++ b/PVE/Storage/FreeNASPlugin.pm
@@ -0,0 +1,1289 @@
+package PVE::Storage::FreeNASPlugin;
+
+use strict;
+use warnings;
+use IO::File;
+use POSIX;
+use PVE::Tools qw(run_command);
+use PVE::Storage::Plugin;
+use PVE::RPCEnvironment;
+use PVE::QemuServer;
+use PVE::LXC;
+use LWP::UserAgent;
+use HTTP::Status;
+use JSON;
+use Data::Dumper;
+
+use base qw(PVE::Storage::Plugin);
+
+my $api = '/api/v1.0';
+my $timeout = 60; # seconds
+my $max_luns = 20; # maximum supported luns per target group in freenas is 25
+ # but reserve 5 for temporary LUNs (snapshots with RAM and
+ # snapshot backup)
+my $active_snaps = 4;
+my $limit = 1; # limit for get requests
+my $version;
+my $fullversion;
+my $target_prefix = 'iqn.2005-10.org.freenas.ctl';
+
+sub freenas_request {
+my ($scfg, $request, $section, $data, $valid_code) = @_;
+my $ua = LWP::UserAgent->new;
+$ua->agent("ProxmoxUA/0.1");
+$ua->ssl_opts( verify_hostname => 0 );
+$ua->timeout($timeout);
+push @{ $ua->requests_redirectable }, 'POST';
+push @{ $ua->requests_redirectable }, 'PUT';
+push @{ $ua->requests_redirectable }, 'DELETE';
+my $req;
+
+my $url = "https://$scfg->{portal}$api/$section";
+
+if ($request eq 'GET') {
+$req = HTTP::Request->new(GET => $url);
+} elsif ($request eq 'POST') {
+$req = HTTP::Request->new(POST => $url);
+$req->content($data);
+} elsif ($request eq 'PUT') {
+$req = HTTP::Request->new(PUT => $url);
+$req->content($data);
+} elsif ($request eq 'DELETE') {
+$req = HTTP::Request->new(DELETE => $url);
+} else {
+die "$request: Unknown request";
+}
+
+$req->content_type('application/json');
+$req->authorization_basic($scfg->{username}, $scfg->{password});
+my $res = $ua->request($req);
+
+ #print Dumper($res);
+return $res->code unless $res->is_success;
+
+return $res->content;
+}
+
+sub freenas_get_version {
+my ($scfg) = @_;
+
+ my $response = freenas_request($scfg, 'GET', "system/version");
+ die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+ my $info = decode_json($response);
+ $fullversion = $info->{fullversion};
+ if ($fullversion =~ /^\w+-(\d+)\.(\d*)\.(\d*)/) {
+ my $minor = $2;
+ my $micro = $3;
+
+ if ($minor) {
+ $minor = "0$minor" unless $minor > 9;
+ } else {
+ $minor = '00';
+ }
+
+ if ($micro) {
+ $micro = "0$micro" unless $micro > 9;
+ } else {
+ $micro = '00';
+ }
+
+ $version = "$1$minor$micro";
+ } else {
+ $version = '90200';
+ }
+}
+
+sub freenas_list_zvol {
+my ($scfg) = @_;
+
+ freenas_get_version($scfg) unless $version;
+
+ my $response = freenas_request($scfg, 'GET',
"storage/volume/$scfg->{pool}/zvols?limit=$limit");
+ die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+my $zvols = decode_json($response);
+ $response = freenas_request($scfg, 'GET',
"storage/snapshot?limit=$limit");
+ die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+my $snapshots = decode_json($response);
+
+my $list
[pve-devel] [PATCH pve-storage 0/3] v3 small fix
This patch series adds a storage plugin for FreeNAS using the FreeNAS API. The plugin supports both Qemu and LXC VM. Features supported, see below: VM CT create YES YES delete YES YES resize (YES) (YES) Note 1 snapshot offline YES YES live YES "YES" (state is not saved) backup snapshot YES Waiting standby YES YES offline YES YES clone full YES YES (Unavailable in GUI) linked YES YES (Unavailable in GUI) Note 1: Due to a bug in the FreeNAS API live resizing has been disabled in the plugin. See https://bugs.freenas.org/issues/24432 Since the plugin attaches disks through the local scsi subsystem via openiscsi and therefore not uses libiscsi there should be support for MPIO and authentication. MPIO is not tested since I don't have the required hardware to do so. Authentication is not enabled but should be easy to do if the requirements exists. Michael Rasmussen (3): First beta of FreeNAS storage plugin. Missing snapshot backup for LXC Proper check if VM/CT is running Fix missing $vmid PVE/Storage.pm |2 + PVE/Storage/FreeNASPlugin.pm | 1294 ++ PVE/Storage/Makefile |2 +- PVE/Storage/Plugin.pm|2 +- 4 files changed, 1298 insertions(+), 2 deletions(-) create mode 100644 PVE/Storage/FreeNASPlugin.pm -- 2.1.4 This mail was virus scanned and spam checked before delivery. This mail is also DKIM signed. See header dkim-signature. ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-storage 2/2] Proper check if VM/CT is running
Signed-off-by: Michael Rasmussen
---
PVE/Storage/FreeNASPlugin.pm | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/PVE/Storage/FreeNASPlugin.pm b/PVE/Storage/FreeNASPlugin.pm
index 771061e..c438519 100644
--- a/PVE/Storage/FreeNASPlugin.pm
+++ b/PVE/Storage/FreeNASPlugin.pm
@@ -1072,8 +1072,13 @@ sub free_image {
sub volume_resize {
my ($class, $scfg, $storeid, $volname, $size, $running) = @_;
+my $run = PVE::QemuServer::check_running($vmid);
+if (!$run) {
+$run = PVE::LXC::check_running($vmid);
+}
+
die 'mode failure - unable to resize disk(s) on a running system due to
FreeNAS bug.
-See bug report: https://bugs.freenas.org/issues/24432"; target="_blank">#24432'
if $running;
+See bug report: https://bugs.freenas.org/issues/24432"; target="_blank">#24432'
if $run;
my ($vtype, $name, $vmid) = $class->parse_volname($volname);
--
2.1.4
This mail was virus scanned and spam checked before delivery.
This mail is also DKIM signed. See header dkim-signature.
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-storage 0/2] v2 FreeNAS storage plugin
v2: Proper check if VM/CT is running when resizing. This patch series adds a storage plugin for FreeNAS using the FreeNAS API. The plugin supports both Qemu and LXC VM. Features supported, see below: VM CT create YES YES delete YES YES resize (YES) (YES) Note 1 snapshot offline YES YES live YES "YES" (state is not saved) backup snapshot YES Waiting standby YES YES offline YES YES clone full YES YES (Unavailable in GUI) linked YES YES (Unavailable in GUI) Note 1: Due to a bug in the FreeNAS API live resizing has been disabled in the plugin. See https://bugs.freenas.org/issues/24432 Since the plugin attaches disks through the local scsi subsystem via openiscsi and therefore not uses libiscsi there should be support for MPIO and authentication. MPIO is not tested since I don't have the required hardware to do so. Authentication is not enabled but should be easy to do if the requirements exists. Michael Rasmussen (2): First beta of FreeNAS storage plugin. Missing snapshot backup for LXC Proper check if VM/CT is running PVE/Storage.pm |2 + PVE/Storage/FreeNASPlugin.pm | 1294 ++ PVE/Storage/Makefile |2 +- PVE/Storage/Plugin.pm|2 +- 4 files changed, 1298 insertions(+), 2 deletions(-) create mode 100644 PVE/Storage/FreeNASPlugin.pm -- 2.1.4 This mail was virus scanned and spam checked before delivery. This mail is also DKIM signed. See header dkim-signature. ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-storage 1/2] First beta of FreeNAS storage plugin. Missing snapshot backup for LXC
Signed-off-by: Michael Rasmussen
---
PVE/Storage.pm |2 +
PVE/Storage/FreeNASPlugin.pm | 1289 ++
PVE/Storage/Makefile |2 +-
PVE/Storage/Plugin.pm|2 +-
4 files changed, 1293 insertions(+), 2 deletions(-)
create mode 100644 PVE/Storage/FreeNASPlugin.pm
diff --git a/PVE/Storage.pm b/PVE/Storage.pm
index ee2295a..f33249f 100755
--- a/PVE/Storage.pm
+++ b/PVE/Storage.pm
@@ -32,6 +32,7 @@ use PVE::Storage::GlusterfsPlugin;
use PVE::Storage::ZFSPoolPlugin;
use PVE::Storage::ZFSPlugin;
use PVE::Storage::DRBDPlugin;
+use PVE::Storage::FreeNASPlugin;
# Storage API version. Icrement it on changes in storage API interface.
use constant APIVER => 1;
@@ -49,6 +50,7 @@ PVE::Storage::GlusterfsPlugin->register();
PVE::Storage::ZFSPoolPlugin->register();
PVE::Storage::ZFSPlugin->register();
PVE::Storage::DRBDPlugin->register();
+PVE::Storage::FreeNASPlugin->register();
# load third-party plugins
if ( -d '/usr/share/perl5/PVE/Storage/Custom' ) {
diff --git a/PVE/Storage/FreeNASPlugin.pm b/PVE/Storage/FreeNASPlugin.pm
new file mode 100644
index 000..771061e
--- /dev/null
+++ b/PVE/Storage/FreeNASPlugin.pm
@@ -0,0 +1,1289 @@
+package PVE::Storage::FreeNASPlugin;
+
+use strict;
+use warnings;
+use IO::File;
+use POSIX;
+use PVE::Tools qw(run_command);
+use PVE::Storage::Plugin;
+use PVE::RPCEnvironment;
+use PVE::QemuServer;
+use PVE::LXC;
+use LWP::UserAgent;
+use HTTP::Status;
+use JSON;
+use Data::Dumper;
+
+use base qw(PVE::Storage::Plugin);
+
+my $api = '/api/v1.0';
+my $timeout = 60; # seconds
+my $max_luns = 20; # maximum supported luns per target group in freenas is 25
+ # but reserve 5 for temporary LUNs (snapshots with RAM and
+ # snapshot backup)
+my $active_snaps = 4;
+my $limit = 1; # limit for get requests
+my $version;
+my $fullversion;
+my $target_prefix = 'iqn.2005-10.org.freenas.ctl';
+
+sub freenas_request {
+my ($scfg, $request, $section, $data, $valid_code) = @_;
+my $ua = LWP::UserAgent->new;
+$ua->agent("ProxmoxUA/0.1");
+$ua->ssl_opts( verify_hostname => 0 );
+$ua->timeout($timeout);
+push @{ $ua->requests_redirectable }, 'POST';
+push @{ $ua->requests_redirectable }, 'PUT';
+push @{ $ua->requests_redirectable }, 'DELETE';
+my $req;
+
+my $url = "https://$scfg->{portal}$api/$section";
+
+if ($request eq 'GET') {
+$req = HTTP::Request->new(GET => $url);
+} elsif ($request eq 'POST') {
+$req = HTTP::Request->new(POST => $url);
+$req->content($data);
+} elsif ($request eq 'PUT') {
+$req = HTTP::Request->new(PUT => $url);
+$req->content($data);
+} elsif ($request eq 'DELETE') {
+$req = HTTP::Request->new(DELETE => $url);
+} else {
+die "$request: Unknown request";
+}
+
+$req->content_type('application/json');
+$req->authorization_basic($scfg->{username}, $scfg->{password});
+my $res = $ua->request($req);
+
+ #print Dumper($res);
+return $res->code unless $res->is_success;
+
+return $res->content;
+}
+
+sub freenas_get_version {
+my ($scfg) = @_;
+
+ my $response = freenas_request($scfg, 'GET', "system/version");
+ die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+ my $info = decode_json($response);
+ $fullversion = $info->{fullversion};
+ if ($fullversion =~ /^\w+-(\d+)\.(\d*)\.(\d*)/) {
+ my $minor = $2;
+ my $micro = $3;
+
+ if ($minor) {
+ $minor = "0$minor" unless $minor > 9;
+ } else {
+ $minor = '00';
+ }
+
+ if ($micro) {
+ $micro = "0$micro" unless $micro > 9;
+ } else {
+ $micro = '00';
+ }
+
+ $version = "$1$minor$micro";
+ } else {
+ $version = '90200';
+ }
+}
+
+sub freenas_list_zvol {
+my ($scfg) = @_;
+
+ freenas_get_version($scfg) unless $version;
+
+ my $response = freenas_request($scfg, 'GET',
"storage/volume/$scfg->{pool}/zvols?limit=$limit");
+ die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+my $zvols = decode_json($response);
+ $response = freenas_request($scfg, 'GET',
"storage/snapshot?limit=$limit");
+ die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+my $snapshots = decode_json($response);
+
+my $list
Re: [pve-devel] pve-manager: bug or feature?
On Mon, 12 Jun 2017 13:20:13 +0200 Fabian Grünbichler wrote: > > you are wrong - vdisk_free is called, but if it fails it is treated as > non-fatal error (since we want to destroy the whole VM), and a simple > warning is logged. this is correct and intended, because once you delete > the first disk, the VM is broken anyhow, even if you can't free the > second(/third/fourth/..) disk... > You are right. vdisk_free is called but bails before the free_image method is called in the storage plugin. die "base volume '$volname' is still in use by linked clones\n" if &$volume_is_base_and_used__no_lock($scfg, $storeid, $plugin, $volname); So if vdisk_free bails why is the vmid.conf file removed? Bailing here means nothing has changed but because vmid.conf is removed the user cannot get to the vm anymore through the gui and are left to handle the situation from the command line. IMHO the user should not be force to go to the command line if there is no reason to do this, command line should always be last resort. > vdisk_free does a check if the volume is a base volume which is still > referenced by a linked clone, which is why you get the warning in the > log. it probably makes sense to add an additional check and iterate over > all disks and check if they are base and used iff the VM is a template. > I don't see the point. If a VM is converted to a template all disks, by convention, will be referenced by linked clones if linked clones exists. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Every country has the government it deserves. -- Joseph De Maistre pgp7mYr4auv0P.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-manager 0/2] v2 of freenas plugin
The GUI part for the FreeNAS storage plugin. v2: Fixes indentation and lint errors Added suggestions by Dominik. Michael Rasmussen (2): First beta of FreeNAS storage plugin Fix indentation, lint errors and add changes suggested by Dominik www/manager6/Makefile | 1 + www/manager6/Utils.js | 4 +- www/manager6/dc/StorageView.js | 13 ++- www/manager6/lxc/ResourceEdit.js| 16 ++-- www/manager6/qemu/Clone.js | 3 +- www/manager6/qemu/HDEdit.js | 3 +- www/manager6/qemu/HDEfi.js | 2 +- www/manager6/qemu/HDMove.js | 3 +- www/manager6/storage/FreeNASEdit.js | 159 9 files changed, 191 insertions(+), 13 deletions(-) create mode 100644 www/manager6/storage/FreeNASEdit.js -- 2.1.4 This mail was virus scanned and spam checked before delivery. This mail is also DKIM signed. See header dkim-signature. ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-manager 1/2] First beta of FreeNAS storage plugin
Signed-off-by: Michael Rasmussen
---
www/manager6/Makefile | 1 +
www/manager6/Utils.js | 4 +-
www/manager6/dc/StorageView.js | 15 +++-
www/manager6/lxc/ResourceEdit.js| 6 +-
www/manager6/qemu/Clone.js | 3 +-
www/manager6/qemu/HDEdit.js | 3 +-
www/manager6/qemu/HDEfi.js | 2 +-
www/manager6/qemu/HDMove.js | 3 +-
www/manager6/storage/FreeNASEdit.js | 152
9 files changed, 180 insertions(+), 9 deletions(-)
create mode 100644 www/manager6/storage/FreeNASEdit.js
diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index 0c266ef..2a4489d 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -186,6 +186,7 @@ JSSRC=
\
storage/SheepdogEdit.js \
storage/ZFSEdit.js \
storage/ZFSPoolEdit.js \
+ storage/FreeNASEdit.js \
ha/StatusView.js\
ha/Status.js\
ha/GroupSelector.js \
diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js
index d1ada77..c6227d0 100644
--- a/www/manager6/Utils.js
+++ b/www/manager6/Utils.js
@@ -799,7 +799,9 @@ Ext.define('PVE.Utils', { utilities: {
return 'iSCSIDirect';
} else if (value === 'drbd') {
return 'DRBD';
- } else {
+ } else if (value === 'freenas') {
+return 'FreeNAS';
+} else {
return PVE.Utils.unknownText;
}
},
diff --git a/www/manager6/dc/StorageView.js b/www/manager6/dc/StorageView.js
index 2940a98..c16d464 100644
--- a/www/manager6/dc/StorageView.js
+++ b/www/manager6/dc/StorageView.js
@@ -58,7 +58,9 @@ Ext.define('PVE.dc.StorageView', {
editor = 'PVE.storage.ZFSEdit';
} else if (type === 'zfspool') {
editor = 'PVE.storage.ZFSPoolEdit';
- } else {
+ } else if (type === 'freenas') {
+editor = 'PVE.storage.FreeNASEdit';
+} else {
return;
}
var win = Ext.create(editor, {
@@ -191,7 +193,16 @@ Ext.define('PVE.dc.StorageView', {
win.on('destroy', reload);
win.show();
}
-}
+},
+{
+text: PVE.Utils.format_storage_type('freenas'),
+iconCls: 'fa fa-fw fa-building',
+handler: function() {
+var win = Ext.create('PVE.storage.FreeNASEdit', {});
+win.on('destroy', reload);
+win.show();
+}
+}
/* the following type are conidered unstable
* so we do not enable that on the GUI for now
diff --git a/www/manager6/lxc/ResourceEdit.js b/www/manager6/lxc/ResourceEdit.js
index 61a5539..8bc3e76 100644
--- a/www/manager6/lxc/ResourceEdit.js
+++ b/www/manager6/lxc/ResourceEdit.js
@@ -387,7 +387,8 @@ Ext.define('PVE.lxc.MountPointInputPanel', {
}
var rec = f.store.getById(value);
if (rec.data.type === 'zfs' ||
- rec.data.type === 'zfspool') {
+ rec.data.type === 'zfspool' ||
+rec.data.type === 'freenas') {
me.quota.setDisabled(true);
me.quota.setValue(false);
} else {
@@ -407,7 +408,8 @@ Ext.define('PVE.lxc.MountPointInputPanel', {
rec.data.type === 'rbd' ||
rec.data.type === 'sheepdog' ||
rec.data.type === 'zfs' ||
- rec.data.type === 'zfspool') {
+ rec.data.type === 'zfspool' ||
+ rec.data.type === 'freenas') {
me.hdfilesel.setDisabled(true);
me.hdfilesel.setVisible(false);
me.hdsizesel.setDisabled(false);
diff --git a/www/manager6/qemu/Clone.js b/www/manager6/qemu/Clone.js
index c5dad4d..9de9777 100644
--- a/www/manager6/qemu/Clone.js
+++ b/www/manager6/qemu/Clone.js
@@ -116,7 +116,8 @@ Ext.define('PVE.window.Clone', {
rec.data.type === 'iscsi' ||
rec.data.type === 'sheepdog' ||
rec.data.type === 'zfs' ||
-
[pve-devel] [PATCH pve-manager 2/2] Fix indentation, lint errors and add changes suggested by Dominik
Signed-off-by: Michael Rasmussen
---
www/manager6/Utils.js | 2 +-
www/manager6/dc/StorageView.js | 20 ++--
www/manager6/lxc/ResourceEdit.js| 16
www/manager6/qemu/Clone.js | 2 +-
www/manager6/qemu/HDEdit.js | 2 +-
www/manager6/qemu/HDMove.js | 2 +-
www/manager6/storage/FreeNASEdit.js | 33 -
7 files changed, 42 insertions(+), 35 deletions(-)
diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js
index c6227d0..068c784 100644
--- a/www/manager6/Utils.js
+++ b/www/manager6/Utils.js
@@ -800,7 +800,7 @@ Ext.define('PVE.Utils', { utilities: {
} else if (value === 'drbd') {
return 'DRBD';
} else if (value === 'freenas') {
-return 'FreeNAS';
+return 'FreeNAS';
} else {
return PVE.Utils.unknownText;
}
diff --git a/www/manager6/dc/StorageView.js b/www/manager6/dc/StorageView.js
index c16d464..586d4a9 100644
--- a/www/manager6/dc/StorageView.js
+++ b/www/manager6/dc/StorageView.js
@@ -59,7 +59,7 @@ Ext.define('PVE.dc.StorageView', {
} else if (type === 'zfspool') {
editor = 'PVE.storage.ZFSPoolEdit';
} else if (type === 'freenas') {
-editor = 'PVE.storage.FreeNASEdit';
+editor = 'PVE.storage.FreeNASEdit';
} else {
return;
}
@@ -194,15 +194,15 @@ Ext.define('PVE.dc.StorageView', {
win.show();
}
},
-{
-text: PVE.Utils.format_storage_type('freenas'),
-iconCls: 'fa fa-fw fa-building',
-handler: function() {
-var win = Ext.create('PVE.storage.FreeNASEdit', {});
-win.on('destroy', reload);
-win.show();
-}
-}
+{
+text: PVE.Utils.format_storage_type('freenas'),
+iconCls: 'fa fa-fw fa-building',
+handler: function() {
+var win =
Ext.create('PVE.storage.FreeNASEdit', {});
+win.on('destroy', reload);
+win.show();
+}
+}
/* the following type are conidered unstable
* so we do not enable that on the GUI for now
diff --git a/www/manager6/lxc/ResourceEdit.js b/www/manager6/lxc/ResourceEdit.js
index 8bc3e76..4336d86 100644
--- a/www/manager6/lxc/ResourceEdit.js
+++ b/www/manager6/lxc/ResourceEdit.js
@@ -388,7 +388,7 @@ Ext.define('PVE.lxc.MountPointInputPanel', {
var rec = f.store.getById(value);
if (rec.data.type === 'zfs' ||
rec.data.type === 'zfspool' ||
-rec.data.type === 'freenas') {
+rec.data.type === 'freenas') {
me.quota.setDisabled(true);
me.quota.setValue(false);
} else {
@@ -403,13 +403,13 @@ Ext.define('PVE.lxc.MountPointInputPanel', {
me.hdfilesel.setVisible(true);
me.hdsizesel.setDisabled(true);
me.hdsizesel.setVisible(false);
- } else if (rec.data.type === 'lvm' ||
- rec.data.type === 'lvmthin' ||
- rec.data.type === 'rbd' ||
- rec.data.type === 'sheepdog' ||
- rec.data.type === 'zfs' ||
- rec.data.type === 'zfspool' ||
- rec.data.type === 'freenas') {
+} else if (rec.data.type === 'lvm' ||
+ rec.data.type === 'lvmthin' ||
+ rec.data.type === 'rbd' ||
+ rec.data.type === 'sheepdog' ||
+ rec.data.type === 'zfs' ||
+ rec.data.type === 'zfspool' ||
+ rec.data.type === 'freenas') {
me.hdfilesel.setDisabled(true);
me.hdfilesel.setVisible(false);
me.hdsizesel.setDisabled(false);
diff --git a/www/manager6/qemu/Clone.js b/www/manager6/qemu/Clone.js
index
Re: [pve-devel] [PATCH pve-manager] First beta of FreeNAS storage plugin
On Tue, 13 Jun 2017 09:39:37 +0200 Dominik Csapak wrote: > > but a better way for the whole plugin would probably be a credentials > file with limited read access (so only root can read it) > The password is written in /etc/pve/storage.cfg which is -rw-r root www-data. I cannot see how this could be changed to a more secure way since www-data will need to be allowed to have access to the password to be able to call the storage. What more security do a credentials file provide? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Don't use conditional branches as a substitute for a logical expression. - The Elements of Programming Style (Kernighan & Plaugher) pgpQmaKD7HE6a.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH pve-manager] First beta of FreeNAS storage plugin
On Mon, 12 Jun 2017 14:18:59 +0200
Dominik Csapak wrote:
> i just gave this patch a quick glance, here a few remarks:
>
> a bit nitpicky, but your indentation needs fixing, the FreeNASEdit.js seems
> mostly ok, but nearly every other file you touched has wrong indentation
>
The reason is that all the *.js files uses a mix of either tabs for
indentation or spaces, in some cases used intertwined so until somebody
normalizes all *.js files it seems the only safe way is to stick
exclusively with spaces as indentation and instruct your editor to
leave tabs untouched for not changes text.
> > + xtype: 'textfield',
> > + name: 'password',
> > + emptyText: '',
> > +inputType: 'password',
> > + fieldLabel: gettext('Password'),
> > + allowBlank: false
> > + },
>
> not really a gui problem, but it is probably a bad idea to save/load
> passwords from/to a textfield, at least do not load and insert it in the
> field, but leave it empty
>
How would you suggest fixing this? If you don't enter something then
the password will be change if the user presses save (I am not a
javascript guru;-)
> > +Ext.define('PVE.storage.FreeNASEdit', {
> > +extend: 'PVE.window.Edit',
> > +
> > +initComponent : function() {
> > + var me = this;
> > +
> > + me.isCreate = !me.storageId;
> > +
> > + if (me.isCreate) {
> > +me.url = '/api2/extjs/storage';
> > +me.method = 'POST';
> > +} else {
> > +me.url = '/api2/extjs/storage/' + me.storageId;
> > +me.method = 'PUT';
> > +}
> > +
> > + var ipanel = Ext.create('PVE.storage.FreeNASInputPanel', {
> > + isCreate: me.isCreate,
> > + storageId: me.storageId
> > + });
> > +
> > + Ext.apply(me, {
> > +subject: 'FreeNAS Storage',
>
> you add the storage type to format_storage_type, but here you do not use it,
> is there a reason?
>
I don't get this. Please elaborate?
--
Hilsen/Regards
Michael Rasmussen
Get my public GnuPG keys:
michael rasmussen cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir datanom net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir miras org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--
/usr/games/fortune -es says:
Government [is] an illusion the governed should not encourage.
-- John Updike, "Couples"
pgpjrHWObWo_P.pgp
Description: OpenPGP digital signature
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] FreeNAS plugin: Status
On Mon, 12 Jun 2017 09:04:38 +0200
Fabian Grünbichler wrote:
>
> but the LXC resize code does not set the running flag when resizing a
> volume, it always passes 0.
>
Strange, it seems to work here? I will investigate further. Maybe use
the same trick I use elsewhere:
my $run = PVE::QemuServer::check_running($vmid);
if (!$run) {
$run = PVE::LXC::check_running($vmid);
}
>
> but a guest running is not the only (potential) long running action
> which needs the disk activated. I don't think this would work (safely)..
>
I have fixed that now. It is only in exceptional corner cases that this
problem exists. It is not a problem for the plugin but more because I
would like to do a proper clean up. Come to think of it I think the
corner cases only come into play in curtain error cases which will
require manual intervention anyway.
--
Hilsen/Regards
Michael Rasmussen
Get my public GnuPG keys:
michael rasmussen cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir datanom net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir miras org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--
/usr/games/fortune -es says:
Go to a movie tonight. Darkness becomes you.
pgpUmNE2_nSoP.pgp
Description: OpenPGP digital signature
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-container] First beta of FreeNAS storage plugin
Signed-off-by: Michael Rasmussen
---
src/PVE/API2/LXC.pm | 9 +
src/PVE/LXC.pm | 6 +-
2 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index d9ad5ff..9cf22de 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -1476,11 +1476,20 @@ __PACKAGE__->register_method({
};
warn "Failed to update the container's filesystem:
$@\n" if $@;
} else {
+ # Some storage backends only exposes disks when VM is
running
+ # and e2fsck and resize2fs requires access to the disk to
work
+ my $scfg = PVE::Storage::storage_config($storage_cfg,
$storeid);
+ if ($scfg->{type} eq 'freenas') {
+ PVE::Storage::activate_volumes($storage_cfg, [$volid]);
+ }
eval {
PVE::Tools::run_command(['e2fsck', '-f', '-y',
$path]);
PVE::Tools::run_command(['resize2fs', $path]);
};
warn "Failed to update the container's filesystem:
$@\n" if $@;
+ if ($scfg->{type} eq 'freenas') {
+ PVE::Storage::deactivate_volumes($storage_cfg,
[$volid]);
+ }
}
}
};
diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index 32b0318..e269e72 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -1268,7 +1268,8 @@ sub mountpoint_mount {
$mounted_dev = run_with_loopdev($domount, $path);
$use_loopdev = 1;
} elsif ($scfg->{type} eq 'drbd' || $scfg->{type} eq 'lvm' ||
-$scfg->{type} eq 'rbd' || $scfg->{type} eq 'lvmthin') {
+$scfg->{type} eq 'rbd' || $scfg->{type} eq 'lvmthin' ||
+ $scfg->{type} eq 'freenas') {
$mounted_dev = $path;
&$domount($path);
} else {
@@ -1387,6 +1388,9 @@ sub create_disks {
die "krbd option must be enabled on storage type
'$scfg->{type}'\n" if !$scfg->{krbd};
$volid = PVE::Storage::vdisk_alloc($storecfg, $storage,
$vmid, 'raw', undef, $size_kb);
format_disk($storecfg, $volid, $rootuid, $rootgid);
+} elsif ($scfg->{type} eq 'freenas') {
+$volid = PVE::Storage::vdisk_alloc($storecfg, $storage, $vmid,
'raw', undef, $size_kb);
+format_disk($storecfg, $volid, $rootuid, $rootgid);
} else {
die "unable to create containers on storage type
'$scfg->{type}'\n";
}
--
2.1.4
This mail was virus scanned and spam checked before delivery.
This mail is also DKIM signed. See header dkim-signature.
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-container] FreeNAS storage plugin
Enable FreeNAS as a valid storage plugin for LXC Michael Rasmussen (1): First beta of FreeNAS storage plugin src/PVE/API2/LXC.pm | 9 + src/PVE/LXC.pm | 6 +- 2 files changed, 14 insertions(+), 1 deletion(-) -- 2.1.4 This mail was virus scanned and spam checked before delivery. This mail is also DKIM signed. See header dkim-signature. ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-manager] FreeNAS storage plugin
The GUI part for the FreeNAS storage plugin. Michael Rasmussen (1): First beta of FreeNAS storage plugin www/manager6/Makefile | 1 + www/manager6/Utils.js | 4 +- www/manager6/dc/StorageView.js | 15 +++- www/manager6/lxc/ResourceEdit.js| 6 +- www/manager6/qemu/Clone.js | 3 +- www/manager6/qemu/HDEdit.js | 3 +- www/manager6/qemu/HDEfi.js | 2 +- www/manager6/qemu/HDMove.js | 3 +- www/manager6/storage/FreeNASEdit.js | 152 9 files changed, 180 insertions(+), 9 deletions(-) create mode 100644 www/manager6/storage/FreeNASEdit.js -- 2.1.4 This mail was virus scanned and spam checked before delivery. This mail is also DKIM signed. See header dkim-signature. ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-manager] First beta of FreeNAS storage plugin
Signed-off-by: Michael Rasmussen
---
www/manager6/Makefile | 1 +
www/manager6/Utils.js | 4 +-
www/manager6/dc/StorageView.js | 15 +++-
www/manager6/lxc/ResourceEdit.js| 6 +-
www/manager6/qemu/Clone.js | 3 +-
www/manager6/qemu/HDEdit.js | 3 +-
www/manager6/qemu/HDEfi.js | 2 +-
www/manager6/qemu/HDMove.js | 3 +-
www/manager6/storage/FreeNASEdit.js | 152
9 files changed, 180 insertions(+), 9 deletions(-)
create mode 100644 www/manager6/storage/FreeNASEdit.js
diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index 0c266ef..2a4489d 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -186,6 +186,7 @@ JSSRC=
\
storage/SheepdogEdit.js \
storage/ZFSEdit.js \
storage/ZFSPoolEdit.js \
+ storage/FreeNASEdit.js \
ha/StatusView.js\
ha/Status.js\
ha/GroupSelector.js \
diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js
index d1ada77..c6227d0 100644
--- a/www/manager6/Utils.js
+++ b/www/manager6/Utils.js
@@ -799,7 +799,9 @@ Ext.define('PVE.Utils', { utilities: {
return 'iSCSIDirect';
} else if (value === 'drbd') {
return 'DRBD';
- } else {
+ } else if (value === 'freenas') {
+return 'FreeNAS';
+} else {
return PVE.Utils.unknownText;
}
},
diff --git a/www/manager6/dc/StorageView.js b/www/manager6/dc/StorageView.js
index 2940a98..c16d464 100644
--- a/www/manager6/dc/StorageView.js
+++ b/www/manager6/dc/StorageView.js
@@ -58,7 +58,9 @@ Ext.define('PVE.dc.StorageView', {
editor = 'PVE.storage.ZFSEdit';
} else if (type === 'zfspool') {
editor = 'PVE.storage.ZFSPoolEdit';
- } else {
+ } else if (type === 'freenas') {
+editor = 'PVE.storage.FreeNASEdit';
+} else {
return;
}
var win = Ext.create(editor, {
@@ -191,7 +193,16 @@ Ext.define('PVE.dc.StorageView', {
win.on('destroy', reload);
win.show();
}
-}
+},
+{
+text: PVE.Utils.format_storage_type('freenas'),
+iconCls: 'fa fa-fw fa-building',
+handler: function() {
+var win = Ext.create('PVE.storage.FreeNASEdit', {});
+win.on('destroy', reload);
+win.show();
+}
+}
/* the following type are conidered unstable
* so we do not enable that on the GUI for now
diff --git a/www/manager6/lxc/ResourceEdit.js b/www/manager6/lxc/ResourceEdit.js
index 61a5539..8bc3e76 100644
--- a/www/manager6/lxc/ResourceEdit.js
+++ b/www/manager6/lxc/ResourceEdit.js
@@ -387,7 +387,8 @@ Ext.define('PVE.lxc.MountPointInputPanel', {
}
var rec = f.store.getById(value);
if (rec.data.type === 'zfs' ||
- rec.data.type === 'zfspool') {
+ rec.data.type === 'zfspool' ||
+rec.data.type === 'freenas') {
me.quota.setDisabled(true);
me.quota.setValue(false);
} else {
@@ -407,7 +408,8 @@ Ext.define('PVE.lxc.MountPointInputPanel', {
rec.data.type === 'rbd' ||
rec.data.type === 'sheepdog' ||
rec.data.type === 'zfs' ||
- rec.data.type === 'zfspool') {
+ rec.data.type === 'zfspool' ||
+ rec.data.type === 'freenas') {
me.hdfilesel.setDisabled(true);
me.hdfilesel.setVisible(false);
me.hdsizesel.setDisabled(false);
diff --git a/www/manager6/qemu/Clone.js b/www/manager6/qemu/Clone.js
index c5dad4d..9de9777 100644
--- a/www/manager6/qemu/Clone.js
+++ b/www/manager6/qemu/Clone.js
@@ -116,7 +116,8 @@ Ext.define('PVE.window.Clone', {
rec.data.type === 'iscsi' ||
rec.data.type === 'sheepdog' ||
rec.data.type === 'zfs' ||
-
[pve-devel] [PATCH pve-storage] FreeNAS storage plugin
This patch series adds a storage plugin for FreeNAS using the FreeNAS API. The plugin supports both Qemu and LXC VM. Features supported, see below: VM CT create YES YES delete YES YES resize (YES) (YES) Note 1 snapshot offline YES YES live YES "YES" (state is not saved) backup snapshot YES Waiting standby YES YES offline YES YES clone full YES YES (Unavailable in GUI) linked YES YES (Unavailable in GUI) Note 1: Due to a bug in the FreeNAS API live resizing has been disabled in the plugin. See https://bugs.freenas.org/issues/24432 Since the plugin attaches disks through the local scsi subsystem via openiscsi and therefore not uses libiscsi there should be support for MPIO and authentication. MPIO is not tested since I don't have the required hardware to do so. Authentication is not enabled but should be easy to do if the requirements exists. Michael Rasmussen (1): First beta of FreeNAS storage plugin. Missing snapshot backup for LXC PVE/Storage.pm |2 + PVE/Storage/FreeNASPlugin.pm | 1289 ++ PVE/Storage/Makefile |2 +- PVE/Storage/Plugin.pm|2 +- 4 files changed, 1293 insertions(+), 2 deletions(-) create mode 100644 PVE/Storage/FreeNASPlugin.pm -- 2.1.4 This mail was virus scanned and spam checked before delivery. This mail is also DKIM signed. See header dkim-signature. ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH pve-storage] First beta of FreeNAS storage plugin. Missing snapshot backup for LXC
Signed-off-by: Michael Rasmussen
---
PVE/Storage.pm |2 +
PVE/Storage/FreeNASPlugin.pm | 1289 ++
PVE/Storage/Makefile |2 +-
PVE/Storage/Plugin.pm|2 +-
4 files changed, 1293 insertions(+), 2 deletions(-)
create mode 100644 PVE/Storage/FreeNASPlugin.pm
diff --git a/PVE/Storage.pm b/PVE/Storage.pm
index ee2295a..f33249f 100755
--- a/PVE/Storage.pm
+++ b/PVE/Storage.pm
@@ -32,6 +32,7 @@ use PVE::Storage::GlusterfsPlugin;
use PVE::Storage::ZFSPoolPlugin;
use PVE::Storage::ZFSPlugin;
use PVE::Storage::DRBDPlugin;
+use PVE::Storage::FreeNASPlugin;
# Storage API version. Icrement it on changes in storage API interface.
use constant APIVER => 1;
@@ -49,6 +50,7 @@ PVE::Storage::GlusterfsPlugin->register();
PVE::Storage::ZFSPoolPlugin->register();
PVE::Storage::ZFSPlugin->register();
PVE::Storage::DRBDPlugin->register();
+PVE::Storage::FreeNASPlugin->register();
# load third-party plugins
if ( -d '/usr/share/perl5/PVE/Storage/Custom' ) {
diff --git a/PVE/Storage/FreeNASPlugin.pm b/PVE/Storage/FreeNASPlugin.pm
new file mode 100644
index 000..771061e
--- /dev/null
+++ b/PVE/Storage/FreeNASPlugin.pm
@@ -0,0 +1,1289 @@
+package PVE::Storage::FreeNASPlugin;
+
+use strict;
+use warnings;
+use IO::File;
+use POSIX;
+use PVE::Tools qw(run_command);
+use PVE::Storage::Plugin;
+use PVE::RPCEnvironment;
+use PVE::QemuServer;
+use PVE::LXC;
+use LWP::UserAgent;
+use HTTP::Status;
+use JSON;
+use Data::Dumper;
+
+use base qw(PVE::Storage::Plugin);
+
+my $api = '/api/v1.0';
+my $timeout = 60; # seconds
+my $max_luns = 20; # maximum supported luns per target group in freenas is 25
+ # but reserve 5 for temporary LUNs (snapshots with RAM and
+ # snapshot backup)
+my $active_snaps = 4;
+my $limit = 1; # limit for get requests
+my $version;
+my $fullversion;
+my $target_prefix = 'iqn.2005-10.org.freenas.ctl';
+
+sub freenas_request {
+my ($scfg, $request, $section, $data, $valid_code) = @_;
+my $ua = LWP::UserAgent->new;
+$ua->agent("ProxmoxUA/0.1");
+$ua->ssl_opts( verify_hostname => 0 );
+$ua->timeout($timeout);
+push @{ $ua->requests_redirectable }, 'POST';
+push @{ $ua->requests_redirectable }, 'PUT';
+push @{ $ua->requests_redirectable }, 'DELETE';
+my $req;
+
+my $url = "https://$scfg->{portal}$api/$section";
+
+if ($request eq 'GET') {
+$req = HTTP::Request->new(GET => $url);
+} elsif ($request eq 'POST') {
+$req = HTTP::Request->new(POST => $url);
+$req->content($data);
+} elsif ($request eq 'PUT') {
+$req = HTTP::Request->new(PUT => $url);
+$req->content($data);
+} elsif ($request eq 'DELETE') {
+$req = HTTP::Request->new(DELETE => $url);
+} else {
+die "$request: Unknown request";
+}
+
+$req->content_type('application/json');
+$req->authorization_basic($scfg->{username}, $scfg->{password});
+my $res = $ua->request($req);
+
+ #print Dumper($res);
+return $res->code unless $res->is_success;
+
+return $res->content;
+}
+
+sub freenas_get_version {
+my ($scfg) = @_;
+
+ my $response = freenas_request($scfg, 'GET', "system/version");
+ die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+ my $info = decode_json($response);
+ $fullversion = $info->{fullversion};
+ if ($fullversion =~ /^\w+-(\d+)\.(\d*)\.(\d*)/) {
+ my $minor = $2;
+ my $micro = $3;
+
+ if ($minor) {
+ $minor = "0$minor" unless $minor > 9;
+ } else {
+ $minor = '00';
+ }
+
+ if ($micro) {
+ $micro = "0$micro" unless $micro > 9;
+ } else {
+ $micro = '00';
+ }
+
+ $version = "$1$minor$micro";
+ } else {
+ $version = '90200';
+ }
+}
+
+sub freenas_list_zvol {
+my ($scfg) = @_;
+
+ freenas_get_version($scfg) unless $version;
+
+ my $response = freenas_request($scfg, 'GET',
"storage/volume/$scfg->{pool}/zvols?limit=$limit");
+ die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+my $zvols = decode_json($response);
+ $response = freenas_request($scfg, 'GET',
"storage/snapshot?limit=$limit");
+ die HTTP::Status::status_message($response) if $response =~ /^\d+$/;
+my $snapshots = decode_json($response);
+
+my $list
Re: [pve-devel] Cloning a CT
On Thu, 8 Jun 2017 21:12:24 +0200 (CEST) Dietmar Maurer wrote: > > Any particular reason to not allow making a clone of a CT? > > I even think that making a linked clone of a CT should be possible, at > > least for zvols. > > AFAIR there was patches for that on this list. But no time to test/apply them > ... > I have done some quick tests here and with backend zfs/zvols there is no difference whether you make clones or linked clones so there is no technical reason not to enable this feature in the gui. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Mieux vaut tard que jamais! [ Better late than never ] pgpWTJq_6C0FX.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] Cloning a CT
Hi all, Any particular reason to not allow making a clone of a CT? I even think that making a linked clone of a CT should be possible, at least for zvols. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Indent to show the logical structure of a program. - The Elements of Programming Style (Kernighan & Plaugher) pgpxaTzZyM4ab.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] FreeNAS plugin: Status
On Thu, 8 Jun 2017 19:38:23 +0200 Michael Rasmussen wrote: > On Thu, 8 Jun 2017 19:21:50 +0200 > Michael Rasmussen wrote: > > > I seem to have found a bug in the API because changing the size through > > the API all though the zvol is properly resized rescanning the session > > on the client side does not show the changed size. If I do the same > > from the FreeNAS GUI then it works as expected! > > > Created a bug report: https://bugs.freenas.org/issues/24432 > To conclude on this long thread: 1) Online resize is not functional due to the bug in FreeNAS API so until this bug is resolved only allow offline resize 2) If I manually resizes the disk in the FreeNAS gui and rescan the iscsi session on the client online resize works with both VM and CT VM: 1) Change size of zvol 2) rescan iscsi session 3) qmp block_resize drive-scsiX +size[KMG] CT: 1) Change size of zvol 2) rescan iscsi session 3) unshare -m -- sh -c 'mount --make-rprivate / && mount /dev/disk/by-path/ip-10.0.1.32:3260-iscsi-iqn.2005-10.org.freenas.ctl:some_lun /tmp && resize2fs /dev/disk/by-path/ip-10.0.1.32:3260-iscsi-iqn.2005-10.org.freenas.ctl:some_lun' -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: If redness or swelling develop, consult physician promptly. pgpbhqqrLLfhb.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] FreeNAS plugin: Status
On Thu, 8 Jun 2017 19:21:50 +0200 Michael Rasmussen wrote: > I seem to have found a bug in the API because changing the size through > the API all though the zvol is properly resized rescanning the session > on the client side does not show the changed size. If I do the same > from the FreeNAS GUI then it works as expected! > Created a bug report: https://bugs.freenas.org/issues/24432 -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Time is a great teacher, but unfortunately it kills all its pupils. -- Hector Berlioz pgpZC24663941.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] FreeNAS plugin: Status
On Thu, 8 Jun 2017 18:15:13 +0200 Michael Rasmussen wrote: > On Thu, 8 Jun 2017 18:07:26 +0200 > Michael Rasmussen wrote: > > > Both host and storage. FreeNAS requires a removal of the > > 'targettoexent' and a new reassigning of targettoextent between > > target and extent. This reassignment brakes the connection sort of like I seem to have found a bug in the API because changing the size through the API all though the zvol is properly resized rescanning the session on the client side does not show the changed size. If I do the same from the FreeNAS GUI then it works as expected! -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Debian Hint #19: If you're interested in building packages from source, you should consider installing the apt-src package. pgpqCXoNwsjkE.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] FreeNAS plugin: Status
On Thu, 8 Jun 2017 17:06:30 +0200 (CEST) Alexandre DERUMIER wrote: > > But I remember to use qmp command to tell the vm the new size of disk (even > if it was already resized on the host) > Is there an equivalent to qmp block_resize for LXC? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Having children is like having a bowling alley installed in your brain. -- Martin Mull pgp7bgUMazTZF.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] FreeNAS plugin: Status
On Thu, 8 Jun 2017 18:07:26 +0200 Michael Rasmussen wrote: > Both host and storage. FreeNAS requires a removal of the > 'targettoexent' and a new reassigning of targettoextent between > target and extent. This reassignment brakes the connection sort of like I stand corrected ;-) Initial assigment $ sudo fdisk -l /dev/sdb Disk /dev/sdb: 5 GiB, 5368709120 bytes, 10485760 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 16384 bytes I/O size (minimum/optimal): 16384 bytes / 1048576 bytes After increasing the size of the zvol but before rescanning the session mir@pve:~/git/pve-storage$ sudo fdisk -l /dev/sdb Disk /dev/sdb: 5 GiB, 5368709120 bytes, 10485760 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 16384 bytes I/O size (minimum/optimal): 16384 bytes / 1048576 bytes Rescanning session mir@pve:~/git/pve-storage$ sudo iscsiadm -m session -r 5 -R Rescanning session [sid: 5, target: iqn.2005-10.org.freenas.ctl:test, portal: 10.0.1.32,3260] mir@pve:~/git/pve-storage$ sudo fdisk -l /dev/sdb Disk /dev/sdb: 10 GiB, 10737418240 bytes, 20971520 sectors Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 16384 bytes I/O size (minimum/optimal): 16384 bytes / 1048576 bytes -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: What do you give a man who has everything? Penicillin. -- Jerry Lester pgpulebrMj9T9.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] FreeNAS plugin: Status
On Thu, 8 Jun 2017 17:06:30 +0200 (CEST) Alexandre DERUMIER wrote: > > do you talk have about lun size on the host ? On the host, a simple iscsi > rescan should be able to detect new size. > Both host and storage. FreeNAS requires a removal of the 'targettoexent' and a new reassigning of targettoextent between target and extent. This reassignment brakes the connection sort of like pulling the power from a disk. In best case it will only cause a slight hang of the VM but could lead to a complete crash and data loss so to be on the safe side I will bail on disk connected to running VM's. A small cost to pay for much higher stability and durability. If a VM/CT most be available 24/7/365 you better make a clustered and/or redundant setup. >I'm sure to be able to do online resize with my nexenta - zfs iscsi plugin in >the past. >First version of my plugin (2010-2012 don't remember) was using scsi on host. >But it was too painfull to maintain (mainly with multipath and cpu usage with >a lot of luns), so I switch >it to libisci. But libiscsi is severely limited. Eg. no support for MPIO, no support for target groups, no support for chap, and most importantly no support for LXC. On the other hand maintenance of the iscsi subsystem has improved a lot making it very easy expose and hide LUN's on demand. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: No evil can happen to a good man. -- Plato pgpDLY1WXz87f.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] FreeNAS plugin: Status
On Thu, 8 Jun 2017 16:00:50 +0200 (CEST) Alexandre DERUMIER wrote: > > it should work if you send qmp block_resize command to vm, > after resize the lun on the host. > It is not the VM as such but more related to the scsi subsystem. The disk geometry is fixed at the time the LUN is attached to the system so the only way to change the geometry is to reload the LUN - reloading means in this context detach and attach again. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Do you think your mother and I should have lived comfortably so long together if ever we had been married? pgpq9YxQYu2Jx.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] FreeNAS plugin: Status
On Wed, 7 Jun 2017 18:57:42 +0200
Michael Rasmussen wrote:
> When CT is online:
> mount.nfs: Failed to resolve server /dev/disk/by-path/ip-10.0.1.32: Name or
> service not known
> Failed to update the container's filesystem: command 'unshare -m -- sh -c
> 'mount --make-rprivate / && mount
> /dev/disk/by-path/ip-10.0.1.32:3260-iscsi-iqn.2005-10.org.freenas.ctl:vm-103-lun-0
> /tmp && resize2fs
> /dev/disk/by-path/ip-10.0.1.32:3260-iscsi-iqn.2005-10.org.freenas.ctl:vm-103-lun-0''
> failed: exit code 32
>
> Should I bail when resize is called on a running CT?
>
It is this part (/usr/share/perl5/PVE/API2/LXC.pm:1455):
if ($format eq 'raw') {
my $path = PVE::Storage::path($storage_cfg, $volid, undef);
if ($running) {
$mp->{mp} = '/';
my $use_loopdev = (PVE::LXC::mountpoint_mount_path($mp,
$storage_cfg))[1];
$path = PVE::LXC::query_loopdev($path) if $use_loopdev;
die "internal error: CT running but mount point not attached to
a loop device"
if !$path;
PVE::Tools::run_command(['losetup', '--set-capacity', $path])
if $use_loopdev;
# In order for resize2fs to know that we need online-resizing a
mountpoint needs
# to be visible to it in its namespace.
# To not interfere with the rest of the system we unshare the
current mount namespace,
# mount over /tmp and then run resize2fs.
# interestingly we don't need to e2fsck on mounted systems...
my $quoted = PVE::Tools::shellquote($path);
my $cmd = "mount --make-rprivate / && mount $quoted /tmp &&
resize2fs $quoted";
eval {
PVE::Tools::run_command(['unshare', '-m', '--', 'sh',
'-c', $cmd]);
};
warn "Failed to update the container's filesystem: $@\n" if $@;
} else {
eval {
PVE::Tools::run_command(['e2fsck', '-f', '-y', $path]);
PVE::Tools::run_command(['resize2fs', $path]);
};
warn "Failed to update the container's filesystem: $@\n" if $@;
}
}
Which brakes.
If I run the command (unshare -m -- sh -c 'mount --make-rprivate / &&
mount
/dev/disk/by-path/ip-10.0.1.32:3260-iscsi-iqn.2005-10.org.freenas.ctl:vm-103-lun-0
/tmp &&
resize2fs
/dev/disk/by-path/ip-10.0.1.32:3260-iscsi-iqn.2005-10.org.freenas.ctl:vm-103-lun-0')
from the command line everything works as expected so I think what is
causing the brake is this mysterious log 'mount.nfs: Failed to resolve
server /dev/disk/by-path/ip-10.0.1.32: Name or service not known' which I don't
understand and cannot find out why a mount.nfs is done and from where it is
executed?
--
Hilsen/Regards
Michael Rasmussen
Get my public GnuPG keys:
michael rasmussen cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir datanom net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir miras org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--
/usr/games/fortune -es says:
Debian Hint #19: If you're interested in building packages from source,
you should consider installing the apt-src package.
pgpWCBlYxEKAt.pgp
Description: OpenPGP digital signature
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] FreeNAS plugin: Status
On Wed, 7 Jun 2017 19:50:17 +0200 Michael Rasmussen wrote: > BTW. Should the disks containing the state of live snapshots be part of > the list displayed when entering storage content? See snapshot > Snapshot: https://snag.gy/U3IysH.jpg -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Anti-trust laws should be approached with exactly that attitude. pgpPHO5MvftDM.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] FreeNAS plugin: Status
BTW. Should the disks containing the state of live snapshots be part of the list displayed when entering storage content? See snapshot -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: You don't have to wait--you can have it in 5.004_54 or so. :-) -- Larry Wall in <199710221740.kaa24...@wall.org> pgpVWaVyARX6v.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] FreeNAS plugin: Status
Hi all, Status for this plugin is in such a way that I hopefully should be able to release a beta later this week;-) VM CT create YES YES delete YES YES resize YES Note 1 (Needs fixing) snapshot offline YES YES live YES "YES" (state is not saved) backup snapshot YES Waiting standby YES YES offline YES YES clone full Waiting Waiting linked Waiting Waiting Note 1: When CT is online: mount.nfs: Failed to resolve server /dev/disk/by-path/ip-10.0.1.32: Name or service not known Failed to update the container's filesystem: command 'unshare -m -- sh -c 'mount --make-rprivate / && mount /dev/disk/by-path/ip-10.0.1.32:3260-iscsi-iqn.2005-10.org.freenas.ctl:vm-103-lun-0 /tmp && resize2fs /dev/disk/by-path/ip-10.0.1.32:3260-iscsi-iqn.2005-10.org.freenas.ctl:vm-103-lun-0'' failed: exit code 32 Should I bail when resize is called on a running CT? When CT is offline: e2fsck 1.42.12 (29-Aug-2014) e2fsck: No such file or directory while trying to open /dev/disk/by-path/ip-10.0.1.32:3260-iscsi-iqn.2005-10.org.freenas.ctl:vm-103-lun-0 Possibly non-existent device? Failed to update the container's filesystem: command 'e2fsck -f -y /dev/disk/by-path/ip-10.0.1.32:3260-iscsi-iqn.2005-10.org.freenas.ctl:vm-103-lun-0' failed: exit code 8 Seems fixable by ensuring to activate volume before hand. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Many changes of mind and mood; do not hesitate too long. pgpPmAF73T2xV.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Problems creating live snapshot
On Wed, 7 Jun 2017 12:36:43 +0200 (CEST)
Dietmar Maurer wrote:
> > We allocate the snapshot file on an active storage, so it should not
> > be necessary to activate it again.
> >
> > Note: see __snapshot_save_vmstate and foreach_writable_storage
>
> So the storage can be assumed to be active. The state volume itself if
> activated before we use it (see posting from fabian).
>
I found the problem. It was Fabian which pointed me in the right
direction.
The problematic part in activate_volume:
# Add new LUN's to session
os_request("iscsiadm -m session -r $sid -R", 0, 60);
# Remove all LUN's from session which is not currently active
deactivate_luns($scfg, $volname, $active_luns);
It seems related to a timing between UDEV is called to create a link and when
this link is actually available so adding a sleep 1 sorts this out. When you do
a --login to target this is no problem since the call to login will not return
until all LUN symlinks are created.
# Add new LUN's to session
os_request("iscsiadm -m session -r $sid -R", 0, 60);
sleep 1;
# Remove all LUN's from session which is not currently active
deactivate_luns($scfg, $volname, $active_luns);
--
Hilsen/Regards
Michael Rasmussen
Get my public GnuPG keys:
michael rasmussen cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir datanom net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir miras org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--
/usr/games/fortune -es says:
The urge to gamble is so universal and its practice so pleasurable
that I assume it must be evil.
-- Heywood Broun
pgpjO1Fj2F0q9.pgp
Description: OpenPGP digital signature
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Problems creating live snapshot
On Wed, 7 Jun 2017 08:16:31 +0200 Fabian Grünbichler wrote: > > maybe an issue with the colons in the path, which potentially require > escaping? > That has not been an issue so far. It is only this particular function that give problems. Why do you think colons should be a problem in this particular case? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: One is often kept in the right road by a rut. -- Gustave Droz pgpMwxn_fgL9Q.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] Problems creating live snapshot
On Wed, 7 Jun 2017 05:47:12 +0200 (CEST) Dietmar Maurer wrote: > > So my questions is: > > 1) Does savevm-start have problems saving to > > a /dev/disk/by-path/some_path? > > No > > > 2) Does savevm-start require a disk with partions and a file system? > > No. I am not aware of such restrictions. > So why does it not work? Any way to debug? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Operator, please trace this call and tell me where I am. pgpicBwmaUv7p.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] Problems creating live snapshot
Hi all, When I try to create a live snapshot of a VM I keep seeing this: TASK ERROR: VM 102 qmp command 'savevm-start' failed - failed to open '/dev/disk/by-path/ip-10.0.1.32:3260-iscsi-iqn.2005-10.org.freenas.ctl:vm-102-lun-20' Above is a link to eg ../../sdd and is present. So my questions is: 1) Does savevm-start have problems saving to a /dev/disk/by-path/some_path? 2) Does savevm-start require a disk with partions and a file system? -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: #define NULL 0 /* silly thing is, we don't even use this */ -- Larry Wall in perl.c from the perl source code pgpafJ4aXpg09.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] perhaps a bug in QemuConfig.pm?
On Tue, 6 Jun 2017 20:41:13 +0200 (CEST) Dietmar Maurer wrote: > > 100/1000 is far better than 25? > This number is the total number of LUN's at any given time. I guess this is the reason for FreeNAS' limit. So if every VM uses 25 LUN's then a FreeNAS storage will support 40 VM's. I have just discovered that ctld supports up to 1024 LUN's: #define MAX_LUNS1024 https://github.com/freebsd/freebsd/blob/master/usr.sbin/ctld/ctld.h -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Why you say you no bunny rabbit when you have little powder-puff tail? -- The Tasmanian Devil pgp65x8Rh1nAZ.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] perhaps a bug in QemuConfig.pm?
On Tue, 6 Jun 2017 20:40:03 +0200 (CEST) Dietmar Maurer wrote: > > If I understand you correctly, we also need to count unused disk (for > example snapshot ram disks) here? If so, this is not unusual. > snapshots does not count since they are not exposed through a LUN so it is only ram disks for live snapshots and actual disks which counts. > > I guess you can/should implement such limit inside the storage driver? > This is what I have done. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: "Microwave oven? Whaddya mean, it's a microwave oven? I've been watching Channel 4 on the thing for two weeks." pgpILmzrVC1Ir.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] perhaps a bug in QemuConfig.pm?
On Tue, 6 Jun 2017 17:53:55 +0200 Michael Rasmussen wrote: > 2) A VM or CT will be limited to have no more than 4 active snapshots > at a time. Forgot to add that VmWare also recommends no more than 4 active snapshots at a time for performance reasons (I have no idea whether this also is true for Proxmox?) -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: With a rubber duck, one's never alone. -- "The Hitchhiker's Guide to the Galaxy" pgpv0Yf6DGLD3.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] perhaps a bug in QemuConfig.pm?
On Tue, 6 Jun 2017 17:18:40 +0200 (CEST) Dietmar Maurer wrote: > > > setting the vmstatestorage once those patches are applied would help > > > then? > > > > > Yes, some. But the underlaying issue is still that FreeNAS puts an > > upper limit to the number of LUNS. > > Why don't you report a BUG to FreeNAS? If already reported, please can you > post > a link to the bug report here? > It is not a bug in FreeNAS per se. It is a limitation in ctld which does not allow an unlimited number of LUNS. I discussed it with the ctld developer some years ago but his would only raise the number of LUN's from 100 to 1000 at that time. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Aliquid melius quam pessimum optimum non est. pgpNntBUUwh3m.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] perhaps a bug in QemuConfig.pm?
On Tue, 6 Jun 2017 17:12:45 +0200 (CEST) Dietmar Maurer wrote: > > IMHO, this is simply not suitable for a virtualization storage backend. > And I don't want to limit the functionality of our software only because > there are some outdated storage boxes around. Do you mean it is a usual situation that a VM has more than 20 disks assigned? > > Or what was the suggestion exactly? > The suggestion was the following: 1) A VM or CT will be limited to have a maximum of 20 disks assigned. 2) A VM or CT will be limited to have no more than 4 active snapshots at a time. 3) One LUN is reserved for snapshot backups of CT's. -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Q: What do you call a blind, deaf-mute, quadraplegic Virginian? A: Trustworthy. pgppIRVO9cIMX.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] perhaps a bug in QemuConfig.pm?
On Tue, 6 Jun 2017 15:39:28 +0200 Fabian Grünbichler wrote: > > setting the vmstatestorage once those patches are applied would help > then? > Yes, some. But the underlaying issue is still that FreeNAS puts an upper limit to the number of LUNS. For now I will circumvent the issue by restricting number of LUN's for disks to a VM to 20 and reserve a pool of 5 LUN's for state and snapshot backup. It's just an annoying thing which makes the situation slightly more complicated, but nothing to worry about;-) -- Hilsen/Regards Michael Rasmussen Get my public GnuPG keys: michael rasmussen cc http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E mir datanom net http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C mir miras org http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917 -- /usr/games/fortune -es says: Health nuts are going to feel stupid someday, lying in hospitals dying of nothing. -- Redd Foxx pgpcvXERtv4iy.pgp Description: OpenPGP digital signature ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
