Re: [pve-devel] [PATCH pve-storage 4/4] Add support for creating LXC snapshot backup

[Michael Rasmussen]

On Thu, 15 Jun 2017 23:03:56 +0200
Andreas Steinel  wrote:

> 
> Compared to what? Of course not local ZFS, because everything applies
> also to local ZFS. TRIM support is the only point I understand in
> comparison to "ordinary" iSCSI backed on "fat" files.
> 
Compared to ZFS over NFS. NFS put a lot of work to the CPU for a
reduced performance compared to ZFS over iSCSI and I also forgot to
mention that ZFS over iSCSI also supports MPIO which is also lacking in
ZFS over NFS.

> I thought, that ZFS-over-iSCSI is superior to iSCSI because of the ZFS
> features, therefore ZFS-over-NFS has the same ZFS features, but as a
> file storage for container. No ordinary NFS has support for snapshots
> or quota directed by PVE, so this would be a real benefit. Also, you
> have a clusterwide-shared ZFS for KVM and LXC. In my opinion: it's the
> dream, isn't it?
My new FreeNAS plugin also supports containers as you can read from the
release notes. This is why I see no usecase for ZFS over NFS. Why use
ZFS over NFS when the FreeNAS plugin supports all the features
supported by Proxmox?

VM  CT
create  YES YES
delete  YES YES
resize  (YES)   (YES) Note 1
snapshot
   offline  YES YES
   live YES "YES" (state is not saved)
backup
   snapshot YES YES
   standby  YES YES
   offline  YES YES
clone
   full YES YES (Unavailable in GUI)
   linked   YES YES (Unavailable in GUI)

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael  rasmussen  cc
http://pgp.mit.edu:11371/pks/lookup?op=get=0xD3C9A00E
mir  datanom  net
http://pgp.mit.edu:11371/pks/lookup?op=get=0xE501F51C
mir  miras  org
http://pgp.mit.edu:11371/pks/lookup?op=get=0xE3E80917
--
/usr/games/fortune -es says:
He who knows, does not speak.  He who speaks, does not know.
-- Lao Tsu


pgpOLyKDkRnHa.pgp
Description: OpenPGP digital signature
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-storage 4/4] Add support for creating LXC snapshot backup

[Andreas Steinel]

Hi mir,

On Thu, Jun 15, 2017 at 7:23 PM, Michael Rasmussen  wrote:
> On Thu, 15 Jun 2017 18:54:29 +0200
> Andreas Steinel  wrote:
>> And I can't see how ZFS-over-NFS would be any different than
>> ZFS-over-iSCSI? Am I see this so wrong? What are the usecases for
>> ZFS-over-iSCSI exactly?
> Three obvious advantages:
> 1) Better performance
> 2) Lower CPU usage
> 3) Support for TRIM/Discard via SCSI UNMAP

Compared to what? Of course not local ZFS, because everything applies
also to local ZFS. TRIM support is the only point I understand in
comparison to "ordinary" iSCSI backed on "fat" files.

I thought, that ZFS-over-iSCSI is superior to iSCSI because of the ZFS
features, therefore ZFS-over-NFS has the same ZFS features, but as a
file storage for container. No ordinary NFS has support for snapshots
or quota directed by PVE, so this would be a real benefit. Also, you
have a clusterwide-shared ZFS for KVM and LXC. In my opinion: it's the
dream, isn't it?
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-storage 4/4] Add support for creating LXC snapshot backup

[Michael Rasmussen]

On Thu, 15 Jun 2017 18:54:29 +0200
Andreas Steinel  wrote:

> 
> And I can't see how ZFS-over-NFS would be any different than
> ZFS-over-iSCSI? Am I see this so wrong? What are the usecases for
> ZFS-over-iSCSI exactly?
Three obvious advantages:
1) Better performance
2) Lower CPU usage
3) Support for TRIM/Discard via SCSI UNMAP

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael  rasmussen  cc
http://pgp.mit.edu:11371/pks/lookup?op=get=0xD3C9A00E
mir  datanom  net
http://pgp.mit.edu:11371/pks/lookup?op=get=0xE501F51C
mir  miras  org
http://pgp.mit.edu:11371/pks/lookup?op=get=0xE3E80917
--
/usr/games/fortune -es says:
Make sure every module hides something.
- The Elements of Programming Style (Kernighan & Plaugher)


pgpuLAMUhTdg0.pgp
Description: OpenPGP digital signature
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-storage 4/4] Add support for creating LXC snapshot backup

[Andreas Steinel]

On Thu, Jun 15, 2017 at 1:04 PM, Michael Rasmussen  wrote:
> I still dont see a usecase for this?

And I can't see how ZFS-over-NFS would be any different than
ZFS-over-iSCSI? Am I see this so wrong? What are the usecases for
ZFS-over-iSCSI exactly?
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-storage 4/4] Add support for creating LXC snapshot backup

[Michael Rasmussen]

I still dont see a usecase for this?

On June 15, 2017 11:09:21 AM GMT+02:00, Andreas Steinel  
wrote:
>On Thu, Jun 15, 2017 at 10:50 AM, Michael Rasmussen 
>wrote:
>> Hi Andreas,
>> Even better, I can sent you the deb packages?
>
>That would also work, I then can create a diff to see what you changed
>and try to incorporate my idea with ZFS-over-NFS.
>___
>pve-devel mailing list
>pve-devel@pve.proxmox.com
>https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-storage 4/4] Add support for creating LXC snapshot backup

[Andreas Steinel]

On Thu, Jun 15, 2017 at 10:50 AM, Michael Rasmussen  wrote:
> Hi Andreas,
> Even better, I can sent you the deb packages?

That would also work, I then can create a diff to see what you changed
and try to incorporate my idea with ZFS-over-NFS.
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH pve-storage 4/4] Add support for creating LXC snapshot backup

[Michael Rasmussen]

Hi Andreas,
Even better, I can sent you the deb packages?

On June 15, 2017 10:20:32 AM GMT+02:00, Andreas Steinel  
wrote:
>Hi mir,
>
>have you a public git with all the changes? I'd like to try but I
>don't want to extract all the patches from this list, try to figure
>out which ones are already taken care of and manually apply them.
>
>Best,
>LnxBil
>
>On Wed, Jun 14, 2017 at 9:55 PM, Michael Rasmussen 
>wrote:
>> Signed-off-by: Michael Rasmussen 
>> ---
>>  PVE/Storage/FreeNASPlugin.pm | 129
>+--
>>  1 file changed, 74 insertions(+), 55 deletions(-)
>>
>> diff --git a/PVE/Storage/FreeNASPlugin.pm
>b/PVE/Storage/FreeNASPlugin.pm
>> index 0f2a56d..fee3d27 100644
>> --- a/PVE/Storage/FreeNASPlugin.pm
>> +++ b/PVE/Storage/FreeNASPlugin.pm
>> @@ -427,40 +427,38 @@ sub freenas_find_free_diskname {
>>  }
>>
>>  sub freenas_get_lun_number {
>> -my ($scfg, $volname, $snap) = @_;
>> +my ($scfg, $volname) = @_;
>>  my $lunid = undef;
>>
>> -if ($volname =~ /^(vm|base)-\d+-disk-(\d+)$/ && ! defined $snap)
>{
>> +if ($volname =~ /^(vm|base)-\d+-disk-(\d+)$/) {
>> $lunid = $2 - 1;
>>  } elsif ($volname =~ /^vm-(\d+)-state/) {
>> # Find id for temporary LUN
>> -   if ($snap) {
>> -   # TODO
>> -   # Required to be able to exposed read-only LUNs for
>snapshot backup CT
>> -   } else {
>> -   my $target = freenas_get_target($scfg, $1);
>> -   my $id = $max_luns;
>> -   my $response = freenas_request($scfg, 'GET',
>"services/iscsi/targettoextent?limit=$limit");
>> -   die HTTP::Status::status_message($response)
>if $response =~ /^\d+$/;
>> -   my $t2extents = decode_json($response);
>> -
>> -   foreach my $t2extent (@$t2extents) {
>> -   next unless $t2extent->{iscsi_target}
>== $target &&
>> -  
>$t2extent->{iscsi_lunid} + 1 > $max_luns &&
>> -  
>$t2extent->{iscsi_lunid} < $max_luns + $active_snaps;
>> -   my $eid = freenas_get_extent($scfg,
>$volname);
>> -   if ($eid) {
>> -   $response =
>freenas_request($scfg, 'GET', "services/iscsi/extent/$eid");
>> -   die
>HTTP::Status::status_message($response) if $response =~ /^\d+$/;
>> -   my $extent =
>decode_json($response);
>> -   # Request to get lunid for an
>existing lun
>> -   last if
>$t2extent->{iscsi_extent} eq $eid;
>> -   }
>> -   $id++;
>> +   my $target = freenas_get_target($scfg, $1);
>> +   my $id = $max_luns;
>> +   my $response = freenas_request($scfg, 'GET',
>"services/iscsi/targettoextent?limit=$limit");
>> +   die HTTP::Status::status_message($response) if
>$response =~ /^\d+$/;
>> +   my $t2extents = decode_json($response);
>> +
>> +   foreach my $t2extent (@$t2extents) {
>> +   next unless $t2extent->{iscsi_target} ==
>$target &&
>> +  
>$t2extent->{iscsi_lunid} + 1 > $max_luns &&
>> +  
>$t2extent->{iscsi_lunid} < $max_luns + $active_snaps;
>> +   my $eid = freenas_get_extent($scfg,
>$volname);
>> +   if ($eid) {
>> +   $response = freenas_request($scfg,
>'GET', "services/iscsi/extent/$eid");
>> +   die
>HTTP::Status::status_message($response) if $response =~ /^\d+$/;
>> +   my $extent = decode_json($response);
>> +   # Request to get lunid for an
>existing lun
>> +   last if $t2extent->{iscsi_extent} eq
>$eid;
>> }
>> -   die "Max snapshots (4) is reached" unless
>($id - $max_luns) < $active_snaps;
>> -   $lunid = $id;
>> -   }
>> +   $id++;
>> +   }
>> +   die "Max snapshots (4) is reached" unless ($id -
>$max_luns) < $active_snaps;
>> +   $lunid = $id;
>> +} elsif ($volname =~ /^(vm|base)-\d+-disk-\d+\@vzdump$/) {
>> +# Required to be able to exposed read-only LUNs for snapshot
>backup CT
>> +$lunid = $max_luns + $active_snaps;
>>  }
>>
>>  return $lunid;
>> @@ -891,12 +889,13 @@ sub list_images {
>>
>>  sub path {
>>  my ($class, $scfg, $volname, $storeid, $snapname) = @_;
>> -
>> -die "direct access to snapshots not implemented"
>> -   if 

Re: [pve-devel] [PATCH pve-storage 4/4] Add support for creating LXC snapshot backup

[Andreas Steinel]

Hi mir,

have you a public git with all the changes? I'd like to try but I
don't want to extract all the patches from this list, try to figure
out which ones are already taken care of and manually apply them.

Best,
LnxBil

On Wed, Jun 14, 2017 at 9:55 PM, Michael Rasmussen  wrote:
> Signed-off-by: Michael Rasmussen 
> ---
>  PVE/Storage/FreeNASPlugin.pm | 129 
> +--
>  1 file changed, 74 insertions(+), 55 deletions(-)
>
> diff --git a/PVE/Storage/FreeNASPlugin.pm b/PVE/Storage/FreeNASPlugin.pm
> index 0f2a56d..fee3d27 100644
> --- a/PVE/Storage/FreeNASPlugin.pm
> +++ b/PVE/Storage/FreeNASPlugin.pm
> @@ -427,40 +427,38 @@ sub freenas_find_free_diskname {
>  }
>
>  sub freenas_get_lun_number {
> -my ($scfg, $volname, $snap) = @_;
> +my ($scfg, $volname) = @_;
>  my $lunid = undef;
>
> -if ($volname =~ /^(vm|base)-\d+-disk-(\d+)$/ && ! defined $snap) {
> +if ($volname =~ /^(vm|base)-\d+-disk-(\d+)$/) {
> $lunid = $2 - 1;
>  } elsif ($volname =~ /^vm-(\d+)-state/) {
> # Find id for temporary LUN
> -   if ($snap) {
> -   # TODO
> -   # Required to be able to exposed read-only LUNs for snapshot 
> backup CT
> -   } else {
> -   my $target = freenas_get_target($scfg, $1);
> -   my $id = $max_luns;
> -   my $response = freenas_request($scfg, 'GET', 
> "services/iscsi/targettoextent?limit=$limit");
> -   die HTTP::Status::status_message($response) if 
> $response =~ /^\d+$/;
> -   my $t2extents = decode_json($response);
> -
> -   foreach my $t2extent (@$t2extents) {
> -   next unless $t2extent->{iscsi_target} == 
> $target &&
> -   
> $t2extent->{iscsi_lunid} + 1 > $max_luns &&
> -   
> $t2extent->{iscsi_lunid} < $max_luns + $active_snaps;
> -   my $eid = freenas_get_extent($scfg, $volname);
> -   if ($eid) {
> -   $response = freenas_request($scfg, 
> 'GET', "services/iscsi/extent/$eid");
> -   die 
> HTTP::Status::status_message($response) if $response =~ /^\d+$/;
> -   my $extent = decode_json($response);
> -   # Request to get lunid for an 
> existing lun
> -   last if $t2extent->{iscsi_extent} eq 
> $eid;
> -   }
> -   $id++;
> +   my $target = freenas_get_target($scfg, $1);
> +   my $id = $max_luns;
> +   my $response = freenas_request($scfg, 'GET', 
> "services/iscsi/targettoextent?limit=$limit");
> +   die HTTP::Status::status_message($response) if $response =~ 
> /^\d+$/;
> +   my $t2extents = decode_json($response);
> +
> +   foreach my $t2extent (@$t2extents) {
> +   next unless $t2extent->{iscsi_target} == $target &&
> +   $t2extent->{iscsi_lunid} + 1 
> > $max_luns &&
> +   $t2extent->{iscsi_lunid} < 
> $max_luns + $active_snaps;
> +   my $eid = freenas_get_extent($scfg, $volname);
> +   if ($eid) {
> +   $response = freenas_request($scfg, 'GET', 
> "services/iscsi/extent/$eid");
> +   die HTTP::Status::status_message($response) 
> if $response =~ /^\d+$/;
> +   my $extent = decode_json($response);
> +   # Request to get lunid for an existing lun
> +   last if $t2extent->{iscsi_extent} eq $eid;
> }
> -   die "Max snapshots (4) is reached" unless ($id - 
> $max_luns) < $active_snaps;
> -   $lunid = $id;
> -   }
> +   $id++;
> +   }
> +   die "Max snapshots (4) is reached" unless ($id - $max_luns) < 
> $active_snaps;
> +   $lunid = $id;
> +} elsif ($volname =~ /^(vm|base)-\d+-disk-\d+\@vzdump$/) {
> +# Required to be able to exposed read-only LUNs for snapshot backup 
> CT
> +$lunid = $max_luns + $active_snaps;
>  }
>
>  return $lunid;
> @@ -891,12 +889,13 @@ sub list_images {
>
>  sub path {
>  my ($class, $scfg, $volname, $storeid, $snapname) = @_;
> -
> -die "direct access to snapshots not implemented"
> -   if defined($snapname);
> +#die "direct access to snapshots not implemented"
> +   #if defined($snapname);
>
>  my ($vtype, $vname, $vmid) = $class->parse_volname($volname);
>
> +   $vname =