Re: [PVE-User] PVE ZFS Over ISCSI

[Gilberto Nunes]

Well... As I am a little stubborn, I try one more time, I get successful
install Iscsi Enterprise Target  in Ubuntu Xenial.
I have set ZFS as well.
And work as a charm...
Give me a little hack, of course...
But i have a Ubuntu machine running with ZFS Over Iscsi.





2017-08-11 15:04 GMT-03:00 Gilberto Nunes :

> hum... nice to hear that... thanks rasmussen
>
>
> Obrigado
>
> Cordialmente
>
>
> Gilberto Ferreira
>
> Consultor TI Linux | IaaS Proxmox, CloudStack, KVM | Zentyal Server |
> Zimbra Mail Server
>
> (47) 3025-5907
> (47) 99676-7530
>
> Skype: gilberto.nunes36
>
>
> konnectati.com.br 
>
>
> https://www.youtube.com/watch?v=dsiTPeNWcSE
>
>
> 2017-08-11 14:50 GMT-03:00 Michael Rasmussen :
>
>> On Fri, 11 Aug 2017 19:55:39 +0300
>> Mikhail  wrote:
>>
>> > FreeNAS and NAS4Free will fail because they're based on FreeBSD which
>> > uses ctld (CAM Target Layer) as a iSCSI target. ctld is not supported by
>> > Proxmox.
>> I have written a storage plugin for FreeNAS which works with the
>> FreeNAS API. The plugin is currently under review by the proxmox devs
>> but hopefully it should be available soon in proxmox 5.0 (I have it
>> running here both under 4.4 and 5.0).
>>
>> This plugin also uses the native SCSI subsystem on the host - no
>> libiscsi, so MPIO is supported.
>>
>> --
>> Hilsen/Regards
>> Michael Rasmussen
>>
>> Get my public GnuPG keys:
>> michael  rasmussen  cc
>> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
>> mir  datanom  net
>> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
>> mir  miras  org
>> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
>> --
>> /usr/games/fortune -es says:
>> If you are good, you will be assigned all the work.  If you are real
>> good, you will get out of it.
>>
>> ___
>> pve-user mailing list
>> pve-user@pve.proxmox.com
>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>>
>>
>
___
pve-user mailing list
pve-user@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

Re: [PVE-User] PVE ZFS Over ISCSI

[Gilberto Nunes]

hum... nice to hear that... thanks rasmussen


Obrigado

Cordialmente


Gilberto Ferreira

Consultor TI Linux | IaaS Proxmox, CloudStack, KVM | Zentyal Server |
Zimbra Mail Server

(47) 3025-5907
(47) 99676-7530

Skype: gilberto.nunes36


konnectati.com.br 


https://www.youtube.com/watch?v=dsiTPeNWcSE


2017-08-11 14:50 GMT-03:00 Michael Rasmussen :

> On Fri, 11 Aug 2017 19:55:39 +0300
> Mikhail  wrote:
>
> > FreeNAS and NAS4Free will fail because they're based on FreeBSD which
> > uses ctld (CAM Target Layer) as a iSCSI target. ctld is not supported by
> > Proxmox.
> I have written a storage plugin for FreeNAS which works with the
> FreeNAS API. The plugin is currently under review by the proxmox devs
> but hopefully it should be available soon in proxmox 5.0 (I have it
> running here both under 4.4 and 5.0).
>
> This plugin also uses the native SCSI subsystem on the host - no
> libiscsi, so MPIO is supported.
>
> --
> Hilsen/Regards
> Michael Rasmussen
>
> Get my public GnuPG keys:
> michael  rasmussen  cc
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
> mir  datanom  net
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
> mir  miras  org
> http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
> --
> /usr/games/fortune -es says:
> If you are good, you will be assigned all the work.  If you are real
> good, you will get out of it.
>
> ___
> pve-user mailing list
> pve-user@pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>
>
___
pve-user mailing list
pve-user@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

Re: [PVE-User] PVE ZFS Over ISCSI

[Michael Rasmussen]

On Fri, 11 Aug 2017 19:55:39 +0300
Mikhail  wrote:

> FreeNAS and NAS4Free will fail because they're based on FreeBSD which
> uses ctld (CAM Target Layer) as a iSCSI target. ctld is not supported by
> Proxmox.
I have written a storage plugin for FreeNAS which works with the
FreeNAS API. The plugin is currently under review by the proxmox devs
but hopefully it should be available soon in proxmox 5.0 (I have it
running here both under 4.4 and 5.0).

This plugin also uses the native SCSI subsystem on the host - no
libiscsi, so MPIO is supported.

-- 
Hilsen/Regards
Michael Rasmussen

Get my public GnuPG keys:
michael  rasmussen  cc
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xD3C9A00E
mir  datanom  net
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE501F51C
mir  miras  org
http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xE3E80917
--
/usr/games/fortune -es says:
If you are good, you will be assigned all the work.  If you are real
good, you will get out of it.


pgpRjJ9vEenYP.pgp
Description: OpenPGP digital signature
___
pve-user mailing list
pve-user@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

Re: [PVE-User] PVE ZFS Over ISCSI

[Gilberto Nunes]

Yep! That's my net shot: Omni with snap-it...

Thank you and good lucky


Obrigado

Cordialmente


Gilberto Ferreira

Consultor TI Linux | IaaS Proxmox, CloudStack, KVM | Zentyal Server |
Zimbra Mail Server

(47) 3025-5907
(47) 99676-7530

Skype: gilberto.nunes36


konnectati.com.br 


https://www.youtube.com/watch?v=dsiTPeNWcSE


2017-08-11 13:55 GMT-03:00 Mikhail :

> FreeNAS and NAS4Free will fail because they're based on FreeBSD which
> uses ctld (CAM Target Layer) as a iSCSI target. ctld is not supported by
> Proxmox.
> There's istgt port, but it is going to be user-space daemon and will
> work slower than native kernel iSCSI target on FreeBSD.
>
> So you can leave these ideas and not waste your time on FreeNAS.
>
> Right now I'm in process of transforming one of the NAS systems to
> Solaris-based OS - OmniOS (you can find in earlier threads on this
> mailing list about it). Proxmox seems to support Comstar as iSCSI
> target, which means it should work.
>
> But.. this is my second attempt to deploy OmniOS for the past year on
> the storage server. My first attempt failed due to missing Intel 10gbit
> NIC driver. And my last attempt also failed because OmniOS (as all
> Solaris OSes) does not support root pool (rpool) on mirrored stripes
> (aka RAID10) - this is due to boot loader limitation. Linux and FreeBSD
> support rpool on mirrored stripes, but ZFS is not native to them.
> So my next attempt is to add 2 USB sticks into server and create
> mirrored rpool on them purely for OS installation. My storage server has
> only 4 disk bays (1U Supermicro) and I do not want to give up with OmniOS.
>
> Good luck!
>
> On 08/11/2017 07:37 PM, Gilberto Nunes wrote:
> > Did you tried FreeNAS or NAS4Free to implement it??
> > Perhaps it's work...
> > That will be my next step.
> >
> >
> > Obrigado
> >
> > Cordialmente
> >
> >
> > Gilberto Ferreira
> >
> > Consultor TI Linux | IaaS Proxmox, CloudStack, KVM | Zentyal Server |
> > Zimbra Mail Server
> >
> > (47) 3025-5907
> > (47) 99676-7530
> >
> > Skype: gilberto.nunes36
> >
> >
> > konnectati.com.br 
> >
> >
> > https://www.youtube.com/watch?v=dsiTPeNWcSE
> >
> >
> > 2017-08-11 13:22 GMT-03:00 Mikhail :
> >
> >> Hello,
> >>
> >> Even if you're able to insert and create the disk, this does not mean
> >> you will be able to start/run VM with that disk. Apart from creating
> >> disk on ZFS storage you also need to export it via iSCSI.
> >>
> >> Unfortunately, ZFS-over-iSCSI is not documented well. I came across same
> >> problem as you 3 weeks ago (posted here on the list), but my questions
> >> left unanswered.
> >>
> >> Mikhail.
> >>
> >> On 08/11/2017 07:14 PM, Gilberto Nunes wrote:
> >>
> >>> If I use the command:
> >>>
> >>> qm set 100 -scsi1 ZFS-OVER:vm-100-disk-1
> >>> update VM 100: -scsi1 ZFS-OVER:vm-100-disk-1
> >>>
> >>> I can insert the disk with sucess
> >>>
> >>> But inside web interface I am no able to!
> >>
> >> ___
> >> pve-user mailing list
> >> pve-user@pve.proxmox.com
> >> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
> >>
> > ___
> > pve-user mailing list
> > pve-user@pve.proxmox.com
> > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
> >
>
> ___
> pve-user mailing list
> pve-user@pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>
___
pve-user mailing list
pve-user@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

Re: [PVE-User] PVE ZFS Over ISCSI

[Mark Adams]

Imo, don't even bother trying to do this with tgt. I got it working some
time ago but it was flakey, and started having dataset naming issues after
about 10 disks.

The only working (stable) zfs over iscsi with proxmox afaik is using
comstar (which to be fair to proxmox devs, is what they say in the docs...)

If you want to get this working I suggest you use omnios for the zfs
storage end. The free version of napp-it is easy to get going for a proof
of concept.

On 11 Aug 2017 19:23, "Mikhail"  wrote:

Hello,

Even if you're able to insert and create the disk, this does not mean
you will be able to start/run VM with that disk. Apart from creating
disk on ZFS storage you also need to export it via iSCSI.

Unfortunately, ZFS-over-iSCSI is not documented well. I came across same
problem as you 3 weeks ago (posted here on the list), but my questions
left unanswered.

Mikhail.

On 08/11/2017 07:14 PM, Gilberto Nunes wrote:

> If I use the command:
>
> qm set 100 -scsi1 ZFS-OVER:vm-100-disk-1
> update VM 100: -scsi1 ZFS-OVER:vm-100-disk-1
>
> I can insert the disk with sucess
>
> But inside web interface I am no able to!

___
pve-user mailing list
pve-user@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
___
pve-user mailing list
pve-user@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

Re: [PVE-User] PVE ZFS Over ISCSI

[Mikhail]

FreeNAS and NAS4Free will fail because they're based on FreeBSD which
uses ctld (CAM Target Layer) as a iSCSI target. ctld is not supported by
Proxmox.
There's istgt port, but it is going to be user-space daemon and will
work slower than native kernel iSCSI target on FreeBSD.

So you can leave these ideas and not waste your time on FreeNAS.

Right now I'm in process of transforming one of the NAS systems to
Solaris-based OS - OmniOS (you can find in earlier threads on this
mailing list about it). Proxmox seems to support Comstar as iSCSI
target, which means it should work.

But.. this is my second attempt to deploy OmniOS for the past year on
the storage server. My first attempt failed due to missing Intel 10gbit
NIC driver. And my last attempt also failed because OmniOS (as all
Solaris OSes) does not support root pool (rpool) on mirrored stripes
(aka RAID10) - this is due to boot loader limitation. Linux and FreeBSD
support rpool on mirrored stripes, but ZFS is not native to them.
So my next attempt is to add 2 USB sticks into server and create
mirrored rpool on them purely for OS installation. My storage server has
only 4 disk bays (1U Supermicro) and I do not want to give up with OmniOS.

Good luck!

On 08/11/2017 07:37 PM, Gilberto Nunes wrote:
> Did you tried FreeNAS or NAS4Free to implement it??
> Perhaps it's work...
> That will be my next step.
> 
> 
> Obrigado
> 
> Cordialmente
> 
> 
> Gilberto Ferreira
> 
> Consultor TI Linux | IaaS Proxmox, CloudStack, KVM | Zentyal Server |
> Zimbra Mail Server
> 
> (47) 3025-5907
> (47) 99676-7530
> 
> Skype: gilberto.nunes36
> 
> 
> konnectati.com.br 
> 
> 
> https://www.youtube.com/watch?v=dsiTPeNWcSE
> 
> 
> 2017-08-11 13:22 GMT-03:00 Mikhail :
> 
>> Hello,
>>
>> Even if you're able to insert and create the disk, this does not mean
>> you will be able to start/run VM with that disk. Apart from creating
>> disk on ZFS storage you also need to export it via iSCSI.
>>
>> Unfortunately, ZFS-over-iSCSI is not documented well. I came across same
>> problem as you 3 weeks ago (posted here on the list), but my questions
>> left unanswered.
>>
>> Mikhail.
>>
>> On 08/11/2017 07:14 PM, Gilberto Nunes wrote:
>>
>>> If I use the command:
>>>
>>> qm set 100 -scsi1 ZFS-OVER:vm-100-disk-1
>>> update VM 100: -scsi1 ZFS-OVER:vm-100-disk-1
>>>
>>> I can insert the disk with sucess
>>>
>>> But inside web interface I am no able to!
>>
>> ___
>> pve-user mailing list
>> pve-user@pve.proxmox.com
>> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>>
> ___
> pve-user mailing list
> pve-user@pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
> 

___
pve-user mailing list
pve-user@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

Re: [PVE-User] PVE ZFS Over ISCSI

[Gilberto Nunes]

Did you tried FreeNAS or NAS4Free to implement it??
Perhaps it's work...
That will be my next step.


Obrigado

Cordialmente


Gilberto Ferreira

Consultor TI Linux | IaaS Proxmox, CloudStack, KVM | Zentyal Server |
Zimbra Mail Server

(47) 3025-5907
(47) 99676-7530

Skype: gilberto.nunes36


konnectati.com.br 


https://www.youtube.com/watch?v=dsiTPeNWcSE


2017-08-11 13:22 GMT-03:00 Mikhail :

> Hello,
>
> Even if you're able to insert and create the disk, this does not mean
> you will be able to start/run VM with that disk. Apart from creating
> disk on ZFS storage you also need to export it via iSCSI.
>
> Unfortunately, ZFS-over-iSCSI is not documented well. I came across same
> problem as you 3 weeks ago (posted here on the list), but my questions
> left unanswered.
>
> Mikhail.
>
> On 08/11/2017 07:14 PM, Gilberto Nunes wrote:
>
> > If I use the command:
> >
> > qm set 100 -scsi1 ZFS-OVER:vm-100-disk-1
> > update VM 100: -scsi1 ZFS-OVER:vm-100-disk-1
> >
> > I can insert the disk with sucess
> >
> > But inside web interface I am no able to!
>
> ___
> pve-user mailing list
> pve-user@pve.proxmox.com
> https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user
>
___
pve-user mailing list
pve-user@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

Re: [PVE-User] PVE ZFS Over ISCSI

[Mikhail]

Hello,

Even if you're able to insert and create the disk, this does not mean
you will be able to start/run VM with that disk. Apart from creating
disk on ZFS storage you also need to export it via iSCSI.

Unfortunately, ZFS-over-iSCSI is not documented well. I came across same
problem as you 3 weeks ago (posted here on the list), but my questions
left unanswered.

Mikhail.

On 08/11/2017 07:14 PM, Gilberto Nunes wrote:

> If I use the command:
> 
> qm set 100 -scsi1 ZFS-OVER:vm-100-disk-1
> update VM 100: -scsi1 ZFS-OVER:vm-100-disk-1
> 
> I can insert the disk with sucess
> 
> But inside web interface I am no able to!

___
pve-user mailing list
pve-user@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user

Re: [PVE-User] PVE ZFS Over ISCSI

[Gilberto Nunes]

Update:

In the Debian Linux, I can see the volume that I create with Proxmox:

zfs list
NAME   USED  AVAIL  REFER  MOUNTPOINT
datastore120,6G  94,7G   149K  /datastore1
datastore1/iscsi1 20,6G   115G43K  -
datastore1/vm-100-disk-1 8K  94,7G 8K  -
datastore1/vm-100-disk-2 8K  94,7G 8K  -
datastore1/vm-100-disk-3 8K  94,7G 8K  -
datastore1/vm-100-disk-4 8K  94,7G 8K  -
datastore1/vm-100-disk-5 8K  94,7G 8K  -
datastore1/vm-101-disk-1 8K  94,7G 8K  -


If I use the command:

qm set 100 -scsi1 ZFS-OVER:vm-100-disk-1
update VM 100: -scsi1 ZFS-OVER:vm-100-disk-1

I can insert the disk with sucess

But inside web interface I am no able to!

So odd!



Obrigado

Cordialmente


Gilberto Ferreira

Consultor TI Linux | IaaS Proxmox, CloudStack, KVM | Zentyal Server |
Zimbra Mail Server

(47) 3025-5907
(47) 99676-7530

Skype: gilberto.nunes36


konnectati.com.br 


https://www.youtube.com/watch?v=dsiTPeNWcSE


2017-08-11 13:09 GMT-03:00 Gilberto Nunes :

> Hi guys
>
> I set up a Linux Debian Box, with istgt installed. Here's the config for
> istgt:
>
> [Global]
>   Comment "Global section"
>   NodeBase "iqn.2017-08.iscsi2.homegiba.com.br"
>   PidFile /var/run/istgt.pid
>   AuthFile /etc/istgt/auth.conf
>   MediaDirectory /var/istgt
>   LogFacility "local7"
>   Timeout 30
>   NopInInterval 20
>   DiscoveryAuthMethod Auto
>   MaxSessions 16
>   MaxConnections 4
>   MaxR2T 32
>   MaxOutstandingR2T 16
>   DefaultTime2Wait 2
>   DefaultTime2Retain 60
>   FirstBurstLength 262144
>   MaxBurstLength 1048576
>   MaxRecvDataSegmentLength 262144
>   InitialR2T Yes
>   ImmediateData Yes
>   DataPDUInOrder Yes
>   DataSequenceInOrder Yes
>   ErrorRecoveryLevel 0
> [UnitControl]
>   Comment "Internal Logical Unit Controller"
>  Portal UC1 127.0.0.1:3261
>   Netmask 127.0.0.1
> [PortalGroup1]
>   Comment "PVE"
>   Portal DA1 10.1.1.200:3260
> [InitiatorGroup1]
>   Comment "Initiator Group1"
>   InitiatorName "ALL"
> [LogicalUnit1]
>   Comment "Hard Disk Sample"
>   TargetName disk1
>   TargetAlias "Data Disk1"
>   Mapping PortalGroup1 InitiatorGroup1
>   AuthMethod Auto
>   AuthGroup AuthGroup1
>   UseDigest Auto
>   UnitType Disk
>   LUN0 Storage /dev/zvol/datastore1/iscsi1 Auto
>
>
> I have set up the ssh key as well and I can log in using that key.
>
> Indee, I have sucessful add the ZFS Over iSCSI storage and I can see the
> disks.
>
> But when I try to add some disk when I create a VM I get the erro below:
>
>
> No configuration found. Install istgt on 10.1.1.200 at
> /usr/share/perl5/PVE/Storage/LunCmd/Istgt.pm line 99. (500)
>
> My istgt is running and up!
>
> What can I doing wrong in this case???
>
> Thanks for any help.
>
___
pve-user mailing list
pve-user@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user