Re: support of PCIe NVME drives

[Miroslav Lachman]

Chris wrote on 04/18/2020 04:33:

On Fri, 17 Apr 2020 08:17:56 +0200 Miroslav Lachman 000.f...@quip.cz said


Chris wrote on 04/17/2020 05:51:
> On Thu, 16 Apr 2020 19:57:21 -0700 Mel Pilgrim > 
list_free...@bluerosetech.com said

> >> On 2020-04-16 12:30, Miroslav Lachman wrote:
>> > Pete Wright wrote on 04/16/2020 20:23:
>> >>
>> >>
>> >> On 4/16/20 11:12 AM, Miroslav Lachman wrote:
>> >>> Kurt Jaeger wrote on 04/16/2020 20:07:
>> > >> I would try booting via UEFI if you can.  I just installed a 
>> laptop >> yesterday which has a nvme root device, it was detected 
by >> the >> 12-STABLE snapshot I used to boot from.  no other 
modifications >> were >> necessary on my end.
>> > > I changed BIOS settings to use UEFI boot method, booted 12.1 >> 
installer > ISO but without luck. Still no NVME disks :(

>> > > You can see it on printscreen from iDRAC https://ibb.co/tPnymL7
>> > > Anything more I can test?
>>
>> Looking at server specs, the R6515's NVME support is only through 
the >> PERC S150 RAID controller.  If that's the case, I'm pretty sure 
you're >> out of luck.  The PERC S-series controllers are 
software-based RAID >> that require Dell's Windows or Linux drivers.  
You'd need a PERC >> H-series card to get support in FreeBSD.


That's something I didn't expect at all. So there is nothing which can 
be fixed on FreeBSD side? :(


>> Someone please correct me if I'm wrong?
> As I mentioned. I was suspicious of this. He should be able to flash 
the > card,
> making it a pass. I do a lot of them. If someone doesn't beat me to 
it. > I'll
> dig through what I have, and see if I can't find the right image(s), 
and > program(s).


This is rented dedicated machine. I cannot flash anything on it.

Apologies for the late reply. Unfortunately $JOB requires me to $WORK. :(
OK Given these are rentals. Probably the most I could supply for your
needs are images that allow you to poll, and change settings of the
controller(s). I have cd9660, and USB flash images of the utilities
for the 100-300 series HBAs as provided by LSI/AvGO. They boot to (MS)DOS,
or to an (U)EFI prompt. Having read what I can; the S150 controller is
Software controlled (likely explaining the "S" in it's model number), 
and as

I understand it *shouldn't* require any flashing. As I also understand you
*should* be able to access control of it through UEFI configs. I *think*
accessed via F2.
Apologies if this comes too late, or you've already solved this.
If you're still interested in the management software I have for the 
100-300

series. Let me know, and I'll attach a copy, or provide a link for you as
needed.


As you already noticed Scott Long is helping me in current@ list. He 
thinks some devices are not probed during boot.


I hope this problem can be solved by some "easy" patch.

Thank you for your help!

Miroslav Lachman
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: support of PCIe NVME drives

[Chris]

On Fri, 17 Apr 2020 08:17:56 +0200 Miroslav Lachman 000.f...@quip.cz said


Chris wrote on 04/17/2020 05:51:
> On Thu, 16 Apr 2020 19:57:21 -0700 Mel Pilgrim 
> list_free...@bluerosetech.com said
> 
>> On 2020-04-16 12:30, Miroslav Lachman wrote:

>> > Pete Wright wrote on 04/16/2020 20:23:
>> >>
>> >>
>> >> On 4/16/20 11:12 AM, Miroslav Lachman wrote:
>> >>> Kurt Jaeger wrote on 04/16/2020 20:07:
>> > >> I would try booting via UEFI if you can.  I just installed a 
>> laptop >> yesterday which has a nvme root device, it was detected by 
>> the >> 12-STABLE snapshot I used to boot from.  no other modifications 
>> were >> necessary on my end.
>> > > I changed BIOS settings to use UEFI boot method, booted 12.1 
>> installer > ISO but without luck. Still no NVME disks :(

>> > > You can see it on printscreen from iDRAC https://ibb.co/tPnymL7
>> > > Anything more I can test?
>>
>> Looking at server specs, the R6515's NVME support is only through the 
>> PERC S150 RAID controller.  If that's the case, I'm pretty sure you're 
>> out of luck.  The PERC S-series controllers are software-based RAID 
>> that require Dell's Windows or Linux drivers.  You'd need a PERC 
>> H-series card to get support in FreeBSD.


That's something I didn't expect at all. So there is nothing which can 
be fixed on FreeBSD side? :(


>> Someone please correct me if I'm wrong?
> As I mentioned. I was suspicious of this. He should be able to flash the 
> card,
> making it a pass. I do a lot of them. If someone doesn't beat me to it. 
> I'll
> dig through what I have, and see if I can't find the right image(s), and 
> program(s).


This is rented dedicated machine. I cannot flash anything on it.

Apologies for the late reply. Unfortunately $JOB requires me to $WORK. :(
OK Given these are rentals. Probably the most I could supply for your
needs are images that allow you to poll, and change settings of the
controller(s). I have cd9660, and USB flash images of the utilities
for the 100-300 series HBAs as provided by LSI/AvGO. They boot to (MS)DOS,
or to an (U)EFI prompt. Having read what I can; the S150 controller is
Software controlled (likely explaining the "S" in it's model number), and as
I understand it *shouldn't* require any flashing. As I also understand you
*should* be able to access control of it through UEFI configs. I *think*
accessed via F2.
Apologies if this comes too late, or you've already solved this.
If you're still interested in the management software I have for the 100-300
series. Let me know, and I'll attach a copy, or provide a link for you as
needed.

--Chris



I can ask the provider if they can install some card. Do you have some 
recommendation? What is supported by Dell and FreeBSD for ZFS?


I really appreciate your help!

Kind regards
Miroslav Lachman
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"



___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: support of PCIe NVME drives

[matti k]

On Fri, 17 Apr 2020 08:41:49 +0200
Miroslav Lachman <000.f...@quip.cz> wrote:

> Miroslav Lachman wrote on 04/17/2020 08:17:
> 
> 
> When I go in to menu Configuration / Controllers there is empty list.
> Is this expected in case of S130?
> 
> I would really like to fulfil the client needs and run FreeBSD on it. 
> But These R6515 are too new too me. The last PowerEdge I have in hand 
> was R610 / T20. Both working fine with FreeBSD.
> 

can you look up the hardware using dell support?  using my local one
https://www.dell.com/support/home/au/en/aubsd1 and enter asset tag or
serial number. It will help identify parts that were shipped. I suspect
if you add an extra PCIe card it may or may not be able to boot from it?

cheers,
matti

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: support of PCIe NVME drives

[Miroslav Lachman]

Miroslav Lachman wrote on 04/17/2020 08:17:

Looking at server specs, the R6515's NVME support is only through the 
PERC S150 RAID controller.  If that's the case, I'm pretty sure 
you're out of luck.  The PERC S-series controllers are software-based 
RAID that require Dell's Windows or Linux drivers.  You'd need a PERC 
H-series card to get support in FreeBSD.


That's something I didn't expect at all. So there is nothing which can 
be fixed on FreeBSD side? :(



Someone please correct me if I'm wrong?
As I mentioned. I was suspicious of this. He should be able to flash 
the card,
making it a pass. I do a lot of them. If someone doesn't beat me to 
it. I'll
dig through what I have, and see if I can't find the right image(s), 
and program(s).


This is rented dedicated machine. I cannot flash anything on it.

I can ask the provider if they can install some card. Do you have some 
recommendation? What is supported by Dell and FreeBSD for ZFS?


I really appreciate your help!


I tried to find info about controller in the iDRAC. I walk through the 
System / Inventory / Hardware Inventory and found nothing about S130


I found this informations

PCIe SSD in Slot 1 in Bay 1
Bus:   82
BusProtocol:  PCIE
Device:  0
DeviceDescription:  PCIe SSD in Slot 1 in Bay 1
DeviceProtocol:  NVMe-MI1.0
DeviceType:  PCIeSSD
DriveFormFactor:  2.5 inch
FailurePredicted:  NO
FQDD:  Disk.Bay.1:Enclosure.Internal.0-1
FreeSizeInBytes:  Information Not Available
Function:  0
HotSpareStatus:  Information Not Available
InstanceID:  Disk.Bay.1:Enclosure.Internal.0-1
Manufacturer:  INTEL
MaximumCapableSpeed:  8 GT/s
MediaType:  Solid State Drive
Model:  Dell Express Flash NVMe P4510 1TB SFF
NegotiatedSpeed:  8 GT/s
PCIeCapableLinkWidth:  x4
PCIeNegotiatedLinkWidth:  x4
PrimaryStatus:  Ok
ProductID:  a54
RaidStatus:  Information Not Available
RAIDType:  Unknown
RemainingRatedWriteEndurance:  100 %
Revision:  VDV1DP23
SerialNumber:  PHLJxxWF1PN
SizeInBytes:  1000204886016
Slot:  1
State:  Ready
SystemEraseCapability:  CryptographicErasePD

PCIe SSD in Slot 1 in Bay 1 - PCI Device
BusNumber:  130
DataBusWidth:  4x or x4
Description:  Express Flash NVMe 1.0 TB 2.5" U.2 (P4510)
DeviceDescription:  PCIe SSD in Slot 1 in Bay 1
DeviceNumber:  0
DeviceType:  PCIDevice
FQDD:  Disk.Bay.1:Enclosure.Internal.0-1
FunctionNumber:  0
InstanceID:  Disk.Bay.1:Enclosure.Internal.0-1
LastSystemInventoryTime:  2020-04-17T03:21:39
LastUpdateTime:  2020-03-31T13:55:06
Manufacturer:  Intel Corporation
PCIDeviceID:  0A54
PCISubDeviceID:  2003
PCISubVendorID:  1028
PCIVendorID:  8086
SlotLength:  2.5 Inch Drive Form Factor
SlotType:  PCI Express Gen 3 SFF-8639


When I go in to menu Configuration / Controllers there is empty list.
Is this expected in case of S130?

I would really like to fulfil the client needs and run FreeBSD on it. 
But These R6515 are too new too me. The last PowerEdge I have in hand 
was R610 / T20. Both working fine with FreeBSD.


Miroslav Lachman
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: support of PCIe NVME drives

[Miroslav Lachman]

Chris wrote on 04/17/2020 05:51:
On Thu, 16 Apr 2020 19:57:21 -0700 Mel Pilgrim 
list_free...@bluerosetech.com said



On 2020-04-16 12:30, Miroslav Lachman wrote:
> Pete Wright wrote on 04/16/2020 20:23:
>>
>>
>> On 4/16/20 11:12 AM, Miroslav Lachman wrote:
>>> Kurt Jaeger wrote on 04/16/2020 20:07:
> >> I would try booting via UEFI if you can.  I just installed a 
laptop >> yesterday which has a nvme root device, it was detected by 
the >> 12-STABLE snapshot I used to boot from.  no other modifications 
were >> necessary on my end.
> > I changed BIOS settings to use UEFI boot method, booted 12.1 
installer > ISO but without luck. Still no NVME disks :(

> > You can see it on printscreen from iDRAC https://ibb.co/tPnymL7
> > Anything more I can test?

Looking at server specs, the R6515's NVME support is only through the 
PERC S150 RAID controller.  If that's the case, I'm pretty sure you're 
out of luck.  The PERC S-series controllers are software-based RAID 
that require Dell's Windows or Linux drivers.  You'd need a PERC 
H-series card to get support in FreeBSD.


That's something I didn't expect at all. So there is nothing which can 
be fixed on FreeBSD side? :(



Someone please correct me if I'm wrong?
As I mentioned. I was suspicious of this. He should be able to flash the 
card,
making it a pass. I do a lot of them. If someone doesn't beat me to it. 
I'll
dig through what I have, and see if I can't find the right image(s), and 
program(s).


This is rented dedicated machine. I cannot flash anything on it.

I can ask the provider if they can install some card. Do you have some 
recommendation? What is supported by Dell and FreeBSD for ZFS?


I really appreciate your help!

Kind regards
Miroslav Lachman
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: support of PCIe NVME drives

[Chris]

On Thu, 16 Apr 2020 19:57:21 -0700 Mel Pilgrim list_free...@bluerosetech.com 
said


On 2020-04-16 12:30, Miroslav Lachman wrote:
> Pete Wright wrote on 04/16/2020 20:23:
>>
>>
>> On 4/16/20 11:12 AM, Miroslav Lachman wrote:
>>> Kurt Jaeger wrote on 04/16/2020 20:07:
> 
>> I would try booting via UEFI if you can.  I just installed a laptop 
>> yesterday which has a nvme root device, it was detected by the 
>> 12-STABLE snapshot I used to boot from.  no other modifications were 
>> necessary on my end.
> 
> I changed BIOS settings to use UEFI boot method, booted 12.1 installer 
> ISO but without luck. Still no NVME disks :(
> 
> You can see it on printscreen from iDRAC https://ibb.co/tPnymL7
> 
> Anything more I can test?


Looking at server specs, the R6515's NVME support is only through the 
PERC S150 RAID controller.  If that's the case, I'm pretty sure you're 
out of luck.  The PERC S-series controllers are software-based RAID that 
require Dell's Windows or Linux drivers.  You'd need a PERC H-series 
card to get support in FreeBSD.


Someone please correct me if I'm wrong?

As I mentioned. I was suspicious of this. He should be able to flash the card,
making it a pass. I do a lot of them. If someone doesn't beat me to it. I'll
dig through what I have, and see if I can't find the right image(s), and 
program(s).


--Chris


___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: support of PCIe NVME drives

[Mel Pilgrim]

On 2020-04-16 12:30, Miroslav Lachman wrote:

Pete Wright wrote on 04/16/2020 20:23:



On 4/16/20 11:12 AM, Miroslav Lachman wrote:

Kurt Jaeger wrote on 04/16/2020 20:07:


I would try booting via UEFI if you can.  I just installed a laptop 
yesterday which has a nvme root device, it was detected by the 
12-STABLE snapshot I used to boot from.  no other modifications were 
necessary on my end.


I changed BIOS settings to use UEFI boot method, booted 12.1 installer 
ISO but without luck. Still no NVME disks :(


You can see it on printscreen from iDRAC https://ibb.co/tPnymL7

Anything more I can test?


Looking at server specs, the R6515's NVME support is only through the 
PERC S150 RAID controller.  If that's the case, I'm pretty sure you're 
out of luck.  The PERC S-series controllers are software-based RAID that 
require Dell's Windows or Linux drivers.  You'd need a PERC H-series 
card to get support in FreeBSD.


Someone please correct me if I'm wrong?
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: support of PCIe NVME drives

[Miroslav Lachman]

Pete Wright wrote on 04/16/2020 23:06:



iDRAC does not allow me to do anything with the drives.
But I booted Linux SystemRescueCd and nvme devices are there visible 
in /dev/

printscreen https://ibb.co/sj22Nwg

So I think the HW is OK, but FreeBSD does not recognize the controller?

might be interesting to see what the dmesg buffer and lspci output for 
this device is under linux.  that should give pointers to how the device 
is being presented to the kernel.


There is verbose output of lspci https://ibb.co/dPZTwV1

dmesg contains:
nvme nvme0: pci function :81:00.0
nvme nvme1: pci function :82:00.0
nvme nvme0: 32/0/0 default/read/poll queues
nvme nvme1: 32/0/0 default/read/poll queues

Kind regards
Miroslav Lachman
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: support of PCIe NVME drives

[Chris]

On Thu, 16 Apr 2020 22:57:10 +0200 Miroslav Lachman 000.f...@quip.cz said


Chuck Tuffli wrote on 04/16/2020 22:29:
> On Thu, Apr 16, 2020 at 12:30 PM Miroslav Lachman <000.f...@quip.cz> wrote:
>>
>> Pete Wright wrote on 04/16/2020 20:23:
>>>
>>>
>>> On 4/16/20 11:12 AM, Miroslav Lachman wrote:
 Kurt Jaeger wrote on 04/16/2020 20:07:
>>
>>> I would try booting via UEFI if you can.  I just installed a laptop
>>> yesterday which has a nvme root device, it was detected by the 12-STABLE
>>> snapshot I used to boot from.  no other modifications were necessary on
>>> my end.
>>
>> I changed BIOS settings to use UEFI boot method, booted 12.1 installer
>> ISO but without luck. Still no NVME disks :(
>>
>> You can see it on printscreen from iDRAC https://ibb.co/tPnymL7
>>
>> Anything more I can test?
> 
> Fair warning, I don't deal much with Dell, but it's curious that iDrac

> lists the device protocol as MI
>> Device Protocol:  NVMe-MI1.0
> 
> MI or the Management Interface is defined by NVMe, but it isn't the

> same as a block interface. MI typically uses SMBUS and not PCIe to
> communicate with the device. So it's possible that the device is
> visible to iDrac via SMBUS but may not be on the PCIe bus for
> $reasons. Does iDrac have any tools to interact with the device via
> MI?

iDRAC does not allow me to do anything with the drives.
But I booted Linux SystemRescueCd and nvme devices are there visible in 
/dev/

printscreen https://ibb.co/sj22Nwg

So I think the HW is OK, but FreeBSD does not recognize the controller?


Does mps(4) or any of the other (often Dell) rebranded LSI drivers expose 
anything?



Kind regards
Miroslav Lachman


--Chris


___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: support of PCIe NVME drives

[Pete Wright]

On 4/16/20 1:57 PM, Miroslav Lachman wrote:

Chuck Tuffli wrote on 04/16/2020 22:29:
On Thu, Apr 16, 2020 at 12:30 PM Miroslav Lachman <000.f...@quip.cz> 
wrote:


Pete Wright wrote on 04/16/2020 20:23:



On 4/16/20 11:12 AM, Miroslav Lachman wrote:

Kurt Jaeger wrote on 04/16/2020 20:07:



I would try booting via UEFI if you can.  I just installed a laptop
yesterday which has a nvme root device, it was detected by the 
12-STABLE
snapshot I used to boot from.  no other modifications were 
necessary on

my end.


I changed BIOS settings to use UEFI boot method, booted 12.1 installer
ISO but without luck. Still no NVME disks :(

You can see it on printscreen from iDRAC https://ibb.co/tPnymL7

Anything more I can test?


Fair warning, I don't deal much with Dell, but it's curious that iDrac
lists the device protocol as MI

Device Protocol:  NVMe-MI1.0


MI or the Management Interface is defined by NVMe, but it isn't the
same as a block interface. MI typically uses SMBUS and not PCIe to
communicate with the device. So it's possible that the device is
visible to iDrac via SMBUS but may not be on the PCIe bus for
$reasons. Does iDrac have any tools to interact with the device via
MI?


iDRAC does not allow me to do anything with the drives.
But I booted Linux SystemRescueCd and nvme devices are there visible 
in /dev/

printscreen https://ibb.co/sj22Nwg

So I think the HW is OK, but FreeBSD does not recognize the controller?

might be interesting to see what the dmesg buffer and lspci output for 
this device is under linux.  that should give pointers to how the device 
is being presented to the kernel.


-p

--
Pete Wright
p...@nomadlogic.org
@nomadlogicLA

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: support of PCIe NVME drives

[Miroslav Lachman]

Chuck Tuffli wrote on 04/16/2020 22:29:

On Thu, Apr 16, 2020 at 12:30 PM Miroslav Lachman <000.f...@quip.cz> wrote:


Pete Wright wrote on 04/16/2020 20:23:



On 4/16/20 11:12 AM, Miroslav Lachman wrote:

Kurt Jaeger wrote on 04/16/2020 20:07:



I would try booting via UEFI if you can.  I just installed a laptop
yesterday which has a nvme root device, it was detected by the 12-STABLE
snapshot I used to boot from.  no other modifications were necessary on
my end.


I changed BIOS settings to use UEFI boot method, booted 12.1 installer
ISO but without luck. Still no NVME disks :(

You can see it on printscreen from iDRAC https://ibb.co/tPnymL7

Anything more I can test?


Fair warning, I don't deal much with Dell, but it's curious that iDrac
lists the device protocol as MI

Device Protocol:  NVMe-MI1.0


MI or the Management Interface is defined by NVMe, but it isn't the
same as a block interface. MI typically uses SMBUS and not PCIe to
communicate with the device. So it's possible that the device is
visible to iDrac via SMBUS but may not be on the PCIe bus for
$reasons. Does iDrac have any tools to interact with the device via
MI?


iDRAC does not allow me to do anything with the drives.
But I booted Linux SystemRescueCd and nvme devices are there visible in 
/dev/

printscreen https://ibb.co/sj22Nwg

So I think the HW is OK, but FreeBSD does not recognize the controller?

Kind regards
Miroslav Lachman
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: support of PCIe NVME drives

[Chuck Tuffli]

On Thu, Apr 16, 2020 at 12:30 PM Miroslav Lachman <000.f...@quip.cz> wrote:
>
> Pete Wright wrote on 04/16/2020 20:23:
> >
> >
> > On 4/16/20 11:12 AM, Miroslav Lachman wrote:
> >> Kurt Jaeger wrote on 04/16/2020 20:07:
>
> > I would try booting via UEFI if you can.  I just installed a laptop
> > yesterday which has a nvme root device, it was detected by the 12-STABLE
> > snapshot I used to boot from.  no other modifications were necessary on
> > my end.
>
> I changed BIOS settings to use UEFI boot method, booted 12.1 installer
> ISO but without luck. Still no NVME disks :(
>
> You can see it on printscreen from iDRAC https://ibb.co/tPnymL7
>
> Anything more I can test?

Fair warning, I don't deal much with Dell, but it's curious that iDrac
lists the device protocol as MI
> Device Protocol:  NVMe-MI1.0

MI or the Management Interface is defined by NVMe, but it isn't the
same as a block interface. MI typically uses SMBUS and not PCIe to
communicate with the device. So it's possible that the device is
visible to iDrac via SMBUS but may not be on the PCIe bus for
$reasons. Does iDrac have any tools to interact with the device via
MI?

--chuck
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: support of PCIe NVME drives

[Miroslav Lachman]

Navdeep Parhar wrote on 04/16/2020 21:51:

On 4/16/20 12:30 PM, Miroslav Lachman wrote:

Pete Wright wrote on 04/16/2020 20:23:



On 4/16/20 11:12 AM, Miroslav Lachman wrote:

Kurt Jaeger wrote on 04/16/2020 20:07:



I would try booting via UEFI if you can.  I just installed a laptop
yesterday which has a nvme root device, it was detected by the
12-STABLE snapshot I used to boot from.  no other modifications were
necessary on my end.


I changed BIOS settings to use UEFI boot method, booted 12.1 installer
ISO but without luck. Still no NVME disks :(

You can see it on printscreen from iDRAC https://ibb.co/tPnymL7

Anything more I can test?


Does the nvme controller show up in pciconf -l?

# pciconf -l | grep nvme


Empty result.

pciconf -l show many things, tome of them are named "noneN@pci..."

The machine is Dell PowerEdge R6515 with AMD EPYC 7302P

Is it possible that the controller is not recognized?

Miroslav Lachman
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: support of PCIe NVME drives

[Navdeep Parhar]

On 4/16/20 12:30 PM, Miroslav Lachman wrote:
> Pete Wright wrote on 04/16/2020 20:23:
>>
>>
>> On 4/16/20 11:12 AM, Miroslav Lachman wrote:
>>> Kurt Jaeger wrote on 04/16/2020 20:07:
> 
>> I would try booting via UEFI if you can.  I just installed a laptop
>> yesterday which has a nvme root device, it was detected by the
>> 12-STABLE snapshot I used to boot from.  no other modifications were
>> necessary on my end.
> 
> I changed BIOS settings to use UEFI boot method, booted 12.1 installer
> ISO but without luck. Still no NVME disks :(
> 
> You can see it on printscreen from iDRAC https://ibb.co/tPnymL7
> 
> Anything more I can test?

Does the nvme controller show up in pciconf -l?

# pciconf -l | grep nvme

Regards,
Navdeep

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: support of PCIe NVME drives

[Miroslav Lachman]

Pete Wright wrote on 04/16/2020 20:23:



On 4/16/20 11:12 AM, Miroslav Lachman wrote:

Kurt Jaeger wrote on 04/16/2020 20:07:


I would try booting via UEFI if you can.  I just installed a laptop 
yesterday which has a nvme root device, it was detected by the 12-STABLE 
snapshot I used to boot from.  no other modifications were necessary on 
my end.


I changed BIOS settings to use UEFI boot method, booted 12.1 installer 
ISO but without luck. Still no NVME disks :(


You can see it on printscreen from iDRAC https://ibb.co/tPnymL7

Anything more I can test?

Kind regards
Miroslav Lachman
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: support of PCIe NVME drives

[Rebecca Cran]

On 4/16/20 12:23 PM, Pete Wright wrote:

I would try booting via UEFI if you can.  I just installed a laptop 
yesterday which has a nvme root device, it was detected by the 
12-STABLE snapshot I used to boot from.  no other modifications were 
necessary on my end.


I can confirm, NVMe drives tend to not have any support under the 
traditional/legacy BIOS, and require UEFI.



--
Rebecca Cran


___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: support of PCIe NVME drives

[Mel Pilgrim]

On 2020-04-16 11:12, Miroslav Lachman wrote:

Kurt Jaeger wrote on 04/16/2020 20:07:

Hi!

I was requested to install FreeBSD 11.3 on a new Dell machine with 
only 2
NVME drives in ZFS mirror. The problem is that installer does not see 
the

drives. Are there any special procedure to use NVME drives for
installation a later for booting?


I use 2 NVMe drives as zfs mirror to boot from on my testbox,
but it runs CURRENT, since approx. November 2018.

So maybe try it with 12.1 ? I know, that does not help if you are asked
to install 11.3, but at least it gives you an idea...



I tried 12.1 few minutes ago but the result is the same - no NVME drives 
listed.

Should I try something with kernel modules, some sysctl tweaks?
Should I try UEFI boot? (I never did)


Make sure the UEFI BIOS can see them and enable dual-mode or UEFI-only 
boot. IIRC I couldn't legacy boot from the NVME drives.


My home and work servers have UEFI-boot gptzfsboot/root-on-zfs NVME 
drives that initially started with 11.2 and now run 12.1.

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: support of PCIe NVME drives

[Karl Denninger]

On 4/16/2020 13:23, Pete Wright wrote:
>
>
> On 4/16/20 11:12 AM, Miroslav Lachman wrote:
>> Kurt Jaeger wrote on 04/16/2020 20:07:
>>> Hi!
>>>
 I was requested to install FreeBSD 11.3 on a new Dell machine with
 only 2
 NVME drives in ZFS mirror. The problem is that installer does not
 see the
 drives. Are there any special procedure to use NVME drives for
 installation a later for booting?
>>>
>>> I use 2 NVMe drives as zfs mirror to boot from on my testbox,
>>> but it runs CURRENT, since approx. November 2018.
>>>
>>> So maybe try it with 12.1 ? I know, that does not help if you are asked
>>> to install 11.3, but at least it gives you an idea...
>>>
>>
>> I tried 12.1 few minutes ago but the result is the same - no NVME
>> drives listed.
>> Should I try something with kernel modules, some sysctl tweaks?
>> Should I try UEFI boot? (I never did)
>>
>
> I would try booting via UEFI if you can.  I just installed a laptop
> yesterday which has a nvme root device, it was detected by the
> 12-STABLE snapshot I used to boot from.  no other modifications were
> necessary on my end.
>
> -pete
>
Yeah my Lenovo Carbon X1 has an nVME drive in it, and nothing else -
12-Stable found it immediately and works fine.

-- 
Karl Denninger
k...@denninger.net 
/The Market Ticker/
/[S/MIME encrypted email preferred]/


smime.p7s
Description: S/MIME Cryptographic Signature

Re: support of PCIe NVME drives

[Pete Wright]

On 4/16/20 11:12 AM, Miroslav Lachman wrote:

Kurt Jaeger wrote on 04/16/2020 20:07:

Hi!

I was requested to install FreeBSD 11.3 on a new Dell machine with 
only 2
NVME drives in ZFS mirror. The problem is that installer does not 
see the

drives. Are there any special procedure to use NVME drives for
installation a later for booting?


I use 2 NVMe drives as zfs mirror to boot from on my testbox,
but it runs CURRENT, since approx. November 2018.

So maybe try it with 12.1 ? I know, that does not help if you are asked
to install 11.3, but at least it gives you an idea...



I tried 12.1 few minutes ago but the result is the same - no NVME 
drives listed.

Should I try something with kernel modules, some sysctl tweaks?
Should I try UEFI boot? (I never did)



I would try booting via UEFI if you can.  I just installed a laptop 
yesterday which has a nvme root device, it was detected by the 12-STABLE 
snapshot I used to boot from.  no other modifications were necessary on 
my end.


-pete

--
Pete Wright
p...@nomadlogic.org
@nomadlogicLA

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

Re: support of PCIe NVME drives

[Miroslav Lachman]

Kurt Jaeger wrote on 04/16/2020 20:07:

Hi!


I was requested to install FreeBSD 11.3 on a new Dell machine with only 2
NVME drives in ZFS mirror. The problem is that installer does not see the
drives. Are there any special procedure to use NVME drives for
installation a later for booting?


I use 2 NVMe drives as zfs mirror to boot from on my testbox,
but it runs CURRENT, since approx. November 2018.

So maybe try it with 12.1 ? I know, that does not help if you are asked
to install 11.3, but at least it gives you an idea...



I tried 12.1 few minutes ago but the result is the same - no NVME drives 
listed.

Should I try something with kernel modules, some sysctl tweaks?
Should I try UEFI boot? (I never did)

Kind regards
Miroslav Lachman

___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"

support of PCIe NVME drives

[Miroslav Lachman]

I was requested to install FreeBSD 11.3 on a new Dell machine with only 
2 NVME drives in ZFS mirror. The problem is that installer does not see 
the drives. Are there any special procedure to use NVME drives for 
installation a later for booting?


Drives are described in iDRAC:

Device Description:   PCIe SSD in Slot 1 in Bay 1
Device Protocol:  NVMe-MI1.0
Model:Dell Express Flash NVMe P4510 1TB SFF
Bus:  130
Manufacturer: INTEL
Product ID:   a54
Revision: VDV1DP23
Enclosure:PCIe SSD Backplane 1

There is no special HW RAID card.

I switched to a shell in an installer and NVME drives are not listen in 
dmesg.boot.


I tried to manually load nvme and nvd modules but the error messages 
says they are already in the installer kernel.


How can I make it work?


Kind regards
Miroslav Lachman
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"