Re: Dell SAS6IR is it worth the effort to convert to IT

[Lee Brown]

On Thu, Sep 13, 2018 at 9:03 AM, Josh Paetzel  wrote:

>
>
> On Thu, Sep 13, 2018, at 10:58 AM, Lee Brown wrote:
> > Hi,
> >
> > I'm in the process of setting up an old PowerEdge R410 which has, I
> > believe, the LSI Logic AS1068, using the mpt driver.
> >
> > I'm not interested in running it as a hardware RAID but rather a
> > geli(authentication only)+gmirror+gjournal.
> >
> > It seems like flashing it requires booting into dos, which frankly I
> can't
> > be bothered with.  mptutil doesn't appear to have that capability.
> >
> > Are there any major downsides to running it in IR mode as two disks?
> > There's no option to turn off the write-cache, but being battery backed
> the
> > only danger I see with that is if the machine is powered with pending
> > writes on the card and the battery gives out before power is restored.
> >
> > The use case is routing/snort IPS with minimal logging, so disk
> performance
> > isn't critical.  It will be the redundant router so mostly doing nothing
> > except the little work OSPF needs.
> >
> > Thanks in advance for any comments -- lee
> > ___
> > freebsd-hardware@freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
> > To unsubscribe, send any mail to "freebsd-hardware-unsubscribe@
> freebsd.org"
>
> On those controllers IR mode is just IT mode with the ability to create
> RAID.  If you don't create a RAID array it behaves exactly like an IT
> controller.
>
> And yes, FreeBSD mptutil doesn't have the ability to erase the flash,
> which you need to be able to do to go from IR to IT.  DOS is the typical
> answer for that.
>
Thanks a lot Josh, most appreciated.
___
freebsd-hardware@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"

Dell SAS6IR is it worth the effort to convert to IT

[Lee Brown]

Hi,

I'm in the process of setting up an old PowerEdge R410 which has, I
believe, the LSI Logic AS1068, using the mpt driver.

I'm not interested in running it as a hardware RAID but rather a
geli(authentication only)+gmirror+gjournal.

It seems like flashing it requires booting into dos, which frankly I can't
be bothered with.  mptutil doesn't appear to have that capability.

Are there any major downsides to running it in IR mode as two disks?
There's no option to turn off the write-cache, but being battery backed the
only danger I see with that is if the machine is powered with pending
writes on the card and the battery gives out before power is restored.

The use case is routing/snort IPS with minimal logging, so disk performance
isn't critical.  It will be the redundant router so mostly doing nothing
except the little work OSPF needs.

Thanks in advance for any comments -- lee
___
freebsd-hardware@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"

Re: Dell R630 unable to detect USB, can't boot

[Lee Brown]

On Thu, Feb 15, 2018 at 12:10 AM, Hans Petter Selasky <h...@selasky.org>
wrote:

> On 02/15/18 06:24, Lee Brown wrote:
>
>> I do find it curious that the loader is able to work the USB hardware but
>> not the kernel.
>>
>
> The loader is using the BIOS's USB code. Usually there is a handover from
> BIOS to OS, maybe that's where it's broken.


Got a little more information from watching slow video:

pcib1:  on apci0
pcib1: _OSC returned error 0x10

I think that may be AE_LIMIT, but I can't be sure.

Do you know if ACPI could be the root cause, I do see the ehci device
created later on, so it does 'see' the USB bus, but maybe that above error
prevents it from finding the hub?  I don't know I'm wildly speculating here.

Maybe I should address the above first, then circle around to this.
___
freebsd-hardware@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"

Re: Dell R630 unable to detect USB, can't boot

[Lee Brown]

q

On Wed, Feb 14, 2018 at 4:33 AM, Hans Petter Selasky <h...@selasky.org>
wrote:

> On 02/13/18 23:00, Lee Brown wrote:
>
>> On Tue, Feb 13, 2018 at 12:56 PM, Hans Petter Selasky <h...@selasky.org>
>> wrote:
>>
>> On 02/13/18 19:52, Lee Brown wrote:
>>>
>>> On Tue, Feb 13, 2018 at 6:00 AM, Hans Petter Selasky <h...@selasky.org>
>>>> wrote:
>>>>
>>>> On 02/13/18 14:45, Hans Petter Selasky wrote:
>>>>
>>>>>
>>>>> On 02/04/18 23:47, Lee Brown wrote:
>>>>>
>>>>>>
>>>>>> Booting FreeBSD11.1-RELENG from a USB stick, loader works fine
>>>>>> keyboard
>>>>>>
>>>>>>> allows selection, kernel loads and starts OK, gets as far as
>>>>>>> detecting
>>>>>>> USB
>>>>>>> devices and stops there.  imgur <https://imgur.com/a/psn0H> for a
>>>>>>> screenshot.  Tried safe mode same occurrence.
>>>>>>>
>>>>>>> CDROM is via a USB, so that's a no go and I've not had luck booting
>>>>>>> UEFI
>>>>>>> iSCSI nor PXE yet, so I'm stuck without a keyboard (no PS/2
>>>>>>> connectors)
>>>>>>> or
>>>>>>> essentially any boot media.
>>>>>>>
>>>>>>> For reference Xen 7.2 is running fine so at first blush it's not
>>>>>>> faulty
>>>>>>> hardware.
>>>>>>> USB3 mode on and off (ie force USB2) in the BIOS makes no difference.
>>>>>>> All Firmwares current.
>>>>>>>
>>>>>>> Any suggestions on how to move forward with this?  I tried turning on
>>>>>>> verbose mode, but I'm not sure how to capture that nor where to look.
>>>>>>>
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>
>>>>>> USB_ERR_TIMEOUT might be an indication of a PCI IRQ issue.
>>>>>>
>>>>>>
>>>>>> Try to set:
>>>>>>
>>>>>
>>>>> hw.usb.xhci.use_polling=1
>>>>>
>>>>> from the loader as a temporary quirk. Then try to dump information
>>>>> about
>>>>> IRQ's like "vmstat -i" "dmesg" and so on.
>>>>>
>>>>> --HPS
>>>>>
>>>>> Thanks Hans, but unfortunately that made no difference.  Here's my full
>>>>>
>>>>> loader.conf
>>>>
>>>> vfs.mountroot.timeout="10"
>>>> boot_multicons="YES"
>>>> boot_serial="YES"
>>>> comconsole_speed="115200"
>>>> console="comconsole,efi"
>>>> kern.cam.boot_delay="1"
>>>> hw.usb.xhci.use_polling=1
>>>>
>>>>
>>> Did you enable USB 3.0 in the BIOS when trying this?
>>>
>>> --HPS
>>>
>>> It was configured with USB3.0 OFF
>>>
>> I just toggled it ON, same result.
>>
>> I verified the sysctl polling value from the loader before booting
>>
>
> Hi,
>
> I suggest you bring this issue up with Dell customer support.
>
> Further, maybe Linux XHCI / EHCI has a quirk for Dell R630.
>
> Alternativly you can try to debug the XHCI/EHCI debug prints on your own.
> If you get a timeout it means something is not communicating like expected
> in the XHCI/EHCI DMA descriptors.
>
>
> As I only have the hardware replacement level of support and FreeBSD isn't
on their list of certified OS's, I'm probably best spending my time
debugging this issue.

I do find it curious that the loader is able to work the USB hardware but
not the kernel.

Once I figure out how to capture the kernel output, I'll crank up the
debugging and post back here.
___
freebsd-hardware@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"

Re: Dell R630 unable to detect USB, can't boot

[Lee Brown]

On Tue, Feb 13, 2018 at 12:56 PM, Hans Petter Selasky <h...@selasky.org>
wrote:

> On 02/13/18 19:52, Lee Brown wrote:
>
>> On Tue, Feb 13, 2018 at 6:00 AM, Hans Petter Selasky <h...@selasky.org>
>> wrote:
>>
>> On 02/13/18 14:45, Hans Petter Selasky wrote:
>>>
>>> On 02/04/18 23:47, Lee Brown wrote:
>>>>
>>>> Booting FreeBSD11.1-RELENG from a USB stick, loader works fine keyboard
>>>>> allows selection, kernel loads and starts OK, gets as far as detecting
>>>>> USB
>>>>> devices and stops there.  imgur <https://imgur.com/a/psn0H> for a
>>>>> screenshot.  Tried safe mode same occurrence.
>>>>>
>>>>> CDROM is via a USB, so that's a no go and I've not had luck booting
>>>>> UEFI
>>>>> iSCSI nor PXE yet, so I'm stuck without a keyboard (no PS/2 connectors)
>>>>> or
>>>>> essentially any boot media.
>>>>>
>>>>> For reference Xen 7.2 is running fine so at first blush it's not faulty
>>>>> hardware.
>>>>> USB3 mode on and off (ie force USB2) in the BIOS makes no difference.
>>>>> All Firmwares current.
>>>>>
>>>>> Any suggestions on how to move forward with this?  I tried turning on
>>>>> verbose mode, but I'm not sure how to capture that nor where to look.
>>>>>
>>>>>
>>>>> Hi,
>>>>
>>>> USB_ERR_TIMEOUT might be an indication of a PCI IRQ issue.
>>>>
>>>>
>>>> Try to set:
>>>
>>> hw.usb.xhci.use_polling=1
>>>
>>> from the loader as a temporary quirk. Then try to dump information about
>>> IRQ's like "vmstat -i" "dmesg" and so on.
>>>
>>> --HPS
>>>
>>> Thanks Hans, but unfortunately that made no difference.  Here's my full
>>>
>> loader.conf
>>
>> vfs.mountroot.timeout="10"
>> boot_multicons="YES"
>> boot_serial="YES"
>> comconsole_speed="115200"
>> console="comconsole,efi"
>> kern.cam.boot_delay="1"
>> hw.usb.xhci.use_polling=1
>>
>
> Did you enable USB 3.0 in the BIOS when trying this?
>
> --HPS
>
> It was configured with USB3.0 OFF
I just toggled it ON, same result.

I verified the sysctl polling value from the loader before booting
___
freebsd-hardware@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"

Re: Dell R630 unable to detect USB, can't boot

[Lee Brown]

On Tue, Feb 13, 2018 at 6:00 AM, Hans Petter Selasky <h...@selasky.org>
wrote:

> On 02/13/18 14:45, Hans Petter Selasky wrote:
>
>> On 02/04/18 23:47, Lee Brown wrote:
>>
>>> Booting FreeBSD11.1-RELENG from a USB stick, loader works fine keyboard
>>> allows selection, kernel loads and starts OK, gets as far as detecting
>>> USB
>>> devices and stops there.  imgur <https://imgur.com/a/psn0H> for a
>>> screenshot.  Tried safe mode same occurrence.
>>>
>>> CDROM is via a USB, so that's a no go and I've not had luck booting UEFI
>>> iSCSI nor PXE yet, so I'm stuck without a keyboard (no PS/2 connectors)
>>> or
>>> essentially any boot media.
>>>
>>> For reference Xen 7.2 is running fine so at first blush it's not faulty
>>> hardware.
>>> USB3 mode on and off (ie force USB2) in the BIOS makes no difference.
>>> All Firmwares current.
>>>
>>> Any suggestions on how to move forward with this?  I tried turning on
>>> verbose mode, but I'm not sure how to capture that nor where to look.
>>>
>>>
>> Hi,
>>
>> USB_ERR_TIMEOUT might be an indication of a PCI IRQ issue.
>>
>>
> Try to set:
>
> hw.usb.xhci.use_polling=1
>
> from the loader as a temporary quirk. Then try to dump information about
> IRQ's like "vmstat -i" "dmesg" and so on.
>
> --HPS
>
> Thanks Hans, but unfortunately that made no difference.  Here's my full
loader.conf

vfs.mountroot.timeout="10"
boot_multicons="YES"
boot_serial="YES"
comconsole_speed="115200"
console="comconsole,efi"
kern.cam.boot_delay="1"
hw.usb.xhci.use_polling=1
___
freebsd-hardware@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"

Dell R630 unable to detect USB, can't boot

[Lee Brown]

Booting FreeBSD11.1-RELENG from a USB stick, loader works fine keyboard
allows selection, kernel loads and starts OK, gets as far as detecting USB
devices and stops there.  imgur  for a
screenshot.  Tried safe mode same occurrence.

CDROM is via a USB, so that's a no go and I've not had luck booting UEFI
iSCSI nor PXE yet, so I'm stuck without a keyboard (no PS/2 connectors) or
essentially any boot media.

For reference Xen 7.2 is running fine so at first blush it's not faulty
hardware.
USB3 mode on and off (ie force USB2) in the BIOS makes no difference.
All Firmwares current.

Any suggestions on how to move forward with this?  I tried turning on
verbose mode, but I'm not sure how to capture that nor where to look.

Thx -- lee
___
freebsd-hardware@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"

Re: Does anybody have experience with a MD1420, mpr(4)

[Lee Brown]

On Thu, Nov 16, 2017 at 10:43 AM, Lee Brown <l...@ratnaling.org> wrote:

>
>
> On Thu, Nov 16, 2017 at 6:28 AM, Kenneth D. Merry <k...@freebsd.org> wrote:
>
>> On Wed, Nov 15, 2017 at 12:24:47 -0800, Lee Brown wrote:
>> > Hi,
>> >
>> > This is my for foray into DAS, so please be gentle :)  Apologies if the
>> > cross-post is inappropriate, let me know.
>> >
>> > I'm looking to build a SAS array with a Dell MD1420.  It will be
>> attached
>> > via a SAS 9300-414E HBA in a Dell R330.
>> >
>> > I've verified that the server / SAS card should be fine, but just
>> wanted to
>> > know if anybody has set this up before, or if there's any obvious reason
>> > why it wouldn't work.  Dell can be a bit weird sometimes.
>> >
>> > Any information is most welcome.
>>
>> In theory it should be fine.  I haven't used a Dell enclosure recently,
>> but
>> from posts here, it seems that they like to turn on Type 2 Protection
>> Information on the drives.  (Assuming you buy the drives from Dell.)
>>
>> If they do that, you can either reformat, or see the patch here for the
>> mpr(4) driver to disable using protection information:
>>
>> https://docs.freebsd.org/cgi/getmsg.cgi?fetch=308820+0+archi
>> ve/2017/freebsd-scsi/20171022.freebsd-scsi
>>
>> I haven't gotten around to changing things so that we detect the PI in the
>> da(4) driver and use 32 byte CDBs when type 2 protection is enabled.  (So
>> if someone else has time and wants to do it, feel free.  It will be
>> somewhat messy because we only allocate enough space for 16 byte CDBs
>> inside
>> the CCB, so you'll need to allocate the CDB separately...)
>>
>> If you use ZFS, you'll have checksum protection on your data anyway, so PI
>> would just be additional protection.
>>
>> I would also suggest that you update to the latest firmware (currently
>> Phase 15) for the SAS controller.  You can do that with LSI's sas3flash or
>> the FreeBSD mprutil(8) utility.
>>
>> Ken
>> --
>> Kenneth Merry
>> k...@freebsd.org
>>
> Thanks Ken,
>
> I'm going to purchase the system.  I had no idea about PI, that's a nice
> feature for sure.
>
> This will probably just be iSCSI and NFS4 over a 10Gb NIC (Xen cluster),
> so having PI detection is certainly of interest to me.  I'll figure out
> what I can do to contribute (code/testing) when things have settled.
>
> Assuming all is well I'll get the 1420 added to the hardware compatibility
> list.
>
> Much appreciated -- lee
>

Following up, I have the MD1420 installed and attached.  The kernel patch
works as expected, errors with the flag set to zero, no errors when set to
1.

First problem I am having is saving firmware from the card to local storage:
root@svr-nas-rl:~ # mprutil flash save firmware
mprutil: Fail to save firmware

Second problem is performance, 20MB/s sequential write to a single spindle,
200MB/s read.  I was expecting at least 100MB/s for the write, maybe my
expectations were wrong?

root@svr-nas-rl:~ # cp /dev/zero /dev/da0
dT: 1.058s  w: 1.000s
 L(q)  ops/sr/s   kBps   ms/rw/s   kBps   ms/w   %busy Name
1158  0  00.0158  202066.3   99.8| da0

root@svr-nas-rl:~ # cp /dev/da0 /dev/null
dT: 1.001s  w: 1.000s
 L(q)  ops/sr/s   kBps   ms/rw/s   kBps   ms/w   %busy Name
1   1587   1587 2031710.6  0  00.0   99.4| da0

root@svr-nas-rl:~ # mprutil show adapter
mpr0 Adapter:
   Board Name: SAS9300-4i4e
   Board Assembly: H3-25515-00G
Chip Name: LSISAS3008
Chip Revision: ALL
BIOS Revision: 6.00.00.00
Firmware Revision: 5.00.00.00
  Integrated RAID: no

root@svr-nas-rl:~ # geom disk list da0
Geom name: da0
Providers:
1. Name: da0
   Mediasize: 3000 (279G)
   Sectorsize: 512
   Mode: r0w0e0
   descr: SEAGATE ST300MM0078
   lunid: 5000c500a0f3f977
   ident: W0K1BE3R
   rotationrate: 1
   fwsectors: 63
   fwheads: 255
___
freebsd-hardware@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"

Re: Does anybody have experience with a MD1420, mpr(4)

[Lee Brown]

On Thu, Nov 16, 2017 at 6:28 AM, Kenneth D. Merry <k...@freebsd.org> wrote:

> On Wed, Nov 15, 2017 at 12:24:47 -0800, Lee Brown wrote:
> > Hi,
> >
> > This is my for foray into DAS, so please be gentle :)  Apologies if the
> > cross-post is inappropriate, let me know.
> >
> > I'm looking to build a SAS array with a Dell MD1420.  It will be attached
> > via a SAS 9300-414E HBA in a Dell R330.
> >
> > I've verified that the server / SAS card should be fine, but just wanted
> to
> > know if anybody has set this up before, or if there's any obvious reason
> > why it wouldn't work.  Dell can be a bit weird sometimes.
> >
> > Any information is most welcome.
>
> In theory it should be fine.  I haven't used a Dell enclosure recently, but
> from posts here, it seems that they like to turn on Type 2 Protection
> Information on the drives.  (Assuming you buy the drives from Dell.)
>
> If they do that, you can either reformat, or see the patch here for the
> mpr(4) driver to disable using protection information:
>
> https://docs.freebsd.org/cgi/getmsg.cgi?fetch=308820+0+
> archive/2017/freebsd-scsi/20171022.freebsd-scsi
>
> I haven't gotten around to changing things so that we detect the PI in the
> da(4) driver and use 32 byte CDBs when type 2 protection is enabled.  (So
> if someone else has time and wants to do it, feel free.  It will be
> somewhat messy because we only allocate enough space for 16 byte CDBs
> inside
> the CCB, so you'll need to allocate the CDB separately...)
>
> If you use ZFS, you'll have checksum protection on your data anyway, so PI
> would just be additional protection.
>
> I would also suggest that you update to the latest firmware (currently
> Phase 15) for the SAS controller.  You can do that with LSI's sas3flash or
> the FreeBSD mprutil(8) utility.
>
> Ken
> --
> Kenneth Merry
> k...@freebsd.org
>
Thanks Ken,

I'm going to purchase the system.  I had no idea about PI, that's a nice
feature for sure.

This will probably just be iSCSI and NFS4 over a 10Gb NIC (Xen cluster), so
having PI detection is certainly of interest to me.  I'll figure out what I
can do to contribute (code/testing) when things have settled.

Assuming all is well I'll get the 1420 added to the hardware compatibility
list.

Much appreciated -- lee
___
freebsd-hardware@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"

Does anybody have experience with a MD1420, mpr(4)

[Lee Brown]

Hi,

This is my for foray into DAS, so please be gentle :)  Apologies if the
cross-post is inappropriate, let me know.

I'm looking to build a SAS array with a Dell MD1420.  It will be attached
via a SAS 9300-414E HBA in a Dell R330.

I've verified that the server / SAS card should be fine, but just wanted to
know if anybody has set this up before, or if there's any obvious reason
why it wouldn't work.  Dell can be a bit weird sometimes.

Any information is most welcome.

Thanks -- lee
___
freebsd-hardware@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"