On 05/09/2020 15:19, Mark Kettenis wrote:
>> From: Daniel Pocock <[email protected]>
>> Date: Sat, 5 Sep 2020 14:37:13 +0200
>>
>> On 05/09/2020 13:26, Mark Kettenis wrote:
>>>> From: Daniel Pocock <[email protected]>
>>>> Date: Fri, 4 Sep 2020 18:32:20 +0200
>>>
>>> Hi Daniel,
>>>
>>>>
>>>> On 04/09/2020 18:22, Mark Kettenis wrote:
>>>>
>>>>> Seriously though, I have an idea why this happens; I'll see if I can
>>>>> come up with a quick fix.
>>>>
>>>> Great, I'll be happy to test it and give some feedback
>>>
>>> I committed a quick fix that (if my calculations are right) should
>>> allow you to boot with 128G of RAM (but probably not with 256G). The
>>> latest snapshot probably already has those fixes. But if you get the
>>> same output, please try again in a day or so.
>>>
>>> In case you're interested, the problem is that the kernel virtual
>>> address space is somewhat limited in the current implementation (4G,
>>> which I now bumped to 6G). With 128G a large fraction of that space
>>> gets used for the data structures that describe the memory pages. I
>>> need to expand the kernel virtual address space further but that is
>>> a bit more work.
>>
>> Thanks for getting on to that so quickly. I will probably only be able
>> to test again at the end of the week, Friday, then I'll give some more
>> feedback.
>>
>>
>>>> It is a new build so I'll be trying a few different things on it before
>>>> putting it into serious use. Is there anything else that would be
>>>> useful for me to exercise in OpenBSD on Talos II?
>>>
>>> It is still early days for OpenBSD/powerpc64 so any testing you can do
>>> is welcome. If you end up testing additonal PCIe hardware in the
>>> machine let me know and I can provide some hints for adding the
>>> necessary drivers.
>>
>> The cards I'm likely to test this week:
>>
>> HBA: LSI SAS 9207-4i4e
>
> Should be supported by mpii(4). I'll pre-emptively add the driver to
> the kernel.
>
Great, it is a very simple card that is useful for legacy SATA SSDs
>> GPU: Sapphire Radeon RX 580 Pulse
>
> I'm working on the amdgpu(4) driver to support this hardware, but I
> first need to imrpove the IOMMU support, so this is not hoing to work
> yet.
OK, I can test with the built-in VGA for now
Is there any other GPU that may already be supported now?
>> I have a couple of older NVIDIA GPUs lying about too, for example,
>> Quadro K2200. I might test them if I have some extra time.
>
> No NVIDIA GPU support in OpenBSD.
OK, I'll skip that one.
>> NVMe: I need to buy a more modern HBA or OCuLink adapter to run a pair
>> of U.2 SSDs together with some legacy SATA SSDs and HDDs. An
>> alternative is to just put in a dual-OCuLink card for the U.2 drives and
>> use my existing 9207-4i4e for the legacy drives. If you can make any
>> suggestions which card or combination of cards to consider that would be
>> helpful, especially if they are cards that other OpenBSD users have
>> already validated on other architectures.
>
> The nvme(4) driver supports NVMe drivers, so I suppose OCuLink will
> just work. I think mpii(4) supports the Broadcom HBA 9400 series
> which supports SAS/SATA/NVMe. As far as I can tell there is no
> support for the HBA 9500 series yet, which is of course what you'd
> want on a machine that supports PCIe 4.0.
>
> The mpii(4) driver is widely used by OpenBSD users and is probably
> your safest bet overall.
OK, thanks for that feedback.
The Broadcom user guide mentions that 9500-16i works on FreeBSD already,
is that a path to OpenBSD support?
You are correct in estimating my enthusiasm for PCIe 4.0. Personally, I
have the dual-CPU model with 5 slots. As many people only chose the
motherboard with 2 slots (Talos II Lite or Blackbird), it is important
to squeeze as much value as possible out of each slot.
The smallest slot in these boards is a PCIe 4.0 x8 slot: that can handle
16 GB/second. Newer and faster[2] U.3 SSDs appear to be offering up to
6.2 GB/s and more common/affordable U.2 models offer 2-3 GB/s now.
Therefore, for somebody who does RAID-1 in their workstation, it should
be possible to get the full benefit of both drives reading or writing
simultaneously at full speed and manage a few legacy drives: all using
the same slot.
Regards,
Daniel
1. https://en.wikipedia.org/wiki/PCI_Express#History_and_revisions
2. https://business.kioxia.com/en-emea/ssd/data-center-ssd/cd6-v.html
