Re: [OmniOS-discuss] new supermicro server
So quick google on "razor cards" came up pretty empty(lots of using a razor blade on your cpu).. I will assume that is some sort of switch? pci switch? Does latency stay in tact? Ok so I read the first part of the manual.. It looks like it has 4 nvme ports via OCuLink, which if I am reading correctly will have/need 4 lanes of PCI Express 4? and a dual sas backplane for like the rest. OCuLink driver in Omni? First pass on Oculink looks pretty cool. 8T I am looking at one of those for a project so I will read and report back :) lk On 3/8/17 1:32 AM, Ergi Thanasko wrote: No LSI controller on those, they connect directly to the CPU with razor cards. In theory they can utilize the entire bus speed but in the backplane you get around 2GB/sec throughout on each nvme drive. Sent from my iPhone On Mar 7, 2017, at 11:25 PM, Stephan Budach wrote: - Ursprüngliche Mail - Von: "Dan McDonald" An: "Geoff Nordli" CC: "omnios-discuss" Gesendet: Mittwoch, 8. März 2017 07:44:30 Betreff: Re: [OmniOS-discuss] new supermicro server You didn't mention anything in the original note about NVMe, and neither did the spec sheet. NVMe 1.0 and 1.1 should work on 020 and later OmniOS. Hang out on the illumos developers list to see the latest on that front. Dan Sent from my iPhone (typos, autocorrect, and all) In regards of NVMe, I am eyeing this particular Supermicro and I intend to get my hands on two of those in Q2: SuperMicro SuperServer SSG-2028R-NR48N I think these will bring a lot of fun to the table. ;) Cheers, Stephan ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] new supermicro server
On Wed, 8 Mar 2017, kevin.s...@colostate.edu wrote: It looks like this MB may have an onboard SAS controller. If you're using that, there may be a similar option somewhere in the system BIOS. I am using a plug-in HBA and direct-connect chassis with 16 channels. I think it is a Broadcom/Avago SAS 9300-16i This controller is definitely not a bottleneck for the 16 SAS drives. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] new supermicro server
On Wed, Mar 8, 2017 at 8:36 AM, Bob Friesenhahn < bfrie...@simple.dallas.tx.us> wrote: > > > Perhaps there is a way to tell the HBA BIOS to not advertize the SAS > drives which are not needed for booting? In the HBA BIOS configuration, set the HBA to disabled. OmniOS will still see the HBA and disks, but the BIOS will not want to list all the disks. You only need it enabled if you boot from a disk attached to the HBA. -Chip ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] new supermicro server
We haven't used this particular system, but we had the same problem with SuperMicro Gen 9 motherboards. The solution for us on those systems was to disable BIOS loading for the particular PCI slot the SAS HBA was installed in. This was a system BIOS setting, rather than an HBA BIOS setting. It looks like this MB may have an onboard SAS controller. If you're using that, there may be a similar option somewhere in the system BIOS. We opened a support case on this issue with SuperMicro via our vendor, they concluded that there was an address space limitation with the BIOS boot process, and suggested that an HBA capable of UEFI boot might resolve the problem. Unfortunately, our LSI2308 HBAs were not UEFI capable, so we were unable to test that theory. If you're currently using BIOS boot mode, it might be worthwhile to try switching to UEFI boot mode to see if that helps. Kevin On 03/08/2017 07:36 AM, Bob Friesenhahn wrote: > On Tue, 7 Mar 2017, Geoff Nordli wrote: > >> Hi. >> >> I am looking at ordering a new 3U supermicro server. as an all-in >> one. I have been using these recently: >> >> https://www.supermicro.com/products/system/3U/6038/SSG-6038R-E1CR16L.cfm >> >> It has the LSI 3008 HBA in IT mode. > > I have one of these. It runs OmniOS very well. Mine has four Intel > SATA SSDs, two for boot, and two for SLOG. Two of the SSDs are fixed > internal and two are accessible via optional rear slots. > > However, I am encountering BIOS issues with booting. The drives > advertized by the HBA overwelm the BIOS so that it does not list > internal SATA drives for booting and currently my system only boots from > internal SATA drives if the Intel network boot agent runs. The BIOS > behavior is unstable. > > This problem did not appear to exist when the system was delivered to me > but appeared as soon as I tried to statically configure the BMC IP address. > > Perhaps there is a way to tell the HBA BIOS to not advertize the SAS > drives which are not needed for booting? > > Bob -- --- Kevin Swab UNIX Systems Administrator ACNSColorado State University Phone: (970)491-6572Email: kevin.s...@colostate.edu GPG Fingerprint: 7026 3F66 A970 67BD 6F17 8EB8 8A7D 142F 2392 791C ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] new supermicro server
SuperMicro SuperServer SSG-2028R-NR48N I have been looking at those too.. Maybe someone can help me understand how you can have nvme on a sata bus? I thought the benefit or primary function of nvme was that it was low latency and sits on the pci bus? It looks like these are nvme modules with sata/sas attachments On 3/8/17 1:25 AM, Stephan Budach wrote: - Ursprüngliche Mail - Von: "Dan McDonald" An: "Geoff Nordli" CC: "omnios-discuss" Gesendet: Mittwoch, 8. März 2017 07:44:30 Betreff: Re: [OmniOS-discuss] new supermicro server You didn't mention anything in the original note about NVMe, and neither did the spec sheet. NVMe 1.0 and 1.1 should work on 020 and later OmniOS. Hang out on the illumos developers list to see the latest on that front. Dan Sent from my iPhone (typos, autocorrect, and all) In regards of NVMe, I am eyeing this particular Supermicro and I intend to get my hands on two of those in Q2: SuperMicro SuperServer SSG-2028R-NR48N I think these will bring a lot of fun to the table. ;) Cheers, Stephan ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] new supermicro server
I have been using the 6037R-E1R16L for several of my builds for archive, but the 6038 is also a nice frame. It always depends on what you are doing with it. lk On 3/7/17 7:21 PM, Dan McDonald wrote: On Mar 7, 2017, at 8:16 PM, Geoff Nordli wrote: Hi. I am looking at ordering a new 3U supermicro server. as an all-in one. I have been using these recently: https://www.supermicro.com/products/system/3U/6038/SSG-6038R-E1CR16L.cfm It has the LSI 3008 HBA in IT mode. Any other suggestions out there? The one you mention is pretty good, especially if you want 2 + 16 drives (2x2.5", 16x3.5") online. Unless you want something smaller, I can think of much worse ways to spend your money. Dan ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] new supermicro server
On Tue, 7 Mar 2017, Geoff Nordli wrote: Hi. I am looking at ordering a new 3U supermicro server. as an all-in one. I have been using these recently: https://www.supermicro.com/products/system/3U/6038/SSG-6038R-E1CR16L.cfm It has the LSI 3008 HBA in IT mode. I have one of these. It runs OmniOS very well. Mine has four Intel SATA SSDs, two for boot, and two for SLOG. Two of the SSDs are fixed internal and two are accessible via optional rear slots. However, I am encountering BIOS issues with booting. The drives advertized by the HBA overwelm the BIOS so that it does not list internal SATA drives for booting and currently my system only boots from internal SATA drives if the Intel network boot agent runs. The BIOS behavior is unstable. This problem did not appear to exist when the system was delivered to me but appeared as soon as I tried to statically configure the BMC IP address. Perhaps there is a way to tell the HBA BIOS to not advertize the SAS drives which are not needed for booting? Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] new supermicro server
No LSI controller on those, they connect directly to the CPU with razor cards. In theory they can utilize the entire bus speed but in the backplane you get around 2GB/sec throughout on each nvme drive. Sent from my iPhone > On Mar 7, 2017, at 11:25 PM, Stephan Budach wrote: > > > > - Ursprüngliche Mail - >> Von: "Dan McDonald" >> An: "Geoff Nordli" >> CC: "omnios-discuss" >> Gesendet: Mittwoch, 8. März 2017 07:44:30 >> Betreff: Re: [OmniOS-discuss] new supermicro server >> >> You didn't mention anything in the original note about NVMe, and >> neither did the spec sheet. >> >> NVMe 1.0 and 1.1 should work on 020 and later OmniOS. Hang out on the >> illumos developers list to see the latest on that front. >> >> Dan >> >> Sent from my iPhone (typos, autocorrect, and all) >> > > In regards of NVMe, I am eyeing this particular Supermicro and I intend to > get my hands on two of those in Q2: > > SuperMicro SuperServer SSG-2028R-NR48N > > I think these will bring a lot of fun to the table. ;) > > Cheers, > Stephan > ___ > OmniOS-discuss mailing list > OmniOS-discuss@lists.omniti.com > http://lists.omniti.com/mailman/listinfo/omnios-discuss ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] new supermicro server
- Ursprüngliche Mail - > Von: "Dan McDonald" > An: "Geoff Nordli" > CC: "omnios-discuss" > Gesendet: Mittwoch, 8. März 2017 07:44:30 > Betreff: Re: [OmniOS-discuss] new supermicro server > > You didn't mention anything in the original note about NVMe, and > neither did the spec sheet. > > NVMe 1.0 and 1.1 should work on 020 and later OmniOS. Hang out on the > illumos developers list to see the latest on that front. > > Dan > > Sent from my iPhone (typos, autocorrect, and all) > In regards of NVMe, I am eyeing this particular Supermicro and I intend to get my hands on two of those in Q2: SuperMicro SuperServer SSG-2028R-NR48N I think these will bring a lot of fun to the table. ;) Cheers, Stephan smime.p7s Description: S/MIME cryptographic signature ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] new supermicro server
You didn't mention anything in the original note about NVMe, and neither did the spec sheet. NVMe 1.0 and 1.1 should work on 020 and later OmniOS. Hang out on the illumos developers list to see the latest on that front. Dan Sent from my iPhone (typos, autocorrect, and all) ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] new supermicro server
On 2017-03-07 09:35 PM, Stephan Budach wrote: - Ursprüngliche Mail - Von: "Dan McDonald" An: "Geoff Nordli" CC: "omnios-discuss" Gesendet: Mittwoch, 8. März 2017 02:21:20 Betreff: Re: [OmniOS-discuss] new supermicro server On Mar 7, 2017, at 8:16 PM, Geoff Nordli wrote: Hi. I am looking at ordering a new 3U supermicro server. as an all-in one. I have been using these recently: https://www.supermicro.com/products/system/3U/6038/SSG-6038R-E1CR16L.cfm It has the LSI 3008 HBA in IT mode. Any other suggestions out there? The one you mention is pretty good, especially if you want 2 + 16 drives (2x2.5", 16x3.5") online. Unless you want something smaller, I can think of much worse ways to spend your money. Dan Yeah - I do have a couple of those, even with the current X10 board and I am really satisfied with those. Just throwed in another Intel 10GbE DP and hooked them up to our Nexus network. Really solid boxes! Cheers, Stephan Thanks Stephan and Dan. One thing I am not sure about is NVMe drives. Are those stable? Geoff ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] new supermicro server
- Ursprüngliche Mail - > Von: "Dan McDonald" > An: "Geoff Nordli" > CC: "omnios-discuss" > Gesendet: Mittwoch, 8. März 2017 02:21:20 > Betreff: Re: [OmniOS-discuss] new supermicro server > > > > On Mar 7, 2017, at 8:16 PM, Geoff Nordli wrote: > > > > Hi. > > > > I am looking at ordering a new 3U supermicro server. as an all-in > > one. I have been using these recently: > > > > https://www.supermicro.com/products/system/3U/6038/SSG-6038R-E1CR16L.cfm > > > > It has the LSI 3008 HBA in IT mode. > > > > Any other suggestions out there? > > The one you mention is pretty good, especially if you want 2 + 16 > drives (2x2.5", 16x3.5") online. > > Unless you want something smaller, I can think of much worse ways to > spend your money. > > Dan > Yeah - I do have a couple of those, even with the current X10 board and I am really satisfied with those. Just throwed in another Intel 10GbE DP and hooked them up to our Nexus network. Really solid boxes! Cheers, Stephan smime.p7s Description: S/MIME cryptographic signature ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
Re: [OmniOS-discuss] new supermicro server
> On Mar 7, 2017, at 8:16 PM, Geoff Nordli wrote: > > Hi. > > I am looking at ordering a new 3U supermicro server. as an all-in one. I > have been using these recently: > > https://www.supermicro.com/products/system/3U/6038/SSG-6038R-E1CR16L.cfm > > It has the LSI 3008 HBA in IT mode. > > Any other suggestions out there? The one you mention is pretty good, especially if you want 2 + 16 drives (2x2.5", 16x3.5") online. Unless you want something smaller, I can think of much worse ways to spend your money. Dan ___ OmniOS-discuss mailing list OmniOS-discuss@lists.omniti.com http://lists.omniti.com/mailman/listinfo/omnios-discuss
