Re: Request for Recommendations: media / transcode server

2019-11-14 Thread Josh Paetzel


> 
> On Nov 14, 2019, at 12:40 AM, Ravi Pokala  wrote:
> 
> Hi all,
> 
> For a few years, I've known that my media server's motherboard[1] was 
> potentially vulnerable to a bricking issue[2]. It looks like that day has 
> come. :-/
> 
> Or it might just be "normal" old-age, since I've had the board for ~4.5 years.
> 
> Either way, I find myself in need of a new motherboard. My requirements are:
> 
> - 2C/4T amd64 (4C/8T preferred)
> - 2x DDR4 slots (4x preferred)
> - 6x SATA ports (8x preferred)
> - 2x 1GbE NICs
> - 1x 80mm M.2 slot (2x preferred, 110mm preferred)
> - 1x PCIe x8 slot (2x preferred)
> - BMC w/ dedicated NIC
> 
> Suggestions?
> 
> Thanks,
> 
> Ravi (rpokala@)
> 
> [1] 
> https://www.amazon.com/Supermicro-A1SAM-2550F-Micro-Processor-Motherboard/dp/B00IRHNKKK/
> [2] 
> https://www.anandtech.com/show/0/semi-critical-intel-atom-c2000-flaw-discovered
> 
> 
> _

The bad news.

No one is making server class boards off the current gen SoC based x86 stuff 
like they did with the atom c2000 based stuff.

If you look for J4000/J5000 based boards all there is are media server boards 
with no BMC, 1 SODIMM slot, two SATA ports, and a single PCI-e slot. 

You’ll need to step up to something like a SM X11SPA which is a lot of board to 
get what you want.

Supposedly the ARM ecosystem was going to fill that void (or pushed x86 out 
depending on who you ask) but I don’t really know if that happened. 

Thanks,

Josh Paetzel
___
freebsd-hardware@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"


Re: LSI/AVGO/Broadcom 9280-16i4e support?

2019-09-03 Thread Josh Paetzel



On Tue, Sep 3, 2019, at 8:13 AM, Ireneusz Pluta/wp.pl wrote:
> W dniu 2019-09-03 o 01:25, Josh Paetzel pisze:
>> The closest you can get to pass through is a bunch of single drive RAID 0 
>> disks with the caching disabled.

> The 9280-16i4e supports JBOD mode.


> Consider:


> MegaCli -AdpSetProp -EnableJBOD -val -aN|-a0,1,2|-aALL
>  val - 0=Disable JBOD mode.
>  1=Enable JBOD mode


> MegaCli -PDMakeJBOD -PhysDrv[E0:S0,E1:S1,...] -aN|-a0,1,2|-aALL


> and:


> $ grep mfi /boot/loader.conf 
> mfip_load="YES"
> hw.mfi.allow_cam_disk_passthrough=1


> (however, not sure if mfip_load is really necessary :-).


> The above works for me in a 11.2-RELEASE machine. However this is a 
> playground machine, used mainly for smoke-testing disks, not for serious 
> production tasks.

> 


Well, you are right about that.

mfip gives you access to SMART. mrsas doesn't have an analogue to that.

--

Thanks,

Josh Paetzel

___
freebsd-hardware@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"


Re: LSI/AVGO/Broadcom 9280-16i4e support?

2019-09-02 Thread Josh Paetzel




> On Sep 1, 2019, at 4:03 PM, Karl Denninger  wrote:
> 
>> On 9/1/2019 15:17, Chris wrote:
>> Hello all,
>> I recently picked up an LSI/AVGO/Broadcom 9280-16i4e card with
>> the intent of flashing it to IT mode (pass through). So as to
>> use it in one of my servers.
>> However; looking through the Hardware (support) section of the
>> release notes for 12; the closest I found was the 9260.
>> Does anyone have any experience with the 9280-16i4e on FreeBSD?
>> Does it work? Well? If not, anytime soon?
>> 
>> Thank you very much.
>> 
>> Chris
>> 
> It will PROBABLY work.
> 
> From Avago: http://www.ssdworks.com/MegaRAID-SAS-9280-16i4e.asp
> 
> LSISAS2108 6Gb/s RAID-on-Chip
> 
> From the driver in sys/dev/mps for 12-STABLE at present:
> 
> mps_pci.c:  { MPI2_MFGPAGE_VENDORID_LSI, MPI2_MFGPAGE_DEVID_SAS2108_1,
> mps_pci.c:  0x, 0x, 0, "Avago Technologies (LSI) SAS2108" },
> mps_pci.c:  { MPI2_MFGPAGE_VENDORID_LSI, MPI2_MFGPAGE_DEVID_SAS2108_2,
> mps_pci.c:  0x, 0x, 0, "Avago Technologies (LSI) SAS2108" },
> mps_pci.c:  { MPI2_MFGPAGE_VENDORID_LSI, MPI2_MFGPAGE_DEVID_SAS2108_3,
> mps_pci.c:  0x, 0x, 0, "Avago Technologies (LSI) SAS2108" },
> 
> I wouldn't call this dispositive but it's easy enough to figure out
> (stick it in a box and boot the install media; see if it shows up during
> the probe.)
> 
> -- 
> Karl Denninger
> k...@denninger.net <mailto:k...@denninger.net>
> /The Market Ticker/
> /[S/MIME encrypted email preferred]/

That card works on FreeBSD. I have personal experience with it. However 
flashing it to IT mode isn’t possible. That’s a RoC not an HBA.  The closest 
you can get to pass through is a bunch of single drive RAID 0 disks with the 
caching disabled.

Thanks,

Josh Paetzel
FreeBSD - The Power to Serve
___
freebsd-hardware@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"


Re: When is sataIII actually sataIII?

2018-11-01 Thread Josh Paetzel



On Thu, Nov 1, 2018, at 8:45 AM, spaml...@mail-on.us wrote:

> I have another Sata 3 drive on the second Sata 3 port, that FreeBSD
> actually treats as what it is:
> ada1 at ahcich1 bus 0 scbus1 target 0 lun 0
> ada1:  ACS-2 ATA SATA 3.x device
> ada1: Serial Number W1F55VT9
> ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)
> ada1: Command Queueing enabled
> ada1: 2861588MB (5860533168 512 byte sectors)
> ada1: quirks=0x1<4K>
> 
> note the "quirks" - is that good, or bad?
> 
> In the end, it appears that I have to purchase a slower rated (Sata 3) 
> drive
> to get FreeBSD to treat it as a Sata 3 drive? It makes no sense to me. 
> Which
> is why I came here; in hopes of finding out *why* it appears as it does. 
> :)
> 
> Thanks again, Frank!
> 
> Oh! In case it matters; this is on 12 (CURRENT)
> 
> -- 
> Chris out...
> ___
> freebsd-hardware@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
> To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"

A full dmesg would be useful.  A dmesg from a verbose boot even more useful.
As well as the make and model of the motherboard.

The most common problem is the controller the drive is plugged into isn't 
actually capable of SATA 3.

-- 

Thanks,

Josh Paetzel
___
freebsd-hardware@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"


Re: Dell SAS6IR is it worth the effort to convert to IT

2018-09-13 Thread Josh Paetzel



On Thu, Sep 13, 2018, at 10:58 AM, Lee Brown wrote:
> Hi,
> 
> I'm in the process of setting up an old PowerEdge R410 which has, I
> believe, the LSI Logic AS1068, using the mpt driver.
> 
> I'm not interested in running it as a hardware RAID but rather a
> geli(authentication only)+gmirror+gjournal.
> 
> It seems like flashing it requires booting into dos, which frankly I can't
> be bothered with.  mptutil doesn't appear to have that capability.
> 
> Are there any major downsides to running it in IR mode as two disks?
> There's no option to turn off the write-cache, but being battery backed the
> only danger I see with that is if the machine is powered with pending
> writes on the card and the battery gives out before power is restored.
> 
> The use case is routing/snort IPS with minimal logging, so disk performance
> isn't critical.  It will be the redundant router so mostly doing nothing
> except the little work OSPF needs.
> 
> Thanks in advance for any comments -- lee
> ___
> freebsd-hardware@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
> To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"

On those controllers IR mode is just IT mode with the ability to create RAID.  
If you don't create a RAID array it behaves exactly like an IT controller.

And yes, FreeBSD mptutil doesn't have the ability to erase the flash, which you 
need to be able to do to go from IR to IT.  DOS is the typical answer for that.

-- 

Thanks,

Josh Paetzel
___
freebsd-hardware@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"


Re: Problem with FreeBSD 11.2 and LSI 9341-8i

2018-09-08 Thread Josh Paetzel



On Sat, Sep 8, 2018, at 1:07 PM, Josh Endries wrote:
> I just got an LSI 9341-8i card and I'm having some trouble getting it to
> work in a new install of 11.2-RELEASE. I'm trying to replace an old 9.x
> FreeNAS box with a new machine. I have one drive connected for testing. I
> haven't set anything up, it's just JBOD (which is what I want anyway). The
> BIOS seems to see the drive and controller, and in FreeBSD when I boot I
> see this:
> 
> AVAGO MegaRAID SAS FreeBSD mrsas driver version: 06.712.04.00-fbsd
> mfi0:  port 0xd000-0xd0ff mem
> 0xfe72-0xfe72,0xfe60-0xfe6f irq 19 at device 0.0 on pci2
> mfi0: Using MSI
> mfi0: Megaraid SAS driver Ver 4.23
> mfi0: FW MaxCmds = 240, limiting to 128
> mfi0: MaxCmd = 240, Drv MaxCmd = 128, MaxSgl = 70, state = 0xbf3c00f0
> mfi0: Init command Failed 0x3
> mfi0: TB Init has failed with error 3
> device_attach: mfi0 attach returned 3
> 
> I haven't found anything on Google. I read on the mfi man page (I think)
> that I should enable mrsas for these cards, so I added
> hwmfi.mrsas_enable="1" to /boot/device.hints, but that didn't help. I have
> nothing related in /dev, no disk or even a controller node, so I can't
> update the firmware or do anything else... I'm not sure what else to try.
> 
> For now I'm going to try 10.x instead and see if that works, though I would
> of course rather use the latest version.
> 
> Josh
> ___
> freebsd-hardware@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
> To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"


mfi(4) doesn't support the 9341, however mrsas(4) does.

Is the snippet of dmesg you pasted after you added hwmfi.mrsas_enable="1" to 
/boot/device.hints?

If so perhaps that isn't just a typo in your email, but rather a typo in your 
device.hints

The correct line is hw.mfi.mrsas_enable="1"

If after activating the mrsas driver it's still not working please attach your 
dmesg and output of pciconf -lv



-- 

Thanks,

Josh Paetzel
___
freebsd-hardware@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"


Re: Recommendations for cheap PCI-E network adapter ?

2018-01-02 Thread Josh Paetzel


On Tue, Jan 2, 2018, at 2:38 PM, Ronald F. Guilmette wrote:
> 
> I need to buy a PCI-E ethernet card.  It won't really matter if it
> is 10/100/1000 or just 10/100 but it has to work with FreeBSD at a
> minimum.  It would be Nice if it was also supported by Linux and
> Windoze7, but that isn't really critical.
> 
> I'm a serious cheapskate, so I'd like to spend as little as possible.
> I don't need anything super-deluxe.  Whatever is cheapest will be fine,
> even if the performance is only so-so.
> 
> Recommendations would be appreciated.
> 
> 
> P.S.  The small amount of research I just now did suggests that Realtek
> based cards should be avoided, but one reviewer said that the Rosewill
> RC-411v3 works just fine on Ubuntu, so I'm not sure what to think about
> Realtek-based cards now.  The price is right (for me) on the Rosewill
> RC-411v3, but various online threads (relating to Realtek chips) give
> me pause...
> 
> https://forums.freebsd.org/threads/60033/
> 
> https://forums.freebsd.org/threads/55861/
> 
> I really can't see blowing fifty bucks on a simple, low-end ethernet card,
> but everything inexpensive seems to be Realtek-based. :-(
> ___
> freebsd-hardware@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
> To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"

https://www.neweggbusiness.com/product/product.aspx?item=9b-33-106-033

-- 

Thanks,

Josh Paetzel
___
freebsd-hardware@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"


Re: Do I need SAS drives?..

2017-08-09 Thread Josh Paetzel


On Wed, Aug 9, 2017, at 09:55 AM, Frank Leonhardt (m) wrote:
> Simple answer is to use either. You're running FreeBSD with ZFS, right?
> BSD will hot plug anything. I suspect 'hot plug' relates to Microsoft
> workaround hardware RAID.
> 
> Hot plug enclosures will also let the host know a drive has been pulled.
> Otherwise ZFS won't know whether it was pulled or is unresponsive due to
> it being on fire or something. With 8 drives in your array you can
> probably figure this out yourself.
> 
> SAS drives use SCSI commands, which are supposedly better than SATA
> commands. Electrically they are the same. SAS drives are more expensive
> and tend to be higher spec mechanically, but not always so. Incidentally,
> nearline SAS is a cheaper SATA drive that understands SAS protocol and
> has dual ports. Marketing.
> 
> Basically, if you really want speed at all costs go for SAS. If you want
> best capacity for your money, go SATA. If in doubt, go for SATA. If you
> don't know you need SAS for some reason, you probably don't.
> 
> Regards, Frank.
> 
> 
> On 9 August 2017 15:27:37 BST, "Mikhail T." <m...@aldan.algebra.com>
> wrote:
> >My server has 8 "hot-plug" slots, that can accept both SATA and SAS
> >drives. SATA ones tend to be cheaper for the same features (like
> >cache-sizes), what am I getting for the extra money spent on SAS?
> >
> >Asking specifically about the protocol differences... It would seem,
> >for example, SATA can not be as easily hot-plugged, but with
> >camcontrol(8) that should not be a problem, right? What else? Thank
> >you!
> >-- 

I have a different take on this.  For starters SAS and SATA aren't
electrically compatible.  There's a reason SAS drives are keyed so you
can't plug them in to a SATA controller.  It keeps the magic smoke
inside the drive.  SAS controllers can tunnel SATA (They confusingly
call this STP (Not Spanning Tree Protocol, but SATA Tunneling Protocol) 
It's imperfect but good enough for 8 drives.  You really do not want to
put 60 SATA drives in a SAS JBOD)

SAS can be a shared fabric, which means a group of drives are like a
room full of people having a conversation.  If someone starts screaming
and spurting blood it can disrupt the conversations of everyone in the
room.  Modern RAID controllers are pretty good at disconnecting drives
that are not working properly but not completely dead.  Modern HBAs not
so much.  If your controller is an HBA trying to keep a SAS fabric
stable with SATA drives can be more problematic than if you use SAS
drives...and as Frank pointed out nearline SAS drives are essentially
SATA drives with a SAS interface (and are typically under a $20 premium)

If performance was an issue we'd be talking about SSDs.  While SAS
drives do have a performance advantage over SATA in
multiuser/multiapplication environments (they have a superior queuing
implementation) it's not worth considering when the real solution is
SSDs.

My recommendation is if you have SAS expanders and an HBA use SAS
drives.  If you have direct wired SAS or a RAID controller you can use
either SAS or SATA.  If your application demands performance or
concurrency get a couple SSDs.  They'll smoke anything any spinning
drive can do.

-- 

Thanks,

Josh Paetzel
___
freebsd-hardware@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"


Re: Server performance & enhancement

2017-02-16 Thread Josh Paetzel


On Thu, Feb 16, 2017, at 05:20 AM, Karl Denninger wrote:
> I'm wondering what people think of a forward path for the following:
> 
> 
> CPU: Intel(R) Xeon(R) CPU   E5620  @ 2.40GHz (2400.14-MHz
> K8-class CPU)
>   Origin="GenuineIntel"  Id=0x206c2  Family=0x6  Model=0x2c  Stepping=2
>  
> Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
>  
> Features2=0x29ee3ff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AESNI>
>   AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
>   AMD Features2=0x1
>   VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
>   TSC: P-state invariant, performance statistics
> real memory  = 25773998080 (24580 MB)
> avail memory = 24915828736 (23761 MB)
> Event timer "LAPIC" quality 600
> ACPI APIC Table: <111612 APIC1749>
> FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs
> FreeBSD/SMP: 2 package(s) x 4 core(s) x 2 hardware threads
> 
> The system board is a Supermicro and has embedded IPMI (which I require
> for remote admin purposes) on a separate interface.  It's plenty fast
> but very power-hungry.
> 
> I'm interested in other's experience with various system board options,
> and processors, that will return /better /performance and which have a
> good history running FreeBSD /but with lower power consumption.
> 
> /Any suggestions welcome.
> 
> -- 
> Karl Denninger
> k...@denninger.net <mailto:k...@denninger.net>
> /The Market Ticker/
> /[S/MIME encrypted email preferred]/
> Email had 1 attachment:
> + smime.p7s
>   4k (application/pkcs7-signature)


You could run a single E5-2640v4 CPU which is 10 cores at 2.4Ghz and
90watts.

Dual 5620's are 8 cores at 2.4Ghz and 170watts.

** E5 v4 is faster clock for clock than Westmere, so the 2.4Ghz isn't
equivalent between the two CPUs.

It's somewhere between 2x and 3x more computing power than dual Westmere
5620's, and uses nearly half the power at full load.  In practice it
will use far less than half the power as E5 is much better at powering
down idle cores and so forth than Westmere was.  Not to mention single
socket, which will reduce your platform cost and power draw there. 
There's an additional win, in that FreeBSD doesn't scale well with
multi-package.

As far as motherboards there are a lot of choices,
https://www.supermicro.com/products/nfo/Xeon_X10_E5.cfm?pg=MB=SELECT=UP#List_MBD
shows a bunch of different PCI and SAS/SATA port and memory
configurations. Something in the X10SR family should work well for you.

That's the "less than half the power draw" option.  You can go with
something that has 1/4 the power draw of your current setup, but it will
likely be an incremental performance improvement (~25%)

-- 

Thanks,

Josh Paetzel
___
freebsd-hardware@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-hardware
To unsubscribe, send any mail to "freebsd-hardware-unsubscr...@freebsd.org"


Re: L2 cache errors???

2015-07-28 Thread Josh Paetzel


On 07/28/2015 13:40, Willem Jan Withagen wrote:
 On 28/07/2015 19:48, Mike Tancsa wrote:
 On 7/28/2015 1:16 PM, Willem Jan Withagen wrote:
 Hi,

 Are these what I think they are?
 Errors in the CPU L2 cache?

 Are the ECC corrected?
 Or is error really data kaput?



 Could be. There is also an erratum issue that triggers these errors on
 certain CPUs when running software like virtualbox.  It was fixed in
 RELENG_10 some time ago. What are you running ?


 https://svnweb.freebsd.org/base?view=revisionrevision=269052

 has some details.
 
 'mmm,
 Not running Haswell stuff, but rather older hardware.
 
 Looked in older logfiles, and there are a few more...
 All with the same data, except that it is detected on different CPUs
 
 And it occurs when running:
   mbuffer -4 -m 1000M -I  | \
 zfs receive -F -d -v zfs
 to receive a full backup from my fileserver.
 
 --WjW
 

You can tell ECC corrected the error because on FreeBSD if ECC can't fix
the error the system will panic.  Other systems (Solaris and HP-UX being
the two I have direct experience with) can detach subsystems that have
sustained uncorrectable errors in some cases. (Yes, even CPUs!)

If a system is generating hundreds or thousands of MCAs a minute you are
dealing with a hardware issue.

If you are getting spurious MCAs to the tune of a few a day there's
nothing abnormal or broken there it's just the system doing what it's
supposed to.

Given the amount of data that flies around inside modern computers I'm
surprised there aren't more MCAs than there are in most systems.


-- 
FreeBSD - The Power To Serve.
___
freebsd-hardware@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hardware
To unsubscribe, send any mail to freebsd-hardware-unsubscr...@freebsd.org


Re: Working and Supported SCSI Controller

2015-06-26 Thread Josh Paetzel
An adaptec 2940 isn't a U160 controller.

The adaptec 19160/29160/39160 are, but I haven't used them since FreeBSD 4.x. 
No idea if they still work.

Thanks,

Josh Paetzel

 On Jun 26, 2015, at 10:54 PM, Bernt Hansson b...@bananmonarki.se wrote:
 
 On 2012-05-24 01:02, vermaden wrote:
 Hi,
 
 as HARDWARE NOTES are next to useless:
 http://lists.freebsd.org/pipermail/freebsd-questions/2012-May/241164.html
 
 What *working* PCI/PCI-X SCSI controller do You guys suggest?
 
 Requirements: PCI/PCI-X Ultra160 or Ultra320 with one or more 68-pin 
 internal connector
 
 Thanks in advance for Your suggestions,
 vermaden
 Adaptech 2940
 ___
 freebsd-hardware@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-hardware
 To unsubscribe, send any mail to freebsd-hardware-unsubscr...@freebsd.org
___
freebsd-hardware@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hardware
To unsubscribe, send any mail to freebsd-hardware-unsubscr...@freebsd.org


Re: Support for Fusion IO drives?

2012-08-28 Thread Josh Paetzel
  Original Message 
 Subject: Support for Fusion IO drives?
 Date: Tue, 28 Aug 2012 16:46:00 -0400
 From: Andy Young ayo...@mosaicarchive.com
 To: freebsd-hardware@freebsd.org


 We are investigating adding SSDs as ZIL devices to boost our ZFS write
 performance. I read an article a while ago about iX Systems teaming up
 with
 Fusion IO to integrate their hardware with FreeBSD. Does anyone know
 anything about supported drivers for Fusion IO's iodrives?

 Thanks!

 Andy
 ___
 freebsd-hardware@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-hardware
 To unsubscribe, send any mail to
 freebsd-hardware-unsubscr...@freebsd.org





I'll put on my iXsystems hat here, as well as my fast storage, ZFS and
Fusion-I/O hat.

The ZFS filesystem supports dedicated ZIL devices, which can accelerate
certain types of write requests, notably related to fsync.  The VMWare
NFS client issues a sync with every write, and most databases do as
well.  In those types of environments having a fast dedicated ZIL device
is almost essential.  In other environments the benefits of a dedicated
ZIL range from non-existent to substantial.

A good dedicated ZIL device is all about latency.  It doesn't need to be
large, in fact it will only ever handle 10 seconds of writes, so 10x
network bandwidth is worst case. (In most environments this means 20GB
is larger than needed).

Fusion-I/O cards are far too large to be cost effective ZIL devices.
Even though they do rock at I/O latency, the really fast ones are also
fairly large, so the $/GB on them isn't so attractive.  There are better
options for ZIL devices.

Another consideration is the Fusion-I/O driver is fairly memory hungry,
which competes with memory ZFS wants to use for read caching.

Now as an L2ARC device, that's a whole different can of worms.

Command line used: iozone -r 4k -s 96g -i 0 -i 1 -i 2 -t 8
Parent sees throughput for  8 readers   = 1712399.95 KB/sec
L2 ARC Breakdown:   197.45m
Hit Ratio:  98.61%  194.71m
L2 ARC Size: (Adaptive) 771.13  GiB
ARC Efficiency: 683.40m
Actual Hit Ratio:   71.09%  485.82m

~ 800GB test data, all served from cache.

If you are considering Fusion-I/O, the FreeBSD driver is generally not
released to the general public by Fusion-I/O, but can be obtained from
various partners. (I believe iXsystems is the only FreeBSD friendly
fusion-i/o partner but could be wrong about that)


-- 
Thanks,

Josh Paetzel

___
freebsd-hardware@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hardware
To unsubscribe, send any mail to freebsd-hardware-unsubscr...@freebsd.org