subject:"\[zfs\-discuss\] verging OT\: how to buy J4500 w\/o overpriced drives"

On Tue, Feb 2, 2010 at 9:52 PM, Toby Thain  wrote:

>
> On 2-Feb-10, at 1:54 PM, Orvar Korvar wrote:
>
>  100% uptime for 20 years?
>>
>> So what makes OpenVMS so much more stable than Unix? What is the
>> difference?
>>
>
>
> The short answer is that uptimes like that are VMS *cluster* uptimes.
> Individual hosts don't necessarily have that uptime, but the cluster
> availability is maintained for extremely long periods.
>
> You can probably find more discussion of this in comp.os.vms.

And the 15MB/sec of I/O throughput on that state-of-the-art cluster is
something to write home about? ;)

Seriously, as someone alluded to earlier, we're not comparing apples to
applies. And a 9000 series VAX Cluster was one of the earlier multi-user
systems I worked on for reference ;)

Making that kind of stuff work with modern expectations and tolerances is a
whole new kettle of fish...

-marc
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

2010-02-02 Thread Toby Thain



On 2-Feb-10, at 1:54 PM, Orvar Korvar wrote:


100% uptime for 20 years?

So what makes OpenVMS so much more stable than Unix? What is the  
difference?



The short answer is that uptimes like that are VMS *cluster* uptimes.  
Individual hosts don't necessarily have that uptime, but the cluster  
availability is maintained for extremely long periods.


You can probably find more discussion of this in comp.os.vms.

--Toby


--
This message posted from opensolaris.org
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

2010-02-02 Thread Bob Friesenhahn


On Tue, 2 Feb 2010, Miles Nordin wrote:


"fc" == Frank Cusack  writes:


   fc> by FCoE are you talking about iSCSI?

FCoE is an L2 design where ethernet ``pause'' frames can be sent
specific to one of the seven CoS levels instead of applying to the
entire port, which makes PAUSE abuseable for other purposes than their


Please redirect this encyclopedia contribution to WikiPedia, where it 
belongs.


Thanks,

Bob


former one.  CoS is an L2 priority/QoS tag inside the VLAN header.
Before this hack, pause frames are not useful for congestion
management because they cause head-of-line blocking, so serious
switches only send them in response to backplane congestion, and for
example serious hosts might send them for PCI contention, if clever
enough.  With the hack, the HOL-blocking effect of a PAUSE still
spreads further than you might ideally like but can be constrained to
one of the seven CoS planes in your fabric (probably, the Storage
plane).  This lets you have an HOL-blocking, lossless storage fabric
in parallel with a buffered TCP fabric that is not lossless (uses
packet drops for congestion control like normal Ethernet).  You will
find some squirrely language from FCoE proponents around these issues
because they are trying to convince you that you have every desireable
buzzword in every part of your network, while in fact what you're
doing is making the same wise trade-off that every other non-Ethernet
LAN fabric has always made.  My parallel point is that the
HOL-blocking lossless fabric is *CHEAPER* to create, not nmore
expensive.  It is less capable.  It has no buffers and therefore no
QoS.  It just happens to be what's best for storage.  so, they want
you to pay the prices of a multi-queued QoSed WRED big-buffered
non-blocking fabric suitable for transit traffic even though you
mostly just need to push storage bits: classic upsell, just like all
those ``XL'' PFC's they try to push off to customers who are not even
in the DFZ.

FCoE also includes a bunch of expensive hocus-pokus to bridge these
frames onto a traditional FC-switched network and do a bunch of other
things I don't understand like FC zoning and F-SPF.  Most of the pitch
dwells on this, trying to convince you they've made things ``simpler''
for you because it's once piece of wire.  This seems like an
anti-feature to me: wire's cheap while understanding things is hard,
and now everyone's forced to catch up and learn Fibre Channel before
it's safe to touch anything.  Good in the long run, absolutely.
Cheaper, fuck no.

but the legitimate pitch for FCoE over iSCSI, to my view right now,
comes from not from this management baloney but from the seven CoS
levels, and the possibility some can be blocking and others buffered.
Internet *transit* traffic (as opposed to end systems), and anything
high-rtt, *must* be buffered, while within the LAN my current thinking
is that you're better off with a 10Gbit/s HOL-blocking bufferless link
than a 1Gbit/s non-blocking buffered link.  The latter applies double
for storage traffic which, made up of UDP-like reads and writes where
you are stuck trying to perfect TCP to avoid blowing the buffers of
normal switches while still getting yourself out of slow-start before
the transaction's over andn doing all this in an environment where you
cannot even convince thick-skulled netadmins they NEED to provide RED,
not this bullshit ``weighted tail drop'' of 3560 u.s.w., and which
besides really need backpressure from the fabric so they can be QoS'ed
in the initiator's stack ahead of the network so that for example
scrubs don't slow down pools (don't you find this happens more over
iSCSI than over SAS?).  I'm saaying, um...shit...saying, ``You need to
think, about what you are trying to accomplish,'' and that Sun might
have a suite of protocols based on ancient IB stuff that accomplishes
more than FCoE, and does it cheaper (to them) and more simply, so,
following their usual annoying plan, step 2 charge FCoE prices minus
, step 3 profit.

meanwhile mellanox, having forseen all this and built open standards
to solve it, is out there desperately trying to push some baloney
called Etherband or something because all you bank admins are too daft
to buy anything that does not have Ether in the name. :(



--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

2010-02-02 Thread Miles Nordin

> "fc" == Frank Cusack  writes:

fc> by FCoE are you talking about iSCSI?

FCoE is an L2 design where ethernet ``pause'' frames can be sent
specific to one of the seven CoS levels instead of applying to the
entire port, which makes PAUSE abuseable for other purposes than their
former one.  CoS is an L2 priority/QoS tag inside the VLAN header.
Before this hack, pause frames are not useful for congestion
management because they cause head-of-line blocking, so serious
switches only send them in response to backplane congestion, and for
example serious hosts might send them for PCI contention, if clever
enough.  With the hack, the HOL-blocking effect of a PAUSE still
spreads further than you might ideally like but can be constrained to
one of the seven CoS planes in your fabric (probably, the Storage
plane).  This lets you have an HOL-blocking, lossless storage fabric
in parallel with a buffered TCP fabric that is not lossless (uses
packet drops for congestion control like normal Ethernet).  You will
find some squirrely language from FCoE proponents around these issues
because they are trying to convince you that you have every desireable
buzzword in every part of your network, while in fact what you're
doing is making the same wise trade-off that every other non-Ethernet
LAN fabric has always made.  My parallel point is that the
HOL-blocking lossless fabric is *CHEAPER* to create, not nmore
expensive.  It is less capable.  It has no buffers and therefore no
QoS.  It just happens to be what's best for storage.  so, they want
you to pay the prices of a multi-queued QoSed WRED big-buffered
non-blocking fabric suitable for transit traffic even though you
mostly just need to push storage bits: classic upsell, just like all
those ``XL'' PFC's they try to push off to customers who are not even
in the DFZ.

FCoE also includes a bunch of expensive hocus-pokus to bridge these
frames onto a traditional FC-switched network and do a bunch of other
things I don't understand like FC zoning and F-SPF.  Most of the pitch
dwells on this, trying to convince you they've made things ``simpler''
for you because it's once piece of wire.  This seems like an
anti-feature to me: wire's cheap while understanding things is hard,
and now everyone's forced to catch up and learn Fibre Channel before
it's safe to touch anything.  Good in the long run, absolutely.
Cheaper, fuck no.

but the legitimate pitch for FCoE over iSCSI, to my view right now,
comes from not from this management baloney but from the seven CoS
levels, and the possibility some can be blocking and others buffered.
Internet *transit* traffic (as opposed to end systems), and anything
high-rtt, *must* be buffered, while within the LAN my current thinking
is that you're better off with a 10Gbit/s HOL-blocking bufferless link
than a 1Gbit/s non-blocking buffered link.  The latter applies double
for storage traffic which, made up of UDP-like reads and writes where
you are stuck trying to perfect TCP to avoid blowing the buffers of
normal switches while still getting yourself out of slow-start before
the transaction's over andn doing all this in an environment where you
cannot even convince thick-skulled netadmins they NEED to provide RED,
not this bullshit ``weighted tail drop'' of 3560 u.s.w., and which
besides really need backpressure from the fabric so they can be QoS'ed
in the initiator's stack ahead of the network so that for example
scrubs don't slow down pools (don't you find this happens more over
iSCSI than over SAS?).  I'm saaying, um...shit...saying, ``You need to
think, about what you are trying to accomplish,'' and that Sun might
have a suite of protocols based on ancient IB stuff that accomplishes
more than FCoE, and does it cheaper (to them) and more simply, so,
following their usual annoying plan, step 2 charge FCoE prices minus
, step 3 profit.

meanwhile mellanox, having forseen all this and built open standards
to solve it, is out there desperately trying to push some baloney
called Etherband or something because all you bank admins are too daft
to buy anything that does not have Ether in the name. :(


pgpyvz3N8H8Ve.pgp
Description: PGP signature
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

I believe magical unicorn controllers and drives are both bug-free and
100% spec compliant. The leprichorns sell them if you're trying to
find them ;)

-marc

On 2/2/10, David Magda  wrote:
> On Feb 2, 2010, at 15:21, Tim Cook wrote:
>
>> How exactly do you suggest the drive manufacturers make their drives
>> "just
>> work" with every SAS/SATA controller on the market, and all of the
>> quirks
>> they have?  You're essentially saying you want the drive
>> manufacturers to do
>> what the storage vendors are doing today (all of the integration
>> work), only
>> not charge you for it.
>
> One way is to have bug-free firmware software in both the disk and the
> controllers that follows the specs perfectly. :)
>
> ___
> zfs-discuss mailing list
> zfs-discuss@opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>

-- 
Sent from my mobile device
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

On Feb 2, 2010, at 2:56 PM, David Magda wrote:
> 
> On Feb 2, 2010, at 15:17, Tim Cook wrote:
> 
>> On Tue, Feb 2, 2010 at 12:54 PM, Orvar Korvar wrote:
>> 
>>> 100% uptime for 20 years?
>>> 
>>> So what makes OpenVMS so much more stable than Unix? What is the
>>> difference?
>>> 
>> 
>> They had/have clustering software that was/is bulletproof.  I don't think
>> anyone in the Unix community has duplicated it to date.  As for differences,
>> google is your friend?
>> 
>> http://www3.sympatico.ca/n.rieck/docs/vms_vs_unix.html
> 
> And by "clustering" we're not talking about something like Sun Cluster where 
> it restarts an application after a node fails. It's more along the lines of 
> multiple machines acting as a single server (though each runs its own copy of 
> the OS--not a single image system):

Did you ever wonder why Solaris Cluster seemed to be overkill for a 
simple failover service?  The original design goals for Solaris Cluster
looked a lot more like VMScluster than what you see today in Solaris
Cluster. You can see the remnants remain in the code and features like
pxfs (no, ZFS won't work with pxfs). The barriers to bringing such 
technology from a simple process model (like VMS) to a modern OS
like Solaris are daunting.
 -- richard

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

2010-02-02 Thread David Magda


On Feb 2, 2010, at 15:21, Tim Cook wrote:

How exactly do you suggest the drive manufacturers make their drives  
"just
work" with every SAS/SATA controller on the market, and all of the  
quirks
they have?  You're essentially saying you want the drive  
manufacturers to do
what the storage vendors are doing today (all of the integration  
work), only

not charge you for it.


One way is to have bug-free firmware software in both the disk and the  
controllers that follows the specs perfectly. :)


___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

2010-02-02 Thread David Magda



On Feb 2, 2010, at 15:17, Tim Cook wrote:


On Tue, Feb 2, 2010 at 12:54 PM, Orvar Korvar wrote:


100% uptime for 20 years?

So what makes OpenVMS so much more stable than Unix? What is the
difference?



They had/have clustering software that was/is bulletproof.  I don't  
think
anyone in the Unix community has duplicated it to date.  As for  
differences,

google is your friend?

http://www3.sympatico.ca/n.rieck/docs/vms_vs_unix.html


And by "clustering" we're not talking about something like Sun Cluster  
where it restarts an application after a node fails. It's more along  
the lines of multiple machines acting as a single server (though each  
runs its own copy of the OS--not a single image system):


http://en.wikipedia.org/wiki/OpenVMS#Clustering
http://en.wikipedia.org/wiki/VMScluster

It was originally released in VMS version 4 back in 1984.

VMS originally ran on VAX, was ported to DEC's Alpha (now dead), and  
is now on Intel's Itanium (not that popular AFAIK).


___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

On Feb 2, 2010, at 1:56 PM, Frank Cusack wrote:

> On February 2, 2010 4:31:47 PM -0500 Miles Nordin  wrote:
>> and FCoE is just dumb if you have IB, honestly.
> 
> by FCoE are you talking about iSCSI?

FCoE is to iSCSI as Netware (IPX/SPX) is to NFS :-)
 -- richard


___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

2010-02-02 Thread Bob Friesenhahn


On Tue, 2 Feb 2010, Frank Cusack wrote:


On February 2, 2010 4:31:47 PM -0500 Miles Nordin  wrote:

 and FCoE is just dumb if you have IB, honestly.


by FCoE are you talking about iSCSI?


No.  They are different.  FCoE uses "raw" ethernet packets and 
ethernet switches can/should be specially designed to support it 
whereas iSCSI is a TCP-based protocol.  FCoE is basically fiber 
channel "SAN" protocol over ethernet.


Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives


On February 2, 2010 4:31:47 PM -0500 Miles Nordin  wrote:

 and FCoE is just dumb if you have IB, honestly.


by FCoE are you talking about iSCSI?
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

On Feb 2, 2010, at 10:54 AM, Orvar Korvar wrote:

> 100% uptime for 20 years? 
> 
> So what makes OpenVMS so much more stable than Unix? What is the difference?

Software reliability studies show that the more reliable software is
old software that hasn't changed :-)

On Feb 2, 2010, at 12:42 PM, David Dyer-Bennet wrote:
> I'm suggesting that the standard for the interface ought to be
> sufficiently standardized and well-enough documented that things meeting
> it just work, in the way that desktop motherboards and disk drives "just
> work", i.e. well enough for nearly everybody.  I understand why people
> pushing the limits would need custom-tuned hardware, but I don't think the
> middle of the market should need it.

Every mobo and disk I own has quirks. The only time things settle down
to a widely accepted norm is when innovation stops.  For example, recently
the SATA TRIM command has received a lot of press.  Next quarter, it will
be some other feature on the buzzword list.

> The controllers shouldn't be full of quirks; companies that routinely make
> them that way need to clean up their act or be driven out of the market. 
> Same for the drives.

I think there are only about 5 HDD companies (Hitachi, Seagate, Western
Digital, Samsung, Toshiba) and 3 controller companies today (LSI, Marvell, 
Intel). The remainder are in the process of getting out of the business or being
bought.  Interesting history here:
http://en.wikipedia.org/wiki/List_of_defunct_hard_disk_manufacturers
 -- richard

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

2010-02-02 Thread Miles Nordin

> "bh" == Brandon High writes:
> "ok" == Orvar Korvar writes:
> "mp" == matthew patton writes:

bh> This one holds "only" 24 drives:
bh> http://www.supermicro.com/products/chassis/4U/846/SC846TQ-R900.cfm
bh> ($950)

This one holds only 20 drives. includes fan, not power supply.
the airflow management seems shit but basically works okay:

http://www.servercase.com/Merchant2/merchant.mvc?Screen=PROD&Product_Code=CK4020&Category_Code=4UBKBLN
($375.38 shipping included)
(pornographer's-grade sleds)

I have one of these and am happy with it. It's 20 plain SATA ports,
passively. Competitors' cases are including SAS switches on the
backplane, and since I don't know how to buy switches apart from cases
you may need to get a competing case if yuo want a SAS switch.

I found it cheaper to use ATI 790fx board that has four 8-lane PCIe
slots (16x for power, but 8 data lanes active when all four slots are
in use) and then use 1068e cards with breakout cables, so basically
the 790fx chip does the multiplexing. Obviously this does not scale
as far as a SAS fabric: the 4 PCIe will handle 16 drives, an 8x NIC of
some kind, and an nVidia card.

ok> Is it possible to have large chassi with lots of
ok> drives, and the opensolaris in another chassi, how do you
ok> connect them both?

SAS. The cabling and the switching chips are eerily reminiscent of
infiniband, but while I think IB has open source drivers and stacks
and relatively generic proprietary firmware, more of the brains of the
SAS fabric seem to be controlled by proprietary software running as
``firmware'' on all the LSI chips involved, so I think this landscape
might make harder smartctl, using a cd burner, or offering a SAS
target into the fabric through COMSTAR (AFAIK none of these things
work now---I guess we'll see what evolves). but I guess no one built
cheap single chips to tunnel SATA inside IB, so here we are with
broken promises and compromises and overpriced sillyness like FCoE.

In the model numbers of the old 3Gbit/s LSI cards, the second digit
was the number of external ports, and the third digit the number of
internal ports. For example LSI SAS3801E-R is a mega_sas-drivered
(raid-on-a-card) with 8 external ports, and LSI SAS3081E-R has 8
internal ports. but if you want a cheaper card with IT firmware for
mpt driver, without RAID-on-a card, yuo may have to hunt some more.
The external ports are offered on one or two single connectors with
four ``ports'' per connector---each of the four can be broken out and
connected to an individual disk using a passive four-legged-octopus
cable, or bonded together in sets of four to form a single faster
logical link to a SAS switch chip. beyond that I don't really know
how it all works. I'm probably telling you stuff you already know but
at least hopefully now everyone's caught up.

mp> I buy a Ferrari for the engine and bodywork and chassis
mp> engineering.

Didn't they go through bankruptcy repeatedly and then get bought by
Fiat? Whatever this folded-sheetmetal crap thing from so-called
``servercase.com'' is, it's probably backed secretly by the chinese
government, and I bet it outlasts your fancy J4500. This seems to me
like a bad situation, but I'm not sure what to do about it.

There are many ways to slice the market vertically. For example you
could also get your integration done by renting whitebox crap through
a server-rental company that rents you dedicated storage or compute
nodes at 10 or 100 at a time pre-connected by network equipment they
won't discuss with you (probably cheaper than the cisco stuff you'd
definitely buy if it were your own ass on the line). Part of Sun's
function is prequalifying but another part is to reach inside their
customer's organizations, extract experience, and then share it among
all customers discretely without any one customer feeling violated. A
hardware rental company can do the same thing, and I bet they can do
it at similar scale, with a lot less political bullshit. I think
there's a big messy underground market of these shady rental companies
in parallel to the above-ground overblogged overpriced flakey EC2-like
stuff. My hope is that the IB stack, in which Sun's also apparently
deeply invested with both Solaris and IB-included blades and switches
and backplanes and onboard MAC's, will start taking a chunk out of
Cisco's pie. Meanwhile the box-renting company extracts money from
you by performing an ass-covering function: they can buy cheap risky
things you can't, and then you say to your minders, ``other people buy
from them too. It was not reckless to become their customer. I've
both saved you money and manoevered you the agility to evade the
problems we've had without writing off a lot of purchases,'' when
really what you are outsourcing here is your own expensive CYA
tendencies.

But back to the potential pie-slice for Sun to steal: the function of
the IB switching chips themselves are far simpler

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives


On February 2, 2010 2:17:30 PM -0600 Tim Cook  wrote:

http://www3.sympatico.ca/n.rieck/docs/vms_vs_unix.html


interesting page, if somewhat dated.  e.g. maybe it wasn't true at the
time but don't we now know from the SCO lawsuit that SCO does indeed
own "UNIX"?

as long as we're OT. :)
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

On Tue, Feb 2, 2010 at 3:45 PM, Peter Jeremy <
peter.jer...@alcatel-lucent.com> wrote:

>
> OTOH, if I'm paying 10x the street drive price upfront, plus roughly
> the street price annually in "support", I can save a fair amount of
> money by just buying a pile of spare drives - when one fails, just
> swap it for a spare and it doesn't matter if it takes weeks for the
> vendor to swap it.
>
>
I can tell you with the Enterprise storage vendor I deal with, the
maintenance over five years (expected lifetime of the system) is equal or
more than initial, discounted purchase price!

-marc
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

2010-02-02 Thread Peter Jeremy

On 2010-Feb-03 00:12:43 +0800, Bob Friesenhahn  
wrote:
>On Tue, 2 Feb 2010, David Dyer-Bennet wrote:
>>
>> Now, I'm sure not ALL drives offered at Newegg could qualify; but the
>> question is, how much do I give up by buying an enterprise-grade drive
>> from a major manufacturer, compared to the Sun-certified drive?
>
>If you have a Sun service contract, you give up quite a lot.  If a Sun 
>drive fails every other day, then Sun will replace that Sun drive 
>every other day, even if the system warranty has expired.  But if it 
>is a non-Sun drive, then you have to deal with a disinterested drive 
>manufacturer, which could take weeks or months.

OTOH, if I'm paying 10x the street drive price upfront, plus roughly
the street price annually in "support", I can save a fair amount of
money by just buying a pile of spare drives - when one fails, just
swap it for a spare and it doesn't matter if it takes weeks for the
vendor to swap it.

>Hopefully Oracle will do better than Sun at explaining the benefits 
>and services provided by a service contract.

I know that trying to get Sun to renew a service contract is like
pulling teeth but Oracle is far worse - as far as I can tell, Oracle
contracts are deliberately designed so you can't be certain whether
you are compliant or not.

-- 
Peter Jeremy
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives


On Tue, February 2, 2010 14:21, Tim Cook wrote:
> On Tue, Feb 2, 2010 at 2:14 PM, David Dyer-Bennet  wrote:
>
>>
>> On Tue, February 2, 2010 11:26, Richard Elling wrote:
>> > On Feb 2, 2010, at 8:49 AM, David Dyer-Bennet wrote:
>> >> On Tue, February 2, 2010 10:21, Marc Nicholas wrote:
>> >>> I agree wholeheartedlyyou're paying to make the problem "go
>> away"
>> >>> in
>> >>> an
>> >>> expedient manner. That said, I see how much we spend on NetApp
>> storage
>> >>> at
>> >>> work and it makes me shudder ;)
>> >>
>> >> Yes, exactly.  Pricing must be about right, people wince but pay it
>> :-).
>> >> If they don't wince it's too low.
>> >
>> > Business 101.
>> > The price will be what the market will bear. If the price seems out of
>> > line with your market, then perhaps you aren't in the same market.
>>
>> Yes, perhaps.  Quite clearly, in this case; I'm not buying enterprise
>> storage myself.  If the market bears it for long, then there's
>> definitely
>> an actual market there; otherwise it might have been a mistake by the
>> company.
>>
>> I want the disk companies to come up with a set of specs for an
>> enterprise-grade drive that can be used in stock form in relatively
>> simple
>> hardware to give good results.  This concept that their enterprise-grade
>> drives need tweaking in the firmware and price to be useful is annoying.
>> Fair enough for people pushing the edges of the envelope, but most
>> people
>> don't, there should be a good solid mainstream solution available.
>>
>> A Solaris-based ZFS box using SAS controllers with 5, 8, 24, and 48
>> drive
>> bay options might just about do it, if it could take a range of stock
>> drives officially.  Kill off a big chunk of people who get forced into
>> enterprise storage against their will.
>>
>
> How exactly do you suggest the drive manufacturers make their drives "just
> work" with every SAS/SATA controller on the market, and all of the quirks
> they have?  You're essentially saying you want the drive manufacturers to
> do
> what the storage vendors are doing today (all of the integration work),
> only not charge you for it.
>
> You can't have your cake and eat it too.

I'm suggesting that the standard for the interface ought to be
sufficiently standardized and well-enough documented that things meeting
it just work, in the way that desktop motherboards and disk drives "just
work", i.e. well enough for nearly everybody.  I understand why people
pushing the limits would need custom-tuned hardware, but I don't think the
middle of the market should need it.

The controllers shouldn't be full of quirks; companies that routinely make
them that way need to clean up their act or be driven out of the market. 
Same for the drives.

>> Yes, my Camry is good for commuting and running across town through
>> unfortunately frequent stop-and-go traffic, and running down to see my
>> mother (about an hour) a lot more than I used to need to.  It can carry
>> 4
>> very comfortably, which we only use every month or so, and it can carry
>> the 3-head studio lighting kit in the trunk very comfortably.
>>
>> Probably still somewhat marginal on your ranch, though better than a
>> Ferrari.  The ground clearance is medium, and it's not mainly a
>> cargo-hauler.
>>
>>
> And how well does your Camry run when you try to replace the Toyota
> transmission with one manufactured by Ford?  A mechanic who knows what
> he's
> doing and has fabrication skills could probably get it to work, and pretty
> darn well, but it isn't ever going to be the same as buying an integrated
> product directly from Toyota...

When I was first in the industry, in 1969, it was fairly normal to only be
able to connect DEC disks to a PDP-11; but even then there were
third-party manufacturers making products and customers buying them.  Now,
forty years down the road, computers are constructed from mostly generic
components.  The disk drive is one of the ones that went generic first. 
It's absurd that we can't handle small enterprise storage on
standards-compliant drives at this point.

-- 
David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/
Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/
Photos: http://dd-b.net/photography/gallery/
Dragaera: http://dragaera.info

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

On Tue, Feb 2, 2010 at 2:14 PM, David Dyer-Bennet  wrote:

>
> On Tue, February 2, 2010 11:26, Richard Elling wrote:
> > On Feb 2, 2010, at 8:49 AM, David Dyer-Bennet wrote:
> >> On Tue, February 2, 2010 10:21, Marc Nicholas wrote:
> >>> I agree wholeheartedlyyou're paying to make the problem "go away"
> >>> in
> >>> an
> >>> expedient manner. That said, I see how much we spend on NetApp storage
> >>> at
> >>> work and it makes me shudder ;)
> >>
> >> Yes, exactly.  Pricing must be about right, people wince but pay it :-).
> >> If they don't wince it's too low.
> >
> > Business 101.
> > The price will be what the market will bear. If the price seems out of
> > line with your market, then perhaps you aren't in the same market.
>
> Yes, perhaps.  Quite clearly, in this case; I'm not buying enterprise
> storage myself.  If the market bears it for long, then there's definitely
> an actual market there; otherwise it might have been a mistake by the
> company.
>
> I want the disk companies to come up with a set of specs for an
> enterprise-grade drive that can be used in stock form in relatively simple
> hardware to give good results.  This concept that their enterprise-grade
> drives need tweaking in the firmware and price to be useful is annoying.
> Fair enough for people pushing the edges of the envelope, but most people
> don't, there should be a good solid mainstream solution available.
>
> A Solaris-based ZFS box using SAS controllers with 5, 8, 24, and 48 drive
> bay options might just about do it, if it could take a range of stock
> drives officially.  Kill off a big chunk of people who get forced into
> enterprise storage against their will.
>

How exactly do you suggest the drive manufacturers make their drives "just
work" with every SAS/SATA controller on the market, and all of the quirks
they have?  You're essentially saying you want the drive manufacturers to do
what the storage vendors are doing today (all of the integration work), only
not charge you for it.

You can't have your cake and eat it too.




> Yes, my Camry is good for commuting and running across town through
> unfortunately frequent stop-and-go traffic, and running down to see my
> mother (about an hour) a lot more than I used to need to.  It can carry 4
> very comfortably, which we only use every month or so, and it can carry
> the 3-head studio lighting kit in the trunk very comfortably.
>
> Probably still somewhat marginal on your ranch, though better than a
> Ferrari.  The ground clearance is medium, and it's not mainly a
> cargo-hauler.
>
>
And how well does your Camry run when you try to replace the Toyota
transmission with one manufactured by Ford?  A mechanic who knows what he's
doing and has fabrication skills could probably get it to work, and pretty
darn well, but it isn't ever going to be the same as buying an integrated
product directly from Toyota...

--Tim
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

On Tue, Feb 2, 2010 at 12:54 PM, Orvar Korvar <
knatte_fnatte_tja...@yahoo.com> wrote:

> 100% uptime for 20 years?
>
> So what makes OpenVMS so much more stable than Unix? What is the
> difference?
>
>
>
They had/have clustering software that was/is bulletproof.  I don't think
anyone in the Unix community has duplicated it to date.  As for differences,
google is your friend?
http://www3.sympatico.ca/n.rieck/docs/vms_vs_unix.html

--Tim
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

On Tue, February 2, 2010 11:26, Richard Elling wrote:
> On Feb 2, 2010, at 8:49 AM, David Dyer-Bennet wrote:
>> On Tue, February 2, 2010 10:21, Marc Nicholas wrote:
>>> I agree wholeheartedlyyou're paying to make the problem "go away"
>>> in
>>> an
>>> expedient manner. That said, I see how much we spend on NetApp storage
>>> at
>>> work and it makes me shudder ;)
>>
>> Yes, exactly.  Pricing must be about right, people wince but pay it :-).
>> If they don't wince it's too low.
>
> Business 101.
> The price will be what the market will bear. If the price seems out of
> line with your market, then perhaps you aren't in the same market.

Yes, perhaps.  Quite clearly, in this case; I'm not buying enterprise
storage myself.  If the market bears it for long, then there's definitely
an actual market there; otherwise it might have been a mistake by the
company.

I want the disk companies to come up with a set of specs for an
enterprise-grade drive that can be used in stock form in relatively simple
hardware to give good results.  This concept that their enterprise-grade
drives need tweaking in the firmware and price to be useful is annoying. 
Fair enough for people pushing the edges of the envelope, but most people
don't, there should be a good solid mainstream solution available.

A Solaris-based ZFS box using SAS controllers with 5, 8, 24, and 48 drive
bay options might just about do it, if it could take a range of stock
drives officially.  Kill off a big chunk of people who get forced into
enterprise storage against their will.

> Personally, I think Ferraris are neat. But here on the ranch, I might be
> able to squeeze a bale of hay into the passenger seat, but the low
> ground clearance means I'll have to keep the tractor nearby to pull it
> out  when it gets stuck. So a Ferrari has $0 market value here at the
> ranch.

Yes, my Camry is good for commuting and running across town through
unfortunately frequent stop-and-go traffic, and running down to see my
mother (about an hour) a lot more than I used to need to.  It can carry 4
very comfortably, which we only use every month or so, and it can carry
the 3-head studio lighting kit in the trunk very comfortably.

Probably still somewhat marginal on your ranch, though better than a
Ferrari.  The ground clearance is medium, and it's not mainly a
cargo-hauler.
-- 
David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/
Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/
Photos: http://dd-b.net/photography/gallery/
Dragaera: http://dragaera.info

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

1) SAS HBA seems to be an I/O card which has SAS cable connection. It sits in 
the OSol server. It is basically just a simple I/O card, right? I hope these 
cards are cheap? 

2) So I can buy a disk chassi with 24 disks, connect all disks to one SAS cable 
and connect that SAS cable to my OSol server, which has a SAS HBA I/O card? Is 
this correct? Or do I need one SAS cable for each disk, hence I need 24 SAS 
cables?

3) Does the SAS HBA card need Solaris drivers? 

4) Will there be performance hit if I connect 24 disks to one SAS cable/HBA? 
Maybe band width in one SAS cable will not suffice for 24 disks?
-- 
This message posted from opensolaris.org
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

2010-02-02 Thread Ed Fang

Also, both of those chassis come in SAS expander version and JBOD.  the SAS 
expander version is the E1 version of the case.  With the SAS Expander, and a 
motherboard using the LSI2008 or LSI1068 chipset, you can attach one cable from 
the SAS port (SFF8087) to the SAS expander and have all the drives online 
rather than connecting 24/36 drives individually. . .
-- 
This message posted from opensolaris.org
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

100% uptime for 20 years? 

So what makes OpenVMS so much more stable than Unix? What is the difference?
-- 
This message posted from opensolaris.org
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives


On February 2, 2010 12:08:13 PM -0600 Tim Cook  wrote:

Not exactly unix, but there's still VMS clusters running around out there
with 100% uptime for over 20 years.  I wouldn't mind seeing it opened up.


Agreed, I'd love to see that opened up.  Might even give it new life.
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

2010-02-02 Thread Brandon High

On Tue, Feb 2, 2010 at 5:41 AM, Orvar Korvar
 wrote:
> I see 24 drives in an external chassi. I presume that chassis does only hold 
> drives, it does not hold a motherboard.
>
> How do you connect all drives to your OpenSolaris server? Do you place them 
> next to each other, and then you have three 8 SATA ports in your OpenSolaris 
> server, and have 24 SATA cables in the air? And the chassis are wide open?
>
> Or, am I wrong, does the chassi also hold a motherboard?

Both of the cases I posted can hold a motherboard. There is also a kit
available for about $100 that lets you use the chassis with just disk
in it.

-B

-- 
Brandon High : bh...@freaks.com
For sale: One moral compass, never used.
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

On Tue, Feb 2, 2010 at 12:00 PM, Frank Cusack
wrote:

> On February 2, 2010 11:58:17 AM -0600 Tim Cook  wrote:
>
>> On Tue, Feb 2, 2010 at 11:53 AM, Frank Cusack
>> wrote:
>>
>>  On February 2, 2010 8:57:32 AM -0800 Orvar Korvar <
>>> knatte_fnatte_tja...@yahoo.com> wrote:
>>>
>>>  I love that Sun shares their products for free. Which other big Unix
 vendor does that?


>>> Who's left?
>>>
>>>
>>>  Pretty sure HP and IBM are still alive and well.
>>
>
> Yeah but who would want it, even for free. :P
>


Not exactly unix, but there's still VMS clusters running around out there
with 100% uptime for over 20 years.  I wouldn't mind seeing it opened up.

--Tim
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives


On February 2, 2010 11:58:17 AM -0600 Tim Cook  wrote:

On Tue, Feb 2, 2010 at 11:53 AM, Frank Cusack
wrote:


On February 2, 2010 8:57:32 AM -0800 Orvar Korvar <
knatte_fnatte_tja...@yahoo.com> wrote:


I love that Sun shares their products for free. Which other big Unix
vendor does that?



Who's left?



Pretty sure HP and IBM are still alive and well.


Yeah but who would want it, even for free. :P
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

2010-02-02 Thread David Champion

* On 02 Feb 2010, Orvar Korvar wrote:
> Ok, I see that the chassi contains a mother board. So never mind that
> question.
>
> Another q:  Is it possible to have large chassi with lots of drives,
> and the opensolaris in another chassi, how do you connect them both?

The J4500 and most other storage products being discussed are not
servers: they are SATA concentrators with SAS uplinks.  You plug in
a bunch of cheap SATA disk, and you connect the chassis to a server
with SAS.  The logic board on the storage tray just converts the SAS
signalling to SATA.  It is not a computer in the usual sense.

In many cases such products also have SAS expander ports, so that you
can link multiple storage trays to a single SAS host bus adapter on your
server by daisy-chaining them.

So you need at least one SAS HBA on your OpenSolaris box, and SAS cables
to hook up the trays containing the SATA drives.

To the original question: you can purchase a J4x00 with a limited
number of drives (empty is generally not an option), but there is no
officially-sanctioned way to obtain the drive adapters except to buy Sun
disks.  You need either a SAS or a SATA drive bracket to adapt the drive
to the J4x00 backplane, but they are not sold separately: one ships with
each drive.

As mentioned there are companies that sell remanufactured or discarded
components, or machine their own substitutes.  (Re)marketing Sun or
compatible drive brackets has always been a lively business for a
few small outfits.  But Sun has no involvement with this, and may be
unwilling to support a frankenstein server.

Sun state that their OEM drives are of higher quality than OTS drives
from manufacturers or retailers, and that they have custom firmware that
improves their performance and reliability in Sun storage trays/arrays.
I see no reason to disbelieve that, but it is quite a steep price to pay
for that premium edge.  When cost is a bigger concern than performance
or reliability, I have generally bought the StorEdge product with the
smallest drives I can (250 GB or 500 GB) and upgraded them myself to the
size I really want.  It's cheaper to buy 20 drives from CDW than 10 from
Sun even when you account for the tiny throwaway drive, and you can keep
the 10 extra as cold spares.  At low enough scale the financial savings
are worth the time to replace them as they fail.

(I wish I could say the same of the StorEdge arrays themselves.  Fully
half of my 2540 controllers have failed, costing me huge amounts of
time in both direct and contractual service, and I'm given up on them
completely as a product line.  I'll be thrilled to switch to JBOD.)

For larger and less fault-tolerant systems, when money is available,
I'm happy to pay Sun's premium.

However, as others say, the other brands sometimes offer decent enough
products to use instead of Sun's enterprise line.  As always, it depends
on your site's requirements and budget.  I assume that a home NAS is
comparatively low on both: therefore I wouldn't even shop with Sun
unless you have a line on cheap castoffs from an enterprise shop.

-- 
 -D.d...@uchicago.eduNSITUniversity of Chicago
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

On Tue, Feb 2, 2010 at 11:53 AM, Frank Cusack
wrote:

> On February 2, 2010 8:57:32 AM -0800 Orvar Korvar <
> knatte_fnatte_tja...@yahoo.com> wrote:
>
>> I love that Sun shares their products for free. Which other big Unix
>> vendor does that?
>>
>
> Who's left?
>
>
Pretty sure HP and IBM are still alive and well.

--Tim
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

On February 2, 2010 8:57:32 AM -0800 Orvar Korvar 
 wrote:

I love that Sun shares their products for free. Which other big Unix
vendor does that?


Who's left?
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

On Feb 2, 2010, at 8:49 AM, David Dyer-Bennet wrote:
> On Tue, February 2, 2010 10:21, Marc Nicholas wrote:
>> I agree wholeheartedlyyou're paying to make the problem "go away" in
>> an
>> expedient manner. That said, I see how much we spend on NetApp storage at
>> work and it makes me shudder ;)
> 
> Yes, exactly.  Pricing must be about right, people wince but pay it :-). 
> If they don't wince it's too low.

Business 101.
The price will be what the market will bear. If the price seems out of
line with your market, then perhaps you aren't in the same market.

Personally, I think Ferraris are neat. But here on the ranch, I might be
able to squeeze a bale of hay into the passenger seat, but the low
ground clearance means I'll have to keep the tractor nearby to pull it
out  when it gets stuck. So a Ferrari has $0 market value here at the ranch.
 -- richard

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

This reminds me of this attorney that charged very much for a contract template 
he copied and gave to a client. To that, he responded:
-You dont pay for me finding this template and copying to you, which took me 5 
minutes. You pay me because I sat 5 years in the university, and have 15 years 
of experience. That is why you pay me.

I agree that Sun hardware is way too pricey for a home user, but this is 
Enterprise stuff. And you should look at IBM prices, they are 5-10x higher than 
Sun's prices.

I think the Enterprise customers, do not pay for Sun bringing you a disk from 
the cellar, which takes 5 minutes. But they pay for all the research, 
development, and Sun making sure everything works as intended.

I love that Sun shares their products for free. Which other big Unix vendor 
does that?
-- 
This message posted from opensolaris.org
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

On Tue, February 2, 2010 10:21, Marc Nicholas wrote:
> I agree wholeheartedlyyou're paying to make the problem "go away" in
> an
> expedient manner. That said, I see how much we spend on NetApp storage at
> work and it makes me shudder ;)

Yes, exactly.  Pricing must be about right, people wince but pay it :-). 
If they don't wince it's too low.

There are certainly places (NASDAQ, say) who have to have absolute
reliability (on both their main and disaster-recovery sites).  That level
of reliability costs money, and they pay it, probably even fairly
cheerfully.

Since spare parts have to be stocked and field service people trained, it
makes sense that service contracts cover limited sets of equipment.  And
that's a strong argument for staying within that set of equipment.

I do think that a lot of companies buy higher up the reliability curve
than they need.  But that's their choice.

> I think someone was wondering if the large storage vendors have their own
> microcode on drives? I can tell you that NetApp do...and that's one way
> they
> "lock you in" (if the drive doesn't report NetApp firmware, the filer will
> "reject" the drive) and also how they do tricks like
> soft-failure/re-validation, 520-byte sectors, etc.

Sigh.  It makes perfect sense that getting some special tricks in the
drives actually pays off.  And yet it's inevitable that they ALSO use it
as a lockin.

I've seen people down extra days while locked-in parts are shipped to
them; the parts were essentially identical to what you could buy that day
at retail locally, but the locally-available version wouldn't work.

-- 
David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/
Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/
Photos: http://dd-b.net/photography/gallery/
Dragaera: http://dragaera.info

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

2010-02-02 Thread Joerg Schilling

Marc Nicholas  wrote:

> I think someone was wondering if the large storage vendors have their own
> microcode on drives? I can tell you that NetApp do...and that's one way they
> "lock you in" (if the drive doesn't report NetApp firmware, the filer will
> "reject" the drive) and also how they do tricks like
> soft-failure/re-validation, 520-byte sectors, etc.

Since IBM started to use SCSI drives more than 20 years ago for their 
mainframes, you can format most drives with any sector size. IBM used 800
or 8000 byte sectors (10 or 100 punch cards ;-), but you also may reformat
a drive with 520 bytes per sector.

Jörg

-- 
 EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
   j...@cs.tu-berlin.de(uni)  
   joerg.schill...@fokus.fraunhofer.de (work) Blog: 
http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

I agree wholeheartedlyyou're paying to make the problem "go away" in an
expedient manner. That said, I see how much we spend on NetApp storage at
work and it makes me shudder ;)

I think someone was wondering if the large storage vendors have their own
microcode on drives? I can tell you that NetApp do...and that's one way they
"lock you in" (if the drive doesn't report NetApp firmware, the filer will
"reject" the drive) and also how they do tricks like
soft-failure/re-validation, 520-byte sectors, etc.

-marc


On Tue, Feb 2, 2010 at 11:12 AM, Bob Friesenhahn <
bfrie...@simple.dallas.tx.us> wrote:

> On Tue, 2 Feb 2010, David Dyer-Bennet wrote:
>
>>
>> Now, I'm sure not ALL drives offered at Newegg could qualify; but the
>> question is, how much do I give up by buying an enterprise-grade drive
>> from a major manufacturer, compared to the Sun-certified drive?
>>
>
> If you have a Sun service contract, you give up quite a lot.  If a Sun
> drive fails every other day, then Sun will replace that Sun drive every
> other day, even if the system warranty has expired.  But if it is a non-Sun
> drive, then you have to deal with a disinterested drive manufacturer, which
> could take weeks or months.
>
> My experiences thus far is that if you pay for a Sun service contract, then
> you should definitely pay extra for Sun branded parts.
>
> Hopefully Oracle will do better than Sun at explaining the benefits and
> services provided by a service contract.
>
> Bob
> --
> Bob Friesenhahn
> bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
> GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
>
> ___
> zfs-discuss mailing list
> zfs-discuss@opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

2010-02-02 Thread Bob Friesenhahn


On Tue, 2 Feb 2010, David Dyer-Bennet wrote:


Now, I'm sure not ALL drives offered at Newegg could qualify; but the
question is, how much do I give up by buying an enterprise-grade drive
from a major manufacturer, compared to the Sun-certified drive?


If you have a Sun service contract, you give up quite a lot.  If a Sun 
drive fails every other day, then Sun will replace that Sun drive 
every other day, even if the system warranty has expired.  But if it 
is a non-Sun drive, then you have to deal with a disinterested drive 
manufacturer, which could take weeks or months.


My experiences thus far is that if you pay for a Sun service contract, 
then you should definitely pay extra for Sun branded parts.


Hopefully Oracle will do better than Sun at explaining the benefits 
and services provided by a service contract.


Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

On Tue, February 2, 2010 09:58, Tim Cook wrote:

> It's called spreading the costs around.  Would you really rather pay 10x
> the price on everything else besides the drives?

This seems to miss the point.  I presented an argument for why I think the
qualified drives are a huge profit-center, not just making a reasonable
profit on the work of qualification.

In general, I'd much rather pay reasonable costs for each piece, rather
than weird costs artificially shoved around to make things come out some
strange way somebody favors.

> This is essentially Sun's
> way
> of tiered pricing.  Rather than charge you a software fee based on how
> much
> storage you have, they increase the price of the drives.  Seems fairly
> reasonable to me... it gives a low point of entry for people that don't
> need
> that much storage without using ridiculous capacity based licensing on
> software.

It works great for me personally -- I'm using the software with other
people's hardware, for free.

But why should people who need a lot of storage pay proportionally more? 
I don't get that, that's grossly wrong.
-- 
David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/
Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/
Photos: http://dd-b.net/photography/gallery/
Dragaera: http://dragaera.info

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

On Tue, Feb 2, 2010 at 9:45 AM, David Dyer-Bennet  wrote:

>
> On Tue, February 2, 2010 01:27, Tim Cook wrote:
>
> > Except you think the original engineering is just a couple grand, and
> > that's
> > where you're wrong.  I hate the prices just as much as the next guy, but
> > they do in fact need to feed their families.  In fact, they need to do a
> > hell of a lot more than that, including paying off what is likely bumping
> > up
> > against a 6-figure education for the engineering degrees they hold to
> > design
> > that hardware.  If it were easy, everyone would be doing it, and it
> > wouldn't
> > be expensive.
>
> I don't think the complaint is mostly about the part Sun engineered,
> though; it seems to me the complaint is about the price of what looks and
> smells to many of us like essentially the same disk drive we can buy for
> 1/10 the price at Newegg.
>
> Now, I'm sure not ALL drives offered at Newegg could qualify; but the
> question is, how much do I give up by buying an enterprise-grade drive
> from a major manufacturer, compared to the Sun-certified drive?
>
> I can easily believe that Sun and the vendors spend many man-months
> qualifying a drive, and that perhaps there are even custom firmware
> versions for the Sun-certified drives.  For a single drive, what's a
> not-crazy estimate?  12 man-months?  Fully-loaded man-months costing about
> $250k/year?  If they sell 10,000 of that drive, the per-drive cost of
> qualification would seem to be $25.  Of course, I made up all the numbers,
> and have no idea if the quantity in particular is sane or not.  Anyway,
> it's thinking like this that leads some of us to feel that a $900 premium
> is not reasonable.  I suppose anybody who really knows sales numbers can't
> talk about them, and the same for some of the other bits.  Still, the way
> to combat misperceptions (if these are in fact misperceptions) is with
> more accurate information.
>
> (I actually worked in the group Thumper came out of for a bit; I was
> working on the user interface software for the video streaming software
> that Thumper was developed to provide storage for.  I was with Sun
> 2005-2008.)
>
> > If you think they're overcharging, you're more than welcome to go into
> > business, undercut the shit out of them, and still make a ton of money,
> > since you think they 're charging 10x market value.
>
> I also think I'm not qualified to do that; I'm a software guy, neither
> management nor marketing nor hardware engineering.
>
> Also, if they really are charging 10x, then they can easily cut prices to
> compete with any upstarts, a fact that potential investors would take note
> of.
>
>
>
It's called spreading the costs around.  Would you really rather pay 10x the
price on everything else besides the drives?  This is essentially Sun's way
of tiered pricing.  Rather than charge you a software fee based on how much
storage you have, they increase the price of the drives.  Seems fairly
reasonable to me... it gives a low point of entry for people that don't need
that much storage without using ridiculous capacity based licensing on
software.

--Tim
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

On Tue, February 2, 2010 01:27, Tim Cook wrote:

> Except you think the original engineering is just a couple grand, and
> that's
> where you're wrong.  I hate the prices just as much as the next guy, but
> they do in fact need to feed their families.  In fact, they need to do a
> hell of a lot more than that, including paying off what is likely bumping
> up
> against a 6-figure education for the engineering degrees they hold to
> design
> that hardware.  If it were easy, everyone would be doing it, and it
> wouldn't
> be expensive.

I don't think the complaint is mostly about the part Sun engineered,
though; it seems to me the complaint is about the price of what looks and
smells to many of us like essentially the same disk drive we can buy for
1/10 the price at Newegg.

Now, I'm sure not ALL drives offered at Newegg could qualify; but the
question is, how much do I give up by buying an enterprise-grade drive
from a major manufacturer, compared to the Sun-certified drive?

I can easily believe that Sun and the vendors spend many man-months
qualifying a drive, and that perhaps there are even custom firmware
versions for the Sun-certified drives.  For a single drive, what's a
not-crazy estimate?  12 man-months?  Fully-loaded man-months costing about
$250k/year?  If they sell 10,000 of that drive, the per-drive cost of
qualification would seem to be $25.  Of course, I made up all the numbers,
and have no idea if the quantity in particular is sane or not.  Anyway,
it's thinking like this that leads some of us to feel that a $900 premium
is not reasonable.  I suppose anybody who really knows sales numbers can't
talk about them, and the same for some of the other bits.  Still, the way
to combat misperceptions (if these are in fact misperceptions) is with
more accurate information.

(I actually worked in the group Thumper came out of for a bit; I was
working on the user interface software for the video streaming software
that Thumper was developed to provide storage for.  I was with Sun
2005-2008.)

> If you think they're overcharging, you're more than welcome to go into
> business, undercut the shit out of them, and still make a ton of money,
> since you think they 're charging 10x market value.

I also think I'm not qualified to do that; I'm a software guy, neither
management nor marketing nor hardware engineering.

Also, if they really are charging 10x, then they can easily cut prices to
compete with any upstarts, a fact that potential investors would take note
of.
-- 
David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/
Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/
Photos: http://dd-b.net/photography/gallery/
Dragaera: http://dragaera.info

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

On Tue, February 2, 2010 01:26, James C. McPherson wrote:

> The engineering ratings are different to what you can buy from
> your local corner PC store, and the firmware is different. The
> qualification is done with the assumption that the disks will be
> spinning every single second for a number of years, and that
> they will have a much, *much* higher duty cycle than consumer
> grade hardware.
>
> Please stop assuming that all this only costs a few pennies.
> It doesn't.

I'm pretty doubtful that the hardware differs from what I can buy from
Newegg or whatever *IF* I buy the same enterprise-grade drive model (WD
S25 or RE-4, say, rather than Caviar Blue) (I don't know what WD drives,
if any, are currently qualified for use in any Sun products.)  Just to be
clear, are you asserting that?  Or are you only asserting that the drives
that get qualified are not the cheap drive models most easily found at
your handy corner PC store?  (I have less trouble believing they might
have non-standard microcode.)

I'm simply not Sun's market (home NAS); $1000 disk drives simply do not
exist in my home; my budget doesn't stretch that far.  And I do find it
somewhat offensive that software, controller, and drives can't find enough
common standards to actually be able to work (and I'll pay the money I
need for the duty cycle I need; but in fact my home gear has run 24/7
since 1985, and I've had very little problem with disk drives.  In fact
drives most often die for me when the equipment is power-cycled).

(I've still got the corpse of at least one 300MB drive from long ago that
I paid $1500 for, come to think of it!)

-- 
David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/
Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/
Photos: http://dd-b.net/photography/gallery/
Dragaera: http://dragaera.info

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

2010-02-02 Thread Rob Logan


> true. but I buy a Ferrari for the engine and bodywork and chassis
> engineering. It is totally criminal what Sun/EMC/Dell/Netapp do charging

its interesting to read this with another thread containing:

> timeout issue is definitely the WD10EARS disks.
> replaced 24 of them with ST32000542AS (f/w CC34), and the problem departed 
with the WD disks.

everyone needs to eat, if Ferrari spreads their NRE over
the wheels, it might be because they are light and have
been tested to not melt from the heat. Sun/EMC/Dell/Netapp
tests each of their components and sells the total "car".

I'm thankful Sun shares their research and we can build on it.
(btw, netapp ontap 8 is freebsd, and runs on std hardware
after alittle bios work :-)

Rob
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

2010-02-02 Thread David Magda

On Tue, February 2, 2010 02:24, matthew patton wrote:

> true. but I buy a Ferrari for the engine and bodywork and chassis
> engineering. It is totally criminal what Sun/EMC/Dell/Netapp do charging
> customers 10x the open-market rate for standard drives. A RE3/4 or NS
> drive is the same damn thing no matter if I buy it from ebay or my local
> distributor. Dell/Sun/Netapp buy drives by the container load. Oh sure, I
> don't mind paying an extra couple pennies/GB for all the strenuous efforts
> the vendors spend on firmware verification (HA!).

Tell that to Intel and their SSD firmware team, or Seagate:

http://mswhs.com/2009/01/21/seagate-hard-drive-firmware-bug/

Heck, even things as "low-end" as Netgear's ReadyNAS product specify using
only certain versions of the firmware on many drives:

http://www.readynas.com/?page_id=82

You buy enterprise drives to make sure they work as advertised and don't
drop SYNC commands on the floor and then lie to ZFS about it.

Disk may be cheap, but redundancy, iops, backups, and testing are not.

As someone else suggested, you may want to skip the "enterprise" enclosure
and go with a "consumer" one instead if price is a concern. From home
workloads it may be sufficient to meet your needs.

___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives

Ok, I see that the chassi contains a mother board. So never mind that question.

Another q:
Is it possible to have large chassi with lots of drives, and the opensolaris in 
another chassi, how do you connect them both?
-- 
This message posted from opensolaris.org
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] verging OT: how to buy J4500 w/o overpriced drives