Re: [sqlite] SQLite database on a certain high-performance "SSD"

[Mark]

It's an ioDrive. We updated the firmware to the just-released version, 
and it made a WORLD of difference. It's now performing at the level we'd 
expect (which is quite impressive!)

Talking with their performance engineer, they do indeed suggest 
disabling the OS cache in some cases. We haven't gone that route yet to 
see what happens.


Ken wrote:
> Interesting
> 
> Mind if we ask what the SSD device brand and model is?
> 
> Is it a disk backed type of device with equal memory in front, I recall 
> seeing devices like this about 7 years ago. I'm thinking that the sync call 
> is causing the device to write its memory contents back out to disk (ie to be 
> persisted). Just a thought...
> 
> 
> 
> --- On Wed, 9/23/09, Pavel Ivanov <paiva...@gmail.com> wrote:
> 
>> From: Pavel Ivanov <paiva...@gmail.com>
>> Subject: Re: [sqlite] SQLite database on a certain high-performance "SSD"
>> To: "General Discussion of SQLite Database" <sqlite-users@sqlite.org>
>> Date: Wednesday, September 23, 2009, 11:21 AM
>>> Is the only change the absence
>> of a call to "fsync()" when turning
>>> synchronous off? If so, I can conclusively say that
>> fsync() is very slow
>>> on this storage device.
>> Yes, the only action of synchronous = off is to turn off
>> calls to
>> fsync() which is called at least twice during each commit.
>>
>> Pavel
>>
>> On Wed, Sep 23, 2009 at 11:40 AM, Mark <godef...@gmail.com>
>> wrote:
>>> On a RAID-5 array of 4x SAS disks, turning the sync
>> off made it about 2x
>>> faster, give or take.
>>>
>>> On the "SSD", it was about 150x faster.
>>>
>>> Is the only change the absence of a call to "fsync()"
>> when turning
>>> synchronous off? If so, I can conclusively say that
>> fsync() is very slow
>>> on this storage device.
>>>
>>> Thanks for the suggestion.
>>>
>>> Mark
>>>
>>>
>>> Pavel Ivanov wrote:
>>>> If you execute
>>>>
>>>> pragma synchronous = off;
>>>>
>>>> you'll be able to compare performance with syncs
>> and without them. So
>>>> if you make this comparison on standard spinning
>> disk and on SSD
>>>> you'll see if syncs on SSD indeed extra-ordinary
>> slow.
>>>> Pavel
>>>>
>>>> On Wed, Sep 23, 2009 at 10:09 AM, Mark <godef...@gmail.com>
>> wrote:
>>>>> It's very possible, but I don't know how to
>> tell. Is there an easy way
>>>>> to know if the sync() calls are taking
>> inordinately long?
>>>>> Mark
>>>>>
>>>>>
>>>>> Thomas Briggs wrote:
>>>>>>Is the sync necessary to commit a
>> transaction slow?  Performance of
>>>>>> that sync depends on the OS, file system,
>> hardwar, etc. IIRC, so IOs
>>>>>> may be fast but it's possible that the
>> syncs are killing you.
>>>>>>-T
>>>>>>
>>>>>> On Tue, Sep 22, 2009 at 5:14 PM, Mark
>> <godef...@gmail.com>
>> wrote:
>>>>>>> Lothar Scholz wrote:
>>>>>>>> Hello Mark,
>>>>>>>>
>>>>>>>> Tuesday, September 22, 2009,
>> 3:53:48 AM, you wrote:
>>>>>>>> M> I've currently got a loaner
>> high-performance flash-based "SSD" (let's
>>>>>>>> M> just say it doesn't connect
>> to any disk controllers) that I'm testing
>>>>>>>> M> for performance. I've run my
>> application against it, and I believe that
>>>>>>>> M> I should see numbers MUCH
>> higher than I do. When I run my test app on a
>>>>>>>> M> normal SATA 7200 RPM disk, I
>> get a certain performance, and on the "SSD"
>>>>>>>> M> I get about 1/10th that
>> speed. On an array of SAS disks I get numbers
>>>>>>>> M> that are about 5x faster
>> than my SATA disk, so my software itself isn't
>>>>>>>> M> (I believe) the bottleneck.
>>>>>>>>
>>>>>>>> M> I'm wondering if anyone has
>> any tips for "optimizing" for this sort of
>>>>>>>> M> storage solution.
>>>>>>>>
>>>>>>>> Throw it into the trash bin and
>> buy a new one which has a 3rd
>>>>>>>> generation controller and at least
>> 64MB fast cache. The old JMicron
>>>>>>>> controller that many low cost SSD
>> still use was developed for Flash
>>>>>>>> USB sticks.
>>>>>>>>
>>>>>>>> With modern SSD like the latest
>> Samsung should give you at least the
>>>>>>>> same performance as the SATA. If
>> it gets better depends on file size
>>>>>>>> and cache. Are you sure that the
>> SAS RAID Controller is not keeping
>>>>>>>> everything in the controller
>> cache?
>>>>>>> This isn't an "SSD". It's connected
>> directly to the PCI Express bus, and
>>>>>>> "low cost" it certainly is NOT. It's
>> much more valuable than the server
>>>>>>> it's plugged into.
>>>>>>>
>>>>>>> I've run benchmark tests (iometer),
>> and the benchmarks show it's as fast
>>>>>>> as the mfgr says it should be
>> (~700MB/sec read and write bandwidth,
>>>>>>>  >115,000 IOPS) but it performs
>> quite poorly when I run my app on it. I
>>>>>>> can't figure out why.
>>>>>>>
>>>>>>> Mark

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite database on a certain high-performance "SSD"

[Kosenko Max]

Mark Parker-4 wrote:
> This isn't an "SSD". It's connected directly to the PCI Express bus, and 
> "low cost" it certainly is NOT. It's much more valuable than the server
> it's plugged into.

Check page size. It might be less than cluster of your ioExtreme. You should
also think about actually turning off filesystem cache on that file (with
increase of sqlite in-memory page cache) and that can require small
adjustments in SQLite sources for you.

-
Best Regards.
Max Kosenko.
-- 
View this message in context: 
http://www.nabble.com/SQLite-database-on-a-certain-high-performance-%22SSD%22-tp25550787p25574692.html
Sent from the SQLite mailing list archive at Nabble.com.

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite database on a certain high-performance "SSD"

[Ken]

Interesting

Mind if we ask what the SSD device brand and model is?

Is it a disk backed type of device with equal memory in front, I recall seeing 
devices like this about 7 years ago. I'm thinking that the sync call is causing 
the device to write its memory contents back out to disk (ie to be persisted). 
Just a thought...



--- On Wed, 9/23/09, Pavel Ivanov <paiva...@gmail.com> wrote:

> From: Pavel Ivanov <paiva...@gmail.com>
> Subject: Re: [sqlite] SQLite database on a certain high-performance "SSD"
> To: "General Discussion of SQLite Database" <sqlite-users@sqlite.org>
> Date: Wednesday, September 23, 2009, 11:21 AM
> > Is the only change the absence
> of a call to "fsync()" when turning
> > synchronous off? If so, I can conclusively say that
> fsync() is very slow
> > on this storage device.
> 
> Yes, the only action of synchronous = off is to turn off
> calls to
> fsync() which is called at least twice during each commit.
> 
> Pavel
> 
> On Wed, Sep 23, 2009 at 11:40 AM, Mark <godef...@gmail.com>
> wrote:
> >
> > On a RAID-5 array of 4x SAS disks, turning the sync
> off made it about 2x
> > faster, give or take.
> >
> > On the "SSD", it was about 150x faster.
> >
> > Is the only change the absence of a call to "fsync()"
> when turning
> > synchronous off? If so, I can conclusively say that
> fsync() is very slow
> > on this storage device.
> >
> > Thanks for the suggestion.
> >
> > Mark
> >
> >
> > Pavel Ivanov wrote:
> >> If you execute
> >>
> >> pragma synchronous = off;
> >>
> >> you'll be able to compare performance with syncs
> and without them. So
> >> if you make this comparison on standard spinning
> disk and on SSD
> >> you'll see if syncs on SSD indeed extra-ordinary
> slow.
> >>
> >> Pavel
> >>
> >> On Wed, Sep 23, 2009 at 10:09 AM, Mark <godef...@gmail.com>
> wrote:
> >>> It's very possible, but I don't know how to
> tell. Is there an easy way
> >>> to know if the sync() calls are taking
> inordinately long?
> >>>
> >>> Mark
> >>>
> >>>
> >>> Thomas Briggs wrote:
> >>>>    Is the sync necessary to commit a
> transaction slow?  Performance of
> >>>> that sync depends on the OS, file system,
> hardwar, etc. IIRC, so IOs
> >>>> may be fast but it's possible that the
> syncs are killing you.
> >>>>
> >>>>    -T
> >>>>
> >>>> On Tue, Sep 22, 2009 at 5:14 PM, Mark
> <godef...@gmail.com>
> wrote:
> >>>>> Lothar Scholz wrote:
> >>>>>> Hello Mark,
> >>>>>>
> >>>>>> Tuesday, September 22, 2009,
> 3:53:48 AM, you wrote:
> >>>>>>
> >>>>>> M> I've currently got a loaner
> high-performance flash-based "SSD" (let's
> >>>>>> M> just say it doesn't connect
> to any disk controllers) that I'm testing
> >>>>>> M> for performance. I've run my
> application against it, and I believe that
> >>>>>> M> I should see numbers MUCH
> higher than I do. When I run my test app on a
> >>>>>> M> normal SATA 7200 RPM disk, I
> get a certain performance, and on the "SSD"
> >>>>>> M> I get about 1/10th that
> speed. On an array of SAS disks I get numbers
> >>>>>> M> that are about 5x faster
> than my SATA disk, so my software itself isn't
> >>>>>> M> (I believe) the bottleneck.
> >>>>>>
> >>>>>> M> I'm wondering if anyone has
> any tips for "optimizing" for this sort of
> >>>>>> M> storage solution.
> >>>>>>
> >>>>>> Throw it into the trash bin and
> buy a new one which has a 3rd
> >>>>>> generation controller and at least
> 64MB fast cache. The old JMicron
> >>>>>> controller that many low cost SSD
> still use was developed for Flash
> >>>>>> USB sticks.
> >>>>>>
> >>>>>> With modern SSD like the latest
> Samsung should give you at least the
> >>>>>> same performance as the SATA. If
> it gets better depends on file size
> >>>>>> and cache. Are you sure that the
> SAS RAID Controller is not keeping
> >>>>>> everything in the controller
> cache?
> >>>>> This isn't an "SSD". It's connected
> directly to the PCI Express bus, and
> >>>>> "low cost" it certainly is NOT. It's
> much more valuable than the server
> >>>>> it's plugged into.
> >>>>>
> >>>>> I've run benchmark tests (iometer),
> and the benchmarks show it's as fast
> >>>>> as the mfgr says it should be
> (~700MB/sec read and write bandwidth,
> >>>>>  >115,000 IOPS) but it performs
> quite poorly when I run my app on it. I
> >>>>> can't figure out why.
> >>>>>
> >>>>> Mark
> >
> > ___
> > sqlite-users mailing list
> > sqlite-users@sqlite.org
> > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> >
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> 
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite database on a certain high-performance "SSD"

[Pavel Ivanov]

> Is the only change the absence of a call to "fsync()" when turning
> synchronous off? If so, I can conclusively say that fsync() is very slow
> on this storage device.

Yes, the only action of synchronous = off is to turn off calls to
fsync() which is called at least twice during each commit.

Pavel

On Wed, Sep 23, 2009 at 11:40 AM, Mark  wrote:
>
> On a RAID-5 array of 4x SAS disks, turning the sync off made it about 2x
> faster, give or take.
>
> On the "SSD", it was about 150x faster.
>
> Is the only change the absence of a call to "fsync()" when turning
> synchronous off? If so, I can conclusively say that fsync() is very slow
> on this storage device.
>
> Thanks for the suggestion.
>
> Mark
>
>
> Pavel Ivanov wrote:
>> If you execute
>>
>> pragma synchronous = off;
>>
>> you'll be able to compare performance with syncs and without them. So
>> if you make this comparison on standard spinning disk and on SSD
>> you'll see if syncs on SSD indeed extra-ordinary slow.
>>
>> Pavel
>>
>> On Wed, Sep 23, 2009 at 10:09 AM, Mark  wrote:
>>> It's very possible, but I don't know how to tell. Is there an easy way
>>> to know if the sync() calls are taking inordinately long?
>>>
>>> Mark
>>>
>>>
>>> Thomas Briggs wrote:
    Is the sync necessary to commit a transaction slow?  Performance of
 that sync depends on the OS, file system, hardwar, etc. IIRC, so IOs
 may be fast but it's possible that the syncs are killing you.

    -T

 On Tue, Sep 22, 2009 at 5:14 PM, Mark  wrote:
> Lothar Scholz wrote:
>> Hello Mark,
>>
>> Tuesday, September 22, 2009, 3:53:48 AM, you wrote:
>>
>> M> I've currently got a loaner high-performance flash-based "SSD" (let's
>> M> just say it doesn't connect to any disk controllers) that I'm testing
>> M> for performance. I've run my application against it, and I believe 
>> that
>> M> I should see numbers MUCH higher than I do. When I run my test app on 
>> a
>> M> normal SATA 7200 RPM disk, I get a certain performance, and on the 
>> "SSD"
>> M> I get about 1/10th that speed. On an array of SAS disks I get numbers
>> M> that are about 5x faster than my SATA disk, so my software itself 
>> isn't
>> M> (I believe) the bottleneck.
>>
>> M> I'm wondering if anyone has any tips for "optimizing" for this sort of
>> M> storage solution.
>>
>> Throw it into the trash bin and buy a new one which has a 3rd
>> generation controller and at least 64MB fast cache. The old JMicron
>> controller that many low cost SSD still use was developed for Flash
>> USB sticks.
>>
>> With modern SSD like the latest Samsung should give you at least the
>> same performance as the SATA. If it gets better depends on file size
>> and cache. Are you sure that the SAS RAID Controller is not keeping
>> everything in the controller cache?
> This isn't an "SSD". It's connected directly to the PCI Express bus, and
> "low cost" it certainly is NOT. It's much more valuable than the server
> it's plugged into.
>
> I've run benchmark tests (iometer), and the benchmarks show it's as fast
> as the mfgr says it should be (~700MB/sec read and write bandwidth,
>  >115,000 IOPS) but it performs quite poorly when I run my app on it. I
> can't figure out why.
>
> Mark
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite database on a certain high-performance "SSD"

[Mark]

On a RAID-5 array of 4x SAS disks, turning the sync off made it about 2x 
faster, give or take.

On the "SSD", it was about 150x faster.

Is the only change the absence of a call to "fsync()" when turning 
synchronous off? If so, I can conclusively say that fsync() is very slow 
on this storage device.

Thanks for the suggestion.

Mark


Pavel Ivanov wrote:
> If you execute
> 
> pragma synchronous = off;
> 
> you'll be able to compare performance with syncs and without them. So
> if you make this comparison on standard spinning disk and on SSD
> you'll see if syncs on SSD indeed extra-ordinary slow.
> 
> Pavel
> 
> On Wed, Sep 23, 2009 at 10:09 AM, Mark  wrote:
>> It's very possible, but I don't know how to tell. Is there an easy way
>> to know if the sync() calls are taking inordinately long?
>>
>> Mark
>>
>>
>> Thomas Briggs wrote:
>>>Is the sync necessary to commit a transaction slow?  Performance of
>>> that sync depends on the OS, file system, hardwar, etc. IIRC, so IOs
>>> may be fast but it's possible that the syncs are killing you.
>>>
>>>-T
>>>
>>> On Tue, Sep 22, 2009 at 5:14 PM, Mark  wrote:
 Lothar Scholz wrote:
> Hello Mark,
>
> Tuesday, September 22, 2009, 3:53:48 AM, you wrote:
>
> M> I've currently got a loaner high-performance flash-based "SSD" (let's
> M> just say it doesn't connect to any disk controllers) that I'm testing
> M> for performance. I've run my application against it, and I believe that
> M> I should see numbers MUCH higher than I do. When I run my test app on a
> M> normal SATA 7200 RPM disk, I get a certain performance, and on the 
> "SSD"
> M> I get about 1/10th that speed. On an array of SAS disks I get numbers
> M> that are about 5x faster than my SATA disk, so my software itself isn't
> M> (I believe) the bottleneck.
>
> M> I'm wondering if anyone has any tips for "optimizing" for this sort of
> M> storage solution.
>
> Throw it into the trash bin and buy a new one which has a 3rd
> generation controller and at least 64MB fast cache. The old JMicron
> controller that many low cost SSD still use was developed for Flash
> USB sticks.
>
> With modern SSD like the latest Samsung should give you at least the
> same performance as the SATA. If it gets better depends on file size
> and cache. Are you sure that the SAS RAID Controller is not keeping
> everything in the controller cache?
 This isn't an "SSD". It's connected directly to the PCI Express bus, and
 "low cost" it certainly is NOT. It's much more valuable than the server
 it's plugged into.

 I've run benchmark tests (iometer), and the benchmarks show it's as fast
 as the mfgr says it should be (~700MB/sec read and write bandwidth,
  >115,000 IOPS) but it performs quite poorly when I run my app on it. I
 can't figure out why.

 Mark

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite database on a certain high-performance "SSD"

[Pavel Ivanov]

If you execute

pragma synchronous = off;

you'll be able to compare performance with syncs and without them. So
if you make this comparison on standard spinning disk and on SSD
you'll see if syncs on SSD indeed extra-ordinary slow.

Pavel

On Wed, Sep 23, 2009 at 10:09 AM, Mark  wrote:
> It's very possible, but I don't know how to tell. Is there an easy way
> to know if the sync() calls are taking inordinately long?
>
> Mark
>
>
> Thomas Briggs wrote:
>>    Is the sync necessary to commit a transaction slow?  Performance of
>> that sync depends on the OS, file system, hardwar, etc. IIRC, so IOs
>> may be fast but it's possible that the syncs are killing you.
>>
>>    -T
>>
>> On Tue, Sep 22, 2009 at 5:14 PM, Mark  wrote:
>>> Lothar Scholz wrote:
 Hello Mark,

 Tuesday, September 22, 2009, 3:53:48 AM, you wrote:

 M> I've currently got a loaner high-performance flash-based "SSD" (let's
 M> just say it doesn't connect to any disk controllers) that I'm testing
 M> for performance. I've run my application against it, and I believe that
 M> I should see numbers MUCH higher than I do. When I run my test app on a
 M> normal SATA 7200 RPM disk, I get a certain performance, and on the "SSD"
 M> I get about 1/10th that speed. On an array of SAS disks I get numbers
 M> that are about 5x faster than my SATA disk, so my software itself isn't
 M> (I believe) the bottleneck.

 M> I'm wondering if anyone has any tips for "optimizing" for this sort of
 M> storage solution.

 Throw it into the trash bin and buy a new one which has a 3rd
 generation controller and at least 64MB fast cache. The old JMicron
 controller that many low cost SSD still use was developed for Flash
 USB sticks.

 With modern SSD like the latest Samsung should give you at least the
 same performance as the SATA. If it gets better depends on file size
 and cache. Are you sure that the SAS RAID Controller is not keeping
 everything in the controller cache?
>>> This isn't an "SSD". It's connected directly to the PCI Express bus, and
>>> "low cost" it certainly is NOT. It's much more valuable than the server
>>> it's plugged into.
>>>
>>> I've run benchmark tests (iometer), and the benchmarks show it's as fast
>>> as the mfgr says it should be (~700MB/sec read and write bandwidth,
>>>  >115,000 IOPS) but it performs quite poorly when I run my app on it. I
>>> can't figure out why.
>>>
>>> Mark
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite database on a certain high-performance "SSD"

[Mark]

It's very possible, but I don't know how to tell. Is there an easy way 
to know if the sync() calls are taking inordinately long?

Mark


Thomas Briggs wrote:
>Is the sync necessary to commit a transaction slow?  Performance of
> that sync depends on the OS, file system, hardwar, etc. IIRC, so IOs
> may be fast but it's possible that the syncs are killing you.
> 
>-T
> 
> On Tue, Sep 22, 2009 at 5:14 PM, Mark  wrote:
>> Lothar Scholz wrote:
>>> Hello Mark,
>>>
>>> Tuesday, September 22, 2009, 3:53:48 AM, you wrote:
>>>
>>> M> I've currently got a loaner high-performance flash-based "SSD" (let's
>>> M> just say it doesn't connect to any disk controllers) that I'm testing
>>> M> for performance. I've run my application against it, and I believe that
>>> M> I should see numbers MUCH higher than I do. When I run my test app on a
>>> M> normal SATA 7200 RPM disk, I get a certain performance, and on the "SSD"
>>> M> I get about 1/10th that speed. On an array of SAS disks I get numbers
>>> M> that are about 5x faster than my SATA disk, so my software itself isn't
>>> M> (I believe) the bottleneck.
>>>
>>> M> I'm wondering if anyone has any tips for "optimizing" for this sort of
>>> M> storage solution.
>>>
>>> Throw it into the trash bin and buy a new one which has a 3rd
>>> generation controller and at least 64MB fast cache. The old JMicron
>>> controller that many low cost SSD still use was developed for Flash
>>> USB sticks.
>>>
>>> With modern SSD like the latest Samsung should give you at least the
>>> same performance as the SATA. If it gets better depends on file size
>>> and cache. Are you sure that the SAS RAID Controller is not keeping
>>> everything in the controller cache?
>> This isn't an "SSD". It's connected directly to the PCI Express bus, and
>> "low cost" it certainly is NOT. It's much more valuable than the server
>> it's plugged into.
>>
>> I've run benchmark tests (iometer), and the benchmarks show it's as fast
>> as the mfgr says it should be (~700MB/sec read and write bandwidth,
>>  >115,000 IOPS) but it performs quite poorly when I run my app on it. I
>> can't figure out why.
>>
>> Mark

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite database on a certain high-performance "SSD"

[Thomas Briggs]

   Is the sync necessary to commit a transaction slow?  Performance of
that sync depends on the OS, file system, hardwar, etc. IIRC, so IOs
may be fast but it's possible that the syncs are killing you.

   -T

On Tue, Sep 22, 2009 at 5:14 PM, Mark  wrote:
> Lothar Scholz wrote:
>> Hello Mark,
>>
>> Tuesday, September 22, 2009, 3:53:48 AM, you wrote:
>>
>> M> I've currently got a loaner high-performance flash-based "SSD" (let's
>> M> just say it doesn't connect to any disk controllers) that I'm testing
>> M> for performance. I've run my application against it, and I believe that
>> M> I should see numbers MUCH higher than I do. When I run my test app on a
>> M> normal SATA 7200 RPM disk, I get a certain performance, and on the "SSD"
>> M> I get about 1/10th that speed. On an array of SAS disks I get numbers
>> M> that are about 5x faster than my SATA disk, so my software itself isn't
>> M> (I believe) the bottleneck.
>>
>> M> I'm wondering if anyone has any tips for "optimizing" for this sort of
>> M> storage solution.
>>
>> Throw it into the trash bin and buy a new one which has a 3rd
>> generation controller and at least 64MB fast cache. The old JMicron
>> controller that many low cost SSD still use was developed for Flash
>> USB sticks.
>>
>> With modern SSD like the latest Samsung should give you at least the
>> same performance as the SATA. If it gets better depends on file size
>> and cache. Are you sure that the SAS RAID Controller is not keeping
>> everything in the controller cache?
>
> This isn't an "SSD". It's connected directly to the PCI Express bus, and
> "low cost" it certainly is NOT. It's much more valuable than the server
> it's plugged into.
>
> I've run benchmark tests (iometer), and the benchmarks show it's as fast
> as the mfgr says it should be (~700MB/sec read and write bandwidth,
>  >115,000 IOPS) but it performs quite poorly when I run my app on it. I
> can't figure out why.
>
> Mark
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite database on a certain high-performance "SSD"

[Mark]

Lothar Scholz wrote:
> Hello Mark,
> 
> Tuesday, September 22, 2009, 3:53:48 AM, you wrote:
> 
> M> I've currently got a loaner high-performance flash-based "SSD" (let's 
> M> just say it doesn't connect to any disk controllers) that I'm testing 
> M> for performance. I've run my application against it, and I believe that
> M> I should see numbers MUCH higher than I do. When I run my test app on a
> M> normal SATA 7200 RPM disk, I get a certain performance, and on the "SSD"
> M> I get about 1/10th that speed. On an array of SAS disks I get numbers 
> M> that are about 5x faster than my SATA disk, so my software itself isn't
> M> (I believe) the bottleneck.
> 
> M> I'm wondering if anyone has any tips for "optimizing" for this sort of
> M> storage solution.
> 
> Throw it into the trash bin and buy a new one which has a 3rd
> generation controller and at least 64MB fast cache. The old JMicron
> controller that many low cost SSD still use was developed for Flash
> USB sticks.
> 
> With modern SSD like the latest Samsung should give you at least the
> same performance as the SATA. If it gets better depends on file size
> and cache. Are you sure that the SAS RAID Controller is not keeping
> everything in the controller cache?

This isn't an "SSD". It's connected directly to the PCI Express bus, and 
"low cost" it certainly is NOT. It's much more valuable than the server 
it's plugged into.

I've run benchmark tests (iometer), and the benchmarks show it's as fast 
as the mfgr says it should be (~700MB/sec read and write bandwidth, 
 >115,000 IOPS) but it performs quite poorly when I run my app on it. I 
can't figure out why.

Mark

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite database on a certain high-performance "SSD"

[Mark]

Simon Slavin wrote:
> On 22 Sep 2009, at 3:07pm, Ken wrote:
> 
>> How is the SSD connected? Could it be an issue with the transport  
>> layer to the device?
> 
> Also, do you have a storage driver that /knows/ its talking to an  
> SSD ?  Because some operating systems do all sorts of clever stuff  
> (write-queueing, read-ahead caching) on the assumption that they're  
> talking to a spinning hard disk.  And these more than make up for how  
> much faster an SSD should be.
> 
> If your SSD is doing a very good impersonation of a SATA-connected  
> hard disk, it might be fooling the OS into treating it a one.
> 
> Simon.

It's connected directly to the PCI Express bus. It's using it's own 
driver, so I assume it knows it's not a hard disk :)

Mark

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite database on a certain high-performance "SSD"

[Lothar Scholz]

Hello Mark,

Tuesday, September 22, 2009, 3:53:48 AM, you wrote:

M> I've currently got a loaner high-performance flash-based "SSD" (let's 
M> just say it doesn't connect to any disk controllers) that I'm testing 
M> for performance. I've run my application against it, and I believe that
M> I should see numbers MUCH higher than I do. When I run my test app on a
M> normal SATA 7200 RPM disk, I get a certain performance, and on the "SSD"
M> I get about 1/10th that speed. On an array of SAS disks I get numbers 
M> that are about 5x faster than my SATA disk, so my software itself isn't
M> (I believe) the bottleneck.

M> I'm wondering if anyone has any tips for "optimizing" for this sort of
M> storage solution.

Throw it into the trash bin and buy a new one which has a 3rd
generation controller and at least 64MB fast cache. The old JMicron
controller that many low cost SSD still use was developed for Flash
USB sticks.

With modern SSD like the latest Samsung should give you at least the
same performance as the SATA. If it gets better depends on file size
and cache. Are you sure that the SAS RAID Controller is not keeping
everything in the controller cache?

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite database on a certain high-performance "SSD"

[Simon Slavin]

On 22 Sep 2009, at 3:07pm, Ken wrote:

> How is the SSD connected? Could it be an issue with the transport  
> layer to the device?

Also, do you have a storage driver that /knows/ its talking to an  
SSD ?  Because some operating systems do all sorts of clever stuff  
(write-queueing, read-ahead caching) on the assumption that they're  
talking to a spinning hard disk.  And these more than make up for how  
much faster an SSD should be.

If your SSD is doing a very good impersonation of a SATA-connected  
hard disk, it might be fooling the OS into treating it a one.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite database on a certain high-performance "SSD"

[Ken]

How is the SSD connected? Could it be an issue with the transport layer to the 
device?


--- On Mon, 9/21/09, Dave Toll <dave.t...@opentv.com> wrote:

> From: Dave Toll <dave.t...@opentv.com>
> Subject: Re: [sqlite] SQLite database on a certain high-performance "SSD"
> To: "General Discussion of SQLite Database" <sqlite-users@sqlite.org>
> Date: Monday, September 21, 2009, 5:31 PM
> Hi Mark
> 
> I've had a little experience working with flash-based
> filesystems - I'd
> recommend playing with the page_size and temp_store PRAGMAs
> (and of
> course make sure you are using transactions to minimise the
> number of
> file writes) to improve performance.
> 
> Cheers,
> Dave.
> 
> 
> -Original Message-
> From: Mark [mailto:godef...@gmail.com]
> 
> Sent: 21 September 2009 13:54
> To: sqlite-users@sqlite.org
> Subject: [sqlite] SQLite database on a certain
> high-performance "SSD"
> 
> I've currently got a loaner high-performance flash-based
> "SSD" (let's 
> just say it doesn't connect to any disk controllers) that
> I'm testing 
> for performance. I've run my application against it, and I
> believe that 
> I should see numbers MUCH higher than I do. When I run my
> test app on a 
> normal SATA 7200 RPM disk, I get a certain performance, and
> on the "SSD"
> 
> I get about 1/10th that speed. On an array of SAS disks I
> get numbers 
> that are about 5x faster than my SATA disk, so my software
> itself isn't 
> (I believe) the bottleneck.
> 
> I'm wondering if anyone has any tips for "optimizing" for
> this sort of 
> storage solution.
> 
> Also, if anyone has any quick-and-dirty test setups they'd
> like me to 
> run through on this rig, just let me know :)
> 
> 
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> 
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite database on a certain high-performance "SSD"

[Jim Showalter]

SSDs aren't necessarily faster than disks:

http://en.wikipedia.org/wiki/Solid-state_drive#Disadvantages

- Original Message - 
From: "Mark" <godef...@gmail.com>
To: <sqlite-users@sqlite.org>
Sent: Monday, September 21, 2009 1:53 PM
Subject: [sqlite] SQLite database on a certain high-performance "SSD"


> I've currently got a loaner high-performance flash-based "SSD" 
> (let's
> just say it doesn't connect to any disk controllers) that I'm 
> testing
> for performance. I've run my application against it, and I believe 
> that
> I should see numbers MUCH higher than I do. When I run my test app 
> on a
> normal SATA 7200 RPM disk, I get a certain performance, and on the 
> "SSD"
> I get about 1/10th that speed. On an array of SAS disks I get 
> numbers
> that are about 5x faster than my SATA disk, so my software itself 
> isn't
> (I believe) the bottleneck.
>
> I'm wondering if anyone has any tips for "optimizing" for this sort 
> of
> storage solution.
>
> Also, if anyone has any quick-and-dirty test setups they'd like me 
> to
> run through on this rig, just let me know :)
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users 

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite database on a certain high-performance "SSD"

[Cory Nelson]

On Mon, Sep 21, 2009 at 1:53 PM, Mark  wrote:
> I've currently got a loaner high-performance flash-based "SSD" (let's
> just say it doesn't connect to any disk controllers) that I'm testing
> for performance. I've run my application against it, and I believe that
> I should see numbers MUCH higher than I do. When I run my test app on a
> normal SATA 7200 RPM disk, I get a certain performance, and on the "SSD"
> I get about 1/10th that speed. On an array of SAS disks I get numbers
> that are about 5x faster than my SATA disk, so my software itself isn't
> (I believe) the bottleneck.
>
> I'm wondering if anyone has any tips for "optimizing" for this sort of
> storage solution.
>
> Also, if anyone has any quick-and-dirty test setups they'd like me to
> run through on this rig, just let me know :)
>

SSDs get their speed by striping together multiple slower NAND
controllers.  If you're getting 1/10th the speed, it seems plausible
that you are only accessing a single stripe at once.  Optimize your
app to spread access across multiple stripes.  Using a larger page
size and/or performing concurrent reads might help.

And of course, make sure you're using transactions to group operations.

-- 
Cory Nelson
http://int64.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite database on a certain high-performance "SSD"

[Dave Toll]

Hi Mark

I've had a little experience working with flash-based filesystems - I'd
recommend playing with the page_size and temp_store PRAGMAs (and of
course make sure you are using transactions to minimise the number of
file writes) to improve performance.

Cheers,
Dave.


-Original Message-
From: Mark [mailto:godef...@gmail.com] 
Sent: 21 September 2009 13:54
To: sqlite-users@sqlite.org
Subject: [sqlite] SQLite database on a certain high-performance "SSD"

I've currently got a loaner high-performance flash-based "SSD" (let's 
just say it doesn't connect to any disk controllers) that I'm testing 
for performance. I've run my application against it, and I believe that 
I should see numbers MUCH higher than I do. When I run my test app on a 
normal SATA 7200 RPM disk, I get a certain performance, and on the "SSD"

I get about 1/10th that speed. On an array of SAS disks I get numbers 
that are about 5x faster than my SATA disk, so my software itself isn't 
(I believe) the bottleneck.

I'm wondering if anyone has any tips for "optimizing" for this sort of 
storage solution.

Also, if anyone has any quick-and-dirty test setups they'd like me to 
run through on this rig, just let me know :)


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] SQLite database on a certain high-performance "SSD"

[Mark]

I've currently got a loaner high-performance flash-based "SSD" (let's 
just say it doesn't connect to any disk controllers) that I'm testing 
for performance. I've run my application against it, and I believe that 
I should see numbers MUCH higher than I do. When I run my test app on a 
normal SATA 7200 RPM disk, I get a certain performance, and on the "SSD" 
I get about 1/10th that speed. On an array of SAS disks I get numbers 
that are about 5x faster than my SATA disk, so my software itself isn't 
(I believe) the bottleneck.

I'm wondering if anyone has any tips for "optimizing" for this sort of 
storage solution.

Also, if anyone has any quick-and-dirty test setups they'd like me to 
run through on this rig, just let me know :)

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users