Hello, Martin.
Martin A. Fink wrote:
TestOpenSuSE(AHCI)
FreeBSD(AHCI)
---
SSD(vfat 25GB)
Hello, Martin.
Martin A. Fink wrote:
TestOpenSuSE(AHCI)
FreeBSD(AHCI)
---
SSD(vfat 25GB)
Arjan van de Ven wrote:
The problem is: FreeBSD is fast, but lacks of some special drivers. Linux has
all drivers but access to harddisk is unpredictable and thus unreliable!
What can I do??
there's several tunables you can do;
1) increase /sys/block//queue/nr_requests
the linux
On 02/12/07 08:37, Martin A. Fink wrote:
> :~> strace -c -T -o trace.out dd if=/dev/zero of=test.txt bs=10MB count=200
>
> 200+0 Datensätze ein
> 200+0 Datensätze aus
> 20 bytes (2,0 GB) copied, 52,8632 seconds, 37,8 MB/s
You might want to check the raw write & read speed to the device
On Tue, Feb 13, 2007 at 11:25:27AM +, Alan wrote:
> > So where is the difference between SATA-I and SATA-II ?
>
> All physical side if they are on the same controller when you do the
> tests. Mostly latency,
SATA-II is a highly confusing marketing term. It is /not/ a technical
term.
In
> data actually did make it to the media, I wouldn't necessary assume
> that it has. Given that it sounds like you really care about this,
> I'd suggest that you explicitly testing this before making
> assumptions.
FreeBSD 6.1 appears to get it right for some subsets of devices so it
seems a
On Tue, Feb 13, 2007 at 01:32:34PM +0100, Martin A. Fink wrote:
> > Does the FreeBSD fsync sync to media ? Also what controller is being used
> > here, and do you have EHCI USB support running ?
>
> Manual of FreeBSD fsync says it syncs to media.
That didn't answer the question. With SATA in
Martin A. Fink wrote:
>> The needed total bandwidth may be to high and at least the incoming part via
> GigE may have serious overhead.
>> 150MB/s in via (at least 2) GigE, without Zero-Copy there is another 150MB/s
> memory to memory.
>> Then there is the next 150MB/s memory to the discs,
Am Dienstag, 13. Februar 2007 13:24 schrieben Sie:
> Martin A. Fink wrote:
>
> >> Also you have skipped the information how the images "arrive" on the
system
> > (PCI(e) card?), that may be important for an "end to end" view of the
> > problem.
> >
> > Images arrive via Gigabit Ethernet. GigE
Am Dienstag, 13. Februar 2007 12:25 schrieben Sie:
> > Well they do. The Flash disk I have (SATA-I) is capable of 48 MB/s and
this
> > value is reached over the whole disk size by windows as well as by
FreeBSD.
> > See my test results in the first thread.
>
> Ok a flash disk should be more
Martin A. Fink wrote:
>> Also you have skipped the information how the images "arrive" on the system
> (PCI(e) card?), that may be important for an "end to end" view of the
> problem.
>
> Images arrive via Gigabit Ethernet. GigE Vision standard. (PCIe x4)
The the next question is:
On Tue, 13 February 2007 11:29:18 +0100, Martin A. Fink wrote:
>
> Please Read Carefully! I talk about flash disk, not normal harddisks. There
> are no mechanical parts in flash disks, only flash memory. And therefore
> 48MB/s is excellent (compared to all other available disks)
>
> [...]
>
>
On Tue, 13 February 2007 11:27:58 +, Alan wrote:
>
> isn't yet a heavily optimised libata path. Secondly erase block size
> matters with flash drives so the bigger each I/O the better erase block
> behaviour we should get.
Although that should max out somewhere between 16KiB and 128KiB,
> there's several tunables you can do;
> 1) increase /sys/block//queue/nr_requests
>the linux default is on the low side
> 5) echo a larger value into /sys/block//queue/max_sectors_kb
>the default seems to be 512 which is... really low. The hw max is in
>another file in that directory;
> Well they do. The Flash disk I have (SATA-I) is capable of 48 MB/s and this
> value is reached over the whole disk size by windows as well as by FreeBSD.
> See my test results in the first thread.
Ok a flash disk should be more stable
> My Seagate Barracuda Harddisk drive (SATA-II) starts
Am Dienstag, 13. Februar 2007 11:16 schrieben Sie:
> Martin A. Fink wrote:
> > Am Dienstag, 13. Februar 2007 00:31 schrieben Sie:
> >> Martin A. Fink wrote:
> >>> I have to store big amounts of data coming from 2 digital cameras to
disk.
> >>> Thus I have to write blocks of around 1 MB at 30 to
On Tue, 2007-02-13 at 12:18 +0100, Andi Kleen wrote:
> Arjan van de Ven <[EMAIL PROTECTED]> writes:
>
> > > >
> > > The problem is: FreeBSD is fast, but lacks of some special drivers. Linux
> > > has
> > > all drivers but access to harddisk is unpredictable and thus unreliable!
> > > What can
Arjan van de Ven <[EMAIL PROTECTED]> writes:
> > >
> > The problem is: FreeBSD is fast, but lacks of some special drivers. Linux
> > has
> > all drivers but access to harddisk is unpredictable and thus unreliable!
> > What can I do??
>
>
> there's several tunables you can do;
[...] Well
Martin A. Fink wrote:
> Am Dienstag, 13. Februar 2007 00:31 schrieben Sie:
>> Martin A. Fink wrote:
>>> I have to store big amounts of data coming from 2 digital cameras to disk.
>>> Thus I have to write blocks of around 1 MB at 30 to 50 frames per second
>>> for
>>> a long period of time. So
> >
> The problem is: FreeBSD is fast, but lacks of some special drivers. Linux has
> all drivers but access to harddisk is unpredictable and thus unreliable!
> What can I do??
there's several tunables you can do;
1) increase /sys/block//queue/nr_requests
the linux default is on the low
Am Montag, 12. Februar 2007 20:08 schrieben Sie:
> On Mon, 12 Feb 2007 18:56:29 +0100
> "Martin A. Fink" <[EMAIL PROTECTED]> wrote:
>
> > I have to store big amounts of data coming from 2 digital cameras to disk.
> > Thus I have to write blocks of around 1 MB at 30 to 50 frames per second
for
Am Dienstag, 13. Februar 2007 00:31 schrieben Sie:
> Martin A. Fink wrote:
> > I have to store big amounts of data coming from 2 digital cameras to disk.
> > Thus I have to write blocks of around 1 MB at 30 to 50 frames per second
for
> > a long period of time. So it is important for me that
Am Dienstag, 13. Februar 2007 00:31 schrieben Sie:
Martin A. Fink wrote:
I have to store big amounts of data coming from 2 digital cameras to disk.
Thus I have to write blocks of around 1 MB at 30 to 50 frames per second
for
a long period of time. So it is important for me that the
Am Montag, 12. Februar 2007 20:08 schrieben Sie:
On Mon, 12 Feb 2007 18:56:29 +0100
Martin A. Fink [EMAIL PROTECTED] wrote:
I have to store big amounts of data coming from 2 digital cameras to disk.
Thus I have to write blocks of around 1 MB at 30 to 50 frames per second
for
a long
The problem is: FreeBSD is fast, but lacks of some special drivers. Linux has
all drivers but access to harddisk is unpredictable and thus unreliable!
What can I do??
there's several tunables you can do;
1) increase /sys/block/device/queue/nr_requests
the linux default is on the low
Martin A. Fink wrote:
Am Dienstag, 13. Februar 2007 00:31 schrieben Sie:
Martin A. Fink wrote:
I have to store big amounts of data coming from 2 digital cameras to disk.
Thus I have to write blocks of around 1 MB at 30 to 50 frames per second
for
a long period of time. So it is important
Arjan van de Ven [EMAIL PROTECTED] writes:
The problem is: FreeBSD is fast, but lacks of some special drivers. Linux
has
all drivers but access to harddisk is unpredictable and thus unreliable!
What can I do??
there's several tunables you can do;
[...] Well Linux certainly
On Tue, 2007-02-13 at 12:18 +0100, Andi Kleen wrote:
Arjan van de Ven [EMAIL PROTECTED] writes:
The problem is: FreeBSD is fast, but lacks of some special drivers. Linux
has
all drivers but access to harddisk is unpredictable and thus unreliable!
What can I do??
Am Dienstag, 13. Februar 2007 11:16 schrieben Sie:
Martin A. Fink wrote:
Am Dienstag, 13. Februar 2007 00:31 schrieben Sie:
Martin A. Fink wrote:
I have to store big amounts of data coming from 2 digital cameras to
disk.
Thus I have to write blocks of around 1 MB at 30 to 50 frames per
Well they do. The Flash disk I have (SATA-I) is capable of 48 MB/s and this
value is reached over the whole disk size by windows as well as by FreeBSD.
See my test results in the first thread.
Ok a flash disk should be more stable
My Seagate Barracuda Harddisk drive (SATA-II) starts with
there's several tunables you can do;
1) increase /sys/block/device/queue/nr_requests
the linux default is on the low side
5) echo a larger value into /sys/block/device/queue/max_sectors_kb
the default seems to be 512 which is... really low. The hw max is in
another file in that
On Tue, 13 February 2007 11:27:58 +, Alan wrote:
isn't yet a heavily optimised libata path. Secondly erase block size
matters with flash drives so the bigger each I/O the better erase block
behaviour we should get.
Although that should max out somewhere between 16KiB and 128KiB,
On Tue, 13 February 2007 11:29:18 +0100, Martin A. Fink wrote:
Please Read Carefully! I talk about flash disk, not normal harddisks. There
are no mechanical parts in flash disks, only flash memory. And therefore
48MB/s is excellent (compared to all other available disks)
[...]
Well.
Martin A. Fink wrote:
Also you have skipped the information how the images arrive on the system
(PCI(e) card?), that may be important for an end to end view of the
problem.
Images arrive via Gigabit Ethernet. GigE Vision standard. (PCIe x4)
The the next question is: ChipSet/Used
Am Dienstag, 13. Februar 2007 12:25 schrieben Sie:
Well they do. The Flash disk I have (SATA-I) is capable of 48 MB/s and
this
value is reached over the whole disk size by windows as well as by
FreeBSD.
See my test results in the first thread.
Ok a flash disk should be more stable
Am Dienstag, 13. Februar 2007 13:24 schrieben Sie:
Martin A. Fink wrote:
Also you have skipped the information how the images arrive on the
system
(PCI(e) card?), that may be important for an end to end view of the
problem.
Images arrive via Gigabit Ethernet. GigE Vision standard.
Martin A. Fink wrote:
The needed total bandwidth may be to high and at least the incoming part via
GigE may have serious overhead.
150MB/s in via (at least 2) GigE, without Zero-Copy there is another 150MB/s
memory to memory.
Then there is the next 150MB/s memory to the discs, without
On Tue, Feb 13, 2007 at 01:32:34PM +0100, Martin A. Fink wrote:
Does the FreeBSD fsync sync to media ? Also what controller is being used
here, and do you have EHCI USB support running ?
Manual of FreeBSD fsync says it syncs to media.
That didn't answer the question. With SATA in
data actually did make it to the media, I wouldn't necessary assume
that it has. Given that it sounds like you really care about this,
I'd suggest that you explicitly testing this before making
assumptions.
FreeBSD 6.1 appears to get it right for some subsets of devices so it
seems a
On Tue, Feb 13, 2007 at 11:25:27AM +, Alan wrote:
So where is the difference between SATA-I and SATA-II ?
All physical side if they are on the same controller when you do the
tests. Mostly latency,
SATA-II is a highly confusing marketing term. It is /not/ a technical
term.
In some
On 02/12/07 08:37, Martin A. Fink wrote:
:~ strace -c -T -o trace.out dd if=/dev/zero of=test.txt bs=10MB count=200
200+0 Datensätze ein
200+0 Datensätze aus
20 bytes (2,0 GB) copied, 52,8632 seconds, 37,8 MB/s
You might want to check the raw write read speed to the device
without
Arjan van de Ven wrote:
The problem is: FreeBSD is fast, but lacks of some special drivers. Linux has
all drivers but access to harddisk is unpredictable and thus unreliable!
What can I do??
there's several tunables you can do;
1) increase /sys/block/device/queue/nr_requests
the
Martin A. Fink wrote:
> I have to store big amounts of data coming from 2 digital cameras to disk.
> Thus I have to write blocks of around 1 MB at 30 to 50 frames per second for
> a long period of time. So it is important for me that the harddisk drive is
> reliable in the sense of "if it is
Hi Alan et al.
On Mon, 2007-02-12 at 19:08 +, Alan wrote:
> I'm not sure you'll get 50MB/sec sustained to work although you might
> with a good current drive used for nothing else, a linear stream of data
> (no seeking and file system overhead), and a non PCI controller (PCI
> Express, host
On Mon, 12 Feb 2007 18:56:29 +0100
"Martin A. Fink" <[EMAIL PROTECTED]> wrote:
> I have to store big amounts of data coming from 2 digital cameras to disk.
> Thus I have to write blocks of around 1 MB at 30 to 50 frames per second for
> a long period of time. So it is important for me that the
Martin A. Fink wrote:
> This means, that the CPU is only 7.3 of 52.8 seconds working.
...
> It looks like
> the SATA driver simply blocks the CPU while doing whatever...
The system sleeps while waiting for the disk (actually, for the SATA
host port) to be done with its work.
As Andi explained,
On 2/12/07, Martin A. Fink <[EMAIL PROTECTED]> wrote:
Am Montag, 12. Februar 2007 19:41 schrieben Sie:
I have to store big amounts of data coming from 2 digital cameras to disk.
Thus I have to write blocks of around 1 MB at 30 to 50 frames per second for
a long period of time. So it is important
Am Montag, 12. Februar 2007 19:41 schrieben Sie:
> "Martin A. Fink" <[EMAIL PROTECTED]> writes:
>
> Your mailer seems to be broken. It drops cc.
> >
> > If you call fsync in BSD then you get what you expect. anything that is
still
> > not on disk will be written. Afterwards fsync returns... So
System Details:
dmesg: (parts)
Bootdata ok (command line is root=/dev/sda7 vga=0x31aresume=/dev/sda5
splash=silent)
Linux version 2.6.18.2-34-default ([EMAIL PROTECTED]) (gcc version 4.1.2
20061115 (prerelease) (SUSE Linux)) #1 SMP Mon Nov 27 11:46:27 UTC 2006
...
Using ACPI (MADT) for SMP
"Martin A. Fink" <[EMAIL PROTECTED]> writes:
Your mailer seems to be broken. It drops cc.
>
> If you call fsync in BSD then you get what you expect. anything that is still
> not on disk will be written. Afterwards fsync returns... So this should be
> the same like with linux?!
Not
Some more info:
:~> strace -c -T -o trace.out dd if=/dev/zero of=test.txt bs=10MB count=200
200+0 Datensätze ein
200+0 Datensätze aus
20 bytes (2,0 GB) copied, 52,8632 seconds, 37,8 MB/s
test.txt:
% time seconds usecs/call callserrors syscall
-- --- ---
Am Montag, 12. Februar 2007 18:04 schrieb Andi Kleen:
> "Martin A. Fink" <[EMAIL PROTECTED]> writes:
> >
> > What I did:
> > I wrote blocks of 1 MB size to file. Each 1 GB I made a fsync and took the
> > time. For those tests with filesystems I wrote files of 1 GB size,
otherwise
> > I just
"Martin A. Fink" <[EMAIL PROTECTED]> writes:
>
> What I did:
> I wrote blocks of 1 MB size to file. Each 1 GB I made a fsync and took the
> time. For those tests with filesystems I wrote files of 1 GB size, otherwise
> I just wrote to the raw device.
Newer Linux versions depending on the disk
Martin A. Fink [EMAIL PROTECTED] writes:
What I did:
I wrote blocks of 1 MB size to file. Each 1 GB I made a fsync and took the
time. For those tests with filesystems I wrote files of 1 GB size, otherwise
I just wrote to the raw device.
Newer Linux versions depending on the disk and the
Am Montag, 12. Februar 2007 18:04 schrieb Andi Kleen:
Martin A. Fink [EMAIL PROTECTED] writes:
What I did:
I wrote blocks of 1 MB size to file. Each 1 GB I made a fsync and took the
time. For those tests with filesystems I wrote files of 1 GB size,
otherwise
I just wrote to the raw
Martin A. Fink [EMAIL PROTECTED] writes:
Your mailer seems to be broken. It drops cc.
If you call fsync in BSD then you get what you expect. anything that is still
not on disk will be written. Afterwards fsync returns... So this should be
the same like with linux?!
Not necessarily. The
System Details:
dmesg: (parts)
Bootdata ok (command line is root=/dev/sda7 vga=0x31aresume=/dev/sda5
splash=silent)
Linux version 2.6.18.2-34-default ([EMAIL PROTECTED]) (gcc version 4.1.2
20061115 (prerelease) (SUSE Linux)) #1 SMP Mon Nov 27 11:46:27 UTC 2006
...
Using ACPI (MADT) for SMP
Am Montag, 12. Februar 2007 19:41 schrieben Sie:
Martin A. Fink [EMAIL PROTECTED] writes:
Your mailer seems to be broken. It drops cc.
If you call fsync in BSD then you get what you expect. anything that is
still
not on disk will be written. Afterwards fsync returns... So this should
On 2/12/07, Martin A. Fink [EMAIL PROTECTED] wrote:
Am Montag, 12. Februar 2007 19:41 schrieben Sie:
I have to store big amounts of data coming from 2 digital cameras to disk.
Thus I have to write blocks of around 1 MB at 30 to 50 frames per second for
a long period of time. So it is important
Martin A. Fink wrote:
This means, that the CPU is only 7.3 of 52.8 seconds working.
...
It looks like
the SATA driver simply blocks the CPU while doing whatever...
The system sleeps while waiting for the disk (actually, for the SATA
host port) to be done with its work.
As Andi explained, if
On Mon, 12 Feb 2007 18:56:29 +0100
Martin A. Fink [EMAIL PROTECTED] wrote:
I have to store big amounts of data coming from 2 digital cameras to disk.
Thus I have to write blocks of around 1 MB at 30 to 50 frames per second for
a long period of time. So it is important for me that the
Hi Alan et al.
On Mon, 2007-02-12 at 19:08 +, Alan wrote:
I'm not sure you'll get 50MB/sec sustained to work although you might
with a good current drive used for nothing else, a linear stream of data
(no seeking and file system overhead), and a non PCI controller (PCI
Express, host
Martin A. Fink wrote:
I have to store big amounts of data coming from 2 digital cameras to disk.
Thus I have to write blocks of around 1 MB at 30 to 50 frames per second for
a long period of time. So it is important for me that the harddisk drive is
reliable in the sense of if it is capable
63 matches
Mail list logo
