Gilboa Davara wrote:
Umm...
As far as I can see, realyfs uses memory buffer for storage, which is
major no-no in my case:
At 50-200MB/sec I'll deplete the system RAM within minutes (even on
AMD64) and as far as I can see, there's no obvious way to commit the
buffers into static storage.
Current me if I'm wrong, but relayfs was designed for fast Kernel/User
transactions and not Kernel/Kernel transactions, right?
I'm looking for 50% match open solution that can be bent to suite my
rather weird requirements.
As the saying goes: Writing you own FS really cuts down on your Doom3
time! :-)
Gilboa,
I'm sorry if I seound harsh, but I don't think you udnerstand your own
needs.
Let's try to analyze them together - you need to store large amount of
data from the network for proccessing by a further entity. What that
data is exactly doesn't matter, but we will note that you might need to
do non trivial handling of the data (encryption).
There obviously is no question of delay involved - you're talking about
writing to disk, using a FIFO to buffer information etc so obviously
there is no serious real time contraints on the data.
What you really care about therefore is - *througput*, NOT delay. For
this, crossing the kernel/userspace and related context switch is NOT a
problem!
The solution boils down to this:
1. You need some way to store information temporary in the kernel
(because it's getting first to the kernel).
2. You then need to handle it (assembly/encryption) and write it to disk.
3. After the data is on disk, you have a different proccess that sends
it to some thrid party, but we really don't care about that.
So long as you didn't perform 2, you want the information to wait in a
temporary buffer (I first wrote "bugger" here, which is rather funny ;-)
until you're done with it so store the data in a temporary in kernel
buffer (or bugger, if you prefer ;-), have a user space proccess read it
from there, proccess it, encrypt it, write it to file.
Have yet another proccess send it to the thrid party (possibly using
the Linux sendfile syscall for efficincy).
Now you can write that "Temporary buffer layer in kernel that can easily
be read by user space: your self but you'll be just wasting DOOM3
cyclesw. This is exactly RelayFS. The rest are simple user space
programs (and if you want to see an example on how to use it see Karim's
most excellent LTT tool that can use RelayFS: http://www.opersys.com/LTT).
Hope this helps,
Gilad
--
Gilad Ben-Yossef <[EMAIL PROTECTED]>
Codefidence. A name you can trust(tm)
Web: http://codefidence.com | SIP: [EMAIL PROTECTED]
IL: +972.9.8650475 ext. 201 | Fax: +972.9.8850643
US: +1.360.2275194 ext. 201 | Cel: +972.52.8260388
"I am Jack's Overwritten Stack Pointer"
-- Hackers Club, the movie
=================================================================
To unsubscribe, send mail to [EMAIL PROTECTED] with
the word "unsubscribe" in the message body, e.g., run the command
echo unsubscribe | mail [EMAIL PROTECTED]
