Gilad, Umm... Interesting. You might be right... but I'm still not convinced. (Though my project manager will love the general idea. To say the least, she doesn't really fancy the idea of writing our own FS :))
> I'm sorry if I seound harsh, but I don't think you udnerstand your own > needs. It has been known to happen.... > > Let's try to analyze them together - you need to store large amount of > data from the network for proccessing by a further entity. What that > data is exactly doesn't matter, but we will note that you might need to > do non trivial handling of the data (encryption). I should add the encryption optional, depending on load and source. (And more important, how fanatical is the client) (There's no way in hell, I'll be able to process and encrypt two OC48 links in real time...) > > There obviously is no question of delay involved - you're talking about > writing to disk, using a FIFO to buffer information etc so obviously > there is no serious real time contraints on the data. As long as the data sequence is maintained, yes. > > What you really care about therefore is - *througput*, NOT delay. For > this, crossing the kernel/userspace and related context switch is NOT a > problem! > > The solution boils down to this: > > 1. You need some way to store information temporary in the kernel > (because it's getting first to the kernel). > 2. You then need to handle it (assembly/encryption) and write it to disk. > 3. After the data is on disk, you have a different proccess that sends > it to some thrid party, but we really don't care about that. No quite: 1. Get data from driver. (Or kernel network stack) 2. (Pre-assemble data), filter data, assemble data. 3. Try sending data to 3'rd party. 4. Optional: encrypt data using a symmetrical encryptions. (Due to performance constrains) 5. Commit data to disk. 6. Read data from disk. 7. Decrypt data. 8. Send data to 3'rd party. > > So long as you didn't perform 2, you want the information to wait in a > temporary buffer (I first wrote "bugger" here, which is rather funny ;-) > until you're done with it so store the data in a temporary in kernel > buffer (or bugger, if you prefer ;-), have a user space proccess read it > from there, proccess it, encrypt it, write it to file. > Have yet another proccess send it to the thrid party (possibly using > the Linux sendfile syscall for efficincy). > Now you can write that "Temporary buffer layer in kernel that can easily > be read by user space: your self but you'll be just wasting DOOM3 > cyclesw. This is exactly RelayFS. The rest are simple user space > programs (and if you want to see an example on how to use it see Karim's > most excellent LTT tool that can use RelayFS: http://www.opersys.com/LTT). > > Hope this helps, > Gilad Interesting... that might work. Let me first point out that once the cells/frames have been processed, I don't care much for timing. (Which bodes well on your solution). However, I'm ****very**** tight on CPU and memory bandwidth. (Even a dual Opteron machine with two memory banks tends to suffocate at a certain point.) No matter what I do I just can't afford to add memcpy's to my system. Here's how I see it: Kernel 1: Device -> SKB -> Reassembly -> Disk. (I can even save the third memcpy [Reassembly -> Disk] I go rewrite the world under me) User: Device -> SKB -> Reassembly ( -> ?) Relayfs -> User: write(2) -> Kernel: sys_write (copy_from_user) -> Disk. Oh... Thanks for the help. I appreciate it. Gilboa ================================================================= To unsubscribe, send mail to [EMAIL PROTECTED] with the word "unsubscribe" in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
