Re: Experiments with send in Linux

[John Hening]

Thanks Avi,
 

> The send doesn't wait, but the kernel has to serve both the send side and 
> the receive side, and if they happen to run on the same CPU and possibly in 
> the same thread, then you'll see them in the flame graph for that CPU or 
> thread.


Ok. I didn't notice that I create a flamegraph for a specififc PID, so it 
is not that case- I mean that I did not sample a CPU but specififc process.

In this case the kernel decided to run the receive processing immediately, 
> so that's what you get.


I see, you right. This is a way kernel implements a loopback.

use unix domain sockets or (better) shared memory for IPC.
>

I cannot, my communication is not interprocess.

Now everything is clear, thanks ;-)

-- 
You received this message because you are subscribed to the Google Groups 
"mechanical-sympathy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to mechanical-sympathy+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Experiments with send in Linux

[Avi Kivity]

On 2018-05-07 01:09, John Hening wrote:




Hello,

I am experimenting with send (from socket to socket through a loopback).



If you want to measure send, don't use loopback. It necessarily includes 
receiving.


I create a flamegraph for that and measure time of execution. It seems 
that is takes relatively much time. (I am not sure whether is it much 
or not). In test I send a 64-byte message in a loop.


send function takes 10-15 microseconds.

1. Is it much or not?


That's a philosophical question that I'll leave to the philosophers.

2. Have you any idea how to improve it? I tried to extend send buffer 
socket, doesn't help- I know that probably it couldn't help.


Don't use loopback, disable meltdown/spectre mitigation, amortize many 
operations on one send(), use unix domain sockets or (better) shared 
memory for IPC.


3. When it comes to a flamegraph: It looks like the kernel waits for a 
response- I mean the function ip_recv and so on. What does the kernel 
try to receive? AFAIK a send function doesn't wait for ACK from receiver.



The send doesn't wait, but the kernel has to serve both the send side 
and the receive side, and if they happen to run on the same CPU and 
possibly in the same thread, then you'll see them in the flame graph for 
that CPU or thread.


In this case the kernel decided to run the receive processing 
immediately, so that's what you get.


--
You received this message because you are subscribed to the Google Groups 
"mechanical-sympathy" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to mechanical-sympathy+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.