Re: [iovisor-dev] How to use USDT probes without having to pass the pid of the traced program?
Just checked python (built from source with --with-dtrace) USDT semaphores:
[yhs@localhost bin]$ nm python3 | grep semaphore
008eb52c d python_function__entry_semaphore
008eb52e d python_function__return_semaphore
008eb53c d python_gc__done_semaphore
008eb53a d python_gc__start_semaphore
008eb540 d python_import__find__load__done_semaphore
008eb53e d python_import__find__load__start_semaphore
008eb536 d python_instance__delete__done_semaphore
008eb534 d python_instance__delete__start_semaphore
008eb532 d python_instance__new__done_semaphore
008eb530 d python_instance__new__start_semaphore
008eb538 d python_line_semaphore
[yhs@localhost bin]$
[25] .probes PROGBITS 008eb52c 002eb52c
0016 WA 0 0 2
[26] .bss NOBITS 008eb560 002eb542
00020d38 WA 0 0 32
Looks like in this specific case, semaphore is file backed.
However, since semaphore is a weak symbole in the library, the main
program can certainly put it into bss section.
-bash-4.2$ cat u.c
#define _SDT_HAS_SEMAPHORES
#include
int test_probe_semaphore;
int main() {
int i;
for (i = 0; i < 10; i++) {
if (test_probe_semaphore)
STAP_PROBE1(test, probe, i);
}
return 0;
}
-bash-4.2$ gcc u.c
-bash-4.2$ nm a.out | grep semaphore
00601030 B test_probe_semaphore
-bash-4.2$ readelf -S a.out
...
[26] .bss NOBITS 0060102c 102c
000c WA 0 0 4
[27] .comment PROGBITS 102c
002d 0001 MS 0 0 1
On Wed, Jan 17, 2018 at 9:20 AM, Alexei Starovoitov via iovisor-dev
wrote:
> yep. uprobe is always 'int 3' so far, since kernel needs to take control.
> we can add something like writing arbitrary value as long as that
> address is in the file
> and accessed via inode. If semaphore is in bss then kernel changes
> would be required.
> I hope we can try such approach without changing the kernel first and
> if it's not enough we'll know exactly what's missing.
>
>
> On Wed, Jan 17, 2018 at 9:08 AM, Sasha Goldshtein wrote:
>> You’re saying that the uprobe installation will replace the semaphore value
>> with something non-zero? Is that guaranteed? If using “int 3” then probably
>> yes but if using an optimized jump? And will it work on non-x86?
>>
>> On Jan 17, 2018, at 17:05, Alexei Starovoitov
>> wrote:
>>
>> I think it should be possible to abuse uprobe kernel logic to flip the
>> semaphore value.
>> Instead of writing into /proc/pid/mem we can uprobe on that exact location.
>> Though it's not text, but data section. It may work ?
>>
>> On Wed, Jan 17, 2018 at 8:57 AM, Sasha Goldshtein via iovisor-dev
>> wrote:
>>
>> Not sure I understand your proposal re the kernel setting the semaphore.
>>
>>
>> Anyway, yes, the semaphore concept and implementation goes back to SystemTap
>> SDT which is basically DTrace probes.
>>
>>
>> On Jan 17, 2018, at 16:55, Kiran T wrote:
>>
>>
>> Is this because this is tied to DTRACE abi? Perhaps another elf note
>>
>> or comment could be implemented, so this semaphore is set on binary
>>
>> load itself by the kernel? Again thinking out loud :)
>>
>>
>> Thanks,
>>
>> Kiran
>>
>>
>> On Tue, Jan 16, 2018 at 8:51 PM, Sasha Goldshtein
>> wrote:
>>
>> Enabling semaphore-enabled probes system-wide requires poking
>>
>> the semaphore’s memory in each process, I’m afraid. We can come
>>
>> up with some implementation that would hook new process creation
>>
>> and try to enable the semaphore when the relevant library gets loaded,
>>
>> but it sounds a bit fragile. It could be interesting to explore
>>
>> implementation alternatives for this. To hermetically enable the probe
>>
>> we would need to synchronously block loading the provider’s library
>>
>> into the target until we can modify the semaphore. This isn’t something
>>
>> a BPF program can do, so it would require some external mechanism.
>>
>>
>> On Jan 17, 2018, at 03:44, Kiran T wrote:
>>
>>
>> Thanks Sasha, Yonghong.
>>
>> Looks like both php and python use semaphores. Is it fundamentally
>>
>> not possible to enable this systemwide or is it a feature which hasn't
>>
>> been implemented?
>>
>> Maybe we get pids of the process as they hit another probe and then
>>
>> get the semaphore addresses and enable them?
>>
>> Or is there a way to start the processes with the semaphore enabled --
>>
>> a elf indicator or something? Just thinking out loud.
>>
>>
>> Thanks,
>>
>> Kiran
>>
>>
>> On Mon, Jan 15, 2018 at 10:30 PM, Sasha Goldshtein
>> wrote:
>>
>> Please note that some USDT probes have an associated "semaphore",
>>
>> which is really just a memory location that the probed code checks
>>
>> before actually invoking the probe. You cannot enable USDT
Re: [iovisor-dev] How to use USDT probes without having to pass the pid of the traced program?
yep. uprobe is always 'int 3' so far, since kernel needs to take control. we can add something like writing arbitrary value as long as that address is in the file and accessed via inode. If semaphore is in bss then kernel changes would be required. I hope we can try such approach without changing the kernel first and if it's not enough we'll know exactly what's missing. On Wed, Jan 17, 2018 at 9:08 AM, Sasha Goldshtein wrote: > You’re saying that the uprobe installation will replace the semaphore value > with something non-zero? Is that guaranteed? If using “int 3” then probably > yes but if using an optimized jump? And will it work on non-x86? > > On Jan 17, 2018, at 17:05, Alexei Starovoitov > wrote: > > I think it should be possible to abuse uprobe kernel logic to flip the > semaphore value. > Instead of writing into /proc/pid/mem we can uprobe on that exact location. > Though it's not text, but data section. It may work ? > > On Wed, Jan 17, 2018 at 8:57 AM, Sasha Goldshtein via iovisor-dev > wrote: > > Not sure I understand your proposal re the kernel setting the semaphore. > > > Anyway, yes, the semaphore concept and implementation goes back to SystemTap > SDT which is basically DTrace probes. > > > On Jan 17, 2018, at 16:55, Kiran T wrote: > > > Is this because this is tied to DTRACE abi? Perhaps another elf note > > or comment could be implemented, so this semaphore is set on binary > > load itself by the kernel? Again thinking out loud :) > > > Thanks, > > Kiran > > > On Tue, Jan 16, 2018 at 8:51 PM, Sasha Goldshtein > wrote: > > Enabling semaphore-enabled probes system-wide requires poking > > the semaphore’s memory in each process, I’m afraid. We can come > > up with some implementation that would hook new process creation > > and try to enable the semaphore when the relevant library gets loaded, > > but it sounds a bit fragile. It could be interesting to explore > > implementation alternatives for this. To hermetically enable the probe > > we would need to synchronously block loading the provider’s library > > into the target until we can modify the semaphore. This isn’t something > > a BPF program can do, so it would require some external mechanism. > > > On Jan 17, 2018, at 03:44, Kiran T wrote: > > > Thanks Sasha, Yonghong. > > Looks like both php and python use semaphores. Is it fundamentally > > not possible to enable this systemwide or is it a feature which hasn't > > been implemented? > > Maybe we get pids of the process as they hit another probe and then > > get the semaphore addresses and enable them? > > Or is there a way to start the processes with the semaphore enabled -- > > a elf indicator or something? Just thinking out loud. > > > Thanks, > > Kiran > > > On Mon, Jan 15, 2018 at 10:30 PM, Sasha Goldshtein > wrote: > > Please note that some USDT probes have an associated "semaphore", > > which is really just a memory location that the probed code checks > > before actually invoking the probe. You cannot enable USDT probes that > > have a semaphore system-wide, without a process ID, because the > > semaphore location has to be poked in each target process. > > > To see if your probes have an associated semaphore, just run tplist with > > -v -v and it will print out the semaphore address for each probe. If it's > > 0, you're good to do system-wide tracing. > > > Sasha > > > > On Tue, Jan 16, 2018 at 8:08 AM Y Song via iovisor-dev > > wrote: > > > Kiran, > > > Yes, tracing through USDT without PID should work. > > You can just remove "-p" parameter and give a try. > > > Please try latest bcc as it fixed a few bugs. > > Let us know if you hit any issues. > > > Yonghong > > > > On Mon, Jan 15, 2018 at 5:11 PM, Kiran T via iovisor-dev > > wrote: > > I meant to say > > > Can one request to monitor binaries -- like with > > uprobes/uretprobes but with USDT? > > > On Mon, Jan 15, 2018 at 5:09 PM, Kiran T wrote: > > Hi > > All the examples on tracing processes with USDT require the pid of the > > traced program: > > > https://github.com/iovisor/bcc/tree/master/examples/tracing > > > Can one not request to monitor binaries -- like with > > uprobes/uretprobes but with USDT? > > > I am trying to trace php scripts running on a webserver, and USDT > > would be ideal. But, I won't have the pid of the process that will be > > spawned by the webserver in advance. > > > Thanks, > > Kiran > > ___ > > iovisor-dev mailing list > > iovisor-dev@lists.iovisor.org > > https://lists.iovisor.org/mailman/listinfo/iovisor-dev > > ___ > > iovisor-dev mailing list > > iovisor-dev@lists.iovisor.org > > https://lists.iovisor.org/mailman/listinfo/iovisor-dev > > ___ > > iovisor-dev mailing list > > iovisor-dev@lists.iovisor.org > > https://lists.iovisor.org/mailman/listinfo/iovisor-dev ___ iovisor-dev mailing list iovisor-dev
Re: [iovisor-dev] How to use USDT probes without having to pass the pid of the traced program?
You’re saying that the uprobe installation will replace the semaphore value with something non-zero? Is that guaranteed? If using “int 3” then probably yes but if using an optimized jump? And will it work on non-x86? > On Jan 17, 2018, at 17:05, Alexei Starovoitov > wrote: > > I think it should be possible to abuse uprobe kernel logic to flip the > semaphore value. > Instead of writing into /proc/pid/mem we can uprobe on that exact location. > Though it's not text, but data section. It may work ? > > On Wed, Jan 17, 2018 at 8:57 AM, Sasha Goldshtein via iovisor-dev > wrote: >> Not sure I understand your proposal re the kernel setting the semaphore. >> >> Anyway, yes, the semaphore concept and implementation goes back to SystemTap >> SDT which is basically DTrace probes. >> >>> On Jan 17, 2018, at 16:55, Kiran T wrote: >>> >>> Is this because this is tied to DTRACE abi? Perhaps another elf note >>> or comment could be implemented, so this semaphore is set on binary >>> load itself by the kernel? Again thinking out loud :) >>> >>> Thanks, >>> Kiran >>> On Tue, Jan 16, 2018 at 8:51 PM, Sasha Goldshtein wrote: Enabling semaphore-enabled probes system-wide requires poking the semaphore’s memory in each process, I’m afraid. We can come up with some implementation that would hook new process creation and try to enable the semaphore when the relevant library gets loaded, but it sounds a bit fragile. It could be interesting to explore implementation alternatives for this. To hermetically enable the probe we would need to synchronously block loading the provider’s library into the target until we can modify the semaphore. This isn’t something a BPF program can do, so it would require some external mechanism. > On Jan 17, 2018, at 03:44, Kiran T wrote: > > Thanks Sasha, Yonghong. > Looks like both php and python use semaphores. Is it fundamentally > not possible to enable this systemwide or is it a feature which hasn't > been implemented? > Maybe we get pids of the process as they hit another probe and then > get the semaphore addresses and enable them? > Or is there a way to start the processes with the semaphore enabled -- > a elf indicator or something? Just thinking out loud. > > Thanks, > Kiran > >> On Mon, Jan 15, 2018 at 10:30 PM, Sasha Goldshtein >> wrote: >> Please note that some USDT probes have an associated "semaphore", >> which is really just a memory location that the probed code checks >> before actually invoking the probe. You cannot enable USDT probes that >> have a semaphore system-wide, without a process ID, because the >> semaphore location has to be poked in each target process. >> >> To see if your probes have an associated semaphore, just run tplist with >> -v -v and it will print out the semaphore address for each probe. If it's >> 0, you're good to do system-wide tracing. >> >> Sasha >> >> >> On Tue, Jan 16, 2018 at 8:08 AM Y Song via iovisor-dev >> wrote: >>> >>> Kiran, >>> >>> Yes, tracing through USDT without PID should work. >>> You can just remove "-p" parameter and give a try. >>> >>> Please try latest bcc as it fixed a few bugs. >>> Let us know if you hit any issues. >>> >>> Yonghong >>> >>> >>> On Mon, Jan 15, 2018 at 5:11 PM, Kiran T via iovisor-dev >>> wrote: I meant to say Can one request to monitor binaries -- like with uprobes/uretprobes but with USDT? > On Mon, Jan 15, 2018 at 5:09 PM, Kiran T wrote: > Hi > All the examples on tracing processes with USDT require the pid of the > traced program: > > https://github.com/iovisor/bcc/tree/master/examples/tracing > > Can one not request to monitor binaries -- like with > uprobes/uretprobes but with USDT? > > I am trying to trace php scripts running on a webserver, and USDT > would be ideal. But, I won't have the pid of the process that will be > spawned by the webserver in advance. > > Thanks, > Kiran ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev >>> ___ >>> iovisor-dev mailing list >>> iovisor-dev@lists.iovisor.org >>> https://lists.iovisor.org/mailman/listinfo/iovisor-dev >> ___ >> iovisor-dev mailing list >> iovisor-dev@lists.iovisor.org >> https://lists.iovisor.org/mailman/listinfo/iovisor-dev ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/li
Re: [iovisor-dev] How to use USDT probes without having to pass the pid of the traced program?
I think it should be possible to abuse uprobe kernel logic to flip the semaphore value. Instead of writing into /proc/pid/mem we can uprobe on that exact location. Though it's not text, but data section. It may work ? On Wed, Jan 17, 2018 at 8:57 AM, Sasha Goldshtein via iovisor-dev wrote: > Not sure I understand your proposal re the kernel setting the semaphore. > > Anyway, yes, the semaphore concept and implementation goes back to SystemTap > SDT which is basically DTrace probes. > >> On Jan 17, 2018, at 16:55, Kiran T wrote: >> >> Is this because this is tied to DTRACE abi? Perhaps another elf note >> or comment could be implemented, so this semaphore is set on binary >> load itself by the kernel? Again thinking out loud :) >> >> Thanks, >> Kiran >> >>> On Tue, Jan 16, 2018 at 8:51 PM, Sasha Goldshtein >>> wrote: >>> Enabling semaphore-enabled probes system-wide requires poking >>> the semaphore’s memory in each process, I’m afraid. We can come >>> up with some implementation that would hook new process creation >>> and try to enable the semaphore when the relevant library gets loaded, >>> but it sounds a bit fragile. It could be interesting to explore >>> implementation alternatives for this. To hermetically enable the probe >>> we would need to synchronously block loading the provider’s library >>> into the target until we can modify the semaphore. This isn’t something >>> a BPF program can do, so it would require some external mechanism. >>> On Jan 17, 2018, at 03:44, Kiran T wrote: Thanks Sasha, Yonghong. Looks like both php and python use semaphores. Is it fundamentally not possible to enable this systemwide or is it a feature which hasn't been implemented? Maybe we get pids of the process as they hit another probe and then get the semaphore addresses and enable them? Or is there a way to start the processes with the semaphore enabled -- a elf indicator or something? Just thinking out loud. Thanks, Kiran > On Mon, Jan 15, 2018 at 10:30 PM, Sasha Goldshtein > wrote: > Please note that some USDT probes have an associated "semaphore", > which is really just a memory location that the probed code checks > before actually invoking the probe. You cannot enable USDT probes that > have a semaphore system-wide, without a process ID, because the > semaphore location has to be poked in each target process. > > To see if your probes have an associated semaphore, just run tplist with > -v -v and it will print out the semaphore address for each probe. If it's > 0, you're good to do system-wide tracing. > > Sasha > > > On Tue, Jan 16, 2018 at 8:08 AM Y Song via iovisor-dev > wrote: >> >> Kiran, >> >> Yes, tracing through USDT without PID should work. >> You can just remove "-p" parameter and give a try. >> >> Please try latest bcc as it fixed a few bugs. >> Let us know if you hit any issues. >> >> Yonghong >> >> >> On Mon, Jan 15, 2018 at 5:11 PM, Kiran T via iovisor-dev >> wrote: >>> I meant to say >>> >>> Can one request to monitor binaries -- like with >>> uprobes/uretprobes but with USDT? >>> On Mon, Jan 15, 2018 at 5:09 PM, Kiran T wrote: Hi All the examples on tracing processes with USDT require the pid of the traced program: https://github.com/iovisor/bcc/tree/master/examples/tracing Can one not request to monitor binaries -- like with uprobes/uretprobes but with USDT? I am trying to trace php scripts running on a webserver, and USDT would be ideal. But, I won't have the pid of the process that will be spawned by the webserver in advance. Thanks, Kiran >>> ___ >>> iovisor-dev mailing list >>> iovisor-dev@lists.iovisor.org >>> https://lists.iovisor.org/mailman/listinfo/iovisor-dev >> ___ >> iovisor-dev mailing list >> iovisor-dev@lists.iovisor.org >> https://lists.iovisor.org/mailman/listinfo/iovisor-dev > ___ > iovisor-dev mailing list > iovisor-dev@lists.iovisor.org > https://lists.iovisor.org/mailman/listinfo/iovisor-dev ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
Re: [iovisor-dev] How to use USDT probes without having to pass the pid of the traced program?
Not sure I understand your proposal re the kernel setting the semaphore. Anyway, yes, the semaphore concept and implementation goes back to SystemTap SDT which is basically DTrace probes. > On Jan 17, 2018, at 16:55, Kiran T wrote: > > Is this because this is tied to DTRACE abi? Perhaps another elf note > or comment could be implemented, so this semaphore is set on binary > load itself by the kernel? Again thinking out loud :) > > Thanks, > Kiran > >> On Tue, Jan 16, 2018 at 8:51 PM, Sasha Goldshtein wrote: >> Enabling semaphore-enabled probes system-wide requires poking >> the semaphore’s memory in each process, I’m afraid. We can come >> up with some implementation that would hook new process creation >> and try to enable the semaphore when the relevant library gets loaded, >> but it sounds a bit fragile. It could be interesting to explore >> implementation alternatives for this. To hermetically enable the probe >> we would need to synchronously block loading the provider’s library >> into the target until we can modify the semaphore. This isn’t something >> a BPF program can do, so it would require some external mechanism. >> >>> On Jan 17, 2018, at 03:44, Kiran T wrote: >>> >>> Thanks Sasha, Yonghong. >>> Looks like both php and python use semaphores. Is it fundamentally >>> not possible to enable this systemwide or is it a feature which hasn't >>> been implemented? >>> Maybe we get pids of the process as they hit another probe and then >>> get the semaphore addresses and enable them? >>> Or is there a way to start the processes with the semaphore enabled -- >>> a elf indicator or something? Just thinking out loud. >>> >>> Thanks, >>> Kiran >>> On Mon, Jan 15, 2018 at 10:30 PM, Sasha Goldshtein wrote: Please note that some USDT probes have an associated "semaphore", which is really just a memory location that the probed code checks before actually invoking the probe. You cannot enable USDT probes that have a semaphore system-wide, without a process ID, because the semaphore location has to be poked in each target process. To see if your probes have an associated semaphore, just run tplist with -v -v and it will print out the semaphore address for each probe. If it's 0, you're good to do system-wide tracing. Sasha On Tue, Jan 16, 2018 at 8:08 AM Y Song via iovisor-dev wrote: > > Kiran, > > Yes, tracing through USDT without PID should work. > You can just remove "-p" parameter and give a try. > > Please try latest bcc as it fixed a few bugs. > Let us know if you hit any issues. > > Yonghong > > > On Mon, Jan 15, 2018 at 5:11 PM, Kiran T via iovisor-dev > wrote: >> I meant to say >> >> Can one request to monitor binaries -- like with >> uprobes/uretprobes but with USDT? >> >>> On Mon, Jan 15, 2018 at 5:09 PM, Kiran T wrote: >>> Hi >>> All the examples on tracing processes with USDT require the pid of the >>> traced program: >>> >>> https://github.com/iovisor/bcc/tree/master/examples/tracing >>> >>> Can one not request to monitor binaries -- like with >>> uprobes/uretprobes but with USDT? >>> >>> I am trying to trace php scripts running on a webserver, and USDT >>> would be ideal. But, I won't have the pid of the process that will be >>> spawned by the webserver in advance. >>> >>> Thanks, >>> Kiran >> ___ >> iovisor-dev mailing list >> iovisor-dev@lists.iovisor.org >> https://lists.iovisor.org/mailman/listinfo/iovisor-dev > ___ > iovisor-dev mailing list > iovisor-dev@lists.iovisor.org > https://lists.iovisor.org/mailman/listinfo/iovisor-dev ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
Re: [iovisor-dev] How to use USDT probes without having to pass the pid of the traced program?
Is this because this is tied to DTRACE abi? Perhaps another elf note or comment could be implemented, so this semaphore is set on binary load itself by the kernel? Again thinking out loud :) Thanks, Kiran On Tue, Jan 16, 2018 at 8:51 PM, Sasha Goldshtein wrote: > Enabling semaphore-enabled probes system-wide requires poking > the semaphore’s memory in each process, I’m afraid. We can come > up with some implementation that would hook new process creation > and try to enable the semaphore when the relevant library gets loaded, > but it sounds a bit fragile. It could be interesting to explore > implementation alternatives for this. To hermetically enable the probe > we would need to synchronously block loading the provider’s library > into the target until we can modify the semaphore. This isn’t something > a BPF program can do, so it would require some external mechanism. > >> On Jan 17, 2018, at 03:44, Kiran T wrote: >> >> Thanks Sasha, Yonghong. >> Looks like both php and python use semaphores. Is it fundamentally >> not possible to enable this systemwide or is it a feature which hasn't >> been implemented? >> Maybe we get pids of the process as they hit another probe and then >> get the semaphore addresses and enable them? >> Or is there a way to start the processes with the semaphore enabled -- >> a elf indicator or something? Just thinking out loud. >> >> Thanks, >> Kiran >> >>> On Mon, Jan 15, 2018 at 10:30 PM, Sasha Goldshtein >>> wrote: >>> Please note that some USDT probes have an associated "semaphore", >>> which is really just a memory location that the probed code checks >>> before actually invoking the probe. You cannot enable USDT probes that >>> have a semaphore system-wide, without a process ID, because the >>> semaphore location has to be poked in each target process. >>> >>> To see if your probes have an associated semaphore, just run tplist with >>> -v -v and it will print out the semaphore address for each probe. If it's >>> 0, you're good to do system-wide tracing. >>> >>> Sasha >>> >>> >>> On Tue, Jan 16, 2018 at 8:08 AM Y Song via iovisor-dev >>> wrote: Kiran, Yes, tracing through USDT without PID should work. You can just remove "-p" parameter and give a try. Please try latest bcc as it fixed a few bugs. Let us know if you hit any issues. Yonghong On Mon, Jan 15, 2018 at 5:11 PM, Kiran T via iovisor-dev wrote: > I meant to say > > Can one request to monitor binaries -- like with > uprobes/uretprobes but with USDT? > >> On Mon, Jan 15, 2018 at 5:09 PM, Kiran T wrote: >> Hi >> All the examples on tracing processes with USDT require the pid of the >> traced program: >> >> https://github.com/iovisor/bcc/tree/master/examples/tracing >> >> Can one not request to monitor binaries -- like with >> uprobes/uretprobes but with USDT? >> >> I am trying to trace php scripts running on a webserver, and USDT >> would be ideal. But, I won't have the pid of the process that will be >> spawned by the webserver in advance. >> >> Thanks, >> Kiran > ___ > iovisor-dev mailing list > iovisor-dev@lists.iovisor.org > https://lists.iovisor.org/mailman/listinfo/iovisor-dev ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev ___ iovisor-dev mailing list iovisor-dev@lists.iovisor.org https://lists.iovisor.org/mailman/listinfo/iovisor-dev
