> starting next year we're planning to invest into building > a prototype where we can inject the code into remote process, > fixup usdt-s in the remote process to point to injected code > and let this injected code interact with master process via > shared memory. > The goal is to build an alternative to uprobe+bpf at much higher speed.
I'm definitely interested in learning more and contributing if possible -- our use case is that we have a database with USDT probes and would want to dynamically trace production workloads, but cannot assume that we are running very modern kernels. ~Alex On Sun, Dec 18, 2016 at 1:54 PM Suchakra via iovisor-dev < [email protected]> wrote: > Hi Joe, > > >> What I did was fork a project called linux-inject here: > >> > >> https://github.com/joe-lawrence/linux-inject > >> > >> This project demonstrates the use of ptrace to attach and then force the > >> target to call __libc_dlopen_mode (this lives in libc unlike the dlopen > >> wrapper in libdl) effectively injecting a given shared object into that > >> program. > > Thanks for your work on linux-inject. Its really cool! You have > already noted the ptrace caveat. > In addition, processes which are security hardened with seccomp etc > may not allow to be ptraced. > > >> I added the patches/ directory to the project to try out this approach > >> in applying a few CVE fixes. I built a base patching shared library > >> that reads and applies a patch description array. New patched code is > >> also included in this shared library. The patching mechanism applies a > >> code-trampoline at given locations routing the binary away from the old > >> code and into the newly loaded patched code. > >> > >> It's only a proof-of-concept, but I did record a few .gif terminal > >> sessions to show what's possible: > > I spent some time researching dynamic instrumentation for tracing a > few years back . We started with a ptrace based approach and developed > kaji (https://github.com/5kg/kaji) as a demo for inserting precompiled > lttng tracepoints. This was also tested with dyninst. Here are some > observations (http://www.dorsal.polymtl.ca/fr/system/files/11Dec2013.pdf). > Here is also some investigation on what happens underneath > ( > https://github.com/tuxology/lttng-ubench/blob/master/analysis/instr/dyninst_kaji > ). > > In addition, some other things to look out for would be : > 1. Fast Tracepoints in GDB that use a similar trampoline approach > 2. SystemTap's Stapdyn approach which uses dyninst > ( > https://research.cs.wisc.edu/htcondor/HTCondorWeek2013/paradyn-slides/stone-stapdyn.pdf > ) > 3. DynamoRIO (http://www.dynamorio.org/) > > Also, some folks I know have complained about dynins't huge memory > consumption in a production simulation. I have not investigated that > myself, but I can if we are heading in this direction. Also, it may be > worthwhile to discuss if "spin-your-own" may be beneficial than an > available and tested framework. For example, Dyninst does recursive > trampoline checks and some other basic safety checks on snippets > before inserting them. It seems overkill in some places, but sometimes > it may add to robustness. > > >> Are the tracing folks discussing userspace upstream on the mailing lists > >> yet? If so, I'd like to participate :) > > > > sorry for delay and thanks a lot for all the links. > > starting next year we're planning to invest into building > > a prototype where we can inject the code into remote process, > > fixup usdt-s in the remote process to point to injected code > > and let this injected code interact with master process via > > shared memory. > > The goal is to build an alternative to uprobe+bpf at much higher speed. > > I am really interested in this now that academic life is getting over :) > Maybe we can try for a Dyninst based demo which can be quick to implement? > > >> libcompel - execute arbitrary code in a context of a foreign process > >> https://criu.org/Compel > > Wow this is cool. I did not know about this. Time to play with this! > > -- > Suchakra > _______________________________________________ > iovisor-dev mailing list > [email protected] > https://lists.iovisor.org/mailman/listinfo/iovisor-dev >
_______________________________________________ iovisor-dev mailing list [email protected] https://lists.iovisor.org/mailman/listinfo/iovisor-dev
