On Thu, 2019-07-25 at 11:40 -0400, Mathieu Desnoyers wrote: > There are a few reasons for using per-uid buffers over per-pid: > > - Lower memory consumption for use-cases with many processes, > - Faster process launch time: no need to allocate buffers for each process. > Useful for use-cases with short-lived processes. > - Keep a flight recorder "snapshot" available for all processes, including > those which recently exited. Indeed, the per-pid buffers don't stay around > for snapshot after a process exits or is killed. > > There are however a few advantages for per-pid buffers: > > - Isolation: if one PID generates corrupted trace data, it does not interfere > with other PIDs buffers, > - If one PID is killed between reserve and commit, it does not make that > specific > per-cpu ring buffer unusable for the rest of the tracing session lifetime. > > Hoping this information helps making the right choice for your deployment!
We recently had this discussion for an embedded product that uses LTTng to gather trace data during operation. In our case, we want to have a flight recorder of the last X seconds of trace data, for the entire device. X seconds times Y byte/sec data generation rate ends up being a very large portion (~30%) of the total memory available. This has to be in RAM, using flash memory for this is not a good idea. If we use per-PID buffers, then the buffer size needed for the largest producer of trace data times the total number of processes is too large: far larger than the device's memory size. Some processes produce trace data at a much higher rate than others. A buffer for X seconds of data on one processes ends up being a buffer for 10*X seconds of data on another. There's not enough RAM for 10*X second buffers. If we use per-UID buffers, then we must run everything as one UID. Which, on an embedded system, is not that bad, but negatively impacts the security of the software. Now all processes, which generate data at different rates, can share one buffer. Much more efficient that having to reserve space the same space for the largest and smallest producers. But there ends up being another problem, the flight recorder data needs to be saved to make use of. To tmpfs in RAM, since the device's flash is not suitable and used elsewhere anyway. So one needs 2x the RAM, one for the ring buffer and one for the trace data dump in tmpfs of the ring buffer. So what we did was not use flight recorder mode. We configured lttng to use a limited number of smaller trace files and trace file rotation. And used small ring buffers, which ended up not needing to be very large to avoid overflow (I imagine saving the data to tmpfs is fast). The trace files are in effect a per-session buffer, which is what we want for greatest efficiency in space utilization. And we can archive those and download them when "something happens" without paying extra cost for space. _______________________________________________ lttng-dev mailing list lttng-dev@lists.lttng.org https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev