[PATCH v2 00/32] 2nd Iteration of Cache QoS Monitoring support.
This series introduces the next iteration of kernel support for the Cache QoS Monitoring (CQM) technology available in Intel Xeon processors. One of the main limitations of the previous version is the inability to simultaneously monitor: 1) cpu event and any other event in that cpu. 2) cgroup events for cgroups in same descendancy line. 3) cgroup events and any thread event of a cgroup in the same descendancy line. Another limitation is that monitoring for a cgroup was enabled/disabled by the existence of a perf event for that cgroup. Since the event llc_occupancy measures changes in occupancy rather than total occupancy, in order to read meaningful llc_occupancy values, an event should be enabled for a long enough period of time. The overhead in context switches caused by the perf events is undesired in some sensitive scenarios. This series of patches addresses the shortcomings mentioned above and, add some other improvements. The main changes are: - No more potential conflicts between different events. New version builds a hierarchy of RMIDs that captures the dependency between monitored cgroups. llc_occupancy for cgroup is the sum of llc_occupancies for that cgroup RMID and all other RMIDs in the cgroups subtree (both monitored cgroups and threads). - A cgroup integration that allows to monitor the a cgroup without creating a perf event, decreasing the context switch overhead. Monitoring is controlled by a boolean cgroup subsystem attribute in each perf cgroup, this is: echo 1 > cgroup_path/perf_event.cqm_cont_monitoring starts CQM monitoring whether or not there is a perf_event attached to the cgroup. Setting the attribute to 0 makes monitoring dependent on the existence of a perf_event. A perf_event is always required in order to read llc_occupancy. This cgroup integration uses Intel's PQR code and is intended to be used by upcoming versions of Intel's CAT. - A more stable rotation algorithm: New algorithm uses SLOs that guarantee: - A minimum of enabled time for monitored cgroups and threads. - A maximum time disabled before error is introduced by reusing dirty RMIDs. - A minimum rate at which RMIDs recycling must progress. - Reduced impact of stealing/rotation of RMIDs: The new algorithm accounts the residual occupancy held by limbo RMIDs towards the former owner of the limbo RMID, decreasing the error introduced by RMID rotation. It also allows a limbo RMID to be reused by its former owner when appropriate, decreasing the potential error of reusing dirty RMIDs and allowing to make progress even if most limbo RMIDs do not drop occupancy fast enough. - Elimination of pmu::count: perf generic's perf_event_count() perform a quick add of atomic types. The introduction of pmu::count in the previous CQM series to read occupancy for thread events changed the behavior of perf_event_count() by performing a potentially slow IPI and write/read to MSR. It also made pmu::read to have different behaviors depending on whether the event was a cpu/cgroup event or a thread. This patches serie removes the custom pmu::count from CQM and provides a consistent behavior for all calls of perf_event_read . - Added error return for pmu::read: Reads to CQM events may fail due to stealing of RMIDs, even after successfully adding an event to a PMU. This patch series expands pmu::read with an int return value and propagates the error to callers that can fail (ie. perf_read). The ability to fail of pmu::read is consistent with the recent changes that allow perf_event_read to fail for transactional reading of event groups. - Introduces the field pmu_event_flags that contain flags set by the PMU to signal variations on the default behavior to perf's generic code. In this series, three flags are introduced: - PERF_CGROUP_NO_RECURSION : Signals generic code to add events of the cgroup ancestors of a cgroup. - PERF_INACTIVE_CPU_READ_PKG: Signals generic coda that this CPU event can be read in any CPU in its event::cpu's package, even if the event is not active. - PERF_INACTIVE_EV_READ_ANY_CPU: Signals generic code that this event can be read in any CPU in any package in the system even if the event is not active. Using the above flags takes advantage of the CQM's hw ability to read llc_occupancy even when the associated perf event is not running in a CPU. This patch series also updates the perf tool to fix
[PATCH v2 00/32] 2nd Iteration of Cache QoS Monitoring support.
This series introduces the next iteration of kernel support for the Cache QoS Monitoring (CQM) technology available in Intel Xeon processors. One of the main limitations of the previous version is the inability to simultaneously monitor: 1) cpu event and any other event in that cpu. 2) cgroup events for cgroups in same descendancy line. 3) cgroup events and any thread event of a cgroup in the same descendancy line. Another limitation is that monitoring for a cgroup was enabled/disabled by the existence of a perf event for that cgroup. Since the event llc_occupancy measures changes in occupancy rather than total occupancy, in order to read meaningful llc_occupancy values, an event should be enabled for a long enough period of time. The overhead in context switches caused by the perf events is undesired in some sensitive scenarios. This series of patches addresses the shortcomings mentioned above and, add some other improvements. The main changes are: - No more potential conflicts between different events. New version builds a hierarchy of RMIDs that captures the dependency between monitored cgroups. llc_occupancy for cgroup is the sum of llc_occupancies for that cgroup RMID and all other RMIDs in the cgroups subtree (both monitored cgroups and threads). - A cgroup integration that allows to monitor the a cgroup without creating a perf event, decreasing the context switch overhead. Monitoring is controlled by a boolean cgroup subsystem attribute in each perf cgroup, this is: echo 1 > cgroup_path/perf_event.cqm_cont_monitoring starts CQM monitoring whether or not there is a perf_event attached to the cgroup. Setting the attribute to 0 makes monitoring dependent on the existence of a perf_event. A perf_event is always required in order to read llc_occupancy. This cgroup integration uses Intel's PQR code and is intended to be used by upcoming versions of Intel's CAT. - A more stable rotation algorithm: New algorithm uses SLOs that guarantee: - A minimum of enabled time for monitored cgroups and threads. - A maximum time disabled before error is introduced by reusing dirty RMIDs. - A minimum rate at which RMIDs recycling must progress. - Reduced impact of stealing/rotation of RMIDs: The new algorithm accounts the residual occupancy held by limbo RMIDs towards the former owner of the limbo RMID, decreasing the error introduced by RMID rotation. It also allows a limbo RMID to be reused by its former owner when appropriate, decreasing the potential error of reusing dirty RMIDs and allowing to make progress even if most limbo RMIDs do not drop occupancy fast enough. - Elimination of pmu::count: perf generic's perf_event_count() perform a quick add of atomic types. The introduction of pmu::count in the previous CQM series to read occupancy for thread events changed the behavior of perf_event_count() by performing a potentially slow IPI and write/read to MSR. It also made pmu::read to have different behaviors depending on whether the event was a cpu/cgroup event or a thread. This patches serie removes the custom pmu::count from CQM and provides a consistent behavior for all calls of perf_event_read . - Added error return for pmu::read: Reads to CQM events may fail due to stealing of RMIDs, even after successfully adding an event to a PMU. This patch series expands pmu::read with an int return value and propagates the error to callers that can fail (ie. perf_read). The ability to fail of pmu::read is consistent with the recent changes that allow perf_event_read to fail for transactional reading of event groups. - Introduces the field pmu_event_flags that contain flags set by the PMU to signal variations on the default behavior to perf's generic code. In this series, three flags are introduced: - PERF_CGROUP_NO_RECURSION : Signals generic code to add events of the cgroup ancestors of a cgroup. - PERF_INACTIVE_CPU_READ_PKG: Signals generic coda that this CPU event can be read in any CPU in its event::cpu's package, even if the event is not active. - PERF_INACTIVE_EV_READ_ANY_CPU: Signals generic code that this event can be read in any CPU in any package in the system even if the event is not active. Using the above flags takes advantage of the CQM's hw ability to read llc_occupancy even when the associated perf event is not running in a CPU. This patch series also updates the perf tool to fix
