Re: [PATCHv1 0/6] rdma controller support
Hello, Parav. On Thu, Jan 07, 2016 at 04:43:20AM +0530, Parav Pandit wrote: > > If different controllers can't agree upon the > > same set of resources, which probably is a pretty good sign that this > > isn't too well thought out to begin with, > > When you said "different controller" you meant "different hw vendors", right? > Or you meant, rdma, mem, cpu as controller here? Different hw vendors. > > at least make all resource > > types defined by the controller itself and let the controllers enable > > them selectively. > > > In this V1 patch, resource is defined by the IB stack and rdma cgroup > is facilitator for same. > By doing so, IB stack modules can define new resource without really > making changes to cgroup. > This design also allows hw vendors to define their own resources which > will be reviewed in rdma mailing list anway. > The idea is different hw versions can have different resource support, > so the whole intention is not about defining different resource but > rather enabling it. > But yes, I equally agree that by doing so, different hw controller > vendors can define different hw resources. How many vendors and resources are we talking about? What I was trying to say was that unless the number is extremely high, it'd be far simpler to hard code them in the rdma controller and let drivers enable the ones which apply to them. It would require updating the rdma cgroup controller to add new resource types but I think that'd actually be an upside, not down. There needs to be some checks and balances against adding new resource types; otherwise, it'll soon become a mess. Thanks. -- tejun -- To unsubscribe from this list: send the line "unsubscribe linux-security-module" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCHv1 0/6] rdma controller support
This patchset adds support for RDMA cgroup by addressing review comments of [1] by implementing published RFC [2]. Overview: Currently user space applications can easily take away all the rdma device specific resources such as AH, CQ, QP, MR etc. Due to which other applications in other cgroup or kernel space ULPs may not even get chance to allocate any rdma resources. This results into service unavailibility. RDMA cgroup addresses this issue by allowing resource accounting, limit enforcement on per cgroup, per rdma device basis. Resources are not defined by the RDMA cgroup. Resources are defined by RDMA/IB stack & optionally by HCA vendor device drivers. This allows rdma cgroup to remain constant while RDMA/IB stack can evolve without the need of rdma cgroup update. A new resource can be easily added by the RDMA/IB stack without touching rdma cgroup. RDMA uverbs layer will enforce limits on well defined RDMA verb resources without any HCA vendor device driver involvement. RDMA uverbs layer will not do accounting of hw vendor specific resources. Instead rdma cgroup provides set of APIs through which vendor specific drivers can define their own resources (upto 64) that can be accounted by rdma cgroup. Resource limit enforcement is hierarchical. When process is migrated with active RDMA resources, rdma cgroup continues to charge original cgroup. Changes from v0: (To address comments from Haggai, Doug, Liran, Tejun, Sean, Jason) * Redesigned to support per device per cgroup limit settings by bringing concept of resource pool. * Redesigned to let IB stack define the resources instead of rdma controller using resource template. * Redesigned to support hw vendor specific limits setting (optional to drivers). * Created new rdma controller instead of piggyback on device cgroup. * Fixed race conditions for multiple tasks sharing rdma resources. * Removed dependency on the task_struct. [1] https://lkml.org/lkml/2015/9/7/476 [2] https://lkml.org/lkml/2015/10/28/144 This patchset is for Tejun's for-4.5 branch. It is not attempted on Doug's rdma tree yet, which I will do once I receive comments for this pathset. Parav Pandit (6): rdmacg: Added rdma cgroup header file IB/core: Added members to support rdma cgroup rdmacg: implements rdma cgroup IB/core: rdmacg support infrastructure APIs IB/core: use rdma cgroup for resource accounting rdmacg: Added documentation for rdma controller. Documentation/cgroup-legacy/rdma.txt | 129 Documentation/cgroup.txt | 79 +++ drivers/infiniband/core/Makefile |1 + drivers/infiniband/core/cgroup.c | 80 +++ drivers/infiniband/core/core_priv.h |5 + drivers/infiniband/core/device.c |8 + drivers/infiniband/core/uverbs_cmd.c | 244 ++- drivers/infiniband/core/uverbs_main.c | 30 + include/linux/cgroup_rdma.h | 91 +++ include/linux/cgroup_subsys.h |4 + include/rdma/ib_verbs.h | 20 + init/Kconfig | 12 + kernel/Makefile |1 + kernel/cgroup_rdma.c | 1220 + 14 files changed, 1907 insertions(+), 17 deletions(-) create mode 100644 Documentation/cgroup-legacy/rdma.txt create mode 100644 drivers/infiniband/core/cgroup.c create mode 100644 include/linux/cgroup_rdma.h create mode 100644 kernel/cgroup_rdma.c -- 1.8.3.1 -- To unsubscribe from this list: send the line "unsubscribe linux-security-module" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCHv1 0/6] rdma controller support
Hello, On Wed, Jan 06, 2016 at 12:28:00AM +0530, Parav Pandit wrote: > Resources are not defined by the RDMA cgroup. Resources are defined > by RDMA/IB stack & optionally by HCA vendor device drivers. As I wrote before, I don't think this is a good idea. Drivers will inevitably add non-sensical "resources" which don't make any sense without much scrutiny. If different controllers can't agree upon the same set of resources, which probably is a pretty good sign that this isn't too well thought out to begin with, at least make all resource types defined by the controller itself and let the controllers enable them selectively. Thanks. -- tejun -- To unsubscribe from this list: send the line "unsubscribe linux-security-module" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
