Re: [patch -mm v2 1/3] mm, memcg: introduce per-memcg oom policy tunable
On Fri, 26 Jan 2018, Michal Hocko wrote: > > The cgroup aware oom killer is needlessly declared for the entire system > > by a mount option. It's unnecessary to force the system into a single > > oom policy: either cgroup aware, or the traditional process aware. > > > > This patch introduces a memory.oom_policy tunable for all mem cgroups. > > It is currently a no-op: it can only be set to "none", which is its > > default policy. It will be expanded in the next patch to define cgroup > > aware oom killer behavior. > > > > This is an extensible interface that can be used to define cgroup aware > > assessment of mem cgroup subtrees or the traditional process aware > > assessment. > > > > So what is the actual semantic and scope of this policy. Does it apply > only down the hierarchy. Also how do you compare cgroups with different > policies? Let's say you have > root > / | \ > A B C >/ \/ \ > D E F G > > Assume A: cgroup, B: oom_group=1, C: tree, G: oom_group=1 > At each level of the hierarchy, memory.oom_policy compares immediate children, it's the only way that an admin can lock in a specific oom policy like "tree" and then delegate the subtree to the user. If you've configured it as above, comparing A and C should be the same based on the cumulative usage of their child mem cgroups. The policy for B hasn't been specified, but since it does not have any children "cgroup" and "tree" should be the same. > Now we have the global OOM killer to choose a victim. From a quick > glance over those patches, it seems that we will be comparing only > tasks because root->oom_policy != MEMCG_OOM_POLICY_CGROUP. A, B and C > policies are ignored. Right, a policy of "none" reverts its subtree back to per-process comparison if you are either not using the cgroup aware oom killer or your subtree is not using the cgroup aware oom killer. > Moreover If I select any of B's tasks then I will > happily kill it breaking the expectation that the whole memcg will go > away. Weird, don't you think? Or did I misunderstand? > It's just as weird as the behavior of memory.oom_group today without using the mount option :) In that case, mem_cgroup_select_oom_victim() always returns false and the value of memory.oom_group is ignored. I agree that it's weird in -mm and there's nothing preventing us from separating memory.oom_group from the cgroup aware oom killer and allowing it to be set regardless of a selection change. If memory.oom_group is set, and the kill originates from that mem cgroup, kill all processes attached to it and its subtree. This is a criticism of the current implementation in -mm, however, my extension only respects its weirdness. > So let's assume that root: cgroup. Then we are finally comparing > cgroups. D, E, B, C. Of those D, E and F do not have any > policy. Do they inherit their policy from the parent? If they don't then > we should be comparing their tasks separately, no? The code disagrees > because once we are in the cgroup mode, we do not care about separate > tasks. > No, perhaps I wasn't clear in the documentation: the policy at each level of the hierarchy is specified by memory.oom_policy and compares its immediate children with that policy. So the per-cgroup usage of A, B, and C and compared regardless of A, B, and C's own oom policies. > Let's say we choose C because it has the largest cumulative consumption. > It is not oom_group so it will select a task from F, G. Again you are > breaking oom_group policy of G if you kill a single task. So you would > have to be recursive here. That sounds fixable though. Just be > recursive. > I fully agree, but that's (another) implementation detail of what is in -mm that isn't specified. I think where you're going is the complete separation of mem cgroup selection from memory.oom_group. I agree, and we can fix that. memory.oom_group also shouldn't depend on any mount option, it can be set or unset depending on the properties of the workload. > Then you say > > > Another benefit of such an approach is that an admin can lock in a > > certain policy for the system or for a mem cgroup subtree and can > > delegate the policy decision to the user to determine if the kill should > > originate from a subcontainer, as indivisible memory consumers > > themselves, or selection should be done per process. > > And the code indeed doesn't check oom_policy on each level of the > hierarchy, unless I am missing something. So the subgroup is simply > locked in to the oom_policy parent has chosen. That is not the case for > the tree policy. > > So look how we are comparing cumulative groups without policy with > groups with policy with subtrees. Either I have grossly misunderstood > something or this is massively inconsistent and it doesn't make much > sense to me. Root memcg without cgroup policy will simply turn off the > whole thing for the global OOM case. So you really
Re: [PATCH v2 1/7] qcom-geni-se: Add QCOM GENI SE Driver summary
On 1/16/2018 9:55 AM, Bjorn Andersson wrote: On Fri 12 Jan 17:05 PST 2018, Karthikeyan Ramasubramanian wrote: Generic Interface (GENI) firmware based Qualcomm Universal Peripheral (QUP) Wrapper is a programmable module that is composed of multiple Serial Engines (SE) and can support various Serial Interfaces like UART, SPI, I2C, I3C, etc. This document provides a high level overview of the GENI based QUP Wrapper. Signed-off-by: Karthikeyan RamasubramanianRather than adding a disconnected chunk of documentation move this into the driver(s), using the format described in https://www.kernel.org/doc/html/latest/doc-guide/kernel-doc.html?highlight=kernel%20doc#overview-documentation-comments I will move the documentation into the driver source file. Regards, Bjorn -- To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Regards, Karthik. -- Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [patch -mm v2 2/3] mm, memcg: replace cgroup aware oom killer mount option with tunable
Hello, Michal. On Mon, Jan 29, 2018 at 11:46:57AM +0100, Michal Hocko wrote: > @@ -1292,7 +1292,11 @@ the memory controller considers only cgroups belonging > to the sub-tree > of the OOM'ing cgroup. > > The root cgroup is treated as a leaf memory cgroup, so it's compared > -with other leaf memory cgroups and cgroups with oom_group option set. > +with other leaf memory cgroups and cgroups with oom_group option > +set. Due to internal implementation restrictions the size of the root > +cgroup is a cumulative sum of oom_badness of all its tasks (in other > +words oom_score_adj of each task is obeyed). This might change in the > +future. Thanks, we can definitely use more documentation. However, it's a bit difficult to follow. Maybe expand it to a separate paragraph on the current behavior with a clear warning that the default OOM heuristics is subject to changes? Thanks. -- tejun -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PULL] Documentation for 4.16
The following changes since commit 50c4c4e268a2d7a3e58ebb698ac74da0de40ae36: Linux 4.15-rc3 (2017-12-10 17:56:26 -0800) are available in the Git repository at: git://git.lwn.net/linux.git tags/docs-4.16 for you to fetch changes up to ae17a87dd7c79fa742ef5dcf06d1095eec4e1925: linux-next: docs-rst: Fix typos in kfigure.py (2018-01-17 16:49:05 -0700) Documentation updates for 4.16. New stuff includes refcount_t documentation, errseq documentation, kernel-doc support for nested structure definitions, the removal of lots of crufty kernel-doc support for unused formats, SPDX tag documentation, the beginnings of a manual for subsystem maintainers, and lots of fixes and updates. As usual, some of the changesets reach outside of Documentation/ to effect kerneldoc comment fixes. It also adds the new LICENSES directory, of which Thomas promises I do not need to be the maintainer. Expect a somewhat unpleasant conflict with the printk tree, which also has changes to printk-formats.rst. The resolution in linux-next is good; please say the word if you'd like me to create a separate resolution for you to apply. Adam Borowski (2): Documentation/filesystems/vfat.txt: fix a remark that implies UCS2 fs/*/Kconfig: drop links to 404-compliant http://acl.bestbits.at Cengiz C (1): i2c: update i2c-dev.h warning in documentation Darren Hart (VMware) (2): Documentation/kbuild: Add guidance for the use of default Documentation/process: Add CONFIG default value to submit-checklist David Sterba (1): Documentation: Add myself to the enforcement statement list Elena Reshetova (1): docs: refcount_t documentation Fabio Estevam (1): doc: usb: chipidea: Fix typo in 'enumerate' Gergo Huszty (1): Fixed typo in onewire generic doc Jonathan Corbet (3): docs: Add an intro note to the maintainers handbook docs: Remove "could not extract kernel version" warning vsprintf: Fix a dangling documentation reference Jonathan Neuschäfer (1): Documentation: mono: Update links and s/CVS/Git/ Kees Cook (1): Documentation: Add Kees Cook to list of enforcement statement endorsers Lu Baolu (1): usb: doc: Update document for USB3 debug port usage Luis R. Rodriguez (1): Documentation: Add Luis R. Rodriguez to list of enforcement statement endorsers Marco Donato Torsello (1): Documentation: kernel-hacking: corrected a typo Markus Heiser (2): docs: ftrace-uses.rst fix varios code-block directives docs: fix process/submit-checklist.rst Sphinx warning Masanari Iida (3): doc: md: Fix a file name to md-fault.c in fault-injection.txt linux-next: DOC: HWPOISON: Fix path to debugfs in hwpoison.txt linux-next: docs-rst: Fix typos in kfigure.py Matthew Wilcox (2): errseq: Add to documentation tree Documentation: Fix misconversion of #if Mauro Carvalho Chehab (20): kernel-doc: parse DECLARE_KFIFO and DECLARE_KFIFO_PTR() docs: kernel-doc.rst: better describe kernel-doc arguments docs: kernel-doc.rst: improve private members description docs: kernel-doc.rst: improve function documentation section docs: kernel-doc.rst: improve structs chapter docs: kernel-doc.rst: improve typedef documentation docs: kernel-doc.rst: add documentation about man pages docs: get rid of kernel-doc-nano-HOWTO.txt scripts: kernel-doc: get rid of unused output formats scripts: kernel-doc: improve argument handling scripts: kernel-doc: change default to ReST format scripts: kernel-doc: replace tabs by spaces scripts: kernel-doc: parse next structs/unions scripts: kernel-doc: get rid of $nested parameter scripts: kernel-doc: print the declaration name on warnings scripts: kernel-doc: handle nested struct function arguments scripts: kernel-doc: improve nested logic to handle multiple identifiers scripts: kernel-doc: apply filtering rules to warnings w1_netlink.h: add support for nested structs scripts: kernel_doc: better handle show warnings logic NeilBrown (1): Documentation: security/credentials.rst: explain need to sort group_list Randy Dunlap (2): Documentation: add Sorting section to kernel-api Documentation: add UUID/GUID to kernel-api Scott Wood (1): Documentation: Better document the hardlockup_panic sysctl Stefan Tatschner (1): Documentation/driver-api/usb: Replace dead link Thomas Gleixner (11): Documentation: Add license-rules.rst to describe how to properly identify file licenses LICENSES: Add the GPL 2.0 license LICENSES: Add the LGPL 2.0 license LICENSES: Add the LGPL-2.1 license LICENSES: Add the BSD 2-clause "Simplified" license LICENSES: Add the BSD 3-clause "New" or "Revised" License
Re: [PATCH resend 6/6] cdrom: wait for drive to become ready
On Fri, 2018-01-26 at 17:58 +0100, Michal Suchanek wrote: > When the drive closes it can take tens of seconds until the disc is > analyzed. Wait for the drive to become ready or report an error. > > Signed-off-by: Michal Suchanek> --- > drivers/cdrom/cdrom.c | 9 + > 1 file changed, 9 insertions(+) > > diff --git a/drivers/cdrom/cdrom.c b/drivers/cdrom/cdrom.c > index 69e85c902373..9994441f5041 100644 > --- a/drivers/cdrom/cdrom.c > +++ b/drivers/cdrom/cdrom.c > @@ -1087,6 +1087,15 @@ int open_for_common(struct cdrom_device_info *cdi, > tracktype *tracks) > } > cd_dbg(CD_OPEN, "the tray is now closed\n"); > } > + /* the door should be closed now, check for the disc */ > + if (ret == CDS_DRIVE_NOT_READY) { > + int poll_res = poll_event_interruptible( > + CDS_DRIVE_NOT_READY != > + (ret = cdo->drive_status(cdi, CDSL_CURRENT)), > + 500); > + if (poll_res == -ERESTARTSYS) > + return poll_res; > + } > if (ret != CDS_DISC_OK) > return -ENOMEDIUM; > } Same comment here as for a previous patch: although interruptible by a signal, I'm not sure potentially infinite loops inside the kernel are really welcome. Thanks, Bart.N�r��yb�X��ǧv�^�){.n�+{�v�"��^n�r���z���h�&���G���h�(�階�ݢj"���m��z�ޖ���f���h���~�m�
Re: [PATCH resend 3/6] cdrom: wait for tray to close
On Fri, 2018-01-26 at 17:58 +0100, Michal Suchanek wrote: > +static int cdrom_tray_close(struct cdrom_device_info *cdi) > +{ > + int ret; > + > + ret = cdi->ops->tray_move(cdi, 0); > + if (ret || !cdi->ops->drive_status) > + return ret; > + > + return poll_event_interruptible(CDS_TRAY_OPEN != > + cdi->ops->drive_status(cdi, CDSL_CURRENT), 500); > +} > + > static > int open_for_common(struct cdrom_device_info *cdi, tracktype *tracks) > { > @@ -1048,7 +1062,9 @@ int open_for_common(struct cdrom_device_info *cdi, > tracktype *tracks) > if (CDROM_CAN(CDC_CLOSE_TRAY) && > cdi->options & CDO_AUTO_CLOSE) { > cd_dbg(CD_OPEN, "trying to close the tray\n"); > - ret = cdo->tray_move(cdi, 0); > + ret = cdrom_tray_close(cdi); > + if (ret == -ERESTARTSYS) > + return ret; > if (ret) { > cd_dbg(CD_OPEN, "bummer. tried to close > the tray but failed.\n"); > /* Ignore the error from the low > @@ -2312,7 +2328,8 @@ static int cdrom_ioctl_closetray(struct > cdrom_device_info *cdi) > > if (!CDROM_CAN(CDC_CLOSE_TRAY)) > return -ENOSYS; > - return cdi->ops->tray_move(cdi, 0); > + > + return cdrom_tray_close(cdi); > } So this patch changes code that does not wait into code that potentially waits forever? Sorry but I don't think that's ideal. Please make sure that after a certain time (a few seconds?) the loop finishes. Thanks, Bart.
Re: [PATCH resend 2/6] cdrom: factor out common open_for_* code
On Fri, 2018-01-26 at 17:58 +0100, Michal Suchanek wrote: > - ret=cdo->tray_move(cdi,0); > + ret = cdo->tray_move(cdi, 0); Please separate whitespace-only changes from functional changes such that this patch series becomes easier to review. Thanks, Bart.
Re: [PATCH resend 1/6] delay: add poll_event_interruptible
On Fri, 2018-01-26 at 17:58 +0100, Michal Suchanek wrote: > Add convenience macro for polling an event that does not have a > waitqueue. > > Signed-off-by: Michal Suchanek> --- > include/linux/delay.h | 12 > 1 file changed, 12 insertions(+) > > diff --git a/include/linux/delay.h b/include/linux/delay.h > index b78bab4395d8..3ae9fa395628 100644 > --- a/include/linux/delay.h > +++ b/include/linux/delay.h > @@ -64,4 +64,16 @@ static inline void ssleep(unsigned int seconds) > msleep(seconds * 1000); > } > > +#define poll_event_interruptible(event, interval) ({ \ > + int ret = 0; \ > + while (!(event)) { \ > + if (signal_pending(current)) { \ > + ret = -ERESTARTSYS; \ > + break; \ > + } \ > + msleep_interruptible(interval); \ > + } \ > + ret; \ > +}) > + > #endif /* defined(_LINUX_DELAY_H) */ Sorry but I'm not sure we should encourage other kernel developers to use busy-waiting by adding the poll_event_interruptible() macro to a system-wide header file. Can that macro be moved into a CDROM-specific .c or .h file? Thanks, Bart.N�r��yb�X��ǧv�^�){.n�+{�v�"��^n�r���z���h�&���G���h�(�階�ݢj"���m��z�ޖ���f���h���~�m�
[PATCH] Documentation: mips: Update AU1xxx_IDE Kconfig dependencies
IDEDMA_AUTO IDEDMA_PCI_AUTO was removed in commit 120b9cfddff2 ("ide: remove CONFIG_IDEDMA_{ICS,PCI}_AUTO config") BLK_DEV_IDEDISK was removed in commit 806f80a6fc20 ("ide: add generic ATA/ATAPI disk driver") BLK_DEV_IDE_AU1XXX_BURSTABLE_ON was removed in commit 8f29e650bffc ("ide: AU1200 IDE update") Remove them from documentation Signed-off-by: Corentin Labbe--- Documentation/mips/AU1xxx_IDE.README | 7 --- 1 file changed, 7 deletions(-) diff --git a/Documentation/mips/AU1xxx_IDE.README b/Documentation/mips/AU1xxx_IDE.README index 52844a58cc8a..ff675a1b1422 100644 --- a/Documentation/mips/AU1xxx_IDE.README +++ b/Documentation/mips/AU1xxx_IDE.README @@ -56,8 +56,6 @@ Following extra configs variables are introduced: CONFIG_BLK_DEV_IDE_AU1XXX_PIO_DBDMA- enable the PIO+DBDMA mode CONFIG_BLK_DEV_IDE_AU1XXX_MDMA2_DBDMA - enable the MWDMA mode - CONFIG_BLK_DEV_IDE_AU1XXX_BURSTABLE_ON - set Burstable FIFO in DBDMA - controller SUPPORTED IDE MODES @@ -82,11 +80,9 @@ CONFIG_IDE_GENERIC=y CONFIG_BLK_DEV_IDEPCI=y CONFIG_BLK_DEV_GENERIC=y CONFIG_BLK_DEV_IDEDMA_PCI=y -CONFIG_IDEDMA_PCI_AUTO=y CONFIG_BLK_DEV_IDE_AU1XXX=y CONFIG_BLK_DEV_IDE_AU1XXX_MDMA2_DBDMA=y CONFIG_BLK_DEV_IDEDMA=y -CONFIG_IDEDMA_AUTO=y Also define 'IDE_AU1XXX_BURSTMODE' in 'drivers/ide/mips/au1xxx-ide.c' to enable the burst support on DBDMA controller. @@ -94,16 +90,13 @@ the burst support on DBDMA controller. If the used system need the USB support enable the following kernel configs for high IDE to USB throughput. -CONFIG_BLK_DEV_IDEDISK=y CONFIG_IDE_GENERIC=y CONFIG_BLK_DEV_IDEPCI=y CONFIG_BLK_DEV_GENERIC=y CONFIG_BLK_DEV_IDEDMA_PCI=y -CONFIG_IDEDMA_PCI_AUTO=y CONFIG_BLK_DEV_IDE_AU1XXX=y CONFIG_BLK_DEV_IDE_AU1XXX_MDMA2_DBDMA=y CONFIG_BLK_DEV_IDEDMA=y -CONFIG_IDEDMA_AUTO=y Also undefine 'IDE_AU1XXX_BURSTMODE' in 'drivers/ide/mips/au1xxx-ide.c' to disable the burst support on DBDMA controller. -- 2.13.6 -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Documentation: networking: ixgb: Remove reference to IXGB_NAPI
NAPI is enabled by default and IXGB_NAPI was removed since commit 6d37ab282e24 ("ixgb: make NAPI the only option and the default") Update the doc accordingly. Signed-off-by: Corentin Labbe--- Documentation/networking/ixgb.txt | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/Documentation/networking/ixgb.txt b/Documentation/networking/ixgb.txt index b10f360c57d4..82f9dc9ba399 100644 --- a/Documentation/networking/ixgb.txt +++ b/Documentation/networking/ixgb.txt @@ -324,8 +324,7 @@ Additional Configurations NAPI - NAPI (Rx polling mode) is supported in the ixgb driver. NAPI is enabled - or disabled based on the configuration of the kernel. see CONFIG_IXGB_NAPI + NAPI (Rx polling mode) is supported in the ixgb driver. See www.cyberus.ca/~hadi/usenix-paper.tgz for more information on NAPI. -- 2.13.6 -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [patch -mm v2 2/3] mm, memcg: replace cgroup aware oom killer mount option with tunable
On Fri 26-01-18 16:17:35, Andrew Morton wrote: > On Fri, 26 Jan 2018 14:52:59 -0800 (PST) David Rientjes> wrote: [...] > > Those use cases are also undocumented such that the user doesn't know the > > behavior they are opting into. Nowhere in the patchset does it mention > > anything about oom_score_adj other than being oom disabled. It doesn't > > mention that a per-process tunable now depends strictly on whether it is > > attached to root or not. It specifies a fair comparison between the root > > mem cgroup and leaf mem cgroups, which is obviously incorrect by the > > implementation itself. So I'm not sure the user would know which use > > cases it is valid for, which is why I've been trying to make it generally > > purposeful and documented. > > Documentation patches are nice. We can cc:stable them too, so no huge > hurry. What about this? >From c02d8bc1396d5ab3785d01f577e2ee74e5dd985e Mon Sep 17 00:00:00 2001 From: Michal Hocko Date: Mon, 29 Jan 2018 11:42:59 +0100 Subject: [PATCH] oom, memcg: clarify root memcg oom accounting David Rientjes has pointed out that the current way how the root memcg is accounted for the cgroup aware OOM killer is undocumented. Unlike regular cgroups there is no accounting going on in the root memcg (mostly for performance reasons). Therefore we are suming up oom_badness of its tasks. This might result in an over accounting because of the oom_score_adj setting. Document this for now. Signed-off-by: Michal Hocko --- Documentation/cgroup-v2.txt | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/Documentation/cgroup-v2.txt b/Documentation/cgroup-v2.txt index 2eaed1e2243d..7dff106bba57 100644 --- a/Documentation/cgroup-v2.txt +++ b/Documentation/cgroup-v2.txt @@ -1292,7 +1292,11 @@ the memory controller considers only cgroups belonging to the sub-tree of the OOM'ing cgroup. The root cgroup is treated as a leaf memory cgroup, so it's compared -with other leaf memory cgroups and cgroups with oom_group option set. +with other leaf memory cgroups and cgroups with oom_group option +set. Due to internal implementation restrictions the size of the root +cgroup is a cumulative sum of oom_badness of all its tasks (in other +words oom_score_adj of each task is obeyed). This might change in the +future. If there are no cgroups with the enabled memory controller, the OOM killer is using the "traditional" process-based approach. -- 2.15.1 -- Michal Hocko SUSE Labs -- To unsubscribe from this list: send the line "unsubscribe linux-doc" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html