[PATCH] mm/mmap: Check for RLIMIT_AS before unmapping
This patch fixes corner case for MAP_FIXED when requested mapping length
is larger than rlimit for virtual memory. In such case any overlapping
mappings are unmapped before we check for the limit and return ENOMEM.
The check is moved before the loop that unmaps overlapping parts of
existing mappings. When we are about to hit the limit (currently mapped
pages + len limit) we scan for overlapping pages and check again
accounting for them.
This fixes situation when userspace program expects that the previous
mappings are preserved after the mmap() syscall has returned with error.
(POSIX clearly states that successfull mapping shall replace any
previous mappings.)
This corner case was found and can be tested with LTP testcase:
testcases/open_posix_testsuite/conformance/interfaces/mmap/24-2.c
In this case the mmap, which is clearly over current limit, unmaps
dynamic libraries and the testcase segfaults right after returning into
userspace.
I've also looked at the second instance of the unmapping loop in the
do_brk(). The do_brk() is called from brk() syscall and from vm_brk().
The brk() syscall checks for overlapping mappings and bails out when
there are any (so it can't be triggered from the brk syscall). The
vm_brk() is called only from binmft handlers so it shouldn't be
triggered unless binmft handler created overlapping mappings.
Signed-off-by: Cyril Hrubis chru...@suse.cz
---
mm/mmap.c | 50 ++
1 file changed, 46 insertions(+), 4 deletions(-)
diff --git a/mm/mmap.c b/mm/mmap.c
index 2664a47..e755080 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -33,6 +33,7 @@
#include linux/uprobes.h
#include linux/rbtree_augmented.h
#include linux/sched/sysctl.h
+#include linux/kernel.h
#include asm/uaccess.h
#include asm/cacheflush.h
@@ -543,6 +544,34 @@ static int find_vma_links(struct mm_struct *mm, unsigned
long addr,
return 0;
}
+static unsigned long count_vma_pages_range(struct mm_struct *mm,
+ unsigned long addr, unsigned long end)
+{
+ unsigned long nr_pages = 0;
+ struct vm_area_struct *vma;
+
+ /* Find first overlaping mapping */
+ vma = find_vma_intersection(mm, addr, end);
+ if (!vma)
+ return 0;
+
+ nr_pages = (min(end, vma-vm_end) -
+ max(addr, vma-vm_start)) PAGE_SHIFT;
+
+ /* Iterate over the rest of the overlaps */
+ for (vma = vma-vm_next; vma; vma = vma-vm_next) {
+ unsigned long overlap_len;
+
+ if (vma-vm_start end)
+ break;
+
+ overlap_len = min(end, vma-vm_end) - vma-vm_start;
+ nr_pages += overlap_len PAGE_SHIFT;
+ }
+
+ return nr_pages;
+}
+
void __vma_link_rb(struct mm_struct *mm, struct vm_area_struct *vma,
struct rb_node **rb_link, struct rb_node *rb_parent)
{
@@ -1433,6 +1462,23 @@ unsigned long mmap_region(struct file *file, unsigned
long addr,
unsigned long charged = 0;
struct inode *inode = file ? file_inode(file) : NULL;
+ /* Check against address space limit. */
+ if (!may_expand_vm(mm, len PAGE_SHIFT)) {
+ unsigned long nr_pages;
+
+ /*
+* MAP_FIXED may remove pages of mappings that intersects with
+* requested mapping. Account for the pages it would unmap.
+*/
+ if (!(vm_flags MAP_FIXED))
+ return -ENOMEM;
+
+ nr_pages = count_vma_pages_range(mm, addr, addr + len);
+
+ if (!may_expand_vm(mm, (len PAGE_SHIFT) - nr_pages))
+ return -ENOMEM;
+ }
+
/* Clear old maps */
error = -ENOMEM;
munmap_back:
@@ -1442,10 +1488,6 @@ munmap_back:
goto munmap_back;
}
- /* Check against address space limit. */
- if (!may_expand_vm(mm, len PAGE_SHIFT))
- return -ENOMEM;
-
/*
* Private writable mapping: check memory availability
*/
--
1.8.1.5
See also a testsuite that exercies the newly added codepaths which is
attached as a tarball (All testcases minus the second that tests
that this patch works succeeds both before and after this patch).
--
Cyril Hrubis
chru...@suse.cz
mm.tar.bz2
Description: BZip2 compressed data
[PATCH] mm/mmap: Check for RLIMIT_AS before unmapping
This patch fixes corner case for MAP_FIXED when requested mapping length
is larger than rlimit for virtual memory. In such case any overlapping
mappings are unmapped before we check for the limit and return ENOMEM.
The check is moved before the loop that unmaps overlapping parts of
existing mappings. When we are about to hit the limit (currently mapped
pages + len limit) we scan for overlapping pages and check again
accounting for them.
This fixes situation when userspace program expects that the previous
mappings are preserved after the mmap() syscall has returned with error.
(POSIX clearly states that successfull mapping shall replace any
previous mappings.)
This corner case was found and can be tested with LTP testcase:
testcases/open_posix_testsuite/conformance/interfaces/mmap/24-2.c
In this case the mmap, which is clearly over current limit, unmaps
dynamic libraries and the testcase segfaults right after returning into
userspace.
I've also looked at the second instance of the unmapping loop in the
do_brk(). The do_brk() is called from brk() syscall and from vm_brk().
The brk() syscall checks for overlapping mappings and bails out when
there are any (so it can't be triggered from the brk syscall). The
vm_brk() is called only from binmft handlers so it shouldn't be
triggered unless binmft handler created overlapping mappings.
Signed-off-by: Cyril Hrubis chru...@suse.cz
---
mm/mmap.c | 50 ++
1 file changed, 46 insertions(+), 4 deletions(-)
diff --git a/mm/mmap.c b/mm/mmap.c
index 2664a47..e755080 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -33,6 +33,7 @@
#include linux/uprobes.h
#include linux/rbtree_augmented.h
#include linux/sched/sysctl.h
+#include linux/kernel.h
#include asm/uaccess.h
#include asm/cacheflush.h
@@ -543,6 +544,34 @@ static int find_vma_links(struct mm_struct *mm, unsigned
long addr,
return 0;
}
+static unsigned long count_vma_pages_range(struct mm_struct *mm,
+ unsigned long addr, unsigned long end)
+{
+ unsigned long nr_pages = 0;
+ struct vm_area_struct *vma;
+
+ /* Find first overlaping mapping */
+ vma = find_vma_intersection(mm, addr, end);
+ if (!vma)
+ return 0;
+
+ nr_pages = (min(end, vma-vm_end) -
+ max(addr, vma-vm_start)) PAGE_SHIFT;
+
+ /* Iterate over the rest of the overlaps */
+ for (vma = vma-vm_next; vma; vma = vma-vm_next) {
+ unsigned long overlap_len;
+
+ if (vma-vm_start end)
+ break;
+
+ overlap_len = min(end, vma-vm_end) - vma-vm_start;
+ nr_pages += overlap_len PAGE_SHIFT;
+ }
+
+ return nr_pages;
+}
+
void __vma_link_rb(struct mm_struct *mm, struct vm_area_struct *vma,
struct rb_node **rb_link, struct rb_node *rb_parent)
{
@@ -1433,6 +1462,23 @@ unsigned long mmap_region(struct file *file, unsigned
long addr,
unsigned long charged = 0;
struct inode *inode = file ? file_inode(file) : NULL;
+ /* Check against address space limit. */
+ if (!may_expand_vm(mm, len PAGE_SHIFT)) {
+ unsigned long nr_pages;
+
+ /*
+* MAP_FIXED may remove pages of mappings that intersects with
+* requested mapping. Account for the pages it would unmap.
+*/
+ if (!(vm_flags MAP_FIXED))
+ return -ENOMEM;
+
+ nr_pages = count_vma_pages_range(mm, addr, addr + len);
+
+ if (!may_expand_vm(mm, (len PAGE_SHIFT) - nr_pages))
+ return -ENOMEM;
+ }
+
/* Clear old maps */
error = -ENOMEM;
munmap_back:
@@ -1442,10 +1488,6 @@ munmap_back:
goto munmap_back;
}
- /* Check against address space limit. */
- if (!may_expand_vm(mm, len PAGE_SHIFT))
- return -ENOMEM;
-
/*
* Private writable mapping: check memory availability
*/
--
1.8.1.5
See also a testsuite that exercies the newly added codepaths which is
attached as a tarball (All testcases minus the second that tests
that this patch works succeeds both before and after this patch).
--
Cyril Hrubis
chru...@suse.cz
mm.tar.bz2
Description: BZip2 compressed data
Re: futex(2) man page update help request
Hi! For this complexity of tests you would just need to call the tst_resm() interface to report success/failure and, at the end of the test, tst_exit() to return the stored overall test status. And ideally call the standard option parsing code and call the test in standard loop so that the test can take advantage of standard options as number of iterations to run, etc. Have a look at: https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines there is simple test example as well as description of the interfaces. Thanks Cyril, I'll follow up with you in a couple weeks most likely. I have some urgent things that will be taking all my time and then some until then. Feel free to poke me though if I lose track of it :-) Do you still plan to work on this? -- Cyril Hrubis chru...@suse.cz -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [LTP] The Linux Test Project has been released for JANUARY 2015
Hi! I will check my iniza setup of Linux v3.19-rc5 the (runltp-)lite way. Attached are the runltp-lite result and my kernel-config. I see a lot of unexpected failures, can you comment on them? Most of the failures comes from EBUSY while trying to umount filesystems. Do you have gvfsd-trash running? That thing is known to probe newly mounted filesystems preventing them from being unmounted for a certain amount of time. -- Cyril Hrubis chru...@suse.cz -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [LTP] The Linux Test Project has been released for JANUARY 2015
Hi! Most of the failures comes from EBUSY while trying to umount filesystems. Do you have gvfsd-trash running? That thing is known to probe newly mounted filesystems preventing them from being unmounted for a certain amount of time. Yupp, that popped up. How can I disable that gvfsd (here: Ubuntu/precise? Just kill the process, or run the testcases via ssh session. And can this issue be documented? Hmm, maybe we can change the test cases to complain loudly if the process is found running. -- Cyril Hrubis chru...@suse.cz -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [LTP] The Linux Test Project has been released for JANUARY 2015
Hi! congrats to the new release! I will check my iniza setup of Linux v3.19-rc5 the (runltp-)lite way. Can you please add checksum files (md5sum, sha256, etc.) to the place where we get the tarballs, please? These are provided by sf.net, if you browse the files on sf.net there is small circle with i on the right, that, when clicked on shows md5 and sha1. These were generated by sf.net servers but I've checked that they match the files I've created. -- Cyril Hrubis chru...@suse.cz -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
The Linux Test Project has been released for JANUARY 2015
God news everyone, the Linux Test Project test suite stable release for *January 2015* has been released. Since the last release 139 patches by 21 authors were merged. Notable changes are: * Fixes and enhancements of network testcases by Alexey Kodanev * New testcases for mount, network and ipc namespaces by Matus Marhefka * New testcase for fanotify ignore masks * New policy and library helpers to install and access testcase datafailes * Use of coccinelle and spatch to fix bugs and cleanup old code * And many more smaller fixes and enhancements If you are not familiar with LTP there is a nice article at LWN.net https://lwn.net/Articles/625969/. The latest version of the test-suite contains 3000+ tests for the Linux and can be downloaded at: http://sourceforge.net/projects/ltp/files/LTP%20Source/ltp-20150119/ The project pages as well as GIT repository are hosted on GitHub for quite some time now: https://github.com/linux-test-project/ltp http://linux-test-project.github.io/ If you ever wondered how to write a LTP testcase, don't miss our developer documentation at: https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines https://github.com/linux-test-project/ltp/wiki/BuildSystem Patches, new tests, bugs, comments or questions should go to to our mailing list at ltp-l...@lists.sourceforge.net. -- Cyril Hrubis chru...@suse.cz -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: futex(2) man page update help request
Hi! I'll follow up with you in a couple weeks most likely. I have some urgent things that will be taking all my time and then some until then. Feel free to poke me though if I lose track of it :-) FYI I've started to work on futex testcases for LTP. The first batch has been commited in: https://github.com/linux-test-project/ltp/commit/6270ba2ebe999ffdb1364e5e814d7e56070a0198 Some of these are losely based on futextest some are written from scratch. The requeue operation, pi futexes and bitset are not covered yet. -- Cyril Hrubis chru...@suse.cz -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [LTP] The Linux Test Project has been released for JANUARY 2015
Hi! Yupp, that popped up. How can I disable that gvfsd (here: Ubuntu/precise? Just kill the process, or run the testcases via ssh session. And can this issue be documented? Hmm, maybe we can change the test cases to complain loudly if the process is found running. Good idea. And should be fixed in latest git. Now the testcases complain about the failure, give a hint what may be causing it, and if the umount() is not the the syscall under test, it's retried a few times with short usleep between. -- Cyril Hrubis chru...@suse.cz -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
LTP] [ANNOUNCE] The Linux Test Project has been released for APRIL 2015
Good news everyone, the Linux Test Project test suite stable release for *April 2015* has been released. Since the last release 183 patches by 23 authors were merged. Notable changes are: * Continued effort to cleanup up old testcases and test library (Special thanks to Coccinelle and spatch for making this easier) * More sleep() calls were changed into proper synchronization which saves a few minutes on test run time * Cleanups, fixes and new tests in network testcases * New testcases for futex() syscall * New testcases for select() and poll() timeouts * New mmap() regression testcase * New regression test for hugepage leak * Fixes all over the codebase The latest version of the test-suite contains 3000+ tests for the Linux and can be downloaded at: http://sourceforge.net/projects/ltp/files/LTP%20Source/ltp-20150420/ The project pages as well as GIT repository are hosted on GitHub for quite some time now: https://github.com/linux-test-project/ltp http://linux-test-project.github.io/ If you ever wondered how to run LTP or write a testcase, don't miss our developer documentation and LWN.net articles at: https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines https://github.com/linux-test-project/ltp/wiki/BuildSystem https://lwn.net/Articles/625969/ https://lwn.net/Articles/630542/ Patches, new tests, bugs, comments or questions should go to to our mailing list at ltp-list@ltp-l...@lists.sourceforge.net. -- Cyril Hrubis chru...@suse.cz -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 1/5] selftests: Add futex functional tests
Hi! I'm happy to do that, but I would like to make sure I'm doing the right thing. The right thing here is to add -pthread to CFLAGS which sets both flags for preprocessor and linker (see man gcc). Hi Cyril, Thanks. I read that, and mentioned it, but my concern with -pthread in the CFLAGS and LDFLAGS is that it is a non-standard compiler flag. I understand we have a number of gcc-isms in our build - but do we want to add more? I'm also struggling to find any kind of prescribed documentation on this beyond the short blurb in the gcc man page which describes what this option does, but not when to use it. I'll need something concrete to justify changes to testcase Makefiles to Shuah. Sorry to mislead you with the pointing at gcc man page. It is a Linux standard. Have a look at pthreads manual page: http://man7.org/linux/man-pages/man7/pthreads.7.html On Linux, programs that use the Pthreads API should be compiled using cc -pthread. Or any pthread_foo() manual page that starts with: Compile and link with -pthread. The portable way i.e. POSIX would be getting compiler flags with getconf but as this is a Linux kernel testsuite I would not bother with that. Hmm, and it looks like this is not implemented on Linux anyway. -- Cyril Hrubis chru...@suse.cz -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 1/5] selftests: Add futex functional tests
Hi! I'm happy to do that, but I would like to make sure I'm doing the right thing. The right thing here is to add -pthread to CFLAGS which sets both flags for preprocessor and linker (see man gcc). -- Cyril Hrubis chru...@suse.cz -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv2 1/1] Documentation: describe how to add a system call
Hi! +Testing +--- + +A new system call should obviously be tested; it is also useful to provide +reviewers with a demonstration of how user space programs will use the system +call. A good way to combine these aims is to include a simple self-test +program in a new directory under tools/testing/selftests/. I know that this is a bit off topic, but since the selftest is now the official place to add kernel testcases to let me rant about it a bit. It's a bit of a pain seeing people reinvent the wheel and trying to figure out consistent test interface while most of the problems has been solved in LTP[1] test library quite some time ago. Especially use of the SAFE_MACROS[2] would simplify writing test setups quite a lot. I wonder if we can at least share the test library, pulling it out of LTP, or at least the interesting parts, wouldn't be hard at all. [1] https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#221-basic-test-structure [2] https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#224-safe-macros -- Cyril Hrubis chru...@suse.cz -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[ANNOUNCE] The Linux Test Project has been released for SEPTEMBER 2015
Good news everyone, the Linux Test Project test suite stable release for *September 2015* has been released. Since the last release 272 patches by 27 authors were merged. Notable changes are: * Network namespace testcases were rewritten from scratch * New user namespaces testcases * New testcases for various virtual network interfaces * New umount2() testcases (for UMOUNT_NOFOLLOW, MNT_EXPIRE and MNT_DETACH flags) * New open() testcase (for O_PATH flag) * New getrandom() testcases * New inotify, cpuset, futex_wake() and recvmsg() regression tests + The usual number of fixes and enhancements The latest version of the test-suite contains 3000+ tests for the Linux and can be downloaded at: https://github.com/linux-test-project/ltp/releases/tag/20150903 The project pages as well as GIT repository are hosted on GitHub: https://github.com/linux-test-project/ltp http://linux-test-project.github.io/ If you ever wondered how to write a LTP testcase, don't miss our (continuously updated) developer documentation at: https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines https://github.com/linux-test-project/ltp/wiki/BuildSystem Patches, new tests, bugs, comments or questions should go to to our mailing list at ltp-l...@lists.sourceforge.net. PS: We had numerous troubles with mailing list on sourceforge, if you are aware of good and free mailing list hosting service please let us know. -- Cyril Hrubis chru...@suse.cz -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: LTP and commit e1d7ba8735551ed7 ("time: Always make sure wall_to_monotonic isn't positive")
Hi!
> While testing v4.3-rc1 with the LTP, we spotted failures in a couple of
> timekeeping tests, which seem to be a result of commit e1d7ba8735551ed7
> ("time: Always make sure wall_to_monotonic isn't positive").
>
> In both cases this is because the new time would be before the boot
> time, so I guess these are covered by the caveat at the end of the
> commit message.
>
> settimeofday01 [1] tries tries to settimeofday() with a timeval 100s and
> 100ns, while clock_gettime02 [2] tries to clock_settime to 1s and
> 100ns. Neither expect -EINVAL, and report failure with the new
> behaviour.
>
> While I don't know that any "real" software is affected, I thought I
> should let you know.
Thanks for the report, we allready have patch for the first testcase on
the mailing list that initializes the the timeval value to current time
+ small amount. I will have a look at the second test as well.
> [1]
> https://github.com/linux-test-project/ltp/blob/20150903/testcases/kernel/syscalls/settimeofday/settimeofday01.c
> [2]
> https://github.com/linux-test-project/ltp/blob/20150903/testcases/kernel/timers/clock_settime/clock_settime02.c
--
Cyril Hrubis
chru...@suse.cz
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [LTP] LTP and commit e1d7ba8735551ed7 ("time: Always make sure wall_to_monotonic isn't positive")
Hi! > Thanks for the report, we allready have patch for the first testcase on > the mailing list that initializes the the timeval value to current time > + small amount. I will have a look at the second test as well. > > > [1] > > https://github.com/linux-test-project/ltp/blob/20150903/testcases/kernel/syscalls/settimeofday/settimeofday01.c > > [2] > > https://github.com/linux-test-project/ltp/blob/20150903/testcases/kernel/timers/clock_settime/clock_settime02.c Both should be fixed in latest git. -- Cyril Hrubis chru...@suse.cz -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH] mm: fix incorrect behavior when process virtual address space limit is exceeded
Hi! > [CCing Cyril] Ah I've confused the vm_flags and flags variables and nobody caught that during the review. But still sorry that I've messed up. Looking at the code I agree with Michal that we try to find the intesection poinlesly even for !MAP_FIXED which slowns down mmap() tiny little bit which should be fixed. The fix looks good to me. -- Cyril Hrubis chru...@suse.cz -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [LTP] [patch V2 00/20] timer: Refactor the timer wheel
Hi!
> > rtbox:~ #
> > /usr/local/ltp/conformance/interfaces/sigtimedwait/sigtimedwait_1-1.run-test
> > Test FAILED: sigtimedwait() did not return in the required time
> > time_elapsed: 1.197057
> > ...come on, you can do it...
> > rtbox:~ #
> > /usr/local/ltp/conformance/interfaces/sigtimedwait/sigtimedwait_1-1.run-test
> > Test PASSED
> >
> > #define ERRORMARGIN 0.1
> > ...
> > if ((time_elapsed > SIGTIMEDWAITSEC + ERRORMARGIN)
> > || (time_elapsed < SIGTIMEDWAITSEC - ERRORMARGIN)) {
> > printf("Test FAILED: sigtimedwait() did not return in "
> > "the required time\n");
> > printf("time_elapsed: %lf\n", time_elapsed);
> > return PTS_FAIL;
> > }
> >
> > Looks hohum to me, but gripe did arrive with patch set, so you get a note.
>
> hohum is a euphemism. That's completely bogus.
>
> The only guarantee a syscall with timers has is: timer does not fire early.
While this is true, checking with reasonable error margin works just
fine 99% of the time. You cannot really test that timer expires, without
setting arbitrary margin.
Looking into POSIX sigtimedwait() timer should run on CLOCK_MONOTONIC so
we can call clock_getres(CLOCK_MONOTOINC, ...) double or tripple the
value and use it for error margin. And also fix the test to use
the CLOCK_MONOTONIC timer.
And of course the error margin must not be used when we check that the
elapsed time wasn't shorter than we expected.
Does that sound reasonable?
--
Cyril Hrubis
chru...@suse.cz
Re: [LTP] [patch V2 00/20] timer: Refactor the timer wheel
Hi! > > While this is true, checking with reasonable error margin works just > > fine 99% of the time. You cannot really test that timer expires, without > > setting arbitrary margin. > > Err. You know that the timer expired because sigtimedwait() returns > EAGAIN. And the only thing you can reliably check for is that the timer did > not expired to early. Anything else is guesswork and voodoo programming. There is quite a lot of things that can happen on mutitasking OS and there are even NMIs in hardware, etc. But seriously is there a reason why OS that is not under heavy load cannot expire timers with reasonable overruns? I.e. if I ask for a second of sleep and expect it to be woken up not much more than half of a second later? If we stick only to guarantees that are defined in POSIX playing music with mplayer would not be possible since it sleeps in futex() and if it wakes too late it will fail to fill buffers. In practice this worked fine for me for years. -- Cyril Hrubis chru...@suse.cz
Re: [LTP] [patch V2 00/20] timer: Refactor the timer wheel
Hi! > Two points: > 1) sigtimedwait() is unusual in that it uses the jiffies timer. Most >system call timeouts (including specifically the one in FUTEX_WAIT) >use the high-resolution timer subsystem, which is a whole different >animal with tighter guarantees, and That is likely POSIX conformance bug, since POSIX explicitly states that sigtimedwait() shall use CLOCK_MONOTONIC to measure the timeout. "If the Monotonic Clock option is supported, the CLOCK_MONOTONIC clock shall be used to measure the time interval specified by the timeout argument." > 2) The worst-case error in tglx's proposal is 1/8 of the requested >timeout: the wakeup is after 112.5% of the requested time, plus >one tick. This is well within your requested accuracy. (For very >short timeouts, the "plus one tick" can dominate the percentage error.) Hmm, that still does not add up to the number in the original email where it says time_elapsed: 1.197057. As far as I can tell the worst case for a tick is CONFIG_HZ=100 so one tick is 0.01s and even after that we get 118.7% since we requested 1s. But that may be caused by the fact that the test uses gettimeofday() to measure the elapsed time, it should use CLOCK_MONOTONIC instead. -- Cyril Hrubis chru...@suse.cz
Re: [LTP] [patch V2 00/20] timer: Refactor the timer wheel
Hi! > > That is likely POSIX conformance bug, since POSIX explicitly states that > > sigtimedwait() shall use CLOCK_MONOTONIC to measure the timeout. > > > > "If the Monotonic Clock option is supported, the CLOCK_MONOTONIC clock > > shall be used to measure the time interval specified by the timeout > > argument." > > That's fine because jiffies is a less granular form of CLOCK_MONOTONIC. Looking into POSIX Realtime Clock and Timers it seems to allow that time service based on CLOCK_* clocks to have different resolution if it's less or equal than 20ms and if this fact is documented. If we wanted to be pedantic about this the man page shoud be patched... Also this gives us reasonably safe upper bound on timer expiration to be something as: sleep_time * 1.125 + 20ms Does this sounds reasonable now? -- Cyril Hrubis chru...@suse.cz
[ANNOUNCE] The Linux Test Project has been released for JANUARY 2016
Good news everyone, the Linux Test Project test suite stable release for *January 2016* has been released. Since the last release 191 patches by 29 authors were merged. Notable changes are: * Rewritten and new cgroup tests for cpuacct and pids controllers * Rewritten basic cgroup functional and stress tests * New userns07 test for user namespaces * New syscall tests for: - renameat2() - sched_getattr() - sched_setattr() - kcmp() - fcntl(fd, F_SETLEASE) - preadv() - pwritev() * New regression tests for: - signal() - mmap() + hugepages, - shmget()/shmat() + hugepages * Open Posix pthread_create() tests were partially rewritten and no longer fail on machines with > 4 CPUS * New tests for mkfs, mkswap, and which utilities * New tests for kernel input stack (utilizing uinput) * Fixes in nfs tests and IPV6 support * New test for dctcp congestion control algorithm The latest version of the test-suite contains 3000+ tests for the Linux and can be downloaded at: https://github.com/linux-test-project/ltp/releases/tag/20160126 The project pages as well as GIT repository are hosted on GitHub: https://github.com/linux-test-project/ltp http://linux-test-project.github.io/ If you ever wondered how to write a LTP testcase, don't miss our (continuously updated) developer documentation at: https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines https://github.com/linux-test-project/ltp/wiki/BuildSystem Patches, new tests, bugs, comments or questions should go to to our new mailing list at l...@lists.linux.it PS: Special thanks to Italian Linux Society (www.ils.org) that provides us with rock solid mailing list hosting. This release wouldn't be possible without their kind help. -- Cyril Hrubis chru...@suse.cz
[LTP] [ANNOUNCE] The Linux Test Project has been released for MAY 2016
Good news everyone, the Linux Test Project test suite stable release for *May 2016* has been released. Since the last release 169 patches by 24 authors were merged. Notable changes are: * New C library test API that significantly eases test writing This is the end result of last few years of cleanups and fixes. Most of the common code has been moved to the library which both helps to avoid common mistakes and lets the programmer concentrate on test itself. It also fixed long standing problems with thread safety, simplified test result propagation from child processes, fixed problems with cleanup callback, etc. If you ever given up on writing a LTP testcase because the test API was incomplete and difficult to use you should really reconsider now. The old and new test API will coexistent for a while until all old testcases are converted, which will take quite some time. New test are written using the new API only. You can have a quick look at new/converted tests to get a feel for the new API: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/syscalls/recvmsg/recvmsg02.c https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/syscalls/creat/creat06.c Or have a look at updated documentation at: https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#22-writing-a-test-in-c * New syscall tests for: - request_key() - preadv() - pwritev(), - fcntl() + F_OFD_* locks - personality() - removexattr() - llistxattr() - epoll_pwait(), - epoll_wait() - open() + O_TMPFILE And regression tests for: - madvise() + MADV_WILLNEED - recvmsg() + MSG_PEEK * Rewritten network stress tests * New test for lsmod * Usual amount of fixes - fixed file corruptions with ltp-pan - added a few return value checks for fork() where the value is later passed to kill() resulting in call to kill(-1, ...) if fork() failed - a few more skipped test on Btrfs on unsupported/unimplemented functionality - fixed test result propagation in five fcntl testcases + much more... The latest version of the test-suite contains 3000+ tests for the Linux and can be downloaded at: https://github.com/linux-test-project/ltp/releases/tag/20160510 The project pages as well as GIT repository are hosted on GitHub: https://github.com/linux-test-project/ltp http://linux-test-project.github.io/ If you ever wondered how to write a LTP testcase, don't miss our (continuously updated) developer documentation at: https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines https://github.com/linux-test-project/ltp/wiki/BuildSystem Patches, new tests, bugs, comments or questions should go to to our mailing list at l...@lists.linux.it. -- Cyril Hrubis chru...@suse.cz
Re: [LTP] [lkp-robot] [fs] 99f64a2676: ltp.creat08.fail & ltp.open10.fail
Hi!
> FYI, we noticed the following commit:
>
> commit: 99f64a2676f0bec4ad32e39fc76eb0914ee091b8 ("fs: Harden against
> open(..., O_CREAT, 02777) in a setgid directory")
> https://git.kernel.org/pub/scm/linux/kernel/git/luto/linux.git vfs/suid
>
> in testcase: ltp
> with following parameters:
>
> test: ltplite
>
> test-description: The LTP testsuite contains a collection of tools for
> testing the Linux kernel and related features.
> test-url: http://linux-test-project.github.io/
Looking at the testcases in question these are testing exactly the
corner case the patch is fixing.
So the tests needs to be fixed once this patch hits mainline. And the
best fix is to expect the SGID flag to be cleared on newer kernels.
--
Cyril Hrubis
chru...@suse.cz
[LTP] [ANNOUNCE] The Linux Test Project has been released for JANUARY 2017
Good news everyone, the Linux Test Project test suite stable release for *January 2017* has been released. Since the last release 211 patches by 28 authors were merged. Notable changes: o New testcases for: - flistxattr() - listxattr() - llistxattr() - getpriority() - setpriority() - sendto() - dirtyc0w - vti and geneve tunnel tests (network tests) o New shell test library was added and many old and broken tests were rewritten to make use of it o 36 syscall testcases were converted to the new library o Many cases of bashism were fixed o Experimental support for network namespaces has been added into network tests, which means that these test can now be optionally executed on a single testing machine o LTP compiled for 32bit should now run flawlessly on 64bit kernel which is useful for testing the 32bit compat syscall interface + Usuall amount of cleanups, fixes and speedups. Another bit worth mentioning is that LTP now uses travis to compile test latest git head. The latest version of the test-suite contains 3000+ tests for the Linux and can be downloaded at: https://github.com/linux-test-project/ltp/releases/tag/20170116 The project pages as well as GIT repository are hosted on GitHub: https://github.com/linux-test-project/ltp http://linux-test-project.github.io/ If you ever wondered how to write LTP testcase, don't miss documentation at: https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines https://github.com/linux-test-project/ltp/wiki/BuildSystem Patches, new tests, bugs, comments or questions should go to to our mailing list at l...@lists.linux.it. -- Cyril Hrubis chru...@suse.cz
[ANNOUNCE] The Linux Test Project has been released for SEPTEMBER 2016
Good news everyone, the Linux Test Project test suite stable release for *September 2016* has been released. Since the last release 236 patches by 28 authors were merged. Notable changes are: o New testcases - tracepath, clockdiff, arpping, ping, wc, insmod - ftrace - access() - fallocate() FALLOC_FL_INSTERT_RANGE - kcmp() - lgetxattr() - sched_setscheduler() - epoll_ctl() - sbrk() regression test - dio_sparse added direct I/O regression test (https://patchwork.ozlabs.org/patch/622065/) o Rewritten/cleaned up tests - waitpid syscall testcases - NFS tests - ftrace tests o More work on memory cgroup testcases - a lot of cleanups, big tests were split into smaller ones - implemented proper synchronization primitives between processes which saves a few minutes on the test execution time o More work on the new test API - test is executed in child processes, parent process watches for timeout, does cleanup, reports results, etc. - Ctrl+C kills all subprocesses correctly and the parent process runs cleanup - new command line option parsing helpers - 25 testcases were cleaned up and rewritten to use the new test API o New EXPECT_PASS and EXPECT_FAIL helpers in shell library o New shell wrappers for checkpoint (futex based) synchronization primitives o New tst_su wrapper (that makes sure that path to LTP binaries is in $PATH) o Open Posix Testsuite changes - parallel build has been fixed - the build process now uses the top level configure parameters (CC, CFLAGS, LDLIBS, LDFLAGS) + The usuall amount of cleanups, fixes, and speedups. The latest version of the test-suite contains 3000+ tests for the Linux and can be downloaded at: https://github.com/linux-test-project/ltp/releases/tag/20160920 The project pages as well as GIT repository are hosted on GitHub: https://github.com/linux-test-project/ltp http://linux-test-project.github.io/ If you ever wondered how to write a LTP testcase, don't miss our (continuously updated) developer documentation at: https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines https://github.com/linux-test-project/ltp/wiki/BuildSystem Patches, new tests, bugs, comments or questions should go to to our mailing list at ltp-l...@lists.linux.it -- Cyril Hrubis chru...@suse.cz
Re: Formal description of system call interface
Hi! > We identified a surprisingly large number of potential users for such > descriptions: > - fuzzers (syzkaller, trinity, iknowthis) > - strace/syscall tracepoints (capturing indirect arguments and >printing human-readable info) > - generation of entry points for C libraries (glibc, liblinux >(raw syscalls), Go runtime, clang/gcc sanitizers) > - valgrind/sanitizers checking of input/output values of syscalls > - seccomp filters (minijail, libseccomp) need to know interfaces >to generate wrappers > - safety certification (requires syscall specifications) > - man pages (could provide actual syscall interface rather than >glibc wrapper interface, it was noted that possible errno values >is an important part here) > - generation of syscall argument validation in kernel (fast version >is enabled all the time, extended is optional) I was thinking of generating LTP testcases from a well defined format for quite some time. Maybe that would be a good way how to keep the descriptions to match the reality. LTP test would however need a bit more information that just syscall parameter anotation. We would need also description of the expected return value in some cases, annotating it as "returns fd" or "zero on success" would be good enough for basic tests. Better testing would need to describe the "side efect" as well (file was created, filesystem mounted, etc.) then we could write a classes of verify functions, something as check_file_exits() and use them to check the results accordingly. I'm not sure if something like this is really doable or in the scope of this project, but it may be worth a try. -- Cyril Hrubis chru...@suse.cz
Re: Formal description of system call interface
Hi! > Description of "returns fd or this set of errors" looks simple and useful. > Any test system or fuzzer will be able to verify that kernel actually returns > claimed return values. Also Carlos expressed interested in errno values > in the context of glibc. > I would do it from day one. > > Re more complex side effects. I always feared that a description suitable > for automatic verification (i.e. zero false positives, otherwise it is > useless) > may be too difficult to achieve. I'm afraid it may be as well. I would expect that we will end up with something quite complex with a large set of exceptions from the rules. > Cyril, Tavis, can you come up with some set of predicates that can be > checked automatically yet still useful? > We can start small, e.g. "must not alter virtual address space". I will try to thing about this a bit. -- Cyril Hrubis chru...@suse.cz
Re: Formal description of system call interface
Hi! > > Looking ahead into the future, I was also thinking that if this becomes > > robust, we could also start an integration specification, that could > > describe how different system calls interact with each other. Like > > open() to read(), write() and close(). > > > > But this is just an idea that popped in my head while reading this > > thread. We want to start small first, but still could keep this in the > > back of our minds for future enhancements. > > > FWIW syzkaller does something along these lines. > It has notion of 'resources' and of input/output arguments. > Then it can figure out that e.g. open creates fd's, so it should be called > before any reads/writes (provided that we want to pass in valid fd's). > It does not have notion of "destructors" for resources (e.g. close > destroys the passed in resource). But it should be easy to describe. Logical extension would be to teach it that creat(), for instance, returns file descriptor and creates a file as a side effect. That file then could be used for a stat() or unlink() concurently, etc. But we should also consider that not all file descriptors or files are equal, so we may end up with some classes of files and file descriptors some of them suitable for different subsets of operations. I think that defining classes of objects and defining how syscalls transform their state may yield something usable. But that would require some serious thinking and a few trial and error implementations. -- Cyril Hrubis chru...@suse.cz
Re: [LTP] [lkp-robot] [KEYS] bdf7c0f8bf: ltp.add_key02.fail
Hi!
> > commit: bdf7c0f8bf282ba44827ce3c7fd7936c8e90a18a ("KEYS: fix dereferencing
> > NULL payload with nonzero length")
> > url:
> > https://github.com/0day-ci/linux/commits/Eric-Biggers/KEYS-fix-dereferencing-NULL-payload-with-nonzero-length/20170403-102013
> > base:
> > https://git.kernel.org/cgit/linux/kernel/git/jmorris/linux-security.git next
> >
> ...
> > caused below changes (please refer to attached dmesg/kmsg for entire
> > log/backtrace):
> >
> >
> > user :notice: [ 45.447047] <<>>
> >
> > user :notice: [ 45.447365] tag=add_key02 stime=1492169102
> >
> > user :notice: [ 45.447567] cmdline="add_key02"
> >
> > user :notice: [ 45.447685] contacts=""
> >
> > user :notice: [ 45.447826] analysis=exit
> >
> > user :notice: [ 45.448011] <<>>
> >
> > user :notice: [ 45.448568] tst_test.c:760: INFO: Timeout per run is 0h
> > 05m 00s
> >
> > user :notice: [ 45.449439] add_key02.c:65: FAIL: add_key() failed
> > unexpectedly, expected EINVAL: EFAULT
>
> In my opinion this is a valid behavior, and the test is just weird; it's
> passing
> in *both* an unaddressable payload and an invalid description, so it's not
> clear
> which case it's meant to be testing. (Generally, if a syscall will fail for
> more than one reason, it's not guaranteed which error code you'll get.)
That is quite common problem with LTP testcases. Do you care to send a
patch or should I fix that?
> In any case, once we have a fix merged, it would be nice for there to be an
> ltp
> test added for the "NULL payload with nonzero length" case with one of the key
> types that crashed the kernel.
Here as well, feel free to send a patch or at least point us to a
reproducer that could be turned into a testcase.
--
Cyril Hrubis
chru...@suse.cz
[ANNOUNCE] The Linux Test Project has been released for May 2017
Good news everyone, the Linux Test Project test suite stable release for *May 2017* has been released. Since the last release 264 patches by 28 authors were merged. Notable changes are: o New tests for: - quotaclt() regression test - quotactl() Q_XGETNEXTQUOTA - lseek() SEEK_HOLE and SEEK_DATA - keyctl() regression tests - memfd_create() tests - madvise() MADV_FREE, MADV_DONTDUMP, MADV_HWPOISON - recvmsg() regression test - getxattr() regression test - ioctl() BLKGETSIZE/BLKGETSIZE64, BLKRAGET/BLKRASET, BLKROSET/BLKROGET - move_pages() regression test o Additional 47 testcases were cleaned up and converted to the new test library. o SAFE_MACROS() can now be used in test cleanup which significantly simplifies test cleanup functions. o Numa memory migration testcases were cleaned up and fixed. o Minimal testing device size was bumped to 256MB in order to support modern filesystems. o Initiall ANDROID support was merged in. o Test output is now colorized in case that we print to terminal. + The usuall amount of fixes all around the place The latest version of the test-suite contains 3000+ tests for the Linux and can be downloaded at: https://github.com/linux-test-project/ltp/releases/tag/20170516 The project pages as well as GIT repository are hosted on GitHub: https://github.com/linux-test-project/ltp http://linux-test-project.github.io/ If you ever wondered how to write a LTP testcase, don't miss our developer documentation at: https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines https://github.com/linux-test-project/ltp/wiki/BuildSystem Patches, new tests, bugs, comments or questions should go to to our mailing list at l...@lists.linux.it. -- Cyril Hrubis chru...@suse.cz
Re: kernel of next-20170602 call trace when run add_key02 in LTP
Hi,
> Compile kernel (next-20170602) and run ltp, find:
>
> / # ./add_key02
> tst_test.c:878: INFO: Timeout per run is 0h 05m 00s
> [ 341.183219] BUG: unable to handle kernel NULL pointer dereference at
> (null)
> [ 341.183850] IP: memset+0x10/0x20
> [ 341.184550] *pdpt = 35441001 *pde =
> [ 341.184550]
> [ 341.184550] Oops: 0002 [#2] SMP
> [ 341.184550] Modules linked in:
> [ 341.184550] CPU: 0 PID: 124 Comm: add_key02 Tainted: G SD W
> 4.12.0-rc3-next-20170602 #3
> [ 341.184550] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
> BIOS Bochs 01/01/2011
> [ 341.184550] task: f5b9ca00 task.stack: f6514000
> [ 341.184550] EIP: memset+0x10/0x20
> [ 341.184550] EFLAGS: 0246 CPU: 0
> [ 341.184550] EAX: EBX: ECX: 0001 EDX:
> [ 341.184550] ESI: EDI: EBP: f6515f24 ESP: f6515f1c
> [ 341.184550] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
> [ 341.184550] CR0: 80050033 CR2: CR3: 36404920 CR4: 06f0
> [ 341.184550] DR0: DR1: DR2: DR3:
> [ 341.184550] DR6: DR7:
> [ 341.184550] Call Trace:
> [ 341.184550] memzero_explicit+0xf/0x20
> [ 341.184550] SyS_add_key+0x11f/0x1c0
> [ 341.184550] ? change_pid+0x13/0x50
> [ 341.184550] do_fast_syscall_32+0x8b/0x130
> [ 341.184550] entry_SYSENTER_32+0x4e/0x7c
> [ 341.184550] EIP: 0xb772ddc1
> [ 341.184550] EFLAGS: 0246 CPU: 0
> [ 341.184550] EAX: ffda EBX: 080de341 ECX: 080de346 EDX:
> [ 341.184550] ESI: 0001 EDI: fffc EBP: 0808aa97 ESP: bfe3636c
> [ 341.184550] DS: 007b ES: 007b FS: GS: 0033 SS: 007b
> [ 341.184550] Code: 8a 0e 88 0f 8d b4 26 00 00 00 00 8b 45 f0 83 c4
> 04 5b 5e 5f 5d c3 90 8d 74 26 00 3e 8d 74 26 00 55 89 e5 57 89 c7 53
> 89 c3 89 d0 aa 89 d8 5b 5f 5d c3 90 90 90 90 90 90 90 90 3e 8d 74
> 26 00
> [ 341.184550] EIP: memset+0x10/0x20 SS:ESP: 0068:f6515f1c
> [ 341.184550] CR2:
> [ 341.219144] ---[ end trace e3963c970d107f91 ]---
> tst_test.c:928: INFO: If you are running on slow machine, try
> exporting LTP_TIMEOUT_MUL > 1
> tst_test.c:929: BROK: Test killed! (timeout?)
>
> I try to use other tags and kernel on next-20170427 is ok, but
> next-20170502 fail.
> Is it bug?
Looks like a kernel bug to me.
The test is a very simple one that just does:
add_key("keyring", "wjkey", NULL, 0, KEY_SPEC_THREAD_KEYRING));
And expects success.
Also CCing LTP ML and relevant maintainers.
--
Cyril Hrubis
chru...@suse.cz
Re: commit f5f99309 (sock: do not set sk_err in sock_dequeue_err_skb) has broken ping
Hi! > Thank you for the confirmation. Could you please try the following > patch to see if it fixes your issue? Does not seem to help, I still got the same bussy loop. -- Cyril Hrubis chru...@suse.cz
Re: commit f5f99309 (sock: do not set sk_err in sock_dequeue_err_skb) has broken ping
Hi! > >> Thank you for the confirmation. Could you please try the following > >> patch to see if it fixes your issue? > > > > Does not seem to help, I still got the same bussy loop. > > Thank you for trying the patch. Unfortunately, I can't reproduce on my > machines here.Would you humor me with another one? Thank you! Still does not help. I've modified the ping binary (on the top of you patch that reads the error queue) to dump the error structure. It seems to repeatedly produce (until I plug the cable back): ee_errno = 113 ee_origin = 2 ee_type = 3 ee_code = 1 ee_info = 0 ee_data = 0 So we get EHOSTUNREACH on SO_EE_ORIGIN_ICMP. -- Cyril Hrubis chru...@suse.cz
Re: commit f5f99309 (sock: do not set sk_err in sock_dequeue_err_skb) has broken ping
Hi! > Thank you very much! I have a wild guess that, when we > have a train of skbs on the error queue starting from a local error, > we will see this issue. > > Ping (without my patch) considers EAGAIN on a normal read as an > indication that there is nothing on the error queue, but that's a > flawed assumption. > > Would you mind trying another shot in the darkness please? Thanks! This patch seems to fix the issue, I've tried several times and poll() just timeouts, haven't seen a single POLLERR in the ping strace. You can add my Tested-by: for this patch as well :-). -- Cyril Hrubis chru...@suse.cz
Re: commit f5f99309 (sock: do not set sk_err in sock_dequeue_err_skb) has broken ping
Hi! > > I've bisected the problem to this commit: > > > > commit f5f99309fa7481f59a500f0d08f3379cd6424c1f (HEAD, refs/bisect/bad) > > Author: Soheil Hassas Yeganeh <soh...@google.com> > > Date: Thu Nov 3 18:24:27 2016 -0400 > > > > sock: do not set sk_err in sock_dequeue_err_skb > > Hi Cyril, > > I'm sorry for the problem, and thank you for the report. > > Two questions: > 1. Could you double check whether you have the following commit in your tree? > > commit 83a1a1a70e87f676fbb6086b26b6ac7f7fdd107d > Author: Soheil Hassas Yeganeh <soh...@google.com> > Date: Wed Nov 30 14:01:08 2016 -0500 > sock: reset sk_err for ICMP packets read from error queue I've started bisecting on v4.11 and see the problem on v4.10 on another machine, the patch should be there in both cases and the bug is easily reproducible. > 2. I've also have sent a fix to iputils on > https://github.com/iputils/iputils/pull/75. Would you be kind to try > that pull request as well? That fixed the problem, you can add: Tested-by: Cyril Hrubis <chru...@suse.cz> -- Cyril Hrubis chru...@suse.cz
commit f5f99309 (sock: do not set sk_err in sock_dequeue_err_skb) has broken ping
Hi! I've started to wonder why is ping eating 100% CPU shortly after I've upgraded my machine to 4.10 and here is what I found: The ping main_loop() sleeps in poll() on its socket, the poll() usually times out, at least that's what strace suggets which causes ping to sleep for ~1s in the kernel. See ping source at: https://github.com/iputils/iputils/blob/master/ping_common.c#L587 The poll() seems to start returning POLLERR immediatelly after poll() is called on the socket in a case that connection has dropped for a short while. It seems to be easily reproducible with: * Starting ping with some ip address i.e. ping 4.2.2.2 * Letting it ping for a minute or so * Disconnection a WAN cable from your AP * After a minute or so ping ends up bussy looping on poll() that returns with POLLERR immediatelly * After plugging the cable back the problem gets only worse since we now spend 99% of the time bussy looping on the poll() syscall * And my CPU fan starts to scream loudly I've bisected the problem to this commit: commit f5f99309fa7481f59a500f0d08f3379cd6424c1f (HEAD, refs/bisect/bad) Author: Soheil Hassas Yeganeh <soh...@google.com> Date: Thu Nov 3 18:24:27 2016 -0400 sock: do not set sk_err in sock_dequeue_err_skb -- Cyril Hrubis chru...@suse.cz
[LTP] [ANNOUNCE] The Linux Test Project has been released for SEPTEMBER 2017
Good news everyone, the Linux Test Project test suite stable release for *September 2017* has been released. Since the last release 290 patches by 39 authors were merged. Notable changes for this release include: - * New tests for: - copy_file_range() - splice() between pipes and sockets - splice() between two pipes - clone() with CLONE_NEWNET - msync() with mmaped file - getsockopt() with SO_PEERCRED - fanotify() with FAN_CLOEXEC - dynamic_debug tracing - basic gdb test - combination of OFD and POSIX locks on a file * New regression tests for: - fanotify() 96d41019e3ac and 05f0e38724e8 - add_key() 5649645d725c - fcntl() 086e774a57fb - sched/autogroup 18f649ef3441 - memcg https://bugzilla.redhat.com/show_bug.cgi?id=1168185 - vdso permissions b6558c4a2378 and e5b97dde514f - keyctl() 63a0b0509e70 - numa MPOL_PREFERRED policy 0867a57c * Removed tests: - long broken dmapi testsuite * 66 testcases were cleaned up and converted to the new test library * KSM (kernel samepage merging) test should be race-free now * So far if only small part of a test reported skipped status the overall test status was reported as skipped as well. Now if test reports both sucessful and skipped results the overall status is reported as suscessful. * Timer mesurement library code was introduced - all syscalls that use kernel timers are measured in a unified way - we take much more samples and use discarded mean to get rid of outliners - the threshold is now defined in the library and can be easily adjusted - for more information see: https://github.com/linux-test-project/ltp/commit/c459654db64cd29177a382ab178fdd5ad59664e4 * Also effort to convert CVE reproducers into LTP testcases started to gain momentum - It starts by introducing thread synchronization library that auto-tunes a spin-in-a-place delay so that we execute two critical sections of code at the same time, which makes tests based on races much faster and stable in reproducing the bugs. - So far implemented CVE tests include: - CVE-2011-0999 transparent hugepages used wrongly for args/env - CVE-2011-2183 NULL pointer dereference in ksm_do_scan() - CVE-2011-2496 address overflow in mremap() - CVE-2012-0957 uname26 memory leak - CVE-2014-0196 PTY echo race - CVE-2015-0235 buffer overflow in gethostbyname_r() - CVE-2015-7550 race between keyctl_read() and keyctl_revoke() - CVE-2016-4470 uninitialized variable in key_reject_and_link() - CVE-2016-5195 aka dirtyc0w - CVE-2016-7042 crash in /proc/keys show function - CVE-2016-7117 use after free between recvmmsg() and close() - CVE-2016-10044 mark AIO pseudo-fs noexec - CVE-2017-2618 off-by-one in setprocattr - CVE-2017-2671 missing rwlock in ping_unhashed() - CVE-2017-5669 NULL page mapped with shmat() and SHM_RND - CVE-2017-6951 NULL dereference in request_key() - CVE-2017-7472 thread keyring memory leak - CVE-2017-1000364 aka stack_clash Note that a few of the testcases were implemented in the previous release as well but these were not counted as CVE related. * The ltp-pan testrunner that executes LTP testcases marks test start in kernel log, which helps with pairing LTP testcase with a kernel trace. * There is new step-by-step C test tutorial at: https://github.com/linux-test-project/ltp/wiki/C-Test-Case-Tutorial * A few EROFS related testcases now use read-only tmpfs instead of the loopback based device, which saves up to half of a minute of runtime per such testcase. The rest of such testcases will be converted after the release. * The nfsv4 locktest source has been translated from French. * Many fixes in numa testcases. * Many fixes in network testcases. + The usuall amount of fixes all over the codebase What havent't made it to this release: -- * A patchset that adds test library support to allow for filesystem related syscalls, such as fallocate(), to be executed against all supported filesystems. * Fixes and speed ups for the CVE synchronization library and a few CVE reproducers. Downloads and links: The latest version of the test-suite contains 3000+ tests for the Linux and can be downloaded at: https://github.com/linux-test-project/ltp/releases/tag/20170930 The project pages as well as GIT repository are hosted on GitHub: https://github.com/linux-test-project/ltp http://linux-test-project.github.io/ If you ever wondered how to write a LTP testcase, don't miss our developer documentation at: https://github.com/linux-test-project/ltp/wiki/C-Test-Case-Tutorial https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines https://github.com/linux-test-project/ltp/wiki/BuildSystem Patches, new tests, bugs, comments or questions should go to to our mailing list at l...@lists.linux.it. -- Cyril Hrubis chru...@suse.cz
Re: [PATCH 2/2] mmap.2: MAP_FIXED updated documentation
Hi! > You selected stupid name for a flag. Everyone and their dog agrees > with that. There's even consensus on better name (and everyone agrees > it is better than .._SAFE). Of course, we could have debate if it is > NOREPLACE or NOREMOVE or ... and that would be bikeshed. This was just > poor naming on your part. Well while everybody agrees that the name is so bad that basically anything else would be better, there does not seem to be consensus on which one to pick. I do understand that this frustrating and fruitless. So what do we do now, roll a dice to choose new name? Or do we ask BFDL[1] to choose the name? [1] https://en.wikipedia.org/wiki/Benevolent_dictator_for_life -- Cyril Hrubis chru...@suse.cz
Re: [PATCH 2/2] mmap.2: MAP_FIXED updated documentation
Hi! > Pretty map everyone agreed MAP_FIXED_SAFE was a bad > name. MAP_FIXED_NOREPLACE (IIRC) was best replacement. For what it's worth I do agree here. -- Cyril Hrubis chru...@suse.cz
Re: [LTP] Towards 4.14 LTS
Hi! > So why didn???t we report these? As mentioned we???ve been tossing out dodgy > test cases to get to a clean baseline. We don???t need or want noise. > > For LTS, I want the system when it detects a failure to enable a quick > bisect involving the affected test bucket. Given the nature of kernel > bugs tho, there is that class of bug which only happens occasionally. >From my experience debugging kernel bugs requires an actuall human interaction and there is only certain level of automation that can be achieved. Don't take me wrong, automatic bisection and other bells and whistles are a nice to have, but at the end of the day you usually need someone to reproduce/look at the problem, possibly check the source code, report a bug, etc. Hence it does not make much sense to have an automated system without dedicated engineers assigned to review the test results. -- Cyril Hrubis chru...@suse.cz
Re: [LTP] Towards 4.14 LTS
Hi! > For instance 4.13-rc just was added to the mix. > > For test buckets, I???m currently dorking around with some make check targets > for a few interesting packages. You may want to look into xfstests as well, we found a few kernel oopses recently related to backported FS patches for SLES kernels. -- Cyril Hrubis chru...@suse.cz
Re: [PATCH 0/2] mm: introduce MAP_FIXED_SAFE
Hi! > > MAP_FIXED_UNIQUE > > MAP_FIXED_ONCE > > MAP_FIXED_FRESH > > Well, I can open a poll for the best name, but none of those you are > proposing sound much better to me. Yeah, naming sucks... Given that MAP_FIXED replaces the previous mapping MAP_FIXED_NOREPLACE would probably be a best fit. -- Cyril Hrubis chru...@suse.cz
Re: [PATCH v2] mmap.2: MAP_FIXED updated documentation
Hi! I know that we are not touching the rest of the existing description for MAP_FIXED however the second sentence in the manual page says that "addr must be a multiple of the page size." Which however is misleading as this is not enough on some architectures. Code in the wild seems to (mis)use SHMLBA for aligment purposes but I'm not sure that we should advise something like that in the manpages. So what about something as: "addr must be suitably aligned, for most architectures multiple of page size is sufficient, however some may impose additional restrictions for page mapping addresses." Which should at least hint the reader that this is architecture specific. -- Cyril Hrubis chru...@suse.cz
Re: [PATCH v2] mmap.2: MAP_FIXED updated documentation
Hi! > >> (It does seem unfortunate that the man page cannot help the programmer > >> actually write correct code here. He or she is forced to read the kernel > >> implementation, in order to figure out the true alignment rules. I was > >> hoping we could avoid that.) > > > > It would be nice if we had this information exported somehere so that we > > do not have to rely on per-architecture ifdefs. > > > > What about adding MapAligment or something similar to the /proc/meminfo? > > > > What's the use case you envision for that? I don't see how that would be > better than using SHMLBA, which is available at compiler time. Because > unless someone expects to be able to run an app that was compiled for > Arch X, on Arch Y (surely that's not requirement here?), I don't see how > the run-time check is any better. I guess that some kind of compile time constant in uapi headers will do as well, I'm really open to any solution that would expose this constant as some kind of official API. There is one problem with using SHMLBA, there are more than one libc implementations and at least two of them (bionic and klibc) does not implement the SysV IPC at all. I know that the chances that you are writing a program that requires MAP_FIXED, is compiled against alternative libc, and expected to run on less common architectures are pretty slim. On the other hand I do not see a reason why this cannot be done properly, this is just about exporting one simple constant to userspace after all. > Or maybe you're thinking that since the SHMLBA cannot be put in the man > pages, we could instead provide MapAlignment as sort of a different > way to document the requirement? This is my intention as well. -- Cyril Hrubis chru...@suse.cz
Re: [PATCH v2] mmap.2: MAP_FIXED updated documentation
Hi! > (It does seem unfortunate that the man page cannot help the programmer > actually write correct code here. He or she is forced to read the kernel > implementation, in order to figure out the true alignment rules. I was > hoping we could avoid that.) It would be nice if we had this information exported somehere so that we do not have to rely on per-architecture ifdefs. What about adding MapAligment or something similar to the /proc/meminfo? -- Cyril Hrubis chru...@suse.cz
Re: [PATCH 0/2] mm: introduce MAP_FIXED_SAFE
Hi! > > If we had a time machine, the right set of flags would be: > > > > - MAP_FIXED: don't treat addr as a hint, fail if addr is not free > > - MAP_REPLACE: replace an existing mapping (or force or clobber) > > Actually, if we had a time machine... would we even provide > MAP_REPLACE functionality? I did a bit of archeology just beacause we can and since there is a git repository of the unix history [1]. The first version of mmap() seems to appear in BSD-4_2-Snapshot there was no MAP_FIXED flag and the addr is expected to be used for the mapping. At least that is what manual seems to say, the kernel code is not written at this point. This seems to correspond to a time when Berkley students were busy rewriting UNIX kernel to take advantage of the VAX's virtual memory. The MAP_FIXED arrived to the manual shortly after, probably someone figured out that passing an address to the call does not make much sense in most of the cases. The first actual implementation that supports MAP_FIXED appeared in the BSD-4_3_Reno-Snapshot and already includes the replace behavior. The original purpose seems to be replacing mappings in the implementation of the execve() call. So the answer would probably be yes but it would probably made sense to keep it as kernel internal flag. And BTW it looks like HPUX got it right before it was changed to follow POSIX. There seems to be HPUX compatibility code in the early BSD codebase that contains both HPUXMAP_FIXED and HPUXMAP_REPLACE. [1] https://github.com/dspinellis/unix-history-repo -- Cyril Hrubis chru...@suse.cz
Re: [LTP] [lkp-robot] [x86/topology] 379a4bb988: dmesg.WARNING:at_arch/x86/events/intel/uncore.c:#uncore_change_type_ctx
Hi! > > bin/lkp install job.yaml > > [root@hpe-dl385gen10-02 lkp-tests]# ls > allotdaemon filters include Makefile pkgrepospec > bin distro Gemfile jobs monitors plot rootfs > stats > cluster doc Gemfile.lock lib pack Rakefile sbin > tests > _config.yml etc hosts lkp-exec paramsREADME.md setup > tools > [root@hpe-dl385gen10-02 lkp-tests]# bin/lkp install ../job.yaml > Not a supported system, cannot install packages. > [root@hpe-dl385gen10-02 lkp-tests]# > > Well that's useful. Sorry for the late reply, you may have figured it out already, you can install and run LTP without the lkp framework and that should be pretty straightforward, then you can just run it by: 'cd /opt/ltp && ./runltp -f cpuhotplug' Looking at the test it may need a kernel source unpacked /usr/src/linux as well. Short HOWTO for building LTP from git is here: https://github.com/linux-test-project/ltp/blob/master/doc/mini-howto-building-ltp-from-git.txt And we do have (unofficional) packages for some SUSE distributions: https://build.opensuse.org/package/show/benchmark/ltp -- Cyril Hrubis chru...@suse.cz
[LTP] [ANNOUNCE] The Linux Test Project has been released for MAY 2018
s release: git shortlog -s -e -n 20180118.. 76 Petr Vorel <pvo...@suse.cz> 38 Cyril Hrubis <chru...@suse.cz> 37 Michael Moese <mmo...@suse.de> 30 Alexey Kodanev <alexey.koda...@oracle.com> 21 Xiao Yang <yangx...@cn.fujitsu.com> 12 yang xu <xuyang...@cn.fujitsu.com> 9 Yixin Zhang <yixin.zh...@intel.com> 7 Jinhui Huang <huangjh...@cn.fujitsu.com> 7 Richard Palethorpe <rpaletho...@suse.com> 6 Jan Stancek <jstan...@redhat.com> 6 Li Wang <liw...@redhat.com> 5 Amir Goldstein <amir7...@gmail.com> 4 Carlo Marcelo Arenas Belon <care...@gmail.com> 4 Stanislav Kholmanskikh <stanislav.kholmansk...@oracle.com> 3 Petr Vorel <petr.vo...@gmail.com> 3 Steve Muckle <smuckle.li...@gmail.com> 2 Anders Roxell <anders.rox...@linaro.org> 2 Eric Biggers <ebigg...@google.com> 2 Khem Raj <raj.k...@gmail.com> 2 Sandeep Patil <sspa...@google.com> 2 Sebastian Chlad <sch...@suse.de> 2 xiao yang <yangx...@cn.fujitsu.com> 1 Christian Lanig <cla...@suse.com> 1 Dmitry Safonov <d...@arista.com> 1 Erick Reyes <erickre...@google.com> 1 Greg Hackmann <ghackm...@google.com> 1 Harish <har...@linux.vnet.ibm.com> 1 Hoang Van Tuyen <tuyen.hoang...@toshiba-tsdv.com> 1 Hridya Valsaraju <hri...@google.com> 1 Joe Konno <joe.ko...@intel.com> 1 Lars Persson <lar...@axis.com> 1 Lianwen Sun <sunlw.f...@cn.fujitsu.com> 1 Pavel Boldin <pbol...@cloudlinux.com> 1 Qiao Zhao <qiaozqj...@gmail.com> 1 Xiong Zhou <xz...@redhat.com> 1 Yang Pengfei <yangpengf...@huawei.com> 1 akodanev <akoda...@gmail.com> 1 xie hui <xieh...@cn.fujitsu.com> 1 yosun <yo...@suse.com> And also thanks to patch reviewers: git log 20180118.. | grep -Ei '(reviewed|acked)-by:' | \ sed 's/.*by: //' | sort | uniq -c | sort -n -r 34 Cyril Hrubis <chru...@suse.cz> 15 Petr Vorel <pvo...@suse.cz> 15 Alexey Kodanev <alexey.koda...@oracle.com> 11 Jan Stancek <jstan...@redhat.com> 7 Richard Palethorpe <rpaletho...@suse.de> 4 Li Wang <liw...@redhat.com> 3 Mimi Zohar <zo...@linux.vnet.ibm.com> 2 Petr Vorel <petr.vo...@gmail.com> 2 James Morris <james.l.mor...@oracle.com> 1 yang xu <xuyang...@cn.fujitsu.com> 1 Xiao Yang <yangx...@cn.fujitsu.com> -- Cyril Hrubis chru...@suse.cz
[LTP] [ANNOUNCE] The Linux Test Project has been released for JANUARY 2018
Good news everyone, the Linux Test Project test suite stable release for *January 2018* has been released. Since the last release 278 patches by 35 authors were merged. Notable changes for this release include: - * New tests for: - unshare(1) command - ioctl07 test for RNDGETENTCNT ioctl() - new network MACsec testcases - new network IPsec SCTP and DCCP testcases * New regression tests for: - CVE-2017-5754 aka meltdown - CVE-2017-12193 (test add_key04) - CVE-2017-15299 and CVE-2017-15951 (test request_key03) - CVE-2017-7308 (test setsockopt02) - CVE-2016-9604 (test keyctl08) - CVE-2017-15537 (test ptrace07) - CVE-2017-12192 (test keyctl07) - add_key03 regression test for kernel commit 237bbd29f7a0 - keyctl06 regression test for kernel commit e645016abc80 * Fixed tests: - brk01 (test rewritten from scratch) - sigwaitinfo01 (fixed and enabled) - openposix aio testcases (uninitialized aiocb) + many smaller fixes * Removed tests: - invalid openposix pthread_barrier_wait_6-1 test - tcp_cmds tests for rwho, echo, finger, and rdist. * The test library gained support to run a particular test against different filesystems including FUSE filesystems such as NTFS or exFAT. The mkfs and kernel/FUSE support for a particular filesystem must be in-place otherwise the tests will skip it automatically. Some of the filesystem specific syscall tests such as fallocate() are executed this way now. We also have a new test that fills up filesystem using several threads and expects the syscalls to fail gracefully. * The fuzzy synchronization library that is used to trigger races mostly in CVE testcases was rewritten to use one thread instead of starting a thread on each iteration, which is not only faster but also more stable since we introduce less random jitter to the timing measurements this way. * Various fixes and enhancements for the network testcases. * Support for NUMA API older than v2 was dropped from the testcases. * The configure script now correctly detects devel libraries on -m32 build. * Another large scale cleanup using coccinelle was done on the code base. We transformed patterns such as: if (scall(...) < 0) tst_brkm(TBROK, ...); into: SAFE_SCALL(); Which will produce unified and more verbose error reporting in case that the call to scall() will fail. * The runltp script now lists test skipped by the skipfile parameter as skipped in the testrun results, these were missing from it previously. * 24 testcases were cleaned up and converted to the new test library + The usual amount of fixes all over the code base Downloads and links: The latest version of the test-suite contains 3000+ tests for the Linux and can be downloaded at: https://github.com/linux-test-project/ltp/releases/tag/20180118 The project pages as well as GIT repository are hosted on GitHub: https://github.com/linux-test-project/ltp http://linux-test-project.github.io/ If you ever wondered how to write a LTP testcase, don't miss our developer documentation at: https://github.com/linux-test-project/ltp/wiki/C-Test-Case-Tutorial https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines https://github.com/linux-test-project/ltp/wiki/BuildSystem Patches, new tests, bugs, comments or questions should go to to our mailing list at l...@lists.linux.it. -- Cyril Hrubis chru...@suse.cz
Re: [LTP] [lkp-robot] [sched/deadline] e0367b1267: WARNING:at_kernel/sched/sched.h:#assert_clock_updated
Hi! > Hummm, wondering how LTP sched tests could trigger this, since a quick > grep into ltp didn't show DEADLINE usage. See kernel/syscalls/sched_setattr/sched_setattr01.c -- Cyril Hrubis chru...@suse.cz
Re: [LTP] [lkp-robot] [sched/deadline] e0367b1267: WARNING:at_kernel/sched/sched.h:#assert_clock_updated
Hi! > > > Hummm, wondering how LTP sched tests could trigger this, since a quick > > > grep into ltp didn't show DEADLINE usage. > > > > See kernel/syscalls/sched_setattr/sched_setattr01.c > > Right, saw that. I was still thinking though why the report seemed to > point to sched, and not syscalls, tests. These collections of tests have sometimes non empty intersections. In this case the particular test is both part of the sched and syscall testruns (grep sched_setattr01 runtest/*). So I suppose that the test fails both in the sched and syscall run but the sched one was simply picked first here. > Thanks for pointing this out. Glad to help :-). -- Cyril Hrubis chru...@suse.cz
[LTP] [ANNOUNCE] The Linux Test Project has been released for SEPTEMBER 2018
Good news everyone,
the Linux Test Project test suite stable release for *September 2018* has been
released.
Since the last release 275 patches by 39 authors were merged.
NOTABLE CHANGES
===
* New tests
- statx tests for:
* basic tests
* invalid inputs
* AT_EMPTY_PATH, AT_SYMLINK_NOFOLLOW
* FS_* flags
* STATX_ATTR_ENCRYPTED (on ext4)
+ more is on the way
- mlock2
* basic tests
* invalid inputs
- memfd_create
* MFD_HUGETLB, MFD_HUGE_2MB, MFD_HUGE_1GB
- fgetxattr
- fsetxattr
- prctl
* PR_{SET,GET}_CHILD_SUBREAPER
- madvise
* MADV_WIPEONFORK, MADV_KEEPONFORK
* New regression tests
- mlock203
b155b4fde5bd mm: mlock: avoid increase mm->locked_vm on mlock() when
already mlock2(,MLOCK_ONFAULT)
- execveat
355139a8dba4 cap_inode_getsecurity: use d_find_any_alias() instead of
d_find_alias()
- pcrypt_aead01 aka CVE-2017-18075
d76c68109f37 crypto: pcrypt - fix freeing pcrypt instances
- shmctl05
3f05317d9889 ipc/shm: fix use-after-free of shm file via remap_file_pages()
- realpath01 aka cve-2018-101
fixed by glibc commit 52a713fdd0a3
- inotify07
d90a10e2444b fsnotify: Fix fsnotify_mark_connector race
- fanotify09
54a307ba8d3c fanotify: fix logic of events on child
- pty02
966031f340185 n_tty: fix EXTPROC vs ICANON interaction with TIOCINQ (aka
FIONREAD)
- migrate_pages03
4b0ece6fa016 mm: migrate: fix remove_migration_pte() for ksm pages
* msgctl IPC test were redesigned and rewritten
* ~50 tests were cleaned up and rewritten to the new library API
* Several network helper libraries and tests has been ported into new API
(busy poll, DHCP, ipneigh, IPsec, iptables, SCTP, TCP fast open, virt, ...)
+ Many fixes all over the place
* Removed code
- pounder21 - broken and wasn't compiled by default for years
* What didn't make it to this release
- second rewrite and simplification of the fuzzy sync library
and this time the user API is much easier to use
- rewrite of the mempolicy NUMA tests that should finally solve
all race conditions and random failures
TODO LIST AND ISSUE TRACKER
===
As noted in the previous release we started to maintain a LTP TODO list in the
form GitHub issues, as expected the list is growing:
https://github.com/linux-test-project/ltp/labels/missing%20coverage
But more importantly we are starting to implement testcases based on that list,
recently we added the fsetxattr, fgetxattr, mlock2, prctl, and statx tests and
more is about to come in the near future.
We do have also list of reproducers to be turned into testcases:
https://github.com/linux-test-project/ltp/labels/reproducer
Lastly but not least I would like to ask everyone to contribute to these lists.
If you have a reproducer or notice missing coverage do not hesitate to open an
issue.
DOWNLOAD AND LINKS
==
The latest version of the test-suite contains 3000+ tests for the Linux
and can be downloaded at:
https://github.com/linux-test-project/ltp/releases/tag/20180926
The project pages as well as GIT repository are hosted on GitHub:
https://github.com/linux-test-project/ltp
http://linux-test-project.github.io/
If you ever wondered how to write a LTP testcase, don't miss our developer
documentation at:
https://github.com/linux-test-project/ltp/wiki/C-Test-Case-Tutorial
https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines
https://github.com/linux-test-project/ltp/wiki/BuildSystem
Patches, new tests, bugs, comments or questions should go to to our mailing
list at l...@lists.linux.it.
CREDITS
===
Many thanks to the people contributing to this release:
git shortlog -s -e -n 20180515..
61 Petr Vorel
31 Xiao Yang
28 Cyril Hrubis
25 Alexey Kodanev
18 Jan Stancek
13 Li Wang
9 Alistair Strachan
9 Jinhui huang
9 Mylene Josserand
8 Yixin Zhang
7 Punit Agrawal
7 Rafael David Tinoco
7 Richard Palethorpe
5 Jinhui Huang
4 Sun Lianwen
4 vaishnavid
3 Michael Moese
3 Sandeep Patil
2 Amir Goldstein
2 Eric Biggers
2 Junchi Chen
2 Kewal Ukunde
2 Wang Long
1 Anton Smorodskyi
1 Chen Rong
1 Christian Lanig
1 Eddie.Horng
1 Fang Guan
1 Fathi Boudra
1 Guangwen Feng
1 He Zhe
1 Kenneth Magic
1 Po-Hsu Lin
1 Stanislav Kholmanskikh
1 Vishnu K
1 Yang Shi
1 xiao yang
And also thanks to patch reviewers:
git log 20180515.. | grep -Ei '(reviewed|acked)-by:' | sed 's/.*by: //' | sort
| uniq -c | sort -n -r
64 Cyril Hrubis
36 Jan Stancek
33 Petr Vorel
18 Alexey Kodanev
17 Li Wang
2 Richard Palethorpe
2 Petr Vorel
1 Xiao Yang
1 Naresh Kamboju
1 Mimi Zohar
1 Jan Kara
--
Cyril Hrubis
chru...@suse.cz
[PATCH] mm/mmap: Check for RLIMIT_AS before unmapping
This patch fixes corner case for MAP_FIXED when requested mapping length
is larger than rlimit for virtual memory. In such case any overlapping
mappings are unmapped before we check for the limit and return ENOMEM.
The check is moved before the loop that unmaps overlapping parts of
existing mappings. When we are about to hit the limit (currently mapped
pages + len > limit) we scan for overlapping pages and check again
accounting for them.
This fixes situation when userspace program expects that the previous
mappings are preserved after the mmap() syscall has returned with error.
(POSIX clearly states that successfull mapping shall replace any
previous mappings.)
This corner case was found and can be tested with LTP testcase:
testcases/open_posix_testsuite/conformance/interfaces/mmap/24-2.c
In this case the mmap, which is clearly over current limit, unmaps
dynamic libraries and the testcase segfaults right after returning into
userspace.
I've also looked at the second instance of the unmapping loop in the
do_brk(). The do_brk() is called from brk() syscall and from vm_brk().
The brk() syscall checks for overlapping mappings and bails out when
there are any (so it can't be triggered from the brk syscall). The
vm_brk() is called only from binmft handlers so it shouldn't be
triggered unless binmft handler created overlapping mappings.
Signed-off-by: Cyril Hrubis
---
mm/mmap.c | 50 ++
1 file changed, 46 insertions(+), 4 deletions(-)
diff --git a/mm/mmap.c b/mm/mmap.c
index 2664a47..e755080 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -33,6 +33,7 @@
#include
#include
#include
+#include
#include
#include
@@ -543,6 +544,34 @@ static int find_vma_links(struct mm_struct *mm, unsigned
long addr,
return 0;
}
+static unsigned long count_vma_pages_range(struct mm_struct *mm,
+ unsigned long addr, unsigned long end)
+{
+ unsigned long nr_pages = 0;
+ struct vm_area_struct *vma;
+
+ /* Find first overlaping mapping */
+ vma = find_vma_intersection(mm, addr, end);
+ if (!vma)
+ return 0;
+
+ nr_pages = (min(end, vma->vm_end) -
+ max(addr, vma->vm_start)) >> PAGE_SHIFT;
+
+ /* Iterate over the rest of the overlaps */
+ for (vma = vma->vm_next; vma; vma = vma->vm_next) {
+ unsigned long overlap_len;
+
+ if (vma->vm_start > end)
+ break;
+
+ overlap_len = min(end, vma->vm_end) - vma->vm_start;
+ nr_pages += overlap_len >> PAGE_SHIFT;
+ }
+
+ return nr_pages;
+}
+
void __vma_link_rb(struct mm_struct *mm, struct vm_area_struct *vma,
struct rb_node **rb_link, struct rb_node *rb_parent)
{
@@ -1433,6 +1462,23 @@ unsigned long mmap_region(struct file *file, unsigned
long addr,
unsigned long charged = 0;
struct inode *inode = file ? file_inode(file) : NULL;
+ /* Check against address space limit. */
+ if (!may_expand_vm(mm, len >> PAGE_SHIFT)) {
+ unsigned long nr_pages;
+
+ /*
+* MAP_FIXED may remove pages of mappings that intersects with
+* requested mapping. Account for the pages it would unmap.
+*/
+ if (!(vm_flags & MAP_FIXED))
+ return -ENOMEM;
+
+ nr_pages = count_vma_pages_range(mm, addr, addr + len);
+
+ if (!may_expand_vm(mm, (len >> PAGE_SHIFT) - nr_pages))
+ return -ENOMEM;
+ }
+
/* Clear old maps */
error = -ENOMEM;
munmap_back:
@@ -1442,10 +1488,6 @@ munmap_back:
goto munmap_back;
}
- /* Check against address space limit. */
- if (!may_expand_vm(mm, len >> PAGE_SHIFT))
- return -ENOMEM;
-
/*
* Private writable mapping: check memory availability
*/
--
1.8.1.5
See also a testsuite that exercies the newly added codepaths which is
attached as a tarball (All testcases minus the second that tests
that this patch works succeeds both before and after this patch).
--
Cyril Hrubis
chru...@suse.cz
mm.tar.bz2
Description: BZip2 compressed data
[PATCH] mm/mmap: Check for RLIMIT_AS before unmapping
This patch fixes corner case for MAP_FIXED when requested mapping length
is larger than rlimit for virtual memory. In such case any overlapping
mappings are unmapped before we check for the limit and return ENOMEM.
The check is moved before the loop that unmaps overlapping parts of
existing mappings. When we are about to hit the limit (currently mapped
pages + len > limit) we scan for overlapping pages and check again
accounting for them.
This fixes situation when userspace program expects that the previous
mappings are preserved after the mmap() syscall has returned with error.
(POSIX clearly states that successfull mapping shall replace any
previous mappings.)
This corner case was found and can be tested with LTP testcase:
testcases/open_posix_testsuite/conformance/interfaces/mmap/24-2.c
In this case the mmap, which is clearly over current limit, unmaps
dynamic libraries and the testcase segfaults right after returning into
userspace.
I've also looked at the second instance of the unmapping loop in the
do_brk(). The do_brk() is called from brk() syscall and from vm_brk().
The brk() syscall checks for overlapping mappings and bails out when
there are any (so it can't be triggered from the brk syscall). The
vm_brk() is called only from binmft handlers so it shouldn't be
triggered unless binmft handler created overlapping mappings.
Signed-off-by: Cyril Hrubis
---
mm/mmap.c | 50 ++
1 file changed, 46 insertions(+), 4 deletions(-)
diff --git a/mm/mmap.c b/mm/mmap.c
index 2664a47..e755080 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -33,6 +33,7 @@
#include
#include
#include
+#include
#include
#include
@@ -543,6 +544,34 @@ static int find_vma_links(struct mm_struct *mm, unsigned
long addr,
return 0;
}
+static unsigned long count_vma_pages_range(struct mm_struct *mm,
+ unsigned long addr, unsigned long end)
+{
+ unsigned long nr_pages = 0;
+ struct vm_area_struct *vma;
+
+ /* Find first overlaping mapping */
+ vma = find_vma_intersection(mm, addr, end);
+ if (!vma)
+ return 0;
+
+ nr_pages = (min(end, vma->vm_end) -
+ max(addr, vma->vm_start)) >> PAGE_SHIFT;
+
+ /* Iterate over the rest of the overlaps */
+ for (vma = vma->vm_next; vma; vma = vma->vm_next) {
+ unsigned long overlap_len;
+
+ if (vma->vm_start > end)
+ break;
+
+ overlap_len = min(end, vma->vm_end) - vma->vm_start;
+ nr_pages += overlap_len >> PAGE_SHIFT;
+ }
+
+ return nr_pages;
+}
+
void __vma_link_rb(struct mm_struct *mm, struct vm_area_struct *vma,
struct rb_node **rb_link, struct rb_node *rb_parent)
{
@@ -1433,6 +1462,23 @@ unsigned long mmap_region(struct file *file, unsigned
long addr,
unsigned long charged = 0;
struct inode *inode = file ? file_inode(file) : NULL;
+ /* Check against address space limit. */
+ if (!may_expand_vm(mm, len >> PAGE_SHIFT)) {
+ unsigned long nr_pages;
+
+ /*
+* MAP_FIXED may remove pages of mappings that intersects with
+* requested mapping. Account for the pages it would unmap.
+*/
+ if (!(vm_flags & MAP_FIXED))
+ return -ENOMEM;
+
+ nr_pages = count_vma_pages_range(mm, addr, addr + len);
+
+ if (!may_expand_vm(mm, (len >> PAGE_SHIFT) - nr_pages))
+ return -ENOMEM;
+ }
+
/* Clear old maps */
error = -ENOMEM;
munmap_back:
@@ -1442,10 +1488,6 @@ munmap_back:
goto munmap_back;
}
- /* Check against address space limit. */
- if (!may_expand_vm(mm, len >> PAGE_SHIFT))
- return -ENOMEM;
-
/*
* Private writable mapping: check memory availability
*/
--
1.8.1.5
See also a testsuite that exercies the newly added codepaths which is
attached as a tarball (All testcases minus the second that tests
that this patch works succeeds both before and after this patch).
--
Cyril Hrubis
chru...@suse.cz
mm.tar.bz2
Description: BZip2 compressed data
[LTP] [ANNOUNCE] The Linux Test Project has been released for MAY 2018
Good news everyone,
the Linux Test Project test suite stable release for *May 2018* has been
released.
Since the last release 297 patches by 38 authors were merged.
NOTABLE CHANGES
===
* New testcases:
- two inotify regression tests for:
764baba80168 ("ovl: hash non-dir by lower inode for fsnotify")
31747eda41ef ("ovl: hash directory inodes for fsnotify")
- sysclt01 a regression test for:
5ccba44ba118 ("sched/sysctl: Check user input value of
sysctl_sched_time_avg")
- CVE regression tests for:
* CVE-2017-17053 - ccd5b3235180 ("x86/mm: Fix use-after-free of
ldt_struct")
* CVE-2017-16939 - 1137b5e2529a ("ipsec: Fix aborted xfrm policy dump
crash")
* CVE-2015-3290 - 9b6e6a8334d5 ("x86/nmi/64: Switch stacks on userspace
NMI entry")
* CVE-2017-17052 - 2b7e8665b4ff ("fork: fix incorrect fput of ->exe_file
causing use-after-free")
* CVE-2017-17807 - 4dca6ea1d943 ("KEYS: add missing permission check for
request_key() destination")
- read_all
Test that aims to stress test /proc, /sys, and safe subset of /dev
filesystems by reading by reading each readable file using several
threads.
- The pwritev03 and preadv03 tests with O_DIRECT executed on all supported
filesystems.
(https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#2214-testing-with-a-block-device)
- Pids controller tests fixed a race that made them fail sporadically and
the the coverage was increased.
- pwrite03, write02
Write syscall tests for special case with NULL buffer and len 0
where the call is expected to return 0 i.e. success.
* Removed outdated and broken tests
- sssd daemon test
- mail command test
- ht_interrupt -- hyperthreading interrupt balancing test
* 54 testcases were cleaned up and converted to the new test library
* The syscalls testrun has been speeded up by about a minute which is 5% of
the runtime.
* Several Android fixes have been applied.
* New macro for automatic retry with exponential backoff has been added to the
test library, which should replace all the ad-hoc retry loops we do have all
around the codebase.
* IMA (Integrity Measurement Architecture) tests
- Were rewritten to use new API, updated, and various bugs has been fixed.
- The most important changes:
* Avoid running on tmpfs ima_measurements.sh and ima_violations.sh
* ima_measurements.sh: support new IMA measurement templates, support
testing
most of IMA supported hash algorithms, fix & update iversion check
* tpm.sh: replace ima_measure with evmctl (external)
* ima_policy.sh: improve check of policy writability
* ima_boot_aggregate: fixing event size for modern BIOS
Many thanks to Mimi Zohar for her review and testing of IMA test cases.
+ The usual number of fixes and cleanups all over the place
NETWORK TESTS
=
Changelog for network tests brought to you by Peter Vorel.
* New testcases:
- for IP_BIND_ADDRESS_NO_PORT, DCCP and UDP ipsec and for UDPLITE protocol
- regression test sctp_big_chunk aka CVE-2018-5803
- netstress gained support for IP_BIND_ADDRESS_NO_PORT, SO_ZEROCOPY flags
and UDPLITE protocol
- Use libtirpc for all RPC tests
* Test cases rewritten into new shell API:
- interface and multicast tests cases
- in6_02
* Removed tests cases
- route{4,6}-ifdown
+ Various fixes and doc cleanups
TODO LIST AND ISSUE TRACKER
===
Just recently we started to maintain a LTP TODO list in the form GitHub issues,
so far there is very coarse list of missing, mostly syscall, coverage:
https://github.com/linux-test-project/ltp/labels/missing%20coverage
Which I suppose will expand quite a bit soon.
And also nearly empty list of reproducers:
https://github.com/linux-test-project/ltp/labels/reproducer
Which I'm about to feed with a list of CVE reproducers I have, that
can be converted into regression testcases.
Lastly but not least I would like to ask everyone to contribute to that list.
DOWNLOAD AND LINKS
==
The latest version of the test-suite contains 3000+ tests for the Linux
and can be downloaded at:
https://github.com/linux-test-project/ltp/releases/tag/20180515
The project pages as well as GIT repository are hosted on GitHub:
https://github.com/linux-test-project/ltp
http://linux-test-project.github.io/
If you ever wondered how to write a LTP testcase, don't miss our developer
documentation at:
https://github.com/linux-test-project/ltp/wiki/C-Test-Case-Tutorial
https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines
https://github.com/linux-test-project/ltp/wiki/BuildSystem
Patches, new tests, bugs, comments or questions should go to to our mailing
list at l...@lists.linux.it.
CREDITS
===
Many thanks to the people contributing to t
[LTP] [ANNOUNCE] The Linux Test Project has been released for SEPTEMBER 2017
Good news everyone, the Linux Test Project test suite stable release for *September 2017* has been released. Since the last release 290 patches by 39 authors were merged. Notable changes for this release include: - * New tests for: - copy_file_range() - splice() between pipes and sockets - splice() between two pipes - clone() with CLONE_NEWNET - msync() with mmaped file - getsockopt() with SO_PEERCRED - fanotify() with FAN_CLOEXEC - dynamic_debug tracing - basic gdb test - combination of OFD and POSIX locks on a file * New regression tests for: - fanotify() 96d41019e3ac and 05f0e38724e8 - add_key() 5649645d725c - fcntl() 086e774a57fb - sched/autogroup 18f649ef3441 - memcg https://bugzilla.redhat.com/show_bug.cgi?id=1168185 - vdso permissions b6558c4a2378 and e5b97dde514f - keyctl() 63a0b0509e70 - numa MPOL_PREFERRED policy 0867a57c * Removed tests: - long broken dmapi testsuite * 66 testcases were cleaned up and converted to the new test library * KSM (kernel samepage merging) test should be race-free now * So far if only small part of a test reported skipped status the overall test status was reported as skipped as well. Now if test reports both sucessful and skipped results the overall status is reported as suscessful. * Timer mesurement library code was introduced - all syscalls that use kernel timers are measured in a unified way - we take much more samples and use discarded mean to get rid of outliners - the threshold is now defined in the library and can be easily adjusted - for more information see: https://github.com/linux-test-project/ltp/commit/c459654db64cd29177a382ab178fdd5ad59664e4 * Also effort to convert CVE reproducers into LTP testcases started to gain momentum - It starts by introducing thread synchronization library that auto-tunes a spin-in-a-place delay so that we execute two critical sections of code at the same time, which makes tests based on races much faster and stable in reproducing the bugs. - So far implemented CVE tests include: - CVE-2011-0999 transparent hugepages used wrongly for args/env - CVE-2011-2183 NULL pointer dereference in ksm_do_scan() - CVE-2011-2496 address overflow in mremap() - CVE-2012-0957 uname26 memory leak - CVE-2014-0196 PTY echo race - CVE-2015-0235 buffer overflow in gethostbyname_r() - CVE-2015-7550 race between keyctl_read() and keyctl_revoke() - CVE-2016-4470 uninitialized variable in key_reject_and_link() - CVE-2016-5195 aka dirtyc0w - CVE-2016-7042 crash in /proc/keys show function - CVE-2016-7117 use after free between recvmmsg() and close() - CVE-2016-10044 mark AIO pseudo-fs noexec - CVE-2017-2618 off-by-one in setprocattr - CVE-2017-2671 missing rwlock in ping_unhashed() - CVE-2017-5669 NULL page mapped with shmat() and SHM_RND - CVE-2017-6951 NULL dereference in request_key() - CVE-2017-7472 thread keyring memory leak - CVE-2017-1000364 aka stack_clash Note that a few of the testcases were implemented in the previous release as well but these were not counted as CVE related. * The ltp-pan testrunner that executes LTP testcases marks test start in kernel log, which helps with pairing LTP testcase with a kernel trace. * There is new step-by-step C test tutorial at: https://github.com/linux-test-project/ltp/wiki/C-Test-Case-Tutorial * A few EROFS related testcases now use read-only tmpfs instead of the loopback based device, which saves up to half of a minute of runtime per such testcase. The rest of such testcases will be converted after the release. * The nfsv4 locktest source has been translated from French. * Many fixes in numa testcases. * Many fixes in network testcases. + The usuall amount of fixes all over the codebase What havent't made it to this release: -- * A patchset that adds test library support to allow for filesystem related syscalls, such as fallocate(), to be executed against all supported filesystems. * Fixes and speed ups for the CVE synchronization library and a few CVE reproducers. Downloads and links: The latest version of the test-suite contains 3000+ tests for the Linux and can be downloaded at: https://github.com/linux-test-project/ltp/releases/tag/20170930 The project pages as well as GIT repository are hosted on GitHub: https://github.com/linux-test-project/ltp http://linux-test-project.github.io/ If you ever wondered how to write a LTP testcase, don't miss our developer documentation at: https://github.com/linux-test-project/ltp/wiki/C-Test-Case-Tutorial https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines https://github.com/linux-test-project/ltp/wiki/BuildSystem Patches, new tests, bugs, comments or questions should go to to our mailing list at l...@lists.linux.it. -- Cyril Hrubis chru...@suse.cz
Re: [PATCH v2] mmap.2: MAP_FIXED updated documentation
Hi! > (It does seem unfortunate that the man page cannot help the programmer > actually write correct code here. He or she is forced to read the kernel > implementation, in order to figure out the true alignment rules. I was > hoping we could avoid that.) It would be nice if we had this information exported somehere so that we do not have to rely on per-architecture ifdefs. What about adding MapAligment or something similar to the /proc/meminfo? -- Cyril Hrubis chru...@suse.cz
Re: [LTP] [lkp-robot] [x86/topology] 379a4bb988: dmesg.WARNING:at_arch/x86/events/intel/uncore.c:#uncore_change_type_ctx
Hi! > > bin/lkp install job.yaml > > [root@hpe-dl385gen10-02 lkp-tests]# ls > allotdaemon filters include Makefile pkgrepospec > bin distro Gemfile jobs monitors plot rootfs > stats > cluster doc Gemfile.lock lib pack Rakefile sbin > tests > _config.yml etc hosts lkp-exec paramsREADME.md setup > tools > [root@hpe-dl385gen10-02 lkp-tests]# bin/lkp install ../job.yaml > Not a supported system, cannot install packages. > [root@hpe-dl385gen10-02 lkp-tests]# > > Well that's useful. Sorry for the late reply, you may have figured it out already, you can install and run LTP without the lkp framework and that should be pretty straightforward, then you can just run it by: 'cd /opt/ltp && ./runltp -f cpuhotplug' Looking at the test it may need a kernel source unpacked /usr/src/linux as well. Short HOWTO for building LTP from git is here: https://github.com/linux-test-project/ltp/blob/master/doc/mini-howto-building-ltp-from-git.txt And we do have (unofficional) packages for some SUSE distributions: https://build.opensuse.org/package/show/benchmark/ltp -- Cyril Hrubis chru...@suse.cz
Re: [LTP] Towards 4.14 LTS
Hi! > So why didn???t we report these? As mentioned we???ve been tossing out dodgy > test cases to get to a clean baseline. We don???t need or want noise. > > For LTS, I want the system when it detects a failure to enable a quick > bisect involving the affected test bucket. Given the nature of kernel > bugs tho, there is that class of bug which only happens occasionally. >From my experience debugging kernel bugs requires an actuall human interaction and there is only certain level of automation that can be achieved. Don't take me wrong, automatic bisection and other bells and whistles are a nice to have, but at the end of the day you usually need someone to reproduce/look at the problem, possibly check the source code, report a bug, etc. Hence it does not make much sense to have an automated system without dedicated engineers assigned to review the test results. -- Cyril Hrubis chru...@suse.cz
commit f5f99309 (sock: do not set sk_err in sock_dequeue_err_skb) has broken ping
Hi! I've started to wonder why is ping eating 100% CPU shortly after I've upgraded my machine to 4.10 and here is what I found: The ping main_loop() sleeps in poll() on its socket, the poll() usually times out, at least that's what strace suggets which causes ping to sleep for ~1s in the kernel. See ping source at: https://github.com/iputils/iputils/blob/master/ping_common.c#L587 The poll() seems to start returning POLLERR immediatelly after poll() is called on the socket in a case that connection has dropped for a short while. It seems to be easily reproducible with: * Starting ping with some ip address i.e. ping 4.2.2.2 * Letting it ping for a minute or so * Disconnection a WAN cable from your AP * After a minute or so ping ends up bussy looping on poll() that returns with POLLERR immediatelly * After plugging the cable back the problem gets only worse since we now spend 99% of the time bussy looping on the poll() syscall * And my CPU fan starts to scream loudly I've bisected the problem to this commit: commit f5f99309fa7481f59a500f0d08f3379cd6424c1f (HEAD, refs/bisect/bad) Author: Soheil Hassas Yeganeh Date: Thu Nov 3 18:24:27 2016 -0400 sock: do not set sk_err in sock_dequeue_err_skb -- Cyril Hrubis chru...@suse.cz
Re: commit f5f99309 (sock: do not set sk_err in sock_dequeue_err_skb) has broken ping
Hi! > > I've bisected the problem to this commit: > > > > commit f5f99309fa7481f59a500f0d08f3379cd6424c1f (HEAD, refs/bisect/bad) > > Author: Soheil Hassas Yeganeh > > Date: Thu Nov 3 18:24:27 2016 -0400 > > > > sock: do not set sk_err in sock_dequeue_err_skb > > Hi Cyril, > > I'm sorry for the problem, and thank you for the report. > > Two questions: > 1. Could you double check whether you have the following commit in your tree? > > commit 83a1a1a70e87f676fbb6086b26b6ac7f7fdd107d > Author: Soheil Hassas Yeganeh > Date: Wed Nov 30 14:01:08 2016 -0500 > sock: reset sk_err for ICMP packets read from error queue I've started bisecting on v4.11 and see the problem on v4.10 on another machine, the patch should be there in both cases and the bug is easily reproducible. > 2. I've also have sent a fix to iputils on > https://github.com/iputils/iputils/pull/75. Would you be kind to try > that pull request as well? That fixed the problem, you can add: Tested-by: Cyril Hrubis -- Cyril Hrubis chru...@suse.cz
Re: commit f5f99309 (sock: do not set sk_err in sock_dequeue_err_skb) has broken ping
Hi! > Thank you for the confirmation. Could you please try the following > patch to see if it fixes your issue? Does not seem to help, I still got the same bussy loop. -- Cyril Hrubis chru...@suse.cz
Re: commit f5f99309 (sock: do not set sk_err in sock_dequeue_err_skb) has broken ping
Hi! > >> Thank you for the confirmation. Could you please try the following > >> patch to see if it fixes your issue? > > > > Does not seem to help, I still got the same bussy loop. > > Thank you for trying the patch. Unfortunately, I can't reproduce on my > machines here.Would you humor me with another one? Thank you! Still does not help. I've modified the ping binary (on the top of you patch that reads the error queue) to dump the error structure. It seems to repeatedly produce (until I plug the cable back): ee_errno = 113 ee_origin = 2 ee_type = 3 ee_code = 1 ee_info = 0 ee_data = 0 So we get EHOSTUNREACH on SO_EE_ORIGIN_ICMP. -- Cyril Hrubis chru...@suse.cz
Re: commit f5f99309 (sock: do not set sk_err in sock_dequeue_err_skb) has broken ping
Hi! > Thank you very much! I have a wild guess that, when we > have a train of skbs on the error queue starting from a local error, > we will see this issue. > > Ping (without my patch) considers EAGAIN on a normal read as an > indication that there is nothing on the error queue, but that's a > flawed assumption. > > Would you mind trying another shot in the darkness please? Thanks! This patch seems to fix the issue, I've tried several times and poll() just timeouts, haven't seen a single POLLERR in the ping strace. You can add my Tested-by: for this patch as well :-). -- Cyril Hrubis chru...@suse.cz
Re: [LTP] Towards 4.14 LTS
Hi! > For instance 4.13-rc just was added to the mix. > > For test buckets, I???m currently dorking around with some make check targets > for a few interesting packages. You may want to look into xfstests as well, we found a few kernel oopses recently related to backported FS patches for SLES kernels. -- Cyril Hrubis chru...@suse.cz
Re: [PATCH v2] mmap.2: MAP_FIXED updated documentation
Hi! > >> (It does seem unfortunate that the man page cannot help the programmer > >> actually write correct code here. He or she is forced to read the kernel > >> implementation, in order to figure out the true alignment rules. I was > >> hoping we could avoid that.) > > > > It would be nice if we had this information exported somehere so that we > > do not have to rely on per-architecture ifdefs. > > > > What about adding MapAligment or something similar to the /proc/meminfo? > > > > What's the use case you envision for that? I don't see how that would be > better than using SHMLBA, which is available at compiler time. Because > unless someone expects to be able to run an app that was compiled for > Arch X, on Arch Y (surely that's not requirement here?), I don't see how > the run-time check is any better. I guess that some kind of compile time constant in uapi headers will do as well, I'm really open to any solution that would expose this constant as some kind of official API. There is one problem with using SHMLBA, there are more than one libc implementations and at least two of them (bionic and klibc) does not implement the SysV IPC at all. I know that the chances that you are writing a program that requires MAP_FIXED, is compiled against alternative libc, and expected to run on less common architectures are pretty slim. On the other hand I do not see a reason why this cannot be done properly, this is just about exporting one simple constant to userspace after all. > Or maybe you're thinking that since the SHMLBA cannot be put in the man > pages, we could instead provide MapAlignment as sort of a different > way to document the requirement? This is my intention as well. -- Cyril Hrubis chru...@suse.cz
Re: [PATCH 0/2] mm: introduce MAP_FIXED_SAFE
Hi! > > MAP_FIXED_UNIQUE > > MAP_FIXED_ONCE > > MAP_FIXED_FRESH > > Well, I can open a poll for the best name, but none of those you are > proposing sound much better to me. Yeah, naming sucks... Given that MAP_FIXED replaces the previous mapping MAP_FIXED_NOREPLACE would probably be a best fit. -- Cyril Hrubis chru...@suse.cz
Re: [PATCH v2] mmap.2: MAP_FIXED updated documentation
Hi! I know that we are not touching the rest of the existing description for MAP_FIXED however the second sentence in the manual page says that "addr must be a multiple of the page size." Which however is misleading as this is not enough on some architectures. Code in the wild seems to (mis)use SHMLBA for aligment purposes but I'm not sure that we should advise something like that in the manpages. So what about something as: "addr must be suitably aligned, for most architectures multiple of page size is sufficient, however some may impose additional restrictions for page mapping addresses." Which should at least hint the reader that this is architecture specific. -- Cyril Hrubis chru...@suse.cz
Re: [PATCH 0/2] mm: introduce MAP_FIXED_SAFE
Hi! > > If we had a time machine, the right set of flags would be: > > > > - MAP_FIXED: don't treat addr as a hint, fail if addr is not free > > - MAP_REPLACE: replace an existing mapping (or force or clobber) > > Actually, if we had a time machine... would we even provide > MAP_REPLACE functionality? I did a bit of archeology just beacause we can and since there is a git repository of the unix history [1]. The first version of mmap() seems to appear in BSD-4_2-Snapshot there was no MAP_FIXED flag and the addr is expected to be used for the mapping. At least that is what manual seems to say, the kernel code is not written at this point. This seems to correspond to a time when Berkley students were busy rewriting UNIX kernel to take advantage of the VAX's virtual memory. The MAP_FIXED arrived to the manual shortly after, probably someone figured out that passing an address to the call does not make much sense in most of the cases. The first actual implementation that supports MAP_FIXED appeared in the BSD-4_3_Reno-Snapshot and already includes the replace behavior. The original purpose seems to be replacing mappings in the implementation of the execve() call. So the answer would probably be yes but it would probably made sense to keep it as kernel internal flag. And BTW it looks like HPUX got it right before it was changed to follow POSIX. There seems to be HPUX compatibility code in the early BSD codebase that contains both HPUXMAP_FIXED and HPUXMAP_REPLACE. [1] https://github.com/dspinellis/unix-history-repo -- Cyril Hrubis chru...@suse.cz
Re: [PATCH 2/2] mmap.2: MAP_FIXED updated documentation
Hi! > Pretty map everyone agreed MAP_FIXED_SAFE was a bad > name. MAP_FIXED_NOREPLACE (IIRC) was best replacement. For what it's worth I do agree here. -- Cyril Hrubis chru...@suse.cz
Re: [PATCH 2/2] mmap.2: MAP_FIXED updated documentation
Hi! > You selected stupid name for a flag. Everyone and their dog agrees > with that. There's even consensus on better name (and everyone agrees > it is better than .._SAFE). Of course, we could have debate if it is > NOREPLACE or NOREMOVE or ... and that would be bikeshed. This was just > poor naming on your part. Well while everybody agrees that the name is so bad that basically anything else would be better, there does not seem to be consensus on which one to pick. I do understand that this frustrating and fruitless. So what do we do now, roll a dice to choose new name? Or do we ask BFDL[1] to choose the name? [1] https://en.wikipedia.org/wiki/Benevolent_dictator_for_life -- Cyril Hrubis chru...@suse.cz
[ANNOUNCE] The Linux Test Project has been released for JANUARY 2021
Good news everyone,
the Linux Test Project test suite stable release for *January 2021* has been
released.
Since the last release 303 patches by 35 authors were merged.
NOTABLE CHANGES
===
* New tests
- finit_module()
- init_module()
- delete_module()
- semctl09 for SEM_STAT_ANY flag
- msgctl06 for MSG_STAT_ANY flag
- shmctl04 for SHM_STAT_ANY flag
- sendmmsg() failure tests
- recvmmsg() failure tests
- open_by_handle_at()
- name_to_handle_at()
- ptrace11 to check if we can trace PID 1
- select04 failure tests
- select04 to check if rfds/wfds flags are clered on empty/full fd
* New regression tests
- inotify10 added test for fecc4559780d (fsnotify: fix events reported to
watching parent and child)
- fanotify09 for 7372e79c9eb9 (fanotify: fix logic of reporting name info
with watched parent)
- ptrace10 bd14406b78e6 (perf/hw_breakpoint: Modify breakpoint even if the
new attr has disabled set)
* Increased coverage
- fanotify09 added case with FAN_CLOSE_NOWRITE on a sibling non-dir child
* Removed tests
- sync01, sync02 these tests were useless so there was no point in keeping
them
* The metadata extraction patchset was merged. LTP now produces metadata.json
with a metadata for new library testcases and also html test catalogue build
from the extracted metadata.
https://github.com/linux-test-project/ltp/blob/master/docparse/README.md
* Kernel .config parser was rewritten to support proper boolean expressions
* LTP now requires pkg-config > 0.23 (working version 0.24 was released in 2010)
* Error handling in test library (mostly SAFE_MACROS()) was unified
* High level test library overview was written:
https://github.com/linux-test-project/ltp/blob/master/lib/README.md
* IMA/EVM
- fixed ima_tpm.sh for TPM 2.0 and for TPM 1.2 on kernel >= v5.8 (commit
6f1a1d103b48)
- rewrote ima_boot_aggregate.c to new API
* 16 testcases were converted to the new test library
+ The usual amount of fixes and cleanups.
NOTABLE CHANGES IN NETWORK TESTS
brought to you by Petr Vorel
* New tests
- wireguard tests
* Fixes
- fix various false-positive failures in tests using tst_netload
(tests in net.features and net_stress.ipsec_* runtest files)
- if-mtu-change.sh on s390x: fix max packet size for IPv4
* Compatibility fixes:
- ping and traceroute from BusyBox
- MUSL (netstress)
- traceroute6 from iputils
+ various other fixes and cleanup
* Rewrite into new API
- host01
- netstat01.sh
DOWNLOAD AND LINKS
==
The latest version of the test-suite contains 3000+ tests for the Linux
and can be downloaded at:
https://github.com/linux-test-project/ltp/releases/tag/20210121
The project pages as well as GIT repository are hosted on GitHub:
https://github.com/linux-test-project/ltp
http://linux-test-project.github.io/
If you ever wondered how to write a LTP testcase, don't miss our developer
documentation at:
https://github.com/linux-test-project/ltp/wiki/C-Test-Case-Tutorial
https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines
https://github.com/linux-test-project/ltp/wiki/BuildSystem
Patches, new tests, bugs, comments or questions should go to to our mailing
list at l...@lists.linux.it.
CREDITS
===
Many thanks to the people contributing to this release:
git shortlog -s -e -n 20200930..
88 Petr Vorel
52 Cyril Hrubis
25 Martin Doucha
19 Yang Xu
18 Viresh Kumar
10 Xiao Yang
8 Alexey Kodanev
8 Amir Goldstein
8 Feiyu Zhu
8 Kory Maincent
7 Radoslav Kolev
5 Richard Palethorpe
4 Cixi Geng
4 Krzysztof Dynowski
4 Li Wang
4 Po-Hsu Lin
4 Joerg Vehlow
3 Alexander Egorenkov
2 Khem Raj
2 Li Zhijian
2 Pengfei Xu
2 Tree Davies
2 Yang Xu
2 Bogdan Lezhepekov
2 Johannes Nixdorf
1 Alexander Egorenkov
1 Deepak Rawat
1 Filip Bozuta
1 Jan Stancek
1 Peter Bee
1 Petr Cervinka via ltp
1 Punit Agrawal
1 Radoslav Kolev via ltp
1 Xinpeng Liu
1 bhargavdas
And also thanks to patch reviewers:
git log 20200930.. | grep -Ei '(reviewed|acked)-by:' | sed 's/.*by: //' | sort
| uniq -c | sort -n -r
114 Cyril Hrubis
58 Petr Vorel
51 Li Wang
13 Yang Xu
11 Amir Goldstein
10 Alexey Kodanev
9 Jan Stancek
5 Xiao Yang
5 Jan Kara
3 Xiao Yang
5 Richard Palethorpe
3 Martin Doucha
2 Mimi Zohar
1 Kory Maincent
1 Joerg Vehlow
--
Cyril Hrubis
chru...@suse.cz
Re: [LTP] [PATCH] selftests: vdso: hash entry size on alpha, s390x is 8 bytes
Hi! As much as it's worth the changes looks good to me. @Jan: I guess that we can as well fix this in LTP first then we can try to get the kernel version fixed... -- Cyril Hrubis chru...@suse.cz
[LTP] [ANNOUNCE] The Linux Test Project has been released for SEPTEMBER 2020
on GitHub:
https://github.com/linux-test-project/ltp
http://linux-test-project.github.io/
If you ever wondered how to write a LTP testcase, don't miss our developer
documentation at:
https://github.com/linux-test-project/ltp/wiki/C-Test-Case-Tutorial
https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines
https://github.com/linux-test-project/ltp/wiki/BuildSystem
Patches, new tests, bugs, comments or questions should go to to our mailing
list at l...@lists.linux.it.
CREDITS
===
Many thanks to the people contributing to this release:
git shortlog -s -e -n 20200515..
70 Petr Vorel
66 Viresh Kumar
37 Cyril Hrubis
35 Yang Xu
33 Martin Doucha
19 Li Wang
13 Jan Stancek
11 Amir Goldstein
10 Xiao Yang
6 Richard Palethorpe
5 Alexey Kodanev
5 Feiyu Zhu
4 Filip Bozuta
4 Lachlan Sneff
4 Po-Hsu Lin
3 Erico Nunes
3 Petr Vorel
3 Yixin Zhang
2 Eric Biggers
2 Khem Raj
2 Michal Kowalczyk
2 Shwetha Subramanian
2 Vikas Kumar
2 Yuan Gao
1 Bird, Tim
1 Fabrice Fontaine
1 Harish
1 He Zhe
1 Kushal Chand
1 Nirav Parmar
1 Qais Yousef
1 Qian Cai
1 Stanislav Kholmanskikh
1 Tree Davies
1 Vishwajith-K <31800616+vishwajit...@users.noreply.github.com>
1 Vitaly Chikunov
1 aidengao
1 pravin
And also thanks to patch reviewers:
git log 20200515.. | grep -Ei '(reviewed|acked)-by:' | sed 's/.*by: //' | sort
| uniq -c | sort -n -r
136 Cyril Hrubis
72 Petr Vorel
48 Li Wang
22 Jan Stancek
11 Xiao Yang
9 Yang Xu
9 Alexey Kodanev
8 Martin Doucha
7 Mimi Zohar
5 Vijay Kumar B.
4 Viresh Kumar
4 Lakshmi Ramasubramanian
2 Tim Bird
2 Petr Vorel
2 Arnd Bergmann
2 Amir Goldstein
1 Richard Palethorpe
1 Jan Kara
1 Christian Brauner
--
Cyril Hrubis
chru...@suse.cz
Re: [LTP] [x86/entry] 2bbc68f837: ltp.ptrace08.fail
Hi! > do_debug is a bit of a red herring here. ptrace should not be able to > put a breakpoint on a kernel address, period. I would just pick a > fixed address that's in the kernel text range or even just in the > pre-KASLR text range and make sure it gets rejected. Maybe try a few > different addresses for good measure. I've looked at the code and it seems like this would be a bit more complicated since the breakpoint is set by an accident in a race and the call still fails. Which is why the test triggers the breakpoint and causes infinite loop in the kernel... I guess that we could instead read back the address with PTRACE_PEEKUSER, so something as: break_addr = ptrace(PTRACE_PEEKUSER, child_pid, (void *)offsetof(struct user, u_debugreg[0]), NULL); if (break_addr == kernel_addr) tst_res(TFAIL, "ptrace() set break on a kernel address"); -- Cyril Hrubis chru...@suse.cz
Re: [LTP] [PATCH v2] syscall/ptrace08: Simplify the test.
Hi!
> looks good and the test passes on older fixed kernels. Just one
> compatibility issue below.
I've fixed that and also the kernel version when the behavior had
changed and pushed, thanks for the review and testing.
...
> > + if (TST_RET != -1) {
> > + tst_res(TFAIL, "ptrace() breakpoint with kernel addr
> > succeeded");
> > + } else {
> > + if (TST_ERR == EINVAL) {
> > + tst_res(TPASS | TTERRNO,
> > + "ptrace() breakpoint with kernel addr failed");
> > + } else {
> > + tst_res(TFAIL | TTERRNO,
> > + "ptrace() breakpoint on kernel addr should
> > return EINVAL, got");
> > + }
> > + }
> > +
> > + unsigned long addr;
>
> AFAICT, we're not compiling with --std=c99 so older compilers may
> complain about the variable declaration here.
The default std for gcc has been at least gnu90 for ages, which includes
subset of c99 features as well including this one. So unless you
explicitly pass --std=c90 or older it will work just fine.
I've moved the declaration to the top of the function nevertheless.
--
Cyril Hrubis
chru...@suse.cz
Re: [LTP] [PATCH] syscalls/ptrace10: Add new regression test
Hi! > the code looks good, though it might make sense to simply integrate the > check into ptrace08. Just 6 extra lines in the existing test should > achieve the same coverage. I would like to avoid triggering the "your system may be vunerable" messages on fixed kernel, hence the separate test. -- Cyril Hrubis chru...@suse.cz
Re: [Y2038][time namespaces] Question regarding CLOCK_REALTIME support plans in Linux time namespaces
Hi! > I do have a question regarding the Linux time namespaces in respect of > adding support for virtualizing the CLOCK_REALTIME. > > According to patch description [1] and time_namespaces documentation > [2] the CLOCK_REALTIME is not supported (for now?) to avoid complexity > and overhead in the kernel. > > Is there any plan to add support for it in a near future? > > Why I'm asking? > > It looks like this kernel feature (with CLOCK_REALTIME support > available) would be very helpful for testing Y2038 compliance for e.g. > glibc 32 bit ports. > > To be more specific - it would be possible to modify time after time_t > 32 bit overflow (i.e. Y2038 bug) on the process running Y2038 > regression tests on the host system (64 bit one). By using Linux time > namespaces the system time will not be affected in any way. And what's exactly wrong with moving the system time forward for a duration of the test? -- Cyril Hrubis chru...@suse.cz
Re: [PATCH 1/2] syscalls: avoid time() using __cvdso_gettimeofday in use-level's VDSO
Hi! > This is a general problem and not really just for this particular test > case. > > Due to the internal implementation of ktime_get_real_seconds(), which is > a 2038 safe replacement for the former get_seconds() function, this > accumulation issue can be observed. (time(2) via syscall and newer > versions of VDSO use the same mechanism). > > clock_gettime(CLOCK_REALTIME, ); > sec = time(); > assert(sec >= ts.tv_sec); > > That assert can trigger for two reasons: > > 1) Clock was set between the clock_gettime() and time(). > > 2) The clock has advanced far enough that: > > timekeeper.tv_nsec + (clock_now_ns() - last_update_ns) > NSEC_PER_SEC > > #1 is just a property of clock REALTIME. There is nothing we can do >about that. > > #2 is due to the optimized get_seconds()/time() access which avoids to >read the clock. This can happen on bare metal as well, but is far >more likely to be exposed on virt. > > The same problem exists for CLOCK_XXX vs. CLOCK_XXX_COARSE > > clock_gettime(CLOCK_XXX, ); > clock_gettime(CLOCK_XXX_COARSE, ); > assert(tc.tv_sec >= ts.tv_sec); > > The _COARSE variants return their associated timekeeper.tv_sec,tv_nsec > pair without reading the clock. Same as #2 above just extended to clock > MONOTONIC. Good hint, I guess that easiest fix would be to switch to coarse timers for these tests. > There is no way to fix this except giving up on the fast accessors and > make everything take the slow path and read the clock, which might make > a lot of people unhappy. That's understandable and reasonable. Thanks a lot for the confirmation. > For clock REALTIME #1 is anyway an issue, so I think documenting this > proper is the right thing to do. > > Thoughts? I guess that ideally BUGS section for time(2) and clock_gettime(2) should be updated with this explanation. -- Cyril Hrubis chru...@suse.cz
Re: Linux-next-20190823: x86_64/i386: prot_hsymlinks.c:325: Failed to run cmd: useradd hsym
Hi! > Do you see this LTP prot_hsymlinks failure on linux next 20190823 on > x86_64 and i386 devices? > > test output log, > useradd: failure while writing changes to /etc/passwd > useradd: /home/hsym was created, but could not be removed This looks like an unrelated problem, failure to write to /etc/passwd probably means that filesystem is full or some problem happend and how is remounted RO. I do not see the kernel messages from this job anywhere at the job pages, is it stored somewhere? -- Cyril Hrubis chru...@suse.cz
[PATCH] syscall/ptrace08: Simplify the test.
The original test was attempting to crash the kernel by setting a
breakpoint on do_debug kernel function which, when triggered, caused an
infinite loop in the kernel. The problem with this approach is that
kernel internal function names are not stable at all and the name was
changed recently, which made the test fail for no good reason.
So this patch changes the test to read the breakpoint address back
instead, which also means that we can drop the /proc/kallsyms parsing as
well.
Signed-off-by: Cyril Hrubis
CC: Andy Lutomirski
CC: Peter Zijlstra
CC: Thomas Gleixner
CC: Alexandre Chartre
---
testcases/kernel/syscalls/ptrace/ptrace08.c | 120 ++--
1 file changed, 60 insertions(+), 60 deletions(-)
diff --git a/testcases/kernel/syscalls/ptrace/ptrace08.c
b/testcases/kernel/syscalls/ptrace/ptrace08.c
index 591aa0dd2..5587f0bbb 100644
--- a/testcases/kernel/syscalls/ptrace/ptrace08.c
+++ b/testcases/kernel/syscalls/ptrace/ptrace08.c
@@ -5,8 +5,17 @@
*
* CVE-2018-1000199
*
- * Test error handling when ptrace(POKEUSER) modifies debug registers.
- * Even if the call returns error, it may create breakpoint in kernel code.
+ * Test error handling when ptrace(POKEUSER) modified x86 debug registers even
+ * when the call returned error.
+ *
+ * When the bug was present we could create breakpoint in the kernel code,
+ * which shoudn't be possible at all. The original CVE caused a kernel crash by
+ * setting a breakpoint on do_debug kernel function which, when triggered,
+ * caused an infinite loop. However we do not have to crash the kernel in order
+ * to assert if kernel has been fixed or not. All we have to do is to try to
+ * set a breakpoint, on any kernel address, then read it back and check if the
+ * value has been set or not.
+ *
* Kernel crash partially fixed in:
*
* commit f67b15037a7a50c57f72e69a6d59941ad90a0f0f
@@ -26,69 +35,42 @@
#include "tst_safe_stdio.h"
#if defined(__i386__) || defined(__x86_64__)
-#define SYMNAME_SIZE 256
-#define KERNEL_SYM "do_debug"
-static unsigned long break_addr;
static pid_t child_pid;
-static void setup(void)
-{
- int fcount;
- char endl, symname[256];
- FILE *fr = SAFE_FOPEN("/proc/kallsyms", "r");
-
- /* Find address of do_debug() in /proc/kallsyms */
- do {
- fcount = fscanf(fr, "%lx %*c %255s%c", _addr, symname,
- );
-
- if (fcount <= 0 && feof(fr))
- break;
-
- if (fcount < 2) {
- fclose(fr);
- tst_brk(TBROK, "Unexpected data in /proc/kallsyms %d",
- fcount);
- }
-
- if (fcount >= 3 && endl != '\n')
- while (!feof(fr) && fgetc(fr) != '\n');
- } while (!feof(fr) && strcmp(symname, KERNEL_SYM));
-
- SAFE_FCLOSE(fr);
-
- if (strcmp(symname, KERNEL_SYM))
- tst_brk(TBROK, "Cannot find address of kernel symbol \"%s\"",
- KERNEL_SYM);
-
- if (!break_addr)
- tst_brk(TCONF, "Addresses in /proc/kallsyms are hidden");
-
- tst_res(TINFO, "Kernel symbol \"%s\" found at 0x%lx", KERNEL_SYM,
- break_addr);
-}
+#if defined(__x86_64__)
+# define KERN_ADDR_MIN 0x8000
+# define KERN_ADDR_MAX 0x
+# define KERN_ADDR_BITS 64
+#elif defined(__i386__)
+# define KERN_ADDR_MIN 0xc000
+# define KERN_ADDR_MAX 0x
+# define KERN_ADDR_BITS 32
+#endif
-static void debug_trap(void)
+static void setup(void)
{
- /* x86 instruction INT1 */
- asm volatile (".byte 0xf1");
+ /*
+* When running in compat mode we can't pass 64 address to ptrace so we
+* have to skip the test.
+*/
+ if (tst_kernel_bits() != KERN_ADDR_BITS)
+ tst_brk(TCONF, "Cannot pass 64bit kernel address in compat
mode");
}
static void child_main(void)
{
raise(SIGSTOP);
- /* wait for SIGCONT from parent */
- debug_trap();
exit(0);
}
-static void run(void)
+static void ptrace_try_kern_addr(unsigned long kern_addr)
{
int status;
- pid_t child;
- child = child_pid = SAFE_FORK();
+ tst_res(TINFO, "Trying address 0x%lx", kern_addr);
+
+ child_pid = SAFE_FORK();
if (!child_pid)
child_main();
@@ -103,22 +85,41 @@ static void run(void)
(void *)offsetof(struct user, u_debugreg[7]), (void *)1);
/* Return value intentionally ignored here */
- ptrace(PTRACE_POKEUSER, child_pid,
+ TEST(ptrace(PTRACE_POKEUSER, child_pid,
(void *)offsetof(struct user, u_debugreg[0]),
- (void *)break_addr);
+ (void *)
Re: [LTP] [PATCH] syscall/ptrace08: Simplify the test.
Hi! > This is failing on our 4.4 and 4.15 kernels, though they passed with the > previous test and have commit f67b15037a7a applied. > > So, this is dependent on some other behavior/commit that has changed. It > passes > on 5.4, for example. I'll try to investigate further. We are looking at it now in SUSE, looks like the initial fix to the kernel postponed the check to the write to the dr7 that enabled the breakpoint, so when the dr0 wasn't enabled you could write anything there. Also looks like somehow the EINVAL is not propagated into the POKEUSER call to the dr7 as it should have been. -- Cyril Hrubis chru...@suse.cz
[PATCH v2] syscall/ptrace08: Simplify the test.
The original test was attempting to crash the kernel by setting a
breakpoint on do_debug kernel function which, when triggered, caused an
infinite loop in the kernel. The problem with this approach is that
kernel internal function names are not stable at all and the name was
changed recently, which made the test fail for no good reason.
The original kernel fix made it however poissible to set a kernel
address as a breakpoint and instead disabled the breakpoint on userspace
modification. The error checks were deffered to write to the dr7 that
enabled the breakpoint again.
So on newer kernels we do not allow to set the breakpoint to the kernel
addres at all, which means that the POKEUSR to dr0 has to fail with an
address in a kernel range and also we read back the breakpoint address
and check that it wasn't set just to be sure.
On older kernels we check that the POKEUSER to dr7 that enables the
breakpoint fails properly after the dr0 has been set to an address in
the kernel range.
Signed-off-by: Cyril Hrubis
CC: Andy Lutomirski
CC: Peter Zijlstra
CC: Thomas Gleixner
CC: Alexandre Chartre
---
testcases/kernel/syscalls/ptrace/ptrace08.c | 136 +++-
1 file changed, 76 insertions(+), 60 deletions(-)
diff --git a/testcases/kernel/syscalls/ptrace/ptrace08.c
b/testcases/kernel/syscalls/ptrace/ptrace08.c
index 591aa0dd2..1b84ce376 100644
--- a/testcases/kernel/syscalls/ptrace/ptrace08.c
+++ b/testcases/kernel/syscalls/ptrace/ptrace08.c
@@ -5,8 +5,17 @@
*
* CVE-2018-1000199
*
- * Test error handling when ptrace(POKEUSER) modifies debug registers.
- * Even if the call returns error, it may create breakpoint in kernel code.
+ * Test error handling when ptrace(POKEUSER) modified x86 debug registers even
+ * when the call returned error.
+ *
+ * When the bug was present we could create breakpoint in the kernel code,
+ * which shoudn't be possible at all. The original CVE caused a kernel crash by
+ * setting a breakpoint on do_debug kernel function which, when triggered,
+ * caused an infinite loop. However we do not have to crash the kernel in order
+ * to assert if kernel has been fixed or not. All we have to do is to try to
+ * set a breakpoint, on any kernel address, then read it back and check if the
+ * value has been set or not.
+ *
* Kernel crash partially fixed in:
*
* commit f67b15037a7a50c57f72e69a6d59941ad90a0f0f
@@ -26,69 +35,54 @@
#include "tst_safe_stdio.h"
#if defined(__i386__) || defined(__x86_64__)
-#define SYMNAME_SIZE 256
-#define KERNEL_SYM "do_debug"
-static unsigned long break_addr;
static pid_t child_pid;
-static void setup(void)
-{
- int fcount;
- char endl, symname[256];
- FILE *fr = SAFE_FOPEN("/proc/kallsyms", "r");
-
- /* Find address of do_debug() in /proc/kallsyms */
- do {
- fcount = fscanf(fr, "%lx %*c %255s%c", _addr, symname,
- );
-
- if (fcount <= 0 && feof(fr))
- break;
-
- if (fcount < 2) {
- fclose(fr);
- tst_brk(TBROK, "Unexpected data in /proc/kallsyms %d",
- fcount);
- }
-
- if (fcount >= 3 && endl != '\n')
- while (!feof(fr) && fgetc(fr) != '\n');
- } while (!feof(fr) && strcmp(symname, KERNEL_SYM));
-
- SAFE_FCLOSE(fr);
-
- if (strcmp(symname, KERNEL_SYM))
- tst_brk(TBROK, "Cannot find address of kernel symbol \"%s\"",
- KERNEL_SYM);
-
- if (!break_addr)
- tst_brk(TCONF, "Addresses in /proc/kallsyms are hidden");
+#if defined(__x86_64__)
+# define KERN_ADDR_MIN 0x8000
+# define KERN_ADDR_MAX 0x
+# define KERN_ADDR_BITS 64
+#elif defined(__i386__)
+# define KERN_ADDR_MIN 0xc000
+# define KERN_ADDR_MAX 0x
+# define KERN_ADDR_BITS 32
+#endif
- tst_res(TINFO, "Kernel symbol \"%s\" found at 0x%lx", KERNEL_SYM,
- break_addr);
-}
+static int deffered_check;
-static void debug_trap(void)
+static void setup(void)
{
- /* x86 instruction INT1 */
- asm volatile (".byte 0xf1");
+ /*
+* When running in compat mode we can't pass 64 address to ptrace so we
+* have to skip the test.
+*/
+ if (tst_kernel_bits() != KERN_ADDR_BITS)
+ tst_brk(TCONF, "Cannot pass 64bit kernel address in compat
mode");
+
+
+ /*
+* The original fix for the kernel haven't rejected the kernel address
+* right away when breakpoint was modified from userspace it was
+* disabled and the EINVAL was returned when dr7 was written to enable
+* it again.
+*/
+ if (tst_kvercmp(4, 17, 0) < 0)
+ deffered_check =
Re: [LTP] [PATCH v2] syscall/ptrace08: Simplify the test.
Hi! > + /* > + * The original fix for the kernel haven't rejected the kernel address > + * right away when breakpoint was modified from userspace it was > + * disabled and the EINVAL was returned when dr7 was written to enable > + * it again. > + */ > + if (tst_kvercmp(4, 17, 0) < 0) > + deffered_check = 1; And this should be 4.19 as the modify_user_hw_breakpoint_check() was made uncoditional (by accident) in: commit bd14406b78e6daa1ea3c1673bda1ffc9efdeead0 Author: Jiri Olsa Date: Mon Aug 27 11:12:25 2018 +0200 perf/hw_breakpoint: Modify breakpoint even if the new attr has disabled set -- Cyril Hrubis chru...@suse.cz
[PATCH] syscalls/ptrace10: Add new regression test
New regression test for a kernel commit:
commit bd14406b78e6daa1ea3c1673bda1ffc9efdeead0
Author: Jiri Olsa
Date: Mon Aug 27 11:12:25 2018 +0200
perf/hw_breakpoint: Modify breakpoint even if the new attr has disabled set
Signed-off-by: Cyril Hrubis
CC: Andy Lutomirski
CC: Peter Zijlstra
CC: Thomas Gleixner
CC: Alexandre Chartre
---
This is a follow up for the ptrace08 fixes.
runtest/syscalls| 1 +
testcases/kernel/syscalls/ptrace/.gitignore | 1 +
testcases/kernel/syscalls/ptrace/ptrace10.c | 86 +
3 files changed, 88 insertions(+)
create mode 100644 testcases/kernel/syscalls/ptrace/ptrace10.c
diff --git a/runtest/syscalls b/runtest/syscalls
index 398145f65..163471bcd 100644
--- a/runtest/syscalls
+++ b/runtest/syscalls
@@ -993,6 +993,7 @@ ptrace05 ptrace05
ptrace07 ptrace07
ptrace08 ptrace08
ptrace09 ptrace09
+ptrace10 ptrace10
pwrite01 pwrite01
pwrite02 pwrite02
diff --git a/testcases/kernel/syscalls/ptrace/.gitignore
b/testcases/kernel/syscalls/ptrace/.gitignore
index 7639e1a9f..7ee3b3c47 100644
--- a/testcases/kernel/syscalls/ptrace/.gitignore
+++ b/testcases/kernel/syscalls/ptrace/.gitignore
@@ -5,3 +5,4 @@
/ptrace07
/ptrace08
/ptrace09
+/ptrace10
diff --git a/testcases/kernel/syscalls/ptrace/ptrace10.c
b/testcases/kernel/syscalls/ptrace/ptrace10.c
new file mode 100644
index 0..b5d6b9f8f
--- /dev/null
+++ b/testcases/kernel/syscalls/ptrace/ptrace10.c
@@ -0,0 +1,86 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2020 Cyril Hrubis
+ *
+ * After fix for CVE-2018-1000199 (see ptrace08.c) subsequent calls to POKEUSER
+ * for x86 debug registers were ignored silently.
+ *
+ * This is a regression test for commit:
+ *
+ * commit bd14406b78e6daa1ea3c1673bda1ffc9efdeead0
+ * Author: Jiri Olsa
+ * Date: Mon Aug 27 11:12:25 2018 +0200
+ *
+ * perf/hw_breakpoint: Modify breakpoint even if the new attr has disabled
set
+ */
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include "tst_test.h"
+
+#if defined(__i386__) || defined(__x86_64__)
+
+static pid_t child_pid;
+
+static void child_main(void)
+{
+ raise(SIGSTOP);
+ exit(0);
+}
+
+static void run(void)
+{
+ int status;
+ unsigned long addr;
+
+ child_pid = SAFE_FORK();
+
+ if (!child_pid)
+ child_main();
+
+ if (SAFE_WAITPID(child_pid, , WUNTRACED) != child_pid)
+ tst_brk(TBROK, "Received event from unexpected PID");
+
+ SAFE_PTRACE(PTRACE_ATTACH, child_pid, NULL, NULL);
+ SAFE_PTRACE(PTRACE_POKEUSER, child_pid,
+ (void *)offsetof(struct user, u_debugreg[0]), (void *)1);
+ SAFE_PTRACE(PTRACE_POKEUSER, child_pid,
+ (void *)offsetof(struct user, u_debugreg[0]), (void *)2);
+
+ addr = ptrace(PTRACE_PEEKUSER, child_pid,
+ (void*)offsetof(struct user, u_debugreg[0]), NULL);
+
+ if (addr == 2)
+ tst_res(TPASS, "The rd0 was set on second PTRACE_POKEUSR");
+ else
+ tst_res(TFAIL, "The rd0 wasn't set on second PTRACE_POKEUSER");
+
+ SAFE_PTRACE(PTRACE_DETACH, child_pid, NULL, NULL);
+ SAFE_KILL(child_pid, SIGCONT);
+ child_pid = 0;
+ tst_reap_children();
+}
+
+static void cleanup(void)
+{
+ /* Main process terminated by tst_brk() with child still paused */
+ if (child_pid)
+ SAFE_KILL(child_pid, SIGKILL);
+}
+
+static struct tst_test test = {
+ .test_all = run,
+ .cleanup = cleanup,
+ .forks_child = 1,
+ .tags = (const struct tst_tag[]) {
+ {"linux-git", "bd14406b78e6"},
+ {}
+ }
+};
+#else
+TST_TEST_TCONF("This test is only supported on x86 systems");
+#endif
--
2.26.2
Re: [LTP] [x86/entry] 2bbc68f837: ltp.ptrace08.fail
Hi! > > do_debug is a bit of a red herring here. ptrace should not be able to > > put a breakpoint on a kernel address, period. I would just pick a > > fixed address that's in the kernel text range or even just in the > > pre-KASLR text range and make sure it gets rejected. Maybe try a few > > different addresses for good measure. > > I've looked at the code and it seems like this would be a bit more > complicated since the breakpoint is set by an accident in a race and the > call still fails. Which is why the test triggers the breakpoint and > causes infinite loop in the kernel... > > I guess that we could instead read back the address with > PTRACE_PEEKUSER, so something as: > > > break_addr = ptrace(PTRACE_PEEKUSER, child_pid, > (void *)offsetof(struct user, u_debugreg[0]), > NULL); > > if (break_addr == kernel_addr) > tst_res(TFAIL, "ptrace() set break on a kernel address"); So this works actually nicely, even better than the original code. Any hints on how to select a fixed address in the kernel range as you pointed out in one of the previous emails? I guess that this would end up as a per-architecture mess of ifdefs if we wanted to hardcode it. -- Cyril Hrubis chru...@suse.cz
Re: [LTP] [PATCH] syscalls/ptrace10: Add new regression test
Hi! > > I would like to avoid triggering the "your system may be vunerable" > > messages on fixed kernel, hence the separate test. > > Good point, go ahead with a separate test then. Thanks for the review, pushed. -- Cyril Hrubis chru...@suse.cz
Re: [Y2038][time namespaces] Question regarding CLOCK_REALTIME support plans in Linux time namespaces
Hi! > Virtualization is the right answer to the testing problem and if people > really insist on running their broken legacy apps past 2038, then stick > them into a VM and charge boatloads of money for that service. Let me just emphasise this with a short story. Before I release LTP I do a lot of pre-release testruns to make sure that all tests works well on a different distributions and kernel versions. Before I wrote a script that automated this[1] i.e. runs all the tests in qemu and filters out the interesting results it took me a few days of manual labor to finish the task. Now I just schedulle the jobs and after a day or two I get the results. Even if the tested kernel crashes, which happens a lot, the machine is just restarted automatically and the testrun carries on with a next test. All in all the work that has been put into the solution wasn't that big to begin with it took me a week to write a first prototype from a scratch. [1] https://github.com/metan-ucw/runltp-ng -- Cyril Hrubis chru...@suse.cz
[LTP] [ANNOUNCE] The Linux Test Project has been released for JANUARY 2018
Good news everyone, the Linux Test Project test suite stable release for *January 2018* has been released. Since the last release 278 patches by 35 authors were merged. Notable changes for this release include: - * New tests for: - unshare(1) command - ioctl07 test for RNDGETENTCNT ioctl() - new network MACsec testcases - new network IPsec SCTP and DCCP testcases * New regression tests for: - CVE-2017-5754 aka meltdown - CVE-2017-12193 (test add_key04) - CVE-2017-15299 and CVE-2017-15951 (test request_key03) - CVE-2017-7308 (test setsockopt02) - CVE-2016-9604 (test keyctl08) - CVE-2017-15537 (test ptrace07) - CVE-2017-12192 (test keyctl07) - add_key03 regression test for kernel commit 237bbd29f7a0 - keyctl06 regression test for kernel commit e645016abc80 * Fixed tests: - brk01 (test rewritten from scratch) - sigwaitinfo01 (fixed and enabled) - openposix aio testcases (uninitialized aiocb) + many smaller fixes * Removed tests: - invalid openposix pthread_barrier_wait_6-1 test - tcp_cmds tests for rwho, echo, finger, and rdist. * The test library gained support to run a particular test against different filesystems including FUSE filesystems such as NTFS or exFAT. The mkfs and kernel/FUSE support for a particular filesystem must be in-place otherwise the tests will skip it automatically. Some of the filesystem specific syscall tests such as fallocate() are executed this way now. We also have a new test that fills up filesystem using several threads and expects the syscalls to fail gracefully. * The fuzzy synchronization library that is used to trigger races mostly in CVE testcases was rewritten to use one thread instead of starting a thread on each iteration, which is not only faster but also more stable since we introduce less random jitter to the timing measurements this way. * Various fixes and enhancements for the network testcases. * Support for NUMA API older than v2 was dropped from the testcases. * The configure script now correctly detects devel libraries on -m32 build. * Another large scale cleanup using coccinelle was done on the code base. We transformed patterns such as: if (scall(...) < 0) tst_brkm(TBROK, ...); into: SAFE_SCALL(); Which will produce unified and more verbose error reporting in case that the call to scall() will fail. * The runltp script now lists test skipped by the skipfile parameter as skipped in the testrun results, these were missing from it previously. * 24 testcases were cleaned up and converted to the new test library + The usual amount of fixes all over the code base Downloads and links: The latest version of the test-suite contains 3000+ tests for the Linux and can be downloaded at: https://github.com/linux-test-project/ltp/releases/tag/20180118 The project pages as well as GIT repository are hosted on GitHub: https://github.com/linux-test-project/ltp http://linux-test-project.github.io/ If you ever wondered how to write a LTP testcase, don't miss our developer documentation at: https://github.com/linux-test-project/ltp/wiki/C-Test-Case-Tutorial https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines https://github.com/linux-test-project/ltp/wiki/BuildSystem Patches, new tests, bugs, comments or questions should go to to our mailing list at l...@lists.linux.it. -- Cyril Hrubis chru...@suse.cz
Re: [LTP] [lkp-robot] [sched/deadline] e0367b1267: WARNING:at_kernel/sched/sched.h:#assert_clock_updated
Hi! > Hummm, wondering how LTP sched tests could trigger this, since a quick > grep into ltp didn't show DEADLINE usage. See kernel/syscalls/sched_setattr/sched_setattr01.c -- Cyril Hrubis chru...@suse.cz
Re: [LTP] [lkp-robot] [sched/deadline] e0367b1267: WARNING:at_kernel/sched/sched.h:#assert_clock_updated
Hi! > > > Hummm, wondering how LTP sched tests could trigger this, since a quick > > > grep into ltp didn't show DEADLINE usage. > > > > See kernel/syscalls/sched_setattr/sched_setattr01.c > > Right, saw that. I was still thinking though why the report seemed to > point to sched, and not syscalls, tests. These collections of tests have sometimes non empty intersections. In this case the particular test is both part of the sched and syscall testruns (grep sched_setattr01 runtest/*). So I suppose that the test fails both in the sched and syscall run but the sched one was simply picked first here. > Thanks for pointing this out. Glad to help :-). -- Cyril Hrubis chru...@suse.cz
Re: [LTP] [mm] 8c7829b04c: ltp.overcommit_memory01.fail
Hi!
> commit: 8c7829b04c523cdc732cb77f59f03320e09f3386 ("mm: fix false-positive
> OVERCOMMIT_GUESS failures")
This has been reported on the LTP ML already, LTP tests needs to be
adjusted, see:
http://lists.linux.it/pipermail/ltp/2019-May/011962.html
--
Cyril Hrubis
chru...@suse.cz
Re: [LTP] 56cbb429d9: ltp.fs_fill.fail
Hi! > tst_test.c:1161: INFO: Testing on vfat > tst_mkfs.c:90: INFO: Formatting /dev/loop0 with vfat opts='' extra opts='' > mkfs.vfat: unable to open /dev/loop0: Device or resource busy > tst_mkfs.c:101: BROK: mkfs.vfat:1: tst_test.c failed with 741 This looks like mkfs.vfat got EBUSY after the loop device was succesfully umounted. We do run the test in a loop for several filesystems and the loop generally does: next: mkfs mount test umount goto next; -- Cyril Hrubis chru...@suse.cz
[ANNOUNCE] The Linux Test Project has been released for SEPTEMBER 2019
Good news everyone,
the Linux Test Project test suite stable release for *September 2019* has been
released.
Since the last release 267 patches by 41 authors were merged.
NOTABLE CHANGES
===
* New tests
- Basic eBPF map and program load tests
- Netlink uevent socket tests
- Basic ioprio() tests
- Basic sendmmsg() test
- mbind() tests
- Memory protection (pkey) tests
- prctl() with PR_CAP_AMBIENT
- prctl() with PR_{GET,SET}_NO_NEW_PRIVS
- prctl() with PR_{SET,GET}_NAME
- prctl() with PR_{SET,GET}_SECCOMP
- futext_cmp_requeue() tests
- fanotify FAN_REPORT_FID tests
- fanotify overlayfs tests
- pidfd_send_signal() tests
- IMA overlay tests
- acct() tests
- preadv2() RWF_NOWAIT tests
* New tests variants for
- clock_getres()
* New regression tests
- bpf_prog02: regression test for 3612af783cf52c (bpf: fix sanitation rewrite
in case of non-pointers)
- timer_create() aka CVE-2017-18344
- alsa timer race aka CVE-2017-1000380
- accept02 aka CVE-2017-8890
- setsockopt04 aka CVE-2016-9793
- move_pages12: regression tests for:
e66f17ff7177 (mm/hugetlb: take page table lock in follow_huge_pmd())
c9d398fa2378 (mm, hugetlb: use pte_present() instead of pmd_present() in
follow_huge_pmd())
4643d67e8cb0 (hugetlbfs: fix hugetlb page migration/fault race causing
SIGBUS)
- crypto_user02: regression test for 21d4120ec6f5 ("crypto: user - prevent
operating on larval algorithms")
- sysctl02: regression test for 32a5ad9c2285 ("sysctl: handle overflow for
file-max")
* Increased coverage
- copy_file_range()
- tests for cross device copy
- tests for invalid files (swapfile, pipe, ...)
- intel PT tests
- sync_file_range()
- added partial range sync
* Added syscall tables for MIPS
* Guarded buffers, which are pages followed by a PROT_NONE and preceeded by a
canary were introduced and 8 tests were changed to make use of the new
feature. Also newly implemented tests are using the same buffers as well.
* Additional 36 tests were converted to the new test library
NOTABLE CHANGES IN NETWORK TESTS
brought to you by Peter Vorel
* New net features:
- introduce TST_GET_UNUSED_PORT() macro into C API
* Rewritten to new API:
- tst_get_unused_port
- bind01
- setdomainname
- sethostname
+ The usuall amount of fixes and cleanups.
DOWNLOAD AND LINKS
==
The latest version of the test-suite contains 3000+ tests for the Linux
and can be downloaded at:
https://github.com/linux-test-project/ltp/releases/tag/20190930
The project pages as well as GIT repository are hosted on GitHub:
https://github.com/linux-test-project/ltp
http://linux-test-project.github.io/
If you ever wondered how to write a LTP testcase, don't miss our developer
documentation at:
https://github.com/linux-test-project/ltp/wiki/C-Test-Case-Tutorial
https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines
https://github.com/linux-test-project/ltp/wiki/BuildSystem
Patches, new tests, bugs, comments or questions should go to to our mailing
list at l...@lists.linux.it.
CREDITS
===
Many thanks to the people contributing to this release:
git shortlog -s -e -n 20190517..
68 Petr Vorel
45 Cyril Hrubis
27 Yang Xu
19 Christian Amann
17 Jan Stancek
9 Richard Palethorpe
8 Li Wang
7 Murphy Zhou
6 Jinhui huang
5 Amir Goldstein
5 Po-Hsu Lin
5 Sandeep Patil
5 Yixin Zhang
4 Joerg Vehlow
4 Xiao Yang
3 He Zhe
3 Matthew Bobrowski
2 Caspar Zhang
2 Eric Biggers
2 Steve Muckle
1 Alexey Kodanev
1 Ammy Yi
1 Hongzhi.Song
1 Khem Raj
1 Linus Walleij
1 Martin Doucha
1 Mathias Fiedler
1 Michael Moese
1 Murphy Zhou
1 Petr Cervinka
1 Ping Fang
1 Piotr Gawel
1 Steven Price
1 Sultan Alsawaf
1 Sumit Garg
1 Thadeu Lima de Souza Cascardo
1 Tobias Jordan
1 Wu,Haiqiang
1 Yann Sionneau
1 Yi Zhao
1 Yu,Siliang
And also thanks to patch reviewers:
git log 20190517.. | grep -Ei '(reviewed|acked)-by:' | sed 's/.*by: //' | sort
| uniq -c | sort -n -r
94 Cyril Hrubis
42 Li Wang
16 Amir Goldstein
13 Jan Stancek
11 Richard Palethorpe
7 Petr Vorel
6 Xiao Yang
6 Clemens Famulla-Conrad
5 Yang Xu
4 Mimi Zohar
4 Alexey Kodanev
3 Sumit Garg
2 Murphy Zhou
2 Ignaz Forster
1 Sandeep Patil
1 Richard Palethorpe
1 Enji Cooper
1 Christian Amann
--
Cyril Hrubis
chru...@suse.cz
Re: [LTP] sched_football: Validity of testcase
Hi! > I was looking thoroughly at the realtime testcase sched_football, > because it sometimes fails and like to know your opinion on the test case. > > A short introduction to how the test works: > It creates nThreads threads called offense and n threads called defense > (all fifo scheduled). The offense threads run at a lower priority than > the defense threads and the main thread has the highest priority. After > all threads are created (validated using an atomic counter). The test > verifies, that the offense threads are never executed by incrementing > a counter in the offense threads, that is zeroed in the main thread. > During the test the main threads sleeps to regularly. > > While the test is totally fine on a single core system, you can > immediately see, that it will fail on a system with nCores > nThreads, > because there will be a core were only an offense thread an no defense > thread is scheduled. In its default setup nThreads = nCores. This should > theoretically work, because there is a defense thread for every score with > a higher priority than the offense threads and they should be scheduled > onto every core. This is indeed what happens. The problem seems to be > the initialization phase. When the threads are created, they are not > evenly scheduled. After pthread_create was called, the threads are scheduled > > too cores where nothing is running. If there is no idle core anymore, they > are > scheduled to any core (the first?, the one with the shortest wait queue?). > At > some point after all threads are created, they are rescheduled to every > core. > It looks like the test fails, when there is initially a core with only an > offense thread scheduled onto it. In perf sched traces I saw, that a defense > thread was migrated to this core, but still the offense thread was executed > for > a short time, until the offense thread runs. From this point onwards only > defense threads are running. > > I tested adding a sleep to the main function, after all threads are created, > to give the system some time for rescheduling. A sleep of around 50ms works > quite well and supports my theory about the migration time being the > problem. > > Now I am not sure if the test case is even valid or if the scheduler is not > working as it is supposed to. Looking at the commits of sched_football it > looks like it was running stable at least at some point, at least it es > reported to have run 15k iterations in e6432e45. > What do you think about the test case? Is it even valid? As far as I can tell the test is designed to check that the realtime threads are distributed evenly on a system. I'm not scheduller expert and as far as I know there are push and pull mechanims to distribute the realtime tasks between per-cpu run-queues so that maximal amount of realtime threads can run at a given time. I guess that there may be a short window until one of the cores pulls the defensive thread to it's run-queue which gives the offensive change to run for a while because the main thread could have been running on that core previously, but I guess that is something that should be answered by scheduller developers, hence CCing them. > Should the cpu affinity be set fixed? Maybe we should pin and isolate the main thread that does the referee to single core and let the test run on the rest of the cores, which would rule out the possibility of the main thread interfering with the rest of the threads. > A note about my testing methodology: > After I realized, that the execution often failed due to the offense thread > running after referee set the_ball to 0, I replaced the loop with just > usleep(1), for faster iteration. > I tested on ubuntu 19.04 with linux 5.0.0-27 running in vmware and > a custom yocto distribution running linux 4.19.59 (with and without rt > patches) -- Cyril Hrubis chru...@suse.cz
Re: [LTP] [fs] ce436509a8: ltp.openat203.fail
Hi!
> > > commit: ce436509a8e109330c56bb4d8ec87d258788f5f4 ("[PATCH v4 2/3] fs:
> > > openat2: Extend open_how to allow userspace-selected fds")
> > > url:
> > > https://github.com/0day-ci/linux/commits/Josh-Triplett/Support-userspace-selected-fds/20200414-102939
> > > base:
> > > https://git.kernel.org/cgit/linux/kernel/git/shuah/linux-kselftest.git
> > > next
> >
> > This commit adds fd parameter to the how structure where LTP test was
> > previously passing garbage, which obviously causes the difference in
> > errno.
> >
> > This could be safely ignored for now, if the patch gets merged the test
> > needs to be updated.
>
> It wouldn't be a bad idea to switch the test to figure out the ksize of
> the struct, so that you only add bad padding after that. But then again,
> this would be a bit ugly -- having CHECK_FIELDS would make this simpler.
Any pointers how can be the size figured out without relying on the
E2BIG we are testing for? Does the kernel export it somewhere?
--
Cyril Hrubis
chru...@suse.cz
EPERM failures for repeated runs
Hi! Lately we started to write BPF testcases for LTP and after writing a first few tests we found out that running more than a few in a row causes them to fail with EPERM. The culprit is deferred cleanup of the bpf maps that are locked in the memory, see: http://lists.linux.it/pipermail/ltp/2019-August/013349.html We worked around that by bumping the limit for the tests in: https://github.com/linux-test-project/ltp/commit/85c4e886b357f7844f6ab8ec5719168c38703a76 But it looks like this value will not scale, especially for architectures that have larger than 4k pages, running four BPF tests in a row still fails on ppc64le even with the increased limit. Perhaps I'm naive but can't we check, in the kernel, if there is deferred cleanup in progress if we fail to lock memory for a map and retry once it's done? Or is this intended behavior and should we retry on EPERM in userspace? -- Cyril Hrubis chru...@suse.cz
Re: [LTP] [blk] 6e6fcbc27e: ltp.fs_fill.fail
Hi! > > tst_test.c:1308: INFO: Testing on vfat > > tst_mkfs.c:90: INFO: Formatting /dev/loop0 with vfat opts='' extra opts='' > > tst_test.c:1247: INFO: Timeout per run is 0h 05m 00s > > fs_fill.c:103: INFO: Running 10 writer threads > > Test timeouted, sending SIGKILL! > > Test timeouted, sending SIGKILL! > > Test timeouted, sending SIGKILL! > > Test timeouted, sending SIGKILL! > > Test timeouted, sending SIGKILL! > > Test timeouted, sending SIGKILL! > The only "error" I found is that fs_fill test timeouted on vfat. That might > indicate some problems. Note, for slow machines, there is LTP_TIMEOUT_MUL > https://github.com/linux-test-project/ltp/wiki/User-Guidelines#1-library-environment-variables Actually you had removed the most important part, it not only timeouted but the process ended up stuck in kernel and could not be killed. So this looks like deadlock somewhere in filesystem code. -- Cyril Hrubis chru...@suse.cz
Re: strace of io_uring events?
Hi! > > - Why aren't the io_uring syscalls in the man-page git? (It seems like > > they're in liburing, but that's should document the _library_ not the > > syscalls, yes?) > > I imagine because using the syscall requires specific memory barriers > which we probably don't want most C programs to be fiddling with > directly. Sort of similar to how iptables doesn't have a syscall-style > man page. Call me old fashioned but I would vote for having all syscalls documented in man pages. At least for me it makes my life much easier as I do not have to fish for documentation or read library source code when debugging. Think of all the poor kernel QA folks that will cry in despair when you decided not to submit manual pages. There is plenty of stuff documented there that most C programmers shouldn't touch, I do not consider this to be a valid excuse. -- Cyril Hrubis chru...@suse.cz
