The branch, v4-7-test has been updated
via d7a0721 s4: torture: kernel oplocks. Add
smb2.kernel-oplocks.kernel_oplocks8
via 6e276ca s3: smbd: kernel oplocks. Replace retry_open() with
setup_kernel_oplock_poll_open().
via 510b11d python: use communicate to fix Popen deadlock
via f1fe68f blackbox tests: method to check specific exit codes
via e45ab18 tevent: version 0.9.34
via c0000bd lib: tevent: Remove select backend.
from 5ec68b2 tevent: Fix a race condition
https://git.samba.org/?p=samba.git;a=shortlog;h=v4-7-test
- Log -----------------------------------------------------------------
commit d7a07213e478e1c30d153c7061eb8ae72dcb75c9
Author: Jeremy Allison <j...@samba.org>
Date: Thu Nov 9 09:59:23 2017 -0800
s4: torture: kernel oplocks. Add smb2.kernel-oplocks.kernel_oplocks8
Test if the server blocks whilst waiting on a kernel lease held by
a non-smbd process.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13121
Signed-off-by: Jeremy Allison <j...@samba.org>
Reviewed-by: Ralph Boehme <s...@samba.org>
Autobuild-User(master): Ralph Böhme <s...@samba.org>
Autobuild-Date(master): Sat Nov 11 20:12:26 CET 2017 on sn-devel-144
(cherry picked from commit ad82557e1355107920ae80fd6a0df0f16d1bdb6c)
Autobuild-User(v4-7-test): Karolin Seeger <ksee...@samba.org>
Autobuild-Date(v4-7-test): Tue Nov 14 16:59:15 CET 2017 on sn-devel-144
commit 6e276cab469da33a4499feca62858451bac63494
Author: Jeremy Allison <j...@samba.org>
Date: Thu Nov 9 12:48:15 2017 -0800
s3: smbd: kernel oplocks. Replace retry_open() with
setup_kernel_oplock_poll_open().
If a O_NONBLOCK open fails with EWOULDBLOCK, this code changes smbd to
do a retry open every second, until either the timeout or we get a
successful
open. If we're opening a file that has a kernel lease set by a non-smbd
process, this is the best we can do.
Prior to this, smbd would block on the second open on such a leased file
(not using O_NONBLOCK) which freezes active clients.
Regression test to follow.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13121
Signed-off-by: Jeremy Allison <j...@samba.org>
Reviewed-by: Ralph Boehme <s...@samba.org>
(cherry picked from commit 47c13fc10a2c9709e9511b2ffcf0e1004497887d)
commit 510b11d0b675f91cd3cad4ea422538d197bbf6f1
Author: Joe Guo <j...@catalyst.net.nz>
Date: Fri Sep 15 16:13:26 2017 +1200
python: use communicate to fix Popen deadlock
`Popen.wait()` will deadlock when using stdout=PIPE and/or stderr=PIPE and
the
child process generates large output to a pipe such that it blocks waiting
for
the OS pipe buffer to accept more data. Use communicate() to avoid that.
Signed-off-by: Joe Guo <j...@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Autobuild-User(master): Andrew Bartlett <abart...@samba.org>
Autobuild-Date(master): Thu Oct 19 09:27:16 CEST 2017 on sn-devel-144
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13127
(cherry picked from commit 5dc773a5b00834c7a53130a73a48f49048bd55e8)
commit f1fe68f724022aa89d8a6c1aeb991257e0a047be
Author: Gary Lockyer <g...@catalyst.net.nz>
Date: Wed Aug 16 13:52:25 2017 +1200
blackbox tests: method to check specific exit codes
Signed-off-by: Gary Lockyer <g...@catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz>
Reviewed-by: Garming Sam <garm...@catalyst.net.nz>
(cherry picked from commit 74ebcf6dfc84b6aab6838fa99e12808eb6b913d9)
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13127
commit e45ab18baeb665b5ec5a82adda288c92c731f11e
Author: Stefan Metzmacher <me...@samba.org>
Date: Mon Nov 13 11:05:04 2017 +0100
tevent: version 0.9.34
* Remove unused select backend
* Fix a race condition in tevent_threaded_schedule_immediate()
(bug #13130)
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Autobuild-User(master): Stefan Metzmacher <me...@samba.org>
Autobuild-Date(master): Mon Nov 13 18:02:46 CET 2017 on sn-devel-144
(cherry picked from commit 2e573eead96b2e98dd8a15c9c8e470679e530392)
commit c0000bd6024ee4208cd7c5356f4c3fb68b8476b8
Author: Jeremy Allison <j...@samba.org>
Date: Tue Sep 12 12:08:38 2017 -0700
lib: tevent: Remove select backend.
select() is no longer useful on modern systems.
Signed-off-by: Jeremy Allison <j...@samba.org>
Reviewed-by: Volker Lendecke <v...@samba.org>
Autobuild-User(master): Volker Lendecke <v...@samba.org>
Autobuild-Date(master): Sat Sep 16 08:35:39 CEST 2017 on sn-devel-144
(cherry picked from commit 2a003b1a576dcbbba0d60bae90427776a5c27867)
-----------------------------------------------------------------------
Summary of changes:
.../ABI/{tevent-0.9.31.sigs => tevent-0.9.34.sigs} | 0
lib/tevent/tevent.c | 1 -
lib/tevent/tevent_internal.h | 1 -
lib/tevent/tevent_select.c | 280 ---------------------
lib/tevent/wscript | 4 +-
python/samba/tests/__init__.py | 25 +-
source3/selftest/tests.py | 4 +-
source3/smbd/open.c | 96 ++++---
source3/torture/torture.c | 10 +-
source4/torture/smb2/oplock.c | 236 +++++++++++++++++
10 files changed, 331 insertions(+), 326 deletions(-)
copy lib/tevent/ABI/{tevent-0.9.31.sigs => tevent-0.9.34.sigs} (100%)
delete mode 100644 lib/tevent/tevent_select.c
Changeset truncated at 500 lines:
diff --git a/lib/tevent/ABI/tevent-0.9.31.sigs
b/lib/tevent/ABI/tevent-0.9.34.sigs
similarity index 100%
copy from lib/tevent/ABI/tevent-0.9.31.sigs
copy to lib/tevent/ABI/tevent-0.9.34.sigs
diff --git a/lib/tevent/tevent.c b/lib/tevent/tevent.c
index 5f44b03..a2d2003 100644
--- a/lib/tevent/tevent.c
+++ b/lib/tevent/tevent.c
@@ -128,7 +128,6 @@ static void tevent_backend_init(void)
done = true;
- tevent_select_init();
tevent_poll_init();
tevent_poll_mt_init();
#if defined(HAVE_EPOLL)
diff --git a/lib/tevent/tevent_internal.h b/lib/tevent/tevent_internal.h
index a5f1ebde..47ea39b 100644
--- a/lib/tevent/tevent_internal.h
+++ b/lib/tevent/tevent_internal.h
@@ -377,7 +377,6 @@ int tevent_common_check_signal(struct tevent_context *ev);
void tevent_cleanup_pending_signal_handlers(struct tevent_signal *se);
bool tevent_standard_init(void);
-bool tevent_select_init(void);
bool tevent_poll_init(void);
void tevent_poll_event_add_fd_internal(struct tevent_context *ev,
struct tevent_fd *fde);
diff --git a/lib/tevent/tevent_select.c b/lib/tevent/tevent_select.c
deleted file mode 100644
index 55dd0b6..0000000
--- a/lib/tevent/tevent_select.c
+++ /dev/null
@@ -1,280 +0,0 @@
-/*
- Unix SMB/CIFS implementation.
- main select loop and event handling
- Copyright (C) Andrew Tridgell 2003-2005
- Copyright (C) Stefan Metzmacher 2005-2009
-
- ** NOTE! The following LGPL license applies to the tevent
- ** library. This does NOT imply that all of Samba is released
- ** under the LGPL
-
- This library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 3 of the License, or (at your option) any later version.
-
- This library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
-
- You should have received a copy of the GNU Lesser General Public
- License along with this library; if not, see <http://www.gnu.org/licenses/>.
-*/
-
-#include "replace.h"
-#include "system/filesys.h"
-#include "system/select.h"
-#include "tevent.h"
-#include "tevent_util.h"
-#include "tevent_internal.h"
-
-struct select_event_context {
- /* a pointer back to the generic event_context */
- struct tevent_context *ev;
-
- /* the maximum file descriptor number in fd_events */
- int maxfd;
-};
-
-/*
- create a select_event_context structure.
-*/
-static int select_event_context_init(struct tevent_context *ev)
-{
- struct select_event_context *select_ev;
-
- /*
- * We might be called during tevent_re_initialise()
- * which means we need to free our old additional_data.
- */
- TALLOC_FREE(ev->additional_data);
-
- select_ev = talloc_zero(ev, struct select_event_context);
- if (!select_ev) return -1;
- select_ev->ev = ev;
-
- ev->additional_data = select_ev;
- return 0;
-}
-
-/*
- recalculate the maxfd
-*/
-static void calc_maxfd(struct select_event_context *select_ev)
-{
- struct tevent_fd *fde;
-
- select_ev->maxfd = 0;
- for (fde = select_ev->ev->fd_events; fde; fde = fde->next) {
- if (fde->fd > select_ev->maxfd) {
- select_ev->maxfd = fde->fd;
- }
- }
-}
-
-
-/* to mark the ev->maxfd invalid
- * this means we need to recalculate it
- */
-#define EVENT_INVALID_MAXFD (-1)
-
-/*
- destroy an fd_event
-*/
-static int select_event_fd_destructor(struct tevent_fd *fde)
-{
- struct tevent_context *ev = fde->event_ctx;
- struct select_event_context *select_ev = NULL;
-
- if (ev) {
- select_ev = talloc_get_type_abort(ev->additional_data,
- struct select_event_context);
-
- if (select_ev->maxfd == fde->fd) {
- select_ev->maxfd = EVENT_INVALID_MAXFD;
- }
- }
-
- return tevent_common_fd_destructor(fde);
-}
-
-/*
- add a fd based event
- return NULL on failure (memory allocation error)
-*/
-static struct tevent_fd *select_event_add_fd(struct tevent_context *ev,
TALLOC_CTX *mem_ctx,
- int fd, uint16_t flags,
- tevent_fd_handler_t handler,
- void *private_data,
- const char *handler_name,
- const char *location)
-{
- struct select_event_context *select_ev =
- talloc_get_type_abort(ev->additional_data,
- struct select_event_context);
- struct tevent_fd *fde;
-
- if (fd < 0 || fd >= FD_SETSIZE) {
- errno = EBADF;
- return NULL;
- }
-
- fde = tevent_common_add_fd(ev, mem_ctx, fd, flags,
- handler, private_data,
- handler_name, location);
- if (!fde) return NULL;
-
- if ((select_ev->maxfd != EVENT_INVALID_MAXFD)
- && (fde->fd > select_ev->maxfd)) {
- select_ev->maxfd = fde->fd;
- }
- talloc_set_destructor(fde, select_event_fd_destructor);
-
- return fde;
-}
-
-/*
- event loop handling using select()
-*/
-static int select_event_loop_select(struct select_event_context *select_ev,
struct timeval *tvalp)
-{
- fd_set r_fds, w_fds;
- struct tevent_fd *fde;
- int selrtn;
- int select_errno;
-
- /* we maybe need to recalculate the maxfd */
- if (select_ev->maxfd == EVENT_INVALID_MAXFD) {
- calc_maxfd(select_ev);
- }
-
- FD_ZERO(&r_fds);
- FD_ZERO(&w_fds);
-
- /* setup any fd events */
- for (fde = select_ev->ev->fd_events; fde; fde = fde->next) {
- if (fde->fd < 0 || fde->fd >= FD_SETSIZE) {
- tevent_debug(select_ev->ev, TEVENT_DEBUG_FATAL,
- "ERROR: EBADF fd[%d] >= %d "
- "select_event_loop_once\n",
- fde->fd, FD_SETSIZE);
- errno = EBADF;
- return -1;
- }
-
- if (fde->flags & TEVENT_FD_READ) {
- FD_SET(fde->fd, &r_fds);
- }
- if (fde->flags & TEVENT_FD_WRITE) {
- FD_SET(fde->fd, &w_fds);
- }
- }
-
- if (select_ev->ev->signal_events &&
- tevent_common_check_signal(select_ev->ev)) {
- return 0;
- }
-
- tevent_trace_point_callback(select_ev->ev, TEVENT_TRACE_BEFORE_WAIT);
- selrtn = select(select_ev->maxfd+1, &r_fds, &w_fds, NULL, tvalp);
- select_errno = errno;
- tevent_trace_point_callback(select_ev->ev, TEVENT_TRACE_AFTER_WAIT);
-
- if (selrtn == -1 && select_errno == EINTR &&
- select_ev->ev->signal_events) {
- tevent_common_check_signal(select_ev->ev);
- return 0;
- }
-
- if (selrtn == -1 && select_errno == EBADF) {
- /* the socket is dead! this should never
- happen as the socket should have first been
- made readable and that should have removed
- the event, so this must be a bug. This is a
- fatal error. */
- tevent_debug(select_ev->ev, TEVENT_DEBUG_FATAL,
- "ERROR: EBADF on select_event_loop_once\n");
- errno = select_errno;
- return -1;
- }
-
- if (selrtn == 0 && tvalp) {
- /* we don't care about a possible delay here */
- tevent_common_loop_timer_delay(select_ev->ev);
- return 0;
- }
-
- if (selrtn > 0) {
- /* at least one file descriptor is ready - check
- which ones and call the handler, being careful to allow
- the handler to remove itself when called */
- for (fde = select_ev->ev->fd_events; fde; fde = fde->next) {
- uint16_t flags = 0;
-
- if (FD_ISSET(fde->fd, &r_fds) && (fde->flags &
TEVENT_FD_READ)) {
- flags |= TEVENT_FD_READ;
- }
- if (FD_ISSET(fde->fd, &w_fds) && (fde->flags &
TEVENT_FD_WRITE)) {
- flags |= TEVENT_FD_WRITE;
- }
- if (flags) {
- DLIST_DEMOTE(select_ev->ev->fd_events, fde);
- fde->handler(select_ev->ev, fde, flags,
fde->private_data);
- break;
- }
- }
- }
-
- return 0;
-}
-
-/*
- do a single event loop using the events defined in ev
-*/
-static int select_event_loop_once(struct tevent_context *ev, const char
*location)
-{
- struct select_event_context *select_ev =
- talloc_get_type_abort(ev->additional_data,
- struct select_event_context);
- struct timeval tval;
-
- if (ev->signal_events &&
- tevent_common_check_signal(ev)) {
- return 0;
- }
-
- if (ev->threaded_contexts != NULL) {
- tevent_common_threaded_activate_immediate(ev);
- }
-
- if (ev->immediate_events &&
- tevent_common_loop_immediate(ev)) {
- return 0;
- }
-
- tval = tevent_common_loop_timer_delay(ev);
- if (tevent_timeval_is_zero(&tval)) {
- return 0;
- }
-
- return select_event_loop_select(select_ev, &tval);
-}
-
-static const struct tevent_ops select_event_ops = {
- .context_init = select_event_context_init,
- .add_fd = select_event_add_fd,
- .set_fd_close_fn = tevent_common_fd_set_close_fn,
- .get_fd_flags = tevent_common_fd_get_flags,
- .set_fd_flags = tevent_common_fd_set_flags,
- .add_timer = tevent_common_add_timer_v2,
- .schedule_immediate = tevent_common_schedule_immediate,
- .add_signal = tevent_common_add_signal,
- .loop_once = select_event_loop_once,
- .loop_wait = tevent_common_loop_wait,
-};
-
-_PRIVATE_ bool tevent_select_init(void)
-{
- return tevent_register_backend("select", &select_event_ops);
-}
diff --git a/lib/tevent/wscript b/lib/tevent/wscript
index bc874bb..31f7ee7 100644
--- a/lib/tevent/wscript
+++ b/lib/tevent/wscript
@@ -1,7 +1,7 @@
#!/usr/bin/env python
APPNAME = 'tevent'
-VERSION = '0.9.33'
+VERSION = '0.9.34'
blddir = 'bin'
@@ -77,7 +77,7 @@ def build(bld):
bld.RECURSE('lib/talloc')
SRC = '''tevent.c tevent_debug.c tevent_fd.c tevent_immediate.c
- tevent_queue.c tevent_req.c tevent_select.c
+ tevent_queue.c tevent_req.c
tevent_poll.c tevent_threads.c
tevent_signal.c tevent_standard.c tevent_timed.c tevent_util.c
tevent_wakeup.c'''
diff --git a/python/samba/tests/__init__.py b/python/samba/tests/__init__.py
index 07c68c4..0b0f567 100644
--- a/python/samba/tests/__init__.py
+++ b/python/samba/tests/__init__.py
@@ -311,19 +311,30 @@ class BlackboxTestCase(TestCaseInTempDir):
return line
def check_run(self, line):
+ self.check_exit_code(line, 0)
+
+ def check_exit_code(self, line, expected):
line = self._make_cmdline(line)
- p = subprocess.Popen(line, stdout=subprocess.PIPE,
stderr=subprocess.PIPE, shell=True)
- retcode = p.wait()
- if retcode:
- raise BlackboxProcessError(retcode, line, p.stdout.read(),
p.stderr.read())
+ p = subprocess.Popen(line,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE,
+ shell=True)
+ stdoutdata, stderrdata = p.communicate()
+ retcode = p.returncode
+ if retcode != expected:
+ raise BlackboxProcessError(retcode,
+ line,
+ stdoutdata,
+ stderrdata)
def check_output(self, line):
line = self._make_cmdline(line)
p = subprocess.Popen(line, stdout=subprocess.PIPE,
stderr=subprocess.PIPE, shell=True, close_fds=True)
- retcode = p.wait()
+ stdoutdata, stderrdata = p.communicate()
+ retcode = p.returncode
if retcode:
- raise BlackboxProcessError(retcode, line, p.stdout.read(),
p.stderr.read())
- return p.stdout.read()
+ raise BlackboxProcessError(retcode, line, stdoutdata, stderrdata)
+ return stdoutdata
def connect_samdb(samdb_url, lp=None, session_info=None, credentials=None,
diff --git a/source3/selftest/tests.py b/source3/selftest/tests.py
index 56b94c4..357970d 100755
--- a/source3/selftest/tests.py
+++ b/source3/selftest/tests.py
@@ -133,7 +133,7 @@ local_tests = [
"LOCAL-sid_to_string",
"LOCAL-binary_to_sid",
"LOCAL-DBTRANS",
- "LOCAL-TEVENT-SELECT",
+ "LOCAL-TEVENT-POLL",
"LOCAL-CONVERT-STRING",
"LOCAL-CONV-AUTH-INFO",
"LOCAL-IDMAP-TDB-COMMON",
@@ -497,7 +497,7 @@ for t in tests:
plansmbtorture4testsuite(t, "simpleserver", '//$SERVER/dosmode
-U$USERNAME%$PASSWORD')
elif t == "smb2.kernel-oplocks":
if have_linux_kernel_oplocks:
- plansmbtorture4testsuite(t, "nt4_dc", '//$SERVER/kernel_oplocks
-U$USERNAME%$PASSWORD')
+ plansmbtorture4testsuite(t, "nt4_dc", '//$SERVER/kernel_oplocks
-U$USERNAME%$PASSWORD --option=torture:localdir=$SELFTEST_PREFIX/nt4_dc/share')
elif t == "smb2.notify-inotify":
if have_inotify:
plansmbtorture4testsuite(t, "fileserver", '//$SERVER_IP/tmp
-U$USERNAME%$PASSWORD')
diff --git a/source3/smbd/open.c b/source3/smbd/open.c
index 89a267b..8c52f4b 100644
--- a/source3/smbd/open.c
+++ b/source3/smbd/open.c
@@ -2410,19 +2410,40 @@ static void defer_open_done(struct tevent_req *req)
}
/**
- * Reschedule an open for immediate execution
+ * Actually attempt the kernel oplock polling open.
+ */
+
+static void kernel_oplock_poll_open_timer(struct tevent_context *ev,
+ struct tevent_timer *te,
+ struct timeval current_time,
+ void *private_data)
+{
+ bool ok;
+ struct smb_request *req = (struct smb_request *)private_data;
+
+ ok = schedule_deferred_open_message_smb(req->xconn, req->mid);
+ if (!ok) {
+ exit_server("schedule_deferred_open_message_smb failed");
+ }
+ DBG_DEBUG("kernel_oplock_poll_open_timer fired. Retying open !\n");
+}
+
+/**
+ * Reschedule an open for 1 second from now, if not timed out.
**/
-static void retry_open(struct timeval request_time,
+static void setup_kernel_oplock_poll_open(struct timeval request_time,
struct smb_request *req,
struct file_id id)
{
- struct deferred_open_record *open_rec = NULL;
+
bool ok;
+ struct deferred_open_record *open_rec = NULL;
+ /* Maximum wait time. */
+ struct timeval timeout = timeval_set(OPLOCK_BREAK_TIMEOUT*2, 0);
- DBG_DEBUG("request time [%s] mid [%" PRIu64 "] file_id [%s]\n",
- timeval_string(talloc_tos(), &request_time, false),
- req->mid,
- file_id_string_tos(&id));
+ if (request_timed_out(request_time, timeout)) {
+ return;
+ }
open_rec = deferred_open_record_create(false, false, id);
if (open_rec == NULL) {
@@ -2431,17 +2452,30 @@ static void retry_open(struct timeval request_time,
ok = push_deferred_open_message_smb(req,
request_time,
- timeval_set(0, 0),
+ timeout,
id,
open_rec);
if (!ok) {
exit_server("push_deferred_open_message_smb failed");
}
- ok = schedule_deferred_open_message_smb(req->xconn, req->mid);
- if (!ok) {
- exit_server("schedule_deferred_open_message_smb failed");
+ /*
+ * As this timer event is owned by req, it will
+ * disappear if req it talloc_freed.
+ */
+ open_rec->te = tevent_add_timer(req->sconn->ev_ctx,
+ req,
+ timeval_current_ofs(1, 0),
+ kernel_oplock_poll_open_timer,
+ req);
+ if (open_rec->te == NULL) {
+ exit_server("tevent_add_timer failed");
}
+
+ DBG_DEBUG("poll request time [%s] mid [%" PRIu64 "] file_id [%s]\n",
+ timeval_string(talloc_tos(), &request_time, false),
+ req->mid,
+ file_id_string_tos(&id));
}
/****************************************************************************
@@ -3160,20 +3194,18 @@ static NTSTATUS open_file_ntcreate(connection_struct
*conn,
flags2 &= ~(O_CREAT|O_TRUNC);
}
- if (first_open_attempt && lp_kernel_oplocks(SNUM(conn))) {
+ if (lp_kernel_oplocks(SNUM(conn))) {
/*
* With kernel oplocks the open breaking an oplock
* blocks until the oplock holder has given up the
- * oplock or closed the file. We prevent this by first
+ * oplock or closed the file. We prevent this by always
* trying to open the file with O_NONBLOCK (see "man
- * fcntl" on Linux). For the second try, triggered by
--
Samba Shared Repository
