The branch, v4-19-stable has been updated
via 7eabe40bfe5 VERSION: Disable GIT_SNAPSHOT for the 4.19.9 release.
via 4059e00109b WHATSNEW: Add release notes for Samba 4.19.9.
via 8d4a277d258 BUG 15590 ldb: Release LDB 2.8.2
via 5a15cbb15da ldb:kv_index: help static analysers to not worry (CID
1615192)
via 9b33893987d ldb:kv_index: realloc away old dn list
via 9a617692d29 ldb_kv_index: dn_list load sub transaction can re-use
keys
via 13cfe36b618 s4:lib/messaging: fix interaction between
imessaging_reinit and irpc_destructor
via 335ce71c636 smbd: remove just created sharemode entry in the error
codepaths
via 92d7e6c6339 smbd: consolidate DH reconnect failure code
via 153f9027c96 s3:tests: let test_durable_handle_reconnect.sh run
smb2.durable-v2-regressions.durable_v2_reconnect_bug15624
via 5f3f8835282 s4:torture/smb2: add
smb2.durable-v2-regressions.durable_v2_reconnect_bug15624
via b0fb5a1d39a vfs_error_inject: add 'error_inject:durable_reconnect =
st_ex_nlink'
via 3e6d740bf3e smbd: add option "smbd:debug events" for tevent
handling duration threshold warnings
via 38242b55d93 smbd: move trace_state variable behind tv variable
via 2c8dfff7d9e smbd: add option "smbd lease break:debug hung procs"
via 71568683ca7 smbd: log share_mode_watch_recv() errors as errors
via b5388c25b88 s3/lib: add option "serverid watch:debug script"
via b365e1068e4 s3/lib: add option "serverid watch:debug = yes" to
print kernel stack of hanging process
via 796d026919f s3/lib: add next helper variable in server_id_watch_*
via 1602d847354 smb2_ioctl: fix truncated FSCTL_QUERY_ALLOCATED_RANGES
responses
via 3455944ce0a s4:torture/smb2: test FSCTL_QUERY_ALLOCATED_RANGES
truncation
via a92076d88bf s3:smbd: fix NULL dereference in case of readlink
failure
via 2d944eb04ad s3:smb2_server: return
NT_STATUS_NETWORK_SESSION_EXPIRED for compound requests
via 3db88bd0527 s4:torture/smb2: let smb2.session.expire2* also check
compound requests
via 9d57d32bba0 VERSION: Bump version up to Samba 4.19.9...
from 204b0f2d2f7 VERSION: Disable GIT_SNAPSHOT for the 4.19.8 release.
https://git.samba.org/?p=samba.git;a=shortlog;h=v4-19-stable
- Log -----------------------------------------------------------------
-----------------------------------------------------------------------
Summary of changes:
VERSION | 2 +-
WHATSNEW.txt | 59 +++++++-
lib/ldb/ABI/{ldb-2.8.0.sigs => ldb-2.8.2.sigs} | 0
...pyldb-util-2.1.0.sigs => pyldb-util-2.8.2.sigs} | 0
lib/ldb/ldb_key_value/ldb_kv_index.c | 100 ++++++++------
lib/ldb/wscript | 2 +-
selftest/skip | 1 +
source3/lib/server_id_watch.c | 128 +++++++++++++++++-
source3/modules/vfs_error_inject.c | 76 +++++++++++
.../script/tests/test_durable_handle_reconnect.sh | 18 +++
source3/smbd/durable.c | 150 ++++++++-------------
source3/smbd/open.c | 119 ++++++++++++++--
source3/smbd/smb2_ioctl.c | 4 +-
source3/smbd/smb2_ioctl_filesys.c | 54 +++++---
source3/smbd/smb2_process.c | 72 +++++++++-
source3/smbd/smb2_server.c | 16 ++-
source4/lib/messaging/messaging.c | 9 ++
source4/libcli/smb2/ioctl.c | 3 +-
source4/torture/smb2/durable_v2_open.c | 118 ++++++++++++++++
source4/torture/smb2/ioctl.c | 149 +++++++++++++++++++-
source4/torture/smb2/session.c | 56 ++++++++
source4/torture/smb2/smb2.c | 2 +
22 files changed, 954 insertions(+), 184 deletions(-)
copy lib/ldb/ABI/{ldb-2.8.0.sigs => ldb-2.8.2.sigs} (100%)
copy lib/ldb/ABI/{pyldb-util-2.1.0.sigs => pyldb-util-2.8.2.sigs} (100%)
Changeset truncated at 500 lines:
diff --git a/VERSION b/VERSION
index 89a60174790..5a12a805f5b 100644
--- a/VERSION
+++ b/VERSION
@@ -27,7 +27,7 @@ SAMBA_COPYRIGHT_STRING="Copyright Andrew Tridgell and the
Samba Team 1992-2023"
########################################################
SAMBA_VERSION_MAJOR=4
SAMBA_VERSION_MINOR=19
-SAMBA_VERSION_RELEASE=8
+SAMBA_VERSION_RELEASE=9
########################################################
# If a official release has a serious bug #
diff --git a/WHATSNEW.txt b/WHATSNEW.txt
index 43b8cafa119..7747fe33732 100644
--- a/WHATSNEW.txt
+++ b/WHATSNEW.txt
@@ -1,3 +1,59 @@
+ ==============================
+ Release Notes for Samba 4.19.9
+ October 17, 2024
+ ==============================
+
+
+This is the latest stable release of the Samba 4.19 release series.
+
+
+Changes since 4.19.8
+--------------------
+
+o Douglas Bagnall <douglas.bagn...@catalyst.net.nz>
+ * BUG 15590: libldb: performance issue with indexes (ldb 2.8.2 is already
+ released).
+
+o Ralph Boehme <s...@samba.org>
+ * BUG 15624: DH reconnect error handling can lead to stale sharemode
entries.
+
+o David Disseldorp <dd...@samba.org>
+ * BUG 15699: Incorrect FSCTL_QUERY_ALLOCATED_RANGES response when truncated.
+
+o Stefan Metzmacher <me...@samba.org>
+ * BUG 15280: irpc_destructor may crash during shutdown.
+ * BUG 15624: DH reconnect error handling can lead to stale sharemode
entries.
+ * BUG 15696: Compound SMB2 requests don't return
+ NT_STATUS_NETWORK_SESSION_EXPIRED for all requests, confuses
+ MacOSX clients.
+
+o Shachar Sharon <ssha...@redhat.com>
+ * BUG 15700: Crash when readlinkat fails.
+
+
+#######################################
+Reporting bugs & Development Discussion
+#######################################
+
+Please discuss this release on the samba-technical mailing list or by
+joining the #samba-technical:matrix.org matrix room, or
+#samba-technical IRC channel on irc.libera.chat.
+
+If you do report problems then please try to send high quality
+feedback. If you don't provide vital information to help us track down
+the problem then you will probably be ignored. All bug reports should
+be filed under the Samba 4.1 and newer product in the project's Bugzilla
+database (https://bugzilla.samba.org/).
+
+
+======================================================================
+== Our Code, Our Bugs, Our Responsibility.
+== The Samba Team
+======================================================================
+
+
+Release notes for older releases follow:
+----------------------------------------
==============================
Release Notes for Samba 4.19.8
August 15, 2024
@@ -97,8 +153,7 @@ database (https://bugzilla.samba.org/).
======================================================================
-Release notes for older releases follow:
-----------------------------------------
+----------------------------------------------------------------------
==============================
Release Notes for Samba 4.19.7
June 10, 2024
diff --git a/lib/ldb/ABI/ldb-2.8.0.sigs b/lib/ldb/ABI/ldb-2.8.2.sigs
similarity index 100%
copy from lib/ldb/ABI/ldb-2.8.0.sigs
copy to lib/ldb/ABI/ldb-2.8.2.sigs
diff --git a/lib/ldb/ABI/pyldb-util-2.1.0.sigs
b/lib/ldb/ABI/pyldb-util-2.8.2.sigs
similarity index 100%
copy from lib/ldb/ABI/pyldb-util-2.1.0.sigs
copy to lib/ldb/ABI/pyldb-util-2.8.2.sigs
diff --git a/lib/ldb/ldb_key_value/ldb_kv_index.c
b/lib/ldb/ldb_key_value/ldb_kv_index.c
index bcbd903f6fb..902fb9b69f2 100644
--- a/lib/ldb/ldb_key_value/ldb_kv_index.c
+++ b/lib/ldb/ldb_key_value/ldb_kv_index.c
@@ -446,34 +446,39 @@ static int ldb_kv_dn_list_load(struct ldb_module *module,
* There is an active index sub transaction, and the record was
* found in the primary index transaction cache. A copy of the
* record needs be taken to prevent the original entry being
- * altered, until the index sub transaction is committed.
+ * altered, until the index sub transaction is committed, but we
+ * don't copy the actual values, just the array of struct ldb_val
+ * that points to the values (which are offsets into a GUID array).
+ *
+ * As a reminder, our primary cache is an in-memory tdb that
+ * maps attributes to struct dn_list objects, which point to
+ * the actual index, which is an array of struct ldb_val, the
+ * contents of which are {.data = <binary GUID>, .length =
+ * 16}. The array is sorted by GUID data, and these GUIDs are
+ * used to look up index entries in the main database. There
+ * are more layers of indirection than necessary, but what
+ * makes the index useful is we can use a binary search to
+ * find if the array contains a GUID.
+ *
+ * What we do in a sub-transaction is make a copy of the struct
+ * dn_list and the array of struct ldb_val, but *not* of the
+ * .data that they point to. This copy is put into a new
+ * in-memory tdb which masks the primary cache for the duration
+ * of the sub-transaction.
+ *
+ * In an add operation in a sub-transaction, the new ldb_val
+ * is a child of the sub-transaction dn_list, which will
+ * become the main dn_list if the transaction succeeds.
+ *
+ * These acrobatics do not affect read-only operations.
*/
-
- {
- struct ldb_val *dns = NULL;
- size_t x = 0;
-
- dns = talloc_array(
- list,
- struct ldb_val,
- list2->count);
- if (dns == NULL) {
- return LDB_ERR_OPERATIONS_ERROR;
- }
- for (x = 0; x < list2->count; x++) {
- dns[x].length = list2->dn[x].length;
- dns[x].data = talloc_memdup(
- dns,
- list2->dn[x].data,
- list2->dn[x].length);
- if (dns[x].data == NULL) {
- TALLOC_FREE(dns);
- return LDB_ERR_OPERATIONS_ERROR;
- }
- }
- list->dn = dns;
- list->count = list2->count;
+ list->dn = talloc_memdup(list,
+ list2->dn,
+ talloc_get_size(list2->dn));
+ if (list->dn == NULL) {
+ return LDB_ERR_OPERATIONS_ERROR;
}
+ list->count = list2->count;
return LDB_SUCCESS;
/*
@@ -3852,9 +3857,7 @@ int ldb_kv_reindex(struct ldb_module *module)
* Copy the contents of the nested transaction index cache record to the
* transaction index cache.
*
- * During this 'commit' of the subtransaction to the main transaction
- * (cache), care must be taken to free any existing index at the top
- * level because otherwise we would leak memory.
+ * This is a 'commit' of the subtransaction to the main transaction cache.
*/
static int ldb_kv_sub_transaction_traverse(
struct tdb_context *tdb,
@@ -3883,8 +3886,7 @@ static int ldb_kv_sub_transaction_traverse(
/*
* Do we already have an entry in the primary transaction cache
- * If so free it's dn_list and replace it with the dn_list from
- * the secondary cache
+ * If so replace dn_list with the one from the subtransaction.
*
* The TDB and so the fetched rec contains NO DATA, just a
* pointer to data held in memory.
@@ -3897,21 +3899,41 @@ static int ldb_kv_sub_transaction_traverse(
abort();
}
/*
- * We had this key at the top level. However we made a copy
- * at the sub-transaction level so that we could possibly
- * roll back. We have to free the top level index memory
- * otherwise we would leak
+ * We had this key at the top level, and made a copy
+ * of the dn list for this sub-transaction level that
+ * borrowed the top level GUID data. We can't free the
+ * original dn list just yet.
+ *
+ * In this diagram, ... is the C pointer structure
+ * and --- is the talloc structure (::: is both).
+ *
+ * index_in_top_level ::: dn orig ..............
+ * | | :
+ * | `--GUID array :
+ * | |----- val1 data
+ * ldb_kv `----- val2 data
+ * | :
+ * index_in_subtransaction :: dn copy ..........:
+ * | :
+ * `------------ new val3 data
+ *
+ * So we don't free the index_in_top_level dn list yet,
+ * because we are (probably) borrowing most of its
+ * children. But we can save memory by discarding the
+ * values and keeping it as an almost empty talloc
+ * node.
*/
- if (index_in_top_level->count > 0) {
- TALLOC_FREE(index_in_top_level->dn);
- }
+ talloc_realloc(index_in_top_level,
+ index_in_top_level->dn, struct ldb_val, 1);
index_in_top_level->dn
= talloc_steal(index_in_top_level,
index_in_subtransaction->dn);
index_in_top_level->count = index_in_subtransaction->count;
return 0;
}
-
+ /*
+ * We found no top level index in the cache, so we put one in.
+ */
index_in_top_level = talloc(ldb_kv->idxptr, struct dn_list);
if (index_in_top_level == NULL) {
ldb_kv->idxptr->error = LDB_ERR_OPERATIONS_ERROR;
diff --git a/lib/ldb/wscript b/lib/ldb/wscript
index 604294eeba9..0e53d4f2952 100644
--- a/lib/ldb/wscript
+++ b/lib/ldb/wscript
@@ -2,7 +2,7 @@
APPNAME = 'ldb'
# For Samba 4.19.x !
-VERSION = '2.8.1'
+VERSION = '2.8.2'
import sys, os
diff --git a/selftest/skip b/selftest/skip
index e808367c00d..056c54ea287 100644
--- a/selftest/skip
+++ b/selftest/skip
@@ -149,3 +149,4 @@ bench # don't run benchmarks in our selftest
^samba.tests.reparsepoints.*
^samba.tests.smb2symlink.*
^samba3.blackbox.open-eintr.*
+smb2.durable-v2-regressions # Only used in blackbox tests
diff --git a/source3/lib/server_id_watch.c b/source3/lib/server_id_watch.c
index f0189e0e896..8ddf9c6b1c8 100644
--- a/source3/lib/server_id_watch.c
+++ b/source3/lib/server_id_watch.c
@@ -17,16 +17,18 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
-#include "replace.h"
-#include <tevent.h>
-#include <talloc.h>
+#include "includes.h"
#include "serverid.h"
#include "server_id_watch.h"
+#include "lib/util/server_id.h"
#include "lib/util/tevent_unix.h"
struct server_id_watch_state {
struct tevent_context *ev;
struct server_id pid;
+ struct timeval start;
+ struct timeval warn;
+ bool debug;
};
static void server_id_watch_waited(struct tevent_req *subreq);
@@ -37,6 +39,7 @@ struct tevent_req *server_id_watch_send(TALLOC_CTX *mem_ctx,
{
struct tevent_req *req, *subreq;
struct server_id_watch_state *state;
+ struct timeval next;
req = tevent_req_create(mem_ctx, &state, struct server_id_watch_state);
if (req == NULL) {
@@ -44,14 +47,21 @@ struct tevent_req *server_id_watch_send(TALLOC_CTX *mem_ctx,
}
state->ev = ev;
state->pid = pid;
+ state->start = tevent_timeval_current();
+ state->warn = tevent_timeval_add(&state->start, 10, 0);
+
+ state->debug = lp_parm_bool(GLOBAL_SECTION_SNUM,
+ "serverid watch",
+ "debug",
+ CHECK_DEBUGLVL(DBGLVL_DEBUG));
if (!serverid_exists(&state->pid)) {
tevent_req_done(req);
return tevent_req_post(req, ev);
}
- subreq = tevent_wakeup_send(
- state, ev, tevent_timeval_current_ofs(0, 500000));
+ next = tevent_timeval_add(&state->start, 0, 500000);
+ subreq = tevent_wakeup_send(state, ev, next);
if (tevent_req_nomem(subreq, req)) {
return tevent_req_post(req, ev);
}
@@ -66,6 +76,8 @@ static void server_id_watch_waited(struct tevent_req *subreq)
subreq, struct tevent_req);
struct server_id_watch_state *state = tevent_req_data(
req, struct server_id_watch_state);
+ struct timeval now;
+ struct timeval next;
bool ok;
ok = tevent_wakeup_recv(subreq);
@@ -80,8 +92,110 @@ static void server_id_watch_waited(struct tevent_req
*subreq)
return;
}
- subreq = tevent_wakeup_send(
- state, state->ev, tevent_timeval_current_ofs(0, 500000));
+ now = tevent_timeval_current();
+
+ if (!state->debug) {
+ goto next;
+ }
+
+ if (timeval_compare(&state->warn, &now) == -1) {
+ double duration = timeval_elapsed2(&state->start, &now);
+ const char *cmd = NULL;
+ char proc_path[64] = { 0, };
+ char *kstack = NULL;
+ struct server_id_buf buf;
+ const char *pid = server_id_str_buf(state->pid, &buf);
+ int ret;
+
+ state->warn = tevent_timeval_add(&now, 10, 0);
+
+ cmd = lp_parm_const_string(GLOBAL_SECTION_SNUM,
+ "serverid watch",
+ "debug script",
+ NULL);
+ if (cmd != NULL) {
+ char *cmdstr = NULL;
+ char *output = NULL;
+ int fd;
+
+ /*
+ * Note in a cluster setup pid will be
+ * a NOTE:PID like '1:3978365'
+ *
+ * Without clustering it is just '3978365'
+ */
+ cmdstr = talloc_asprintf(state, "%s %s", cmd, pid);
+ if (cmdstr == NULL) {
+ DBG_ERR("Process %s hanging for %f seconds?\n"
+ "talloc_asprintf failed\n",
+ pid, duration);
+ goto next;
+ }
+
+ become_root();
+ ret = smbrun(cmdstr, &fd, NULL);
+ unbecome_root();
+ if (ret != 0) {
+ DBG_ERR("Process %s hanging for %f seconds?\n"
+ "smbrun('%s') failed\n",
+ pid, duration, cmdstr);
+ TALLOC_FREE(cmdstr);
+ goto next;
+ }
+
+ output = fd_load(fd, NULL, 0, state);
+ close(fd);
+ if (output == NULL) {
+ DBG_ERR("Process %s hanging for %f seconds?\n"
+ "fd_load() of smbrun('%s') failed\n",
+ pid, duration, cmdstr);
+ TALLOC_FREE(cmdstr);
+ goto next;
+ }
+ DBG_ERR("Process %s hanging for %f seconds?\n"
+ "%s returned:\n%s",
+ pid, duration, cmdstr, output);
+ TALLOC_FREE(cmdstr);
+ TALLOC_FREE(output);
+ goto next;
+ }
+
+ if (!procid_is_local(&state->pid) || !sys_have_proc_fds()) {
+ DBG_ERR("Process %s hanging for %f seconds?\n",
+ pid, duration);
+ goto next;
+ }
+
+ ret = snprintf(proc_path,
+ ARRAY_SIZE(proc_path),
+ "/proc/%" PRIu64 "/stack",
+ state->pid.pid);
+ if (ret < 0) {
+ DBG_ERR("Process %s hanging for %f seconds?\n"
+ "snprintf failed\n",
+ pid, duration);
+ goto next;
+ }
+
+ become_root();
+ kstack = file_load(proc_path, NULL, 0, state);
+ unbecome_root();
+ if (kstack == NULL) {
+ DBG_ERR("Process %s hanging for %f seconds?\n"
+ "file_load [%s] failed\n",
+ pid, duration, proc_path);
+ goto next;
+ }
+
+ DBG_ERR("Process %s hanging for %f seconds?\n"
+ "%s:\n%s",
+ pid, duration, proc_path, kstack);
+ TALLOC_FREE(kstack);
+ }
+
+next:
+ next = tevent_timeval_add(&now, 0, 500000);
+ subreq = tevent_wakeup_send(state, state->ev, next);
if (tevent_req_nomem(subreq, req)) {
return;
}
diff --git a/source3/modules/vfs_error_inject.c
b/source3/modules/vfs_error_inject.c
index 529504fd8d5..dcf0de0a2d9 100644
--- a/source3/modules/vfs_error_inject.c
+++ b/source3/modules/vfs_error_inject.c
@@ -19,6 +19,7 @@
#include "includes.h"
#include "smbd/smbd.h"
+#include "librpc/gen_ndr/ndr_open_files.h"
#undef DBGC_CLASS
#define DBGC_CLASS DBGC_VFS
@@ -204,11 +205,86 @@ static int vfs_error_inject_unlinkat(struct
vfs_handle_struct *handle,
return -1;
}
+static NTSTATUS vfs_error_inject_durable_reconnect(struct vfs_handle_struct
*handle,
+ struct smb_request *smb1req,
+ struct smbXsrv_open *op,
+ const DATA_BLOB old_cookie,
+ TALLOC_CTX *mem_ctx,
+ struct files_struct **fsp,
+ DATA_BLOB *new_cookie)
+{
+ const char *vfs_func = "durable_reconnect";
+ const char *err_str = NULL;
+ NTSTATUS status;
+ enum ndr_err_code ndr_err;
+ struct vfs_default_durable_cookie cookie;
+ DATA_BLOB modified_cookie = data_blob_null;
+
+ err_str = lp_parm_const_string(SNUM(handle->conn),
+ "error_inject",
+ vfs_func,
+ NULL);
+ if (err_str == NULL) {
+ return SMB_VFS_NEXT_DURABLE_RECONNECT(handle,
+ smb1req,
+ op,
+ old_cookie,
+ mem_ctx,
+ fsp,
+ new_cookie);
+ }
+
+ ndr_err = ndr_pull_struct_blob(&old_cookie, talloc_tos(), &cookie,
+
(ndr_pull_flags_fn_t)ndr_pull_vfs_default_durable_cookie);
+ if (!NDR_ERR_CODE_IS_SUCCESS(ndr_err)) {
+ status = ndr_map_error2ntstatus(ndr_err);
+ return status;
+ }
+
+ if (strcmp(cookie.magic, VFS_DEFAULT_DURABLE_COOKIE_MAGIC) != 0) {
+ return NT_STATUS_INVALID_PARAMETER;
+ }
+
+ if (cookie.version != VFS_DEFAULT_DURABLE_COOKIE_VERSION) {
+ return NT_STATUS_INVALID_PARAMETER;
+ }
+
+ if (strequal(err_str, "st_ex_nlink")) {
+ cookie.stat_info.st_ex_nlink += 1;
--
Samba Shared Repository
