The branch, v4-14-test has been updated
via 8f7ab597969 smbXsrv_{open,session,tcon}: protect
smbXsrv_{open,session,tcon}_global_traverse_fn against invalid records
via c1662a81220 gensec_krb5: restore ipv6 support for kpasswd
via a6447a1dce1 netcmd: Use next_free_rid() function to calculate a SID
for restoring a backup
via 69d8b64fdc1 python/tests/dsdb: Add tests for RID allocation
functions
via 94ca97bd121 dsdb: Add next_free_rid() function to allocate a RID
without modifying the database
via f9d2652a0b4 netcmd: Add tests for performing an offline backup
immediately after joining a domain
via b226e83a3dc netcmd: Ignore rIDUsedPool attribute in offline domain
backup test
via 79029224ee0 netcmd: Fix error-checking condition
via c1ac591c197 netcmd: Avoid database corruption by opting not to
create database files during an offline domain backup
via 7a7bfba1d98 netcmd: Determine which files are to be copied for an
offline domain backup
via 5b361227e7c netcmd: Add test for an offline backup of nested
directories
via b095932a303 netcmd: Add test for an offline backup of a directory
containing hardlinks
via 60714069b2c mdssvc: avoid direct filesystem access, use the VFS
via 19115477256 mdssvc: chdir() to the conn of the RPC request
via f8e857aeed3 mdssvc: maintain a connection struct in the mds_ctx
via 9439cfe7142 smbd: add create_conn_struct_cwd()
via 5ee1c6a0b01 smbd: pass tevent context to
create_conn_struct_as_root()
via b1cb178ab9d mdssvc: pass messaging context to mds_init_ctx()
via db5326a7f7a mdssvc: don't fail mds_add_result() if result is not
found in CNID set
via 6ce42a067f3 mdssvc: use a helper variable in mds_add_result()
via 858a116e796 smbd: add synthetic_pathref()
via 4936ad99859 s3: smbd: Remove erroneous
TALLOC_FREE(smb_fname_parent) in change_file_owner_to_parent() error path.
via f8c4bcb0b22 s3/modules: fchmod: fallback to path based chmod if
pathref
via 866efccfa90 s3: VFS: default: Add proc_fd's fallback for
vfswrap_fchown().
via 35d7a23d720 s3: lib: Fix talloc heirarcy error in
parent_smb_fname().
from 42fa9f800fd smbd: fix pathref unlinking in create_file_unixpath()
https://git.samba.org/?p=samba.git;a=shortlog;h=v4-14-test
- Log -----------------------------------------------------------------
commit 8f7ab597969e6e834ef333d5cf314f770325d6a9
Author: Stefan Metzmacher <me...@samba.org>
Date: Mon Jul 5 17:17:30 2021 +0200
smbXsrv_{open,session,tcon}: protect
smbXsrv_{open,session,tcon}_global_traverse_fn against invalid records
I saw systems with locking.tdb records being part of:
ctdb catdb smbXsrv_tcon_global.tdb
It's yet unknown how that happened, but we should not panic in srvsvc_*
calls because the info0 pointer was NULL.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14752
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Volker Lendecke <v...@samba.org>
Autobuild-User(master): Stefan Metzmacher <me...@samba.org>
Autobuild-Date(master): Tue Jul 6 11:08:43 UTC 2021 on sn-devel-184
(cherry picked from commit 00bab5b3c821f272153a25ded9743460887a7907)
Autobuild-User(v4-14-test): Karolin Seeger <ksee...@samba.org>
Autobuild-Date(v4-14-test): Mon Jul 12 13:52:06 UTC 2021 on sn-devel-184
commit c1662a8122011aa550b2ae2325de97c6f57e1485
Author: Stefan Metzmacher <me...@samba.org>
Date: Fri Jul 2 09:37:25 2021 +0200
gensec_krb5: restore ipv6 support for kpasswd
We need to offer as much space we have in order to
get the address out of tsocket_address_bsd_sockaddr().
This fixes a regression in commit
43c808f2ff907497dfff0988ff90a48fdcfc16ef.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14750
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
(cherry picked from commit 0388a8f33bdde49f1cc805a0291859203c1a52b4)
commit a6447a1dce1bed1a33ab6aa729f5837acc3895f6
Author: Joseph Sutton <josephsut...@catalyst.net.nz>
Date: Thu May 27 15:35:35 2021 +1200
netcmd: Use next_free_rid() function to calculate a SID for restoring a
backup
This means we won't get errors if the DC doesn't have a rIDNextRID
attribute, but we will still error if there is no RID Set or if all its
pools are exhausted.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14669
Signed-off-by: Joseph Sutton <josephsut...@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz>
(cherry picked from commit 59d293b60608172ae61551c642d13d3b215924e4)
commit 69d8b64fdc1b2d9b5ac88385af704e2935d6ca4e
Author: Joseph Sutton <josephsut...@catalyst.net.nz>
Date: Mon May 24 16:46:28 2021 +1200
python/tests/dsdb: Add tests for RID allocation functions
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14669
Signed-off-by: Joseph Sutton <josephsut...@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz>
(cherry picked from commit 7c7cad81844950c3efe9a540a47b9d4e1ce1b2a1)
commit 94ca97bd121993b461082286ae24d2510211d36e
Author: Joseph Sutton <josephsut...@catalyst.net.nz>
Date: Mon May 24 12:59:59 2021 +1200
dsdb: Add next_free_rid() function to allocate a RID without modifying the
database
If used to generate SIDs for objects, care should be taken, as the
possibility for having duplicate objectSIDs can arise.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14669
Signed-off-by: Joseph Sutton <josephsut...@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz>
(cherry picked from commit cc98e03e7a0f2bf7a1ace2950fe6500f53640c1b)
commit f9d2652a0b42c37acff3ce16f0c430f3522012e5
Author: Joseph Sutton <josephsut...@catalyst.net.nz>
Date: Mon May 24 14:58:40 2021 +1200
netcmd: Add tests for performing an offline backup immediately after
joining a domain
This currently fails due to the DC not having a rIDNextRID attribute,
which is required for the restore process.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14669
Signed-off-by: Joseph Sutton <josephsut...@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz>
(cherry picked from commit b7e6a1c5da7283c49586dc29f85ab19e0e57b0f6)
commit b226e83a3dc3c74c06e48469ec02c19cbbfb963b
Author: Joseph Sutton <josephsut...@catalyst.net.nz>
Date: Wed May 26 13:40:30 2021 +1200
netcmd: Ignore rIDUsedPool attribute in offline domain backup test
The RID Set of the newly created DC account has all its values
initialised to zero. If the rIDUsedPool attribute was previously
non-zero, then the restore process will cause its value to change.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14669
Signed-off-by: Joseph Sutton <josephsut...@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz>
(cherry picked from commit 658e5a6cc20b57f48477affd370fe25458178b92)
commit 79029224ee02ff9f240ab1a81a3372a193b0879d
Author: Joseph Sutton <josephsut...@catalyst.net.nz>
Date: Mon May 24 16:40:55 2021 +1200
netcmd: Fix error-checking condition
This condition probably meant to check the argument of the most recently
thrown exception, rather than the previous one again.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14669
Signed-off-by: Joseph Sutton <josephsut...@catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz>
(cherry picked from commit e8c242bed19432d96e78dc345ab5f06422c5b104)
commit c1ac591c1976278aac7c26f91d87acdf37bdfb77
Author: Joseph Sutton <josephsut...@catalyst.net.nz>
Date: Tue Mar 16 22:20:21 2021 +1300
netcmd: Avoid database corruption by opting not to create database files
during an offline domain backup
If backup dirs contain hardlinks, the backup process could previously
attempt to open an LMDB database already opened during the backup,
causing it to be recreated as a new TDB database. This commit ensures
that new database files are not created during this operation, and that
the main SamDB database is not modified.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14027
Signed-off-by: Joseph Sutton <josephsut...@catalyst.net.nz>
(cherry picked from commit 4cf773591d49166b8c7ef8d637d7edfe755d48aa)
commit 7a7bfba1d981edca2dc54db9c43c7f6b9310bc39
Author: Joseph Sutton <josephsut...@catalyst.net.nz>
Date: Tue Mar 16 16:22:40 2021 +1300
netcmd: Determine which files are to be copied for an offline domain backup
The old behaviour attempted to check for and remove files with duplicate
names, but did not do so due to a bug, and would have left undetermined
which files were given priority when duplicate filenames were present.
Now when hardlinks are present, only one instance of each file is
chosen, with files in the private directory having priority. If one
backup dir is nested inside another, the files contained in the nested
directory are only added once. Additionally, the BIND DNS database is
omitted from the backup.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14027
Signed-off-by: Joseph Sutton <josephsut...@catalyst.net.nz>
(cherry picked from commit 3723148e7aa7e6d4a48a1a38112f121f52b6ee6f)
commit 5b361227e7cd259b5585b4e9ad0df7ccd1021580
Author: Joseph Sutton <josephsut...@catalyst.net.nz>
Date: Thu Mar 18 10:52:52 2021 +1300
netcmd: Add test for an offline backup of nested directories
This test verifies that when performing an offline backup of a domain
where one of the directories to be backed up is nested inside another,
the contained files are only included once in the backup.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14027
Signed-off-by: Joseph Sutton <josephsut...@catalyst.net.nz>
(cherry picked from commit f994783f4279884ec4d2ee3e7db80fb7af267d1c)
commit b095932a303c881b459ba73e222b6ed199f673dc
Author: Joseph Sutton <josephsut...@catalyst.net.nz>
Date: Tue Mar 16 16:13:05 2021 +1300
netcmd: Add test for an offline backup of a directory containing hardlinks
This test verifies that when performing an offline backup of a domain
where the directories to be backed up contain hardlinks, only one
instance of each file is backed up, and that files in the private
directory take precedence.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14027
Signed-off-by: Joseph Sutton <josephsut...@catalyst.net.nz>
(cherry picked from commit 0e5738887524b467bfebcf657bcb00ed71827784)
commit 60714069b2c8550de7527336713f8e8905c5869b
Author: Ralph Boehme <s...@samba.org>
Date: Mon May 10 12:34:32 2021 +0200
mdssvc: avoid direct filesystem access, use the VFS
This ensures mdssvc uses the same FileIDs as the fileserver as well as
Spotlight
can be used working on a virtual filesystem like GlusterFS.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14740
RN: Spotlight RPC service doesn't work with vfs_glusterfs
Signed-off-by: Ralph Boehme <s...@samba.org>
Reviewed-by: Jeremy Allison <j...@samba.org>
Autobuild-User(master): Ralph Böhme <s...@samba.org>
Autobuild-Date(master): Wed Jun 16 05:59:13 UTC 2021 on sn-devel-184
(backported from commit 620b99144359f45aa69c13731db8d793cfbba197)
[s...@samba.org: smbd_check_access_rights_fsp() doesn't take dirfsp arg]
commit 1911547725601a728617e0677486a78ed99e4169
Author: Ralph Boehme <s...@samba.org>
Date: Tue Jun 15 14:14:52 2021 +0200
mdssvc: chdir() to the conn of the RPC request
In preperation of calling VFS functions.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14740
Reviewed-by: Ralph Boehme <s...@samba.org>
Reviewed-by: Jeremy Allison <j...@samba.org>
(cherry picked from commit 6de3a88494b5932d0fd10f5c8c8ec57916aeefc5)
commit f8e857aeed30bc2c42f42c8692ce207a3f2d4a19
Author: Ralph Boehme <s...@samba.org>
Date: Mon May 10 12:10:08 2021 +0200
mdssvc: maintain a connection struct in the mds_ctx
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14740
Signed-off-by: Ralph Boehme <s...@samba.org>
Reviewed-by: Jeremy Allison <j...@samba.org>
(cherry picked from commit 8b681cfb5d9b1ece03f7e7b9d3a08ae6c461d679)
commit 9439cfe71422d8f29e5b998ff5b1007de6dcec8c
Author: Ralph Boehme <s...@samba.org>
Date: Fri May 28 09:25:22 2021 +0200
smbd: add create_conn_struct_cwd()
Compared to create_conn_struct_tos_cwd() this takes a TALLOC_CTX and
tevent_context as additional arguments and the resulting connection_struct
is
stable across the lifetime of mem_ctx and ev.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14740
Signed-off-by: Ralph Boehme <s...@samba.org>
Reviewed-by: Jeremy Allison <j...@samba.org>
(cherry picked from commit 9a2d6bcfd5797dd4db764921548c8dca6dd0eb21)
commit 5ee1c6a0b01d03c0d697bd797474b0d35e49b512
Author: Ralph Boehme <s...@samba.org>
Date: Tue Jun 15 11:17:57 2021 +0200
smbd: pass tevent context to create_conn_struct_as_root()
The next commit will add another caller of create_conn_struct_as_root()
that is
going to pass a long-lived tevent context.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14740
Signed-off-by: Ralph Boehme <s...@samba.org>
Reviewed-by: Jeremy Allison <j...@samba.org>
(cherry picked from commit 16c39b81d6f2c7d75cfe72bbbe2f6a5bde42c7b0)
commit b1cb178ab9de83bfee55f8a7145a978bb736feed
Author: Ralph Boehme <s...@samba.org>
Date: Mon May 10 12:08:17 2021 +0200
mdssvc: pass messaging context to mds_init_ctx()
This is needed in a subsequent commit. Note that I prefer to do the event
context unwrapping in the caller and pass both the event and messaging
context
explicitly to mds_init_ctx().
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14740
Signed-off-by: Ralph Boehme <s...@samba.org>
Reviewed-by: Jeremy Allison <j...@samba.org>
(cherry picked from commit 1ef2828e1025e4c89292df1dfa6161c4453b3afe)
commit db5326a7f7a0ba877b0b21685fa825b2794dce81
Author: Ralph Boehme <s...@samba.org>
Date: Mon May 10 11:07:27 2021 +0200
mdssvc: don't fail mds_add_result() if result is not found in CNID set
Just skip adding the result to the pending results set, don't return an
error. Returning an error triggers an error at the MDSSVC RPC error which
is NOT
what we want here.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14740
Signed-off-by: Ralph Boehme <s...@samba.org>
Reviewed-by: Jeremy Allison <j...@samba.org>
(cherry picked from commit 8847f46f75ac5c1a753a0e7da88c522be25ef681)
commit 6ce42a067f34d40e5c26712413405f591a3b7b67
Author: Ralph Boehme <s...@samba.org>
Date: Mon May 10 11:04:38 2021 +0200
mdssvc: use a helper variable in mds_add_result()
No change in behaviour.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14740
Signed-off-by: Ralph Boehme <s...@samba.org>
Reviewed-by: Jeremy Allison <j...@samba.org>
(cherry picked from commit e2486d76b611f07b85b26c54fe14da7b76bd01c2)
commit 858a116e796e35bbe52ff4e60cdeb255c408b497
Author: Ralph Boehme <s...@samba.org>
Date: Thu Jan 21 15:04:57 2021 +0100
smbd: add synthetic_pathref()
Similar to synthetic_smb_fname(), but also opens a pathref fsp.
Signed-off-by: Ralph Boehme <s...@samba.org>
Reviewed-by: Jeremy Allison <j...@samba.org>
(cherry picked from commit d9f95b8cefe2d1c8020592434481025aa1045e2f)
commit 4936ad998598325f1b472d7ecb2af82c08a1ef8f
Author: Jeremy Allison <j...@samba.org>
Date: Wed Jun 9 12:22:26 2021 -0700
s3: smbd: Remove erroneous TALLOC_FREE(smb_fname_parent) in
change_file_owner_to_parent() error path.
Caller is still using this !
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14736
Signed-off-by: Jeremy Allison <j...@samba.org>
Reviewed-by: Noel Power<npo...@samba.org>
Autobuild-User(master): Noel Power <npo...@samba.org>
Autobuild-Date(master): Fri Jun 11 10:17:46 UTC 2021 on sn-devel-184
(cherry picked from commit 4f20d310af2bb1f96dea4810a7130492cc4cfc55)
commit f8c4bcb0b22d3c20419bec2f175785266e629f3a
Author: Ralph Boehme <s...@samba.org>
Date: Fri Apr 9 14:58:34 2021 +0200
s3/modules: fchmod: fallback to path based chmod if pathref
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14734
Signed-off-by: Noel Power <noel.po...@suse.com>
Signed-off-by: Ralph Boehme <s...@samba.org>
Back-ported from master commit 6ad10836d6e04d8c95773e9122b63f5a5e040487)
commit 866efccfa9061a5b3468af74fc5d0791feffa5fa
Author: Jeremy Allison <j...@samba.org>
Date: Wed Jun 9 15:57:38 2021 -0700
s3: VFS: default: Add proc_fd's fallback for vfswrap_fchown().
https://bugzilla.samba.org/show_bug.cgi?id=14734
Signed-off-by: Jeremy Allison <j...@samba.org>
Reviewed-by: Noel Power <npo...@samba.org>
Autobuild-User(master): Noel Power <npo...@samba.org>
Autobuild-Date(master): Thu Jun 10 09:16:22 UTC 2021 on sn-devel-184
(cherry picked from commit f44918e6c83c89936156eb24c982a897c9c45f61)
commit 35d7a23d72054363ec659b1d89e2e6fbd850a7a3
Author: Jeremy Allison <j...@samba.org>
Date: Tue Jun 1 13:27:47 2021 -0700
s3: lib: Fix talloc heirarcy error in parent_smb_fname().
BUG: https://bugzilla.samba.org/show_bug.cgi?id=14722
Signed-off-by: Jeremy Allison <j...@samba.org>
Reviewed-by: Ralph Boehme <s...@samba.org>
(cherry picked from commit c500d99e2f5aaec102bf952b7941a2596b3e35a1)
-----------------------------------------------------------------------
Summary of changes:
python/samba/netcmd/domain_backup.py | 108 +++++-----
python/samba/samdb.py | 105 ++++++++++
python/samba/tests/domain_backup_offline.py | 162 ++++++++++++---
python/samba/tests/dsdb.py | 305 +++++++++++++++++++++++++++-
source3/lib/filename_util.c | 2 +-
source3/modules/vfs_default.c | 58 +++++-
source3/rpc_server/mdssvc/mdssvc.c | 132 ++++++++++--
source3/rpc_server/mdssvc/mdssvc.h | 2 +
source3/rpc_server/mdssvc/srv_mdssvc_nt.c | 1 +
source3/smbd/files.c | 52 +++++
source3/smbd/msdfs.c | 54 ++++-
source3/smbd/open.c | 2 -
source3/smbd/proto.h | 17 ++
source3/smbd/smbXsrv_open.c | 9 +
source3/smbd/smbXsrv_session.c | 7 +
source3/smbd/smbXsrv_tcon.c | 7 +
source4/auth/gensec/gensec_krb5.c | 6 +-
source4/selftest/tests.py | 2 +-
18 files changed, 913 insertions(+), 118 deletions(-)
Changeset truncated at 500 lines:
diff --git a/python/samba/netcmd/domain_backup.py
b/python/samba/netcmd/domain_backup.py
index 799fd0593e5..f441e7407ee 100644
--- a/python/samba/netcmd/domain_backup.py
+++ b/python/samba/netcmd/domain_backup.py
@@ -62,50 +62,21 @@ from samba.credentials import SMB_SIGNING_REQUIRED
# This ensures that the restored DC's SID won't clash with any other RIDs
# already in use in the domain
def get_sid_for_restore(samdb, logger):
- # Find the DN of the RID set of the server
- res = samdb.search(base=ldb.Dn(samdb, samdb.get_serverName()),
- scope=ldb.SCOPE_BASE, attrs=["serverReference"])
- server_ref_dn = ldb.Dn(samdb, str(res[0]['serverReference'][0]))
- res = samdb.search(base=server_ref_dn,
- scope=ldb.SCOPE_BASE,
- attrs=['rIDSetReferences'])
- rid_set_dn = ldb.Dn(samdb, str(res[0]['rIDSetReferences'][0]))
-
- # Get the alloc pools and next RID of the RID set
- res = samdb.search(base=rid_set_dn,
- scope=ldb.SCOPE_SUBTREE,
- expression="(rIDNextRID=*)",
- attrs=['rIDAllocationPool',
- 'rIDPreviousAllocationPool',
- 'rIDNextRID'])
-
- # Decode the bounds of the RID allocation pools
+ # Allocate a new RID without modifying the database. This should be safe,
+ # because we acquire the RID master role after creating an account using
+ # this RID during the restore process. Acquiring the RID master role
+ # creates a new RID pool which we will fetch RIDs from, so we shouldn't get
+ # duplicates.
try:
- rid = int(res[0].get('rIDNextRID')[0])
- except IndexError:
- logger.info("The RID pool for this DC is not initalized "
- "(e.g. it may be a fairly new DC).")
- logger.info("To initialize it, create a temporary user on this DC "
- "(you can delete it later).")
- raise CommandError("Cannot create backup - "
- "please initialize this DC's RID pool first.")
-
- def split_val(num):
- high = (0xFFFFFFFF00000000 & int(num)) >> 32
- low = 0x00000000FFFFFFFF & int(num)
- return low, high
- pool_l, pool_h = split_val(res[0].get('rIDPreviousAllocationPool')[0])
- npool_l, npool_h = split_val(res[0].get('rIDAllocationPool')[0])
-
- # Calculate next RID based on pool bounds
- if rid == npool_h:
- raise CommandError('Out of RIDs, finished AllocPool')
- if rid == pool_h:
- if pool_h == npool_h:
- raise CommandError('Out of RIDs, finished PrevAllocPool.')
- rid = npool_l
- else:
- rid += 1
+ rid = samdb.next_free_rid()
+ except LdbError as err:
+ logger.info("A SID could not be allocated for restoring the domain. "
+ "Either no RID Set was found on this DC, "
+ "or the RID Set was not usable.")
+ logger.info("To initialise this DC's RID pools, obtain a RID Set from "
+ "this domain's RID master, or run samba-tool dbcheck "
+ "to fix the existing RID Set.")
+ raise CommandError("Cannot create backup", err)
# Construct full SID
sid = dom_sid(samdb.get_domain_sid())
@@ -313,7 +284,8 @@ class cmd_domain_backup_online(samba.netcmd.Command):
shutil.rmtree(paths.sysvol)
# Edit the downloaded sam.ldb to mark it as a backup
- samdb = SamDB(url=paths.samdb, session_info=system_session(),
lp=lp)
+ samdb = SamDB(url=paths.samdb, session_info=system_session(),
lp=lp,
+ flags=ldb.FLG_DONT_CREATE_DB)
time_str = get_timestamp()
add_backup_marker(samdb, "backupDate", time_str)
add_backup_marker(samdb, "sidForRestore", new_sid)
@@ -537,7 +509,8 @@ class cmd_domain_backup_restore(cmd_fsmo_seize):
# open a DB connection to the restored DB
private_dir = os.path.join(targetdir, 'private')
samdb_path = os.path.join(private_dir, 'sam.ldb')
- samdb = SamDB(url=samdb_path, session_info=system_session(), lp=lp)
+ samdb = SamDB(url=samdb_path, session_info=system_session(), lp=lp,
+ flags=ldb.FLG_DONT_CREATE_DB)
backup_type = self.get_backup_type(samdb)
if site is None:
@@ -600,11 +573,12 @@ class cmd_domain_backup_restore(cmd_fsmo_seize):
controls=["show_deleted:0",
"show_recycled:0"])
except LdbError as dup_e:
- if enum != ldb.ERR_NO_SUCH_OBJECT:
- raise e
+ (dup_enum, _) = dup_e.args
+ if dup_enum != ldb.ERR_NO_SUCH_OBJECT:
+ raise
if (len(dup_res) != 1):
- raise e
+ raise
objectguid = samdb.schema_format_value("objectGUID",
dup_res[0]["objectGUID"][0])
@@ -645,7 +619,8 @@ class cmd_domain_backup_restore(cmd_fsmo_seize):
host_ip, host_ip6, site)
secrets_path = os.path.join(private_dir, 'secrets.ldb')
- secrets_ldb = Ldb(secrets_path, session_info=system_session(), lp=lp)
+ secrets_ldb = Ldb(secrets_path, session_info=system_session(), lp=lp,
+ flags=ldb.FLG_DONT_CREATE_DB)
secretsdb_self_join(secrets_ldb, domain=ctx.domain_name,
realm=ctx.realm, dnsdomain=ctx.dnsdomain,
netbiosname=ctx.myname, domainsid=ctx.domsid,
@@ -937,7 +912,8 @@ class cmd_domain_backup_rename(samba.netcmd.Command):
# connect to the local DB (making sure we use the new/renamed config)
lp.load(paths.smbconf)
- samdb = SamDB(url=paths.samdb, session_info=system_session(), lp=lp)
+ samdb = SamDB(url=paths.samdb, session_info=system_session(), lp=lp,
+ flags=ldb.FLG_DONT_CREATE_DB)
# Edit the cloned sam.ldb to mark it as a backup
time_str = get_timestamp()
@@ -1025,7 +1001,8 @@ class cmd_domain_backup_offline(samba.netcmd.Command):
# on the secrets.ldb file before backing up that file and secrets.tdb
def backup_secrets(self, private_dir, lp, logger):
secrets_path = os.path.join(private_dir, 'secrets')
- secrets_obj = Ldb(secrets_path + '.ldb', lp=lp)
+ secrets_obj = Ldb(secrets_path + '.ldb', lp=lp,
+ flags=ldb.FLG_DONT_CREATE_DB)
logger.info('Starting transaction on ' + secrets_path)
secrets_obj.transaction_start()
self.offline_tdb_copy(secrets_path + '.ldb')
@@ -1050,7 +1027,7 @@ class cmd_domain_backup_offline(samba.netcmd.Command):
else:
logger.info('Starting transaction on ' + sam_ldb_path)
copy_function = self.offline_tdb_copy
- sam_obj = Ldb(sam_ldb_path, lp=lp)
+ sam_obj = Ldb(sam_ldb_path, lp=lp, flags=ldb.FLG_DONT_CREATE_DB)
sam_obj.transaction_start()
logger.info(' backing up ' + sam_ldb_path)
@@ -1102,9 +1079,14 @@ class cmd_domain_backup_offline(samba.netcmd.Command):
check_targetdir(logger, targetdir)
- samdb = SamDB(url=paths.samdb, session_info=system_session(), lp=lp)
+ samdb = SamDB(url=paths.samdb, session_info=system_session(), lp=lp,
+ flags=ldb.FLG_RDONLY)
sid = get_sid_for_restore(samdb, logger)
+ # Iterating over the directories in this specific order ensures that
+ # when the private directory contains hardlinks that are also contained
+ # in other directories to be backed up (such as in paths.binddns_dir),
+ # the hardlinks in the private directory take precedence.
backup_dirs = [paths.private_dir, paths.state_dir,
os.path.dirname(paths.smbconf)] # etc dir
logger.info('running backup on dirs: {0}'.format('
'.join(backup_dirs)))
@@ -1117,22 +1099,31 @@ class cmd_domain_backup_offline(samba.netcmd.Command):
continue
if working_dir.endswith('.sock') or '.sock/' in working_dir:
continue
+ # The BIND DNS database can be regenerated, so it doesn't need
+ # to be backed up.
+ if working_dir.startswith(os.path.join(paths.binddns_dir,
'dns')):
+ continue
for filename in filenames:
- if filename in all_files:
+ full_path = os.path.join(working_dir, filename)
+
+ # Ignore files that have already been added. This prevents
+ # duplicates if one backup dir is a subdirectory of
another,
+ # or if backup dirs contain hardlinks.
+ if any(os.path.samefile(full_path, file) for file in
all_files):
continue
# Assume existing backup files are from a previous backup.
# Delete and ignore.
if filename.endswith(self.backup_ext):
- os.remove(os.path.join(working_dir, filename))
+ os.remove(full_path)
continue
# Sock files are autogenerated at runtime, ignore.
if filename.endswith('.sock'):
continue
- all_files.append(os.path.join(working_dir, filename))
+ all_files.append(full_path)
# Backup secrets, sam.ldb and their downstream files
self.backup_secrets(paths.private_dir, lp, logger)
@@ -1144,7 +1135,8 @@ class cmd_domain_backup_offline(samba.netcmd.Command):
# Writing to a .bak file only works because the DN being
# written to happens to be top level.
samdb = SamDB(url=paths.samdb + self.backup_ext,
- session_info=system_session(), lp=lp)
+ session_info=system_session(), lp=lp,
+ flags=ldb.FLG_DONT_CREATE_DB)
time_str = get_timestamp()
add_backup_marker(samdb, "backupDate", time_str)
add_backup_marker(samdb, "sidForRestore", sid)
@@ -1156,7 +1148,7 @@ class cmd_domain_backup_offline(samba.netcmd.Command):
if not os.path.exists(path + self.backup_ext):
if path.endswith('.ldb'):
logger.info('Starting transaction on solo db: ' + path)
- ldb_obj = Ldb(path, lp=lp)
+ ldb_obj = Ldb(path, lp=lp, flags=ldb.FLG_DONT_CREATE_DB)
ldb_obj.transaction_start()
logger.info(' running tdbbackup on the same file')
self.offline_tdb_copy(path)
diff --git a/python/samba/samdb.py b/python/samba/samdb.py
index f95709ab7c8..d65c6f2e3b1 100644
--- a/python/samba/samdb.py
+++ b/python/samba/samdb.py
@@ -1385,6 +1385,111 @@ schemaUpdateNow: 1
'''return a new RID from the RID Pool on this DSA'''
return dsdb._dsdb_allocate_rid(self)
+ def next_free_rid(self):
+ '''return the next free RID from the RID Pool on this DSA.
+
+ :note: This function is not intended for general use, and care must be
+ taken if it is used to generate objectSIDs. The returned RID is not
+ formally reserved for use, creating the possibility of duplicate
+ objectSIDs.
+ '''
+ rid, _ = self.free_rid_bounds()
+ return rid
+
+ def free_rid_bounds(self):
+ '''return the low and high bounds (inclusive) of RIDs that are
+ available for use in this DSA's current RID pool.
+
+ :note: This function is not intended for general use, and care must be
+ taken if it is used to generate objectSIDs. The returned range of
+ RIDs is not formally reserved for use, creating the possibility of
+ duplicate objectSIDs.
+ '''
+ # Get DN of this server's RID Set
+ server_name_dn = ldb.Dn(self, self.get_serverName())
+ res = self.search(base=server_name_dn,
+ scope=ldb.SCOPE_BASE,
+ attrs=["serverReference"])
+ try:
+ server_ref = res[0]["serverReference"]
+ except KeyError:
+ raise ldb.LdbError(
+ ldb.ERR_NO_SUCH_ATTRIBUTE,
+ "No RID Set DN - "
+ "Cannot find attribute serverReference of %s "
+ "to calculate reference dn" % server_name_dn) from None
+ server_ref_dn = ldb.Dn(self, server_ref[0].decode("utf-8"))
+
+ res = self.search(base=server_ref_dn,
+ scope=ldb.SCOPE_BASE,
+ attrs=["rIDSetReferences"])
+ try:
+ rid_set_refs = res[0]["rIDSetReferences"]
+ except KeyError:
+ raise ldb.LdbError(
+ ldb.ERR_NO_SUCH_ATTRIBUTE,
+ "No RID Set DN - "
+ "Cannot find attribute rIDSetReferences of %s "
+ "to calculate reference dn" % server_ref_dn) from None
+ rid_set_dn = ldb.Dn(self, rid_set_refs[0].decode("utf-8"))
+
+ # Get the alloc pools and next RID of this RID Set
+ res = self.search(base=rid_set_dn,
+ scope=ldb.SCOPE_BASE,
+ attrs=["rIDAllocationPool",
+ "rIDPreviousAllocationPool",
+ "rIDNextRID"])
+
+ uint32_max = 2**32 - 1
+ uint64_max = 2**64 - 1
+
+ try:
+ alloc_pool = int(res[0]["rIDAllocationPool"][0])
+ except KeyError:
+ alloc_pool = uint64_max
+ if alloc_pool == uint64_max:
+ raise ldb.LdbError(ldb.ERR_OPERATIONS_ERROR,
+ "Bad RID Set %s" % rid_set_dn)
+
+ try:
+ prev_pool = int(res[0]["rIDPreviousAllocationPool"][0])
+ except KeyError:
+ prev_pool = uint64_max
+ try:
+ next_rid = int(res[0]["rIDNextRID"][0])
+ except KeyError:
+ next_rid = uint32_max
+
+ # If we never used a pool, set up our first pool
+ if prev_pool == uint64_max or next_rid == uint32_max:
+ prev_pool = alloc_pool
+ next_rid = prev_pool & uint32_max
+
+ next_rid += 1
+
+ # Now check if our current pool is still usable
+ prev_pool_lo = prev_pool & uint32_max
+ prev_pool_hi = prev_pool >> 32
+ if next_rid > prev_pool_hi:
+ # We need a new pool, check if we already have a new one
+ # Otherwise we return an error code.
+ if alloc_pool == prev_pool:
+ raise ldb.LdbError(ldb.ERR_OPERATIONS_ERROR,
+ "RID pools out of RIDs")
+
+ # Now use the new pool
+ prev_pool = alloc_pool
+ prev_pool_lo = prev_pool & uint32_max
+ prev_pool_hi = prev_pool >> 32
+ next_rid = prev_pool_lo
+
+ if next_rid < prev_pool_lo or next_rid > prev_pool_hi:
+ raise ldb.LdbError(ldb.ERR_OPERATIONS_ERROR,
+ "Bad RID chosen %d from range %d-%d" %
+ (next_rid, prev_pool_lo, prev_pool_hi))
+
+ return next_rid, prev_pool_hi
+
def normalize_dn_in_domain(self, dn):
'''return a new DN expanded by adding the domain DN
diff --git a/python/samba/tests/domain_backup_offline.py
b/python/samba/tests/domain_backup_offline.py
index 8b7209ec24d..21f42c6dab8 100644
--- a/python/samba/tests/domain_backup_offline.py
+++ b/python/samba/tests/domain_backup_offline.py
@@ -19,8 +19,12 @@ import tarfile
import os
import shutil
import tempfile
-from samba.tests import BlackboxTestCase
+from samba.tests import BlackboxTestCase, BlackboxProcessError
from samba.netcmd import CommandError
+from samba.param import LoadParm
+from samba.join import join_DC
+from samba.credentials import Credentials
+from samba.logger import get_samba_logger
# The backup tests require that a completely clean LoadParm object gets used
# for the restore. Otherwise the same global LP gets re-used, and the LP
@@ -31,6 +35,81 @@ from samba.netcmd import CommandError
# so that we never inadvertently use .runcmd() by accident.
class DomainBackupOfflineCmp(BlackboxTestCase):
+ def test_domain_backup_offline_nested_tdb(self):
+ self.nested_testcase('tdb')
+
+ def test_domain_backup_offline_nested_mdb(self):
+ self.nested_testcase('mdb')
+
+ def nested_testcase(self, backend):
+ self.prov_dir = self.provision(backend)
+ self.extract_dir = None
+
+ src = os.path.join(self.prov_dir, "private")
+ dst = os.path.join(self.prov_dir, "state", "private")
+
+ # Move private directory inside state directory
+ shutil.move(src, dst)
+
+ smbconf = os.path.join(self.prov_dir, "etc", "smb.conf")
+
+ # Update the conf file
+ lp = LoadParm(filename_for_non_global_lp=smbconf)
+ lp.set("private dir", dst)
+ lp.dump(False, smbconf)
+
+ backup_file = self.backup(self.prov_dir)
+
+ # Ensure each file is only present once in the tar file
+ tf = tarfile.open(backup_file)
+ names = tf.getnames()
+ self.assertEqual(len(names), len(set(names)))
+
+ def test_domain_backup_offline_join_restore_tdb(self):
+ self.join_restore_testcase('tdb')
+
+ def test_domain_backup_offline_join_restore_mdb(self):
+ self.join_restore_testcase('mdb')
+
+ def join_restore_testcase(self, backend):
+ self.prov_dir = self.join(backend)
+ self.extract_dir = None
+
+ try:
+ backup_file = self.backup(self.prov_dir)
+ except BlackboxProcessError as e:
+ self.fail(e)
+
+ self.extract_dir = self.restore(backup_file)
+
+ def test_domain_backup_offline_hard_link_tdb(self):
+ self.hard_link_testcase('tdb')
+
+ def test_domain_backup_offline_hard_link_mdb(self):
+ self.hard_link_testcase('mdb')
+
+ def hard_link_testcase(self, backend):
+ self.prov_dir = self.provision(backend)
+ self.extract_dir = None
+
+ # Create hard links in the private and state directories
+ os.link(os.path.join(self.prov_dir, "private", "krb5.conf"),
+ os.path.join(self.prov_dir, "state", "krb5.conf"))
+
+ backup_file = self.backup(self.prov_dir)
+
+ # Extract the backup
+ self.extract_dir = tempfile.mkdtemp(dir=self.tempdir)
+ tf = tarfile.open(backup_file)
+ tf.extractall(self.extract_dir)
+
+ # Ensure that the hard link in the private directory was backed up,
+ # while the one in the state directory was not.
+ self.assertTrue(os.path.exists(os.path.join(self.extract_dir,
+ "private", "krb5.conf")))
+ self.assertFalse(os.path.exists(os.path.join(self.extract_dir,
+ "statedir", "krb5.conf")))
+
def test_domain_backup_offline_untar_tdb(self):
self.untar_testcase('tdb')
@@ -44,39 +123,33 @@ class DomainBackupOfflineCmp(BlackboxTestCase):
self.restore_testcase('mdb')
def restore_testcase(self, backend):
- prov_dir, backup_file = self.provision_and_backup(backend)
+ self.prov_dir = self.provision(backend)
+ self.extract_dir = None
+ backup_file = self.backup(self.prov_dir)
- extract_dir = tempfile.mkdtemp(dir=self.tempdir)
- cmd = ("samba-tool domain backup restore --backup-file={f}"
- " --targetdir={d} "
- "--newservername=NEWSERVER").format(f=backup_file,
d=extract_dir)
- self.check_output(cmd)
+ self.extract_dir = self.restore(backup_file)
# attrs that are altered by the restore process
ignore_attrs = ["servicePrincipalName", "lastLogonTimestamp",
- "rIDAllocationPool", "rIDAvailablePool",
+ "rIDAllocationPool", "rIDAvailablePool", "rIDUsedPool",
"localPolicyFlags", "operatingSystem", "displayName",
"dnsRecord", "dNSTombstoned",
"msDS-NC-Replica-Locations", "msDS-HasInstantiatedNCs",
"interSiteTopologyGenerator"]
filter_arg = "--filter=" + ",".join(ignore_attrs)
args = ["--two", filter_arg]
- self.ldapcmp(prov_dir, extract_dir, args)
-
- shutil.rmtree(prov_dir)
- shutil.rmtree(extract_dir)
+ self.ldapcmp(self.prov_dir, self.extract_dir, args)
def untar_testcase(self, backend):
- prov_dir, backup_file = self.provision_and_backup(backend)
+ self.prov_dir = self.provision(backend)
+ self.extract_dir = None
+ backup_file = self.backup(self.prov_dir)
- extract_dir = tempfile.mkdtemp(dir=self.tempdir)
+ self.extract_dir = tempfile.mkdtemp(dir=self.tempdir)
tf = tarfile.open(backup_file)
- tf.extractall(extract_dir)
-
- self.ldapcmp(prov_dir, extract_dir)
+ tf.extractall(self.extract_dir)
- shutil.rmtree(prov_dir)
- shutil.rmtree(extract_dir)
+ self.ldapcmp(self.prov_dir, self.extract_dir)
def ldapcmp(self, prov_dir, ex_dir, args=[]):
sam_fn = os.path.join("private", "sam.ldb")
@@ -90,8 +163,8 @@ class DomainBackupOfflineCmp(BlackboxTestCase):
self.check_output(cmd)
# Test the "samba-tool domain backup" command with ldapcmp
- def provision_and_backup(self, backend):
- prov_dir = tempfile.mkdtemp(dir=self.tempdir)
+ def provision(self, backend):
+ target = tempfile.mkdtemp(dir=self.tempdir)
# Provision domain. Use fake ACLs and store xattrs in tdbs so that
# NTACL backup will work inside the testenv.
@@ -100,13 +173,37 @@ class DomainBackupOfflineCmp(BlackboxTestCase):
# circumstances, causing the ldapcmp to fail.
prov_cmd = "samba-tool domain provision " +\
"--domain FOO --realm foo.example.com " +\
- "--targetdir {prov_dir} " +\
+ "--targetdir {target} " +\
"--backend-store {backend} " +\
--
Samba Shared Repository
