The branch, master has been updated via 70e1b61 tsocket_bsd: Attempt to increase the SO_SNDBUF if we get EMSGSIZE in sendto() via 50b42d1 s4-lib/socket: Return the original EMSGSIZE when sendto() and setsockopt() both fail via b9b6375 selftest: Remove output directories to save disk space via d5d88bd samba_upgradeprovision: Do not reset every DN when changing an SD via 0f247dc samba_upgradeprovision: do not maintain dnNotToRecalculate as a list via 9bc32bf samba_upgradeprovision: only run rebuild_sd in --full mode via 81cda85 samba_upgradeprovision: Remove alwaysRecalculate, this is too dangerous via 09b82d5 samba_upgradeprovision: Remove unused checkKeepAttributeOldMtd via 9b8d5bb samba_upgradeprovision: Remove inherited ACEs before comparing the SDs via 5074b98 scripting: Rework samba.upgradehelpers.get_diff_sddls to be get_diff_sds via 787a6aa samba_upgradeprovision: Remove auto-detection of pre-alpha9 databases via 9d6af49 selftest: Rename samba4.blackbox.upgradeprovision.py to samba4.blackbox.upgradeprovision.current via 08f0562 selftest: Run dbcheck and improved upgrdeprovision tests against release-4-0-0 via d7936ee selftest: Add ldapcmp to ensure upgradeprovision of a fresh DB is a no-op via f1f36ad selftest: Add in a provision from 4.0.0 to run tests against via 72f73eb selftest: Do an ldapcmp run against the upgraded domain via 24c4d81 samba-tool ldapcmp: Add support for checking DNSDOMAIN and DNSFOREST by default via f508435 samba-tool dbcheck: fix msDS-HasInstantiatedNCs attributes to match instanceType on our ntdsDSA via 97389c3 scripting: Correct parsing of binary DN via 0180a02 subunit: Add a sh macro for skipping a test from c692bb0 Handle EMSGSIZE on UNIX domain sockets.
http://gitweb.samba.org/?p=samba.git;a=shortlog;h=master - Log ----------------------------------------------------------------- commit 70e1b6185e3fb35fdc72eeb529ffb4b50122dc40 Author: Andrew Bartlett <abart...@samba.org> Date: Mon Mar 4 14:06:14 2013 +1100 tsocket_bsd: Attempt to increase the SO_SNDBUF if we get EMSGSIZE in sendto() This matches what was done for lib/socket/socket_unix.c in c692bb02b039ae8fef6ba968fd13b36ad7d62a72. (and is based on that patch by Landon Fuller <land...@bikemonkey.org>) Andrew Bartlett Reviewed-by: Stefan Metzmacher <me...@samba.org> Autobuild-User(master): Stefan Metzmacher <me...@samba.org> Autobuild-Date(master): Mon Mar 4 11:15:35 CET 2013 on sn-devel-104 commit 50b42d1c5bb19e3a5050d7d23ac96e273d3974ee Author: Andrew Bartlett <abart...@samba.org> Date: Mon Mar 4 14:07:38 2013 +1100 s4-lib/socket: Return the original EMSGSIZE when sendto() and setsockopt() both fail This ensures that should we be unable to increase the socket size, we return an error that the application layer above might expect and be able to make as reasonable response to (such as switching to a stream-based transport). This fixes up c692bb02b039ae8fef6ba968fd13b36ad7d62a72. As suggested by metze in https://bugzilla.samba.org/show_bug.cgi?id=9697#c4 Andrew Bartlett Reviewed-by: Stefan Metzmacher <me...@samba.org> commit b9b637569960ae7eef5ee12436624af34a718a9a Author: Andrew Bartlett <abart...@samba.org> Date: Sun Feb 17 22:45:59 2013 +1100 selftest: Remove output directories to save disk space Reviewed-by: Stefan Metzmacher <me...@samba.org> commit d5d88bd82b1cb51da09cf3b3dec40f180f5ed29f Author: Andrew Bartlett <abart...@samba.org> Date: Mon Feb 18 15:56:18 2013 +1100 samba_upgradeprovision: Do not reset every DN when changing an SD SD propogation is handled by an LDB module, we do not need to touch each and every DN to make it happen. Now that we do not need to put this via a hash, the dnToRecalculate list is changed to be a list of Dn objects, not strings so that: if dn in listWellknown is handled using a schema comparison (avoiding different case forms tripping it up). Andrew Bartlett Reviewed-by: Stefan Metzmacher <me...@samba.org> commit 0f247dce00fd26230cdb0566ce4f51a2ea8cfc2b Author: Andrew Bartlett <abart...@samba.org> Date: Mon Feb 18 15:15:52 2013 +1100 samba_upgradeprovision: do not maintain dnNotToRecalculate as a list We only need a boolean indication, not the actual values. Andrew Bartlett Reviewed-by: Stefan Metzmacher <me...@samba.org> commit 9bc32bfd65700c816ebb2a3004ad568327218f86 Author: Andrew Bartlett <abart...@samba.org> Date: Mon Feb 18 15:05:00 2013 +1100 samba_upgradeprovision: only run rebuild_sd in --full mode This is a potentially destructive routine, and should not be run by default. Andrew Bartlett Reviewed-by: Stefan Metzmacher <me...@samba.org> commit 81cda856faf2a5efd38965fd4c3b1f5551ad94d9 Author: Andrew Bartlett <abart...@samba.org> Date: Mon Feb 18 13:00:31 2013 +1100 samba_upgradeprovision: Remove alwaysRecalculate, this is too dangerous I am unclear on why this was added, but the idea that we ever always reset data in the directory is not reasonable to me, so I am removing it. Andrew Bartlett Reviewed-by: Stefan Metzmacher <me...@samba.org> commit 09b82d5fdc05a1f440aa96a690c202d4b0df134b Author: Andrew Bartlett <abart...@samba.org> Date: Mon Feb 18 12:28:23 2013 +1100 samba_upgradeprovision: Remove unused checkKeepAttributeOldMtd lastProvisionUSNs is never None, instead the code requries the administrator to populate this attribute in the directory. Andrew Bartlett Reviewed-by: Stefan Metzmacher <me...@samba.org> commit 9b8d5bba507615aee95a46fd9ae75aa782fd7e66 Author: Andrew Bartlett <abart...@samba.org> Date: Sun Feb 17 22:44:56 2013 +1100 samba_upgradeprovision: Remove inherited ACEs before comparing the SDs This avoids changing an SD when it is not really required. Andrew Bartlett Reviewed-by: Stefan Metzmacher <me...@samba.org> commit 5074b98714c9e038cc31872111508c1d92562841 Author: Andrew Bartlett <abart...@samba.org> Date: Sun Feb 17 22:03:18 2013 +1100 scripting: Rework samba.upgradehelpers.get_diff_sddls to be get_diff_sds This moves the SDDL conversion inside the get_diff_sds function and prepares for removing inherited ACEs from the SD before comparison. Andrew Bartlett Reviewed-by: Stefan Metzmacher <me...@samba.org> commit 787a6aacc3003731784b29fd92c683036c8730a7 Author: Andrew Bartlett <abart...@samba.org> Date: Sat Feb 16 21:58:57 2013 +1100 samba_upgradeprovision: Remove auto-detection of pre-alpha9 databases These are incredibly rare, and administrators running such databases not only ask the Samba Team for help personally, they can read --help. Andrew Bartlett Reviewed-by: Stefan Metzmacher <me...@samba.org> commit 9d6af4938f7bc80b10202d7055c2c32a483bbb5f Author: Andrew Bartlett <abart...@samba.org> Date: Sat Feb 16 13:12:53 2013 +1100 selftest: Rename samba4.blackbox.upgradeprovision.py to samba4.blackbox.upgradeprovision.current This name matches the other upgradeprovision tests for older saved provisions. Andrew Bartlett Reviewed-by: Stefan Metzmacher <me...@samba.org> commit 08f0562240155a871bd2a78d217db660e8ee3c91 Author: Andrew Bartlett <abart...@samba.org> Date: Sat Feb 16 01:07:27 2013 +1100 selftest: Run dbcheck and improved upgrdeprovision tests against release-4-0-0 The improved upgradeprovision tests now call ldapcmp to verify the changes made do actually bring the database in line with a fresh provision. Andrew Bartlett Reviewed-by: Stefan Metzmacher <me...@samba.org> commit d7936ee20c20635d62657cb821ff6dc4eb5fe33c Author: Andrew Bartlett <abart...@samba.org> Date: Sat Feb 16 01:08:20 2013 +1100 selftest: Add ldapcmp to ensure upgradeprovision of a fresh DB is a no-op Reviewed-by: Stefan Metzmacher <me...@samba.org> commit f1f36ad3517cd0e6bceb4b0cc37721a15be4d588 Author: Andrew Bartlett <abart...@samba.org> Date: Sat Feb 16 01:05:56 2013 +1100 selftest: Add in a provision from 4.0.0 to run tests against Reviewed-by: Stefan Metzmacher <me...@samba.org> commit 72f73ebaff8d75fc39770ec785964b0d3c9738cc Author: Andrew Bartlett <abart...@samba.org> Date: Thu Feb 14 15:01:10 2013 +1100 selftest: Do an ldapcmp run against the upgraded domain This checks (with a set of known issues marked in the --filter attribute) that the upgraded domain matches a fresh provision. Andrew Bartlett Reviewed-by: Stefan Metzmacher <me...@samba.org> commit 24c4d818d14c3931cf0cbff3070685fe409e66c6 Author: Andrew Bartlett <abart...@samba.org> Date: Thu Feb 14 15:00:01 2013 +1100 samba-tool ldapcmp: Add support for checking DNSDOMAIN and DNSFOREST by default Reviewed-by: Stefan Metzmacher <me...@samba.org> commit f508435d23445a8b3076f89cbe042e2da1ac0701 Author: Andrew Bartlett <abart...@samba.org> Date: Mon Feb 11 08:25:41 2013 +1100 samba-tool dbcheck: fix msDS-HasInstantiatedNCs attributes to match instanceType on our ntdsDSA This value is only a link to the local value of intanceType on our server, so only fix it for our server. Andrew Bartlett Reviewed-by: Stefan Metzmacher <me...@samba.org> commit 97389c3ec24526837e91fcfcaf7439491fcdb214 Author: Andrew Bartlett <abart...@samba.org> Date: Fri Mar 1 17:29:09 2013 +1100 scripting: Correct parsing of binary DN The DN is of the form B:8:01020304:DC=samba,DC=example,DC=com. We need to account for the case where the 8 is actually (say) 16, and so not just one character. Andrew Bartlett Signed-off-by: Stefan Metzmacher <me...@samba.org> commit 0180a027cbc9725ae13023ddfdb8079f147864c5 Author: Andrew Bartlett <abart...@samba.org> Date: Sat Feb 16 09:36:07 2013 +1100 subunit: Add a sh macro for skipping a test Reviewed-by: Stefan Metzmacher <me...@samba.org> ----------------------------------------------------------------------- Summary of changes: lib/tsocket/tsocket_bsd.c | 26 + python/samba/common.py | 2 +- python/samba/dbchecker.py | 36 +- python/samba/netcmd/ldapcmp.py | 8 +- python/samba/tests/upgradeprovision.py | 65 +- python/samba/upgradehelpers.py | 49 +- selftest/knownfail | 4 + selftest/tests.py | 6 +- source4/lib/socket/socket_unix.c | 2 +- source4/scripting/bin/samba_upgradeprovision | 177 +- .../provisions/release-4-0-0/etc/smb.conf.template | 17 + .../release-4-0-0/private}/dns_update_list | 0 .../provisions/release-4-0-0/private/eadb.tdb.dump | 96 + .../provisions/release-4-0-0/private/hklm.ldb.dump | 80 + .../release-4-0-0/private/idmap.ldb.dump | 48 + .../provisions/release-4-0-0/private/krb5.conf | 4 + .../release-4-0-0/private/privilege.ldb.dump | 156 + ...C%3DRELEASE-4-0-0,DC%3DSAMBA,DC%3DCORP.ldb.dump |28980 +++++++++++++ ...C%3DRELEASE-4-0-0,DC%3DSAMBA,DC%3DCORP.ldb.dump |43468 ++++++++++++++++++++ ...C%3DRELEASE-4-0-0,DC%3DSAMBA,DC%3DCORP.ldb.dump | 928 + ...C%3DRELEASE-4-0-0,DC%3DSAMBA,DC%3DCORP.ldb.dump | 488 + ...C%3DRELEASE-4-0-0,DC%3DSAMBA,DC%3DCORP.ldb.dump | 5736 +++ .../private/sam.ldb.d/metadata.tdb.dump | 4 + .../provisions/release-4-0-0/private/sam.ldb.dump | 40 + .../release-4-0-0/private/secrets.keytab | Bin 0 -> 1317 bytes .../release-4-0-0/private/secrets.ldb.dump | 44 + .../release-4-0-0/private/secrets.tdb.dump | 16 + .../release-4-0-0/private/share.ldb.dump | 32 + .../release-4-0-0/private}/spn_update_list | 0 .../{31B2F340-016D-11D2-945F-00C04FB984F9}/GPT.INI | 2 + .../{6AC1786C-016F-11D2-945F-00C04FB984F9}/GPT.INI | 2 + source4/selftest/tests.py | 2 +- source4/setup/tests/blackbox_upgradeprovision.sh | 42 +- testprogs/blackbox/dbcheck-alpha13.sh | 64 - testprogs/blackbox/dbcheck-oldrelease.sh | 65 + testprogs/blackbox/subunit.sh | 10 + testprogs/blackbox/upgradeprovision-alpha13.sh | 135 - testprogs/blackbox/upgradeprovision-oldrelease.sh | 212 + 38 files changed, 80676 insertions(+), 370 deletions(-) create mode 100644 source4/selftest/provisions/release-4-0-0/etc/smb.conf.template copy source4/{setup => selftest/provisions/release-4-0-0/private}/dns_update_list (100%) create mode 100644 source4/selftest/provisions/release-4-0-0/private/eadb.tdb.dump create mode 100644 source4/selftest/provisions/release-4-0-0/private/hklm.ldb.dump create mode 100644 source4/selftest/provisions/release-4-0-0/private/idmap.ldb.dump create mode 100644 source4/selftest/provisions/release-4-0-0/private/krb5.conf create mode 100644 source4/selftest/provisions/release-4-0-0/private/privilege.ldb.dump create mode 100644 source4/selftest/provisions/release-4-0-0/private/sam.ldb.d/CN%3DCONFIGURATION,DC%3DRELEASE-4-0-0,DC%3DSAMBA,DC%3DCORP.ldb.dump create mode 100644 source4/selftest/provisions/release-4-0-0/private/sam.ldb.d/CN%3DSCHEMA,CN%3DCONFIGURATION,DC%3DRELEASE-4-0-0,DC%3DSAMBA,DC%3DCORP.ldb.dump create mode 100644 source4/selftest/provisions/release-4-0-0/private/sam.ldb.d/DC%3DDOMAINDNSZONES,DC%3DRELEASE-4-0-0,DC%3DSAMBA,DC%3DCORP.ldb.dump create mode 100644 source4/selftest/provisions/release-4-0-0/private/sam.ldb.d/DC%3DFORESTDNSZONES,DC%3DRELEASE-4-0-0,DC%3DSAMBA,DC%3DCORP.ldb.dump create mode 100644 source4/selftest/provisions/release-4-0-0/private/sam.ldb.d/DC%3DRELEASE-4-0-0,DC%3DSAMBA,DC%3DCORP.ldb.dump create mode 100644 source4/selftest/provisions/release-4-0-0/private/sam.ldb.d/metadata.tdb.dump create mode 100644 source4/selftest/provisions/release-4-0-0/private/sam.ldb.dump create mode 100644 source4/selftest/provisions/release-4-0-0/private/secrets.keytab create mode 100644 source4/selftest/provisions/release-4-0-0/private/secrets.ldb.dump create mode 100644 source4/selftest/provisions/release-4-0-0/private/secrets.tdb.dump create mode 100644 source4/selftest/provisions/release-4-0-0/private/share.ldb.dump copy source4/{setup => selftest/provisions/release-4-0-0/private}/spn_update_list (100%) create mode 100644 source4/selftest/provisions/release-4-0-0/sysvol/release-4-0-0.samba.corp/Policies/{31B2F340-016D-11D2-945F-00C04FB984F9}/GPT.INI create mode 100644 source4/selftest/provisions/release-4-0-0/sysvol/release-4-0-0.samba.corp/Policies/{6AC1786C-016F-11D2-945F-00C04FB984F9}/GPT.INI delete mode 100755 testprogs/blackbox/dbcheck-alpha13.sh create mode 100755 testprogs/blackbox/dbcheck-oldrelease.sh delete mode 100755 testprogs/blackbox/upgradeprovision-alpha13.sh create mode 100755 testprogs/blackbox/upgradeprovision-oldrelease.sh Changeset truncated at 500 lines: diff --git a/lib/tsocket/tsocket_bsd.c b/lib/tsocket/tsocket_bsd.c index 56dff68..4b54d31 100644 --- a/lib/tsocket/tsocket_bsd.c +++ b/lib/tsocket/tsocket_bsd.c @@ -1102,6 +1102,32 @@ static void tdgram_bsd_sendto_handler(void *private_data) /* retry later */ return; } + + if (err == EMSGSIZE) { + /* round up in 1K increments */ + int bufsize = ((state->len + 1023) & (~1023)); + + ret = setsockopt(bsds->fd, SOL_SOCKET, SO_SNDBUF, &bufsize, + sizeof(bufsize)); + if (ret == 0) { + /* + * We do the rety here, rather then via the + * handler, as we only want to retry once for + * this condition, so if there is a mismatch + * between what setsockopt() accepts and what can + * actually be sent, we do not end up in a + * loop. + */ + + ret = sendto(bsds->fd, state->buf, state->len, + 0, sa, sa_socklen); + err = tsocket_bsd_error_from_errno(ret, errno, &retry); + if (retry) { /* retry later */ + return; + } + } + } + if (tevent_req_error(req, err)) { return; } diff --git a/python/samba/common.py b/python/samba/common.py index e47f276..c2a3584 100644 --- a/python/samba/common.py +++ b/python/samba/common.py @@ -81,7 +81,7 @@ class dsdb_Dn(object): raise RuntimeError("Invalid DN %s" % dnstring) prefix_len = 4 + len(colons[1]) + int(colons[1]) self.prefix = dnstring[0:prefix_len] - self.binary = self.prefix[4:-1] + self.binary = self.prefix[3+len(colons[1]):-1] self.dnstring = dnstring[prefix_len:] else: self.dnstring = dnstring diff --git a/python/samba/dbchecker.py b/python/samba/dbchecker.py index 297a065..fd42a78 100644 --- a/python/samba/dbchecker.py +++ b/python/samba/dbchecker.py @@ -43,6 +43,7 @@ class dbcheck(object): self.remove_all_empty_attributes = False self.fix_all_normalisation = False self.fix_all_DN_GUIDs = False + self.fix_all_binary_dn = False self.remove_all_deleted_DN_links = False self.fix_all_target_mismatch = False self.fix_all_metadata = False @@ -59,7 +60,7 @@ class dbcheck(object): self.naming_dn = ldb.Dn(samdb, "CN=Partitions,%s" % samdb.get_config_basedn()) self.schema_dn = samdb.get_schema_basedn() self.rid_dn = ldb.Dn(samdb, "CN=RID Manager$,CN=System," + samdb.domain_dn()) - self.ntds_dsa = samdb.get_dsServiceName() + self.ntds_dsa = ldb.Dn(samdb, samdb.get_dsServiceName()) self.class_schemaIDGUID = {} res = self.samdb.search(base=self.ntds_dsa, scope=ldb.SCOPE_BASE, attrs=['msDS-hasMasterNCs', 'hasMasterNCs']) @@ -283,6 +284,23 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base))) "Failed to fix %s on attribute %s" % (errstr, attrname)): self.report("Fixed %s on attribute %s" % (errstr, attrname)) + def err_incorrect_binary_dn(self, dn, attrname, val, dsdb_dn, errstr): + """handle an incorrect binary DN component""" + self.report("ERROR: %s binary component for %s in object %s - %s" % (errstr, attrname, dn, val)) + controls=["extended_dn:1:1", "show_recycled:1"] + + if not self.confirm_all('Change DN to %s?' % str(dsdb_dn), 'fix_all_binary_dn'): + self.report("Not fixing %s" % errstr) + return + m = ldb.Message() + m.dn = dn + m['old_value'] = ldb.MessageElement(val, ldb.FLAG_MOD_DELETE, attrname) + m['new_value'] = ldb.MessageElement(str(dsdb_dn), ldb.FLAG_MOD_ADD, attrname) + + if self.do_modify(m, ["show_recycled:1"], + "Failed to fix %s on attribute %s" % (errstr, attrname)): + self.report("Fixed %s on attribute %s" % (errstr, attrname)) + def err_dn_target_mismatch(self, dn, attrname, val, dsdb_dn, correct_dn, errstr): """handle a DN string being incorrect""" self.report("ERROR: incorrect DN string component for %s in object %s - %s" % (attrname, dn, val)) @@ -449,6 +467,13 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base))) guidstr = str(misc.GUID(guid)) attrs = ['isDeleted'] + + if (str(attrname).lower() == 'msds-hasinstantiatedncs') and (obj.dn == self.ntds_dsa): + fixing_msDS_HasInstantiatedNCs = True + attrs.append("instanceType") + else: + fixing_msDS_HasInstantiatedNCs = False + linkID = self.samdb_schema.get_linkId_from_lDAPDisplayName(attrname) reverse_link_name = self.samdb_schema.get_backlink_from_lDAPDisplayName(attrname) if reverse_link_name is not None: @@ -463,6 +488,15 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base))) self.err_incorrect_dn_GUID(obj.dn, attrname, val, dsdb_dn, "incorrect GUID") continue + if fixing_msDS_HasInstantiatedNCs: + dsdb_dn.prefix = "B:8:%08X:" % int(res[0]['instanceType'][0]) + dsdb_dn.binary = "%08X" % int(res[0]['instanceType'][0]) + + if str(dsdb_dn) != val: + error_count +=1 + self.err_incorrect_binary_dn(obj.dn, attrname, val, dsdb_dn, "incorrect instanceType part of Binary DN") + continue + # now we have two cases - the source object might or might not be deleted is_deleted = 'isDeleted' in obj and obj['isDeleted'][0].upper() == 'TRUE' target_is_deleted = 'isDeleted' in res[0] and res[0]['isDeleted'][0].upper() == 'TRUE' diff --git a/python/samba/netcmd/ldapcmp.py b/python/samba/netcmd/ldapcmp.py index 3c6c5f1..6e025a2 100644 --- a/python/samba/netcmd/ldapcmp.py +++ b/python/samba/netcmd/ldapcmp.py @@ -882,7 +882,7 @@ class cmd_ldapcmp(Command): "credopts": options.CredentialsOptionsDouble, } - takes_args = ["URL1", "URL2", "context1?", "context2?", "context3?"] + takes_args = ["URL1", "URL2", "context1?", "context2?", "context3?", "context4?", "context5?"] takes_options = [ Option("-w", "--two", dest="two", action="store_true", default=False, @@ -910,7 +910,7 @@ class cmd_ldapcmp(Command): ] def run(self, URL1, URL2, - context1=None, context2=None, context3=None, + context1=None, context2=None, context3=None, context4=None, context5=None, two=False, quiet=False, verbose=False, descriptor=False, sort_aces=False, view="section", base="", base2="", scope="SUB", filter="", credopts=None, sambaopts=None, versionopts=None, skip_missing_dn=False): @@ -941,9 +941,9 @@ class cmd_ldapcmp(Command): contexts = ["DOMAIN"] else: # if no argument given, we compare all contexts - contexts = ["DOMAIN", "CONFIGURATION", "SCHEMA"] + contexts = ["DOMAIN", "CONFIGURATION", "SCHEMA", "DNSDOMAIN", "DNSFOREST"] else: - for c in [context1, context2, context3]: + for c in [context1, context2, context3, context4, context5]: if c is None: continue if not c.upper() in ["DOMAIN", "CONFIGURATION", "SCHEMA", "DNSDOMAIN", "DNSFOREST"]: diff --git a/python/samba/tests/upgradeprovision.py b/python/samba/tests/upgradeprovision.py index 93a6731..bc3509e 100644 --- a/python/samba/tests/upgradeprovision.py +++ b/python/samba/tests/upgradeprovision.py @@ -19,7 +19,7 @@ import os from samba.upgradehelpers import (usn_in_range, dn_sort, - get_diff_sddls, update_secrets, + get_diff_sds, update_secrets, construct_existor_expr) from samba.tests.provision import create_dummy_secretsdb @@ -27,6 +27,7 @@ from samba.tests import TestCaseInTempDir from samba import Ldb from ldb import SCOPE_BASE import samba.tests +from samba.dcerpc import security def dummymessage(a=None, b=None): pass @@ -59,33 +60,53 @@ class UpgradeProvisionTestCase(TestCaseInTempDir): self.assertEquals(dn_sort("cn=bar, dc=toto,dc=tata", "cn=foo, dc=toto,dc=tata"), -1) - def test_get_diff_sddl(self): - sddl = "O:SAG:DUD:AI(A;CIID;RPWPCRCCLCLORCWOWDSW;;;SA)\ -(A;CIID;RP LCLORC;;;AU)(A;CIID;RPWPCRCCDCLCLORCWOWDSDDTSW;;;SY)S:AI(AU;CIIDSA;WP;;;WD)" - sddl1 = "O:SAG:DUD:AI(A;CIID;RPWPCRCCLCLORCWOWDSW;;;SA)\ -(A;CIID;RP LCLORC;;;AU)(A;CIID;RPWPCRCCDCLCLORCWOWDSDDTSW;;;SY)S:AI(AU;CIIDSA;WP;;;WD)" - sddl2 = "O:BAG:DUD:AI(A;CIID;RPWPCRCCLCLORCWOWDSW;;;SA)\ -(A;CIID;RP LCLORC;;;AU)(A;CIID;RPWPCRCCDCLCLORCWOWDSDDTSW;;;SY)S:AI(AU;CIIDSA;WP;;;WD)" - sddl3 = "O:SAG:BAD:AI(A;CIID;RPWPCRCCLCLORCWOWDSW;;;SA)\ -(A;CIID;RP LCLORC;;;AU)(A;CIID;RPWPCRCCDCLCLORCWOWDSDDTSW;;;SY)S:AI(AU;CIIDSA;WP;;;WD)" - sddl4 = "O:SAG:DUD:AI(A;CIID;RPWPCRCCLCLORCWOWDSW;;;BA)\ -(A;CIID;RP LCLORC;;;AU)(A;CIID;RPWPCRCCDCLCLORCWOWDSDDTSW;;;SY)S:AI(AU;CIIDSA;WP;;;WD)" - sddl5 = "O:SAG:DUD:AI(A;CIID;RPWPCRCCLCLORCWOWDSW;;;SA)\ -(A;CIID;RP LCLORC;;;AU)(A;CIID;RPWPCRCCDCLCLORCWOWDSDDTSW;;;SY)" - - self.assertEquals(get_diff_sddls(sddl, sddl1), "") - txt = get_diff_sddls(sddl, sddl2) + def test_get_diff_sds(self): + domsid = security.dom_sid('S-1-5-21') + + sddl = "O:SAG:DUD:AI(A;CI;RPWPCRCCLCLORCWOWDSW;;;SA)\ +(A;CI;RP LCLORC;;;AU)(A;CI;RPWPCRCCDCLCLORCWOWDSDDTSW;;;SY)S:AI(AU;CISA;WP;;;WD)" + sddl1 = "O:SAG:DUD:AI(A;CI;RPWPCRCCLCLORCWOWDSW;;;SA)\ +(A;CI;RP LCLORC;;;AU)(A;CI;RPWPCRCCDCLCLORCWOWDSDDTSW;;;SY)S:AI(AU;CISA;WP;;;WD)" + sddl2 = "O:BAG:DUD:AI(A;CI;RPWPCRCCLCLORCWOWDSW;;;SA)\ +(A;CI;RP LCLORC;;;AU)(A;CI;RPWPCRCCDCLCLORCWOWDSDDTSW;;;SY)S:AI(AU;CISA;WP;;;WD)" + sddl3 = "O:SAG:BAD:AI(A;CI;RPWPCRCCLCLORCWOWDSW;;;SA)\ +(A;CI;RP LCLORC;;;AU)(A;CI;RPWPCRCCDCLCLORCWOWDSDDTSW;;;SY)S:AI(AU;CISA;WP;;;WD)" + sddl4 = "O:SAG:DUD:AI(A;CI;RPWPCRCCLCLORCWOWDSW;;;BA)\ +(A;CI;RP LCLORC;;;AU)(A;CI;RPWPCRCCDCLCLORCWOWDSDDTSW;;;SY)S:AI(AU;CISA;WP;;;WD)" + sddl5 = "O:SAG:DUD:AI(A;CI;RPWPCRCCLCLORCWOWDSW;;;SA)\ +(A;CI;RP LCLORC;;;AU)(A;CI;RPWPCRCCDCLCLORCWOWDSDDTSW;;;SY)" + sddl6 = "O:SAG:DUD:AI(A;CIID;RPWPCRCCLCLORCWOWDSW;;;SA)\ +(A;CIID;RP LCLORC;;;AU)(A;CIID;RPWPCRCCDCLCLORCWOWDSDDTSW;;;SY)\ +(A;CI;RPWPCRCCLCLORCWOWDSW;;;SA)\ +(A;CI;RP LCLORC;;;AU)(A;CI;RPWPCRCCDCLCLORCWOWDSDDTSW;;;SY)S:AI(AU;CISA;WP;;;WD)(AU;CIIDSA;WP;;;WD)" + + self.assertEquals(get_diff_sds(security.descriptor.from_sddl(sddl, domsid), + security.descriptor.from_sddl(sddl1, domsid), + domsid), "") + txt = get_diff_sds(security.descriptor.from_sddl(sddl, domsid), + security.descriptor.from_sddl(sddl2, domsid), + domsid) self.assertEquals(txt, "\tOwner mismatch: SA (in ref) BA(in current)\n") - txt = get_diff_sddls(sddl, sddl3) + txt = get_diff_sds(security.descriptor.from_sddl(sddl, domsid), + security.descriptor.from_sddl(sddl3, domsid), + domsid) self.assertEquals(txt, "\tGroup mismatch: DU (in ref) BA(in current)\n") - txt = get_diff_sddls(sddl, sddl4) + txt = get_diff_sds(security.descriptor.from_sddl(sddl, domsid), + security.descriptor.from_sddl(sddl4, domsid), + domsid) txtmsg = "\tPart dacl is different between reference and current here\ - is the detail:\n\t\t(A;CIID;RPWPCRCCLCLORCWOWDSW;;;BA) ACE is not present in\ - the reference\n\t\t(A;CIID;RPWPCRCCLCLORCWOWDSW;;;SA) ACE is not present in\ + is the detail:\n\t\t(A;CI;RPWPCRCCLCLORCWOWDSW;;;BA) ACE is not present in\ + the reference\n\t\t(A;CI;RPWPCRCCLCLORCWOWDSW;;;SA) ACE is not present in\ the current\n" self.assertEquals(txt, txtmsg) - txt = get_diff_sddls(sddl, sddl5) + + txt = get_diff_sds(security.descriptor.from_sddl(sddl, domsid), + security.descriptor.from_sddl(sddl5, domsid), + domsid) self.assertEquals(txt, "\tCurrent ACL hasn't a sacl part\n") + self.assertEquals(get_diff_sds(security.descriptor.from_sddl(sddl, domsid), + security.descriptor.from_sddl(sddl6, domsid), + domsid), "") def test_construct_existor_expr(self): res = construct_existor_expr([]) diff --git a/python/samba/upgradehelpers.py b/python/samba/upgradehelpers.py index 1ec19d4..298e767 100644 --- a/python/samba/upgradehelpers.py +++ b/python/samba/upgradehelpers.py @@ -33,7 +33,7 @@ from samba.provision import (provision_paths_from_lp, getpolicypath, set_gpos_acl, create_gpo_struct, FILL_FULL, provision, ProvisioningError, setsysvolacl, secretsdb_self_join) -from samba.dcerpc import xattr, drsblobs +from samba.dcerpc import xattr, drsblobs, security from samba.dcerpc.misc import SEC_CHAN_BDC from samba.ndr import ndr_unpack from samba.samdb import SamDB @@ -346,8 +346,8 @@ def chunck_sddl(sddl): return hash -def get_diff_sddls(refsddl, cursddl, checkSacl = True): - """Get the difference between 2 sddl +def get_clean_sd(sd): + """Get the SD without difference between 2 sddl This function split the textual representation of ACL into smaller chunck in order to not to report a simple permutation as a difference @@ -358,6 +358,49 @@ def get_diff_sddls(refsddl, cursddl, checkSacl = True): :return: A string that explain difference between sddls """ + sd_clean = security.descriptor() + sd_clean.owner_sid = sd.owner_sid + sd_clean.group_sid = sd.group_sid + sd_clean.type = sd.type + sd_clean.revision = sd.revision + + aces = [] + if sd.sacl is not None: + aces = sd.sacl.aces + for i in range(0, len(aces)): + ace = aces[i] + + if not ace.flags & security.SEC_ACE_FLAG_INHERITED_ACE: + sd_clean.sacl_add(ace) + continue + + aces = [] + if sd.dacl is not None: + aces = sd.dacl.aces + for i in range(0, len(aces)): + ace = aces[i] + + if not ace.flags & security.SEC_ACE_FLAG_INHERITED_ACE: + sd_clean.dacl_add(ace) + continue + return sd_clean + + +def get_diff_sds(refsd, cursd, domainsid, checkSacl = True): + """Get the difference between 2 sd + + This function split the textual representation of ACL into smaller + chunck in order to not to report a simple permutation as a difference + + :param refsddl: First sddl to compare + :param cursddl: Second sddl to compare + :param checkSacl: If false we skip the sacl checks + :return: A string that explain difference between sddls + """ + + cursddl = get_clean_sd(cursd).as_sddl(domainsid) + refsddl = get_clean_sd(refsd).as_sddl(domainsid) + txt = "" hash_cur = chunck_sddl(cursddl) hash_ref = chunck_sddl(refsddl) diff --git a/selftest/knownfail b/selftest/knownfail index 39485af..180a543 100644 --- a/selftest/knownfail +++ b/selftest/knownfail @@ -159,6 +159,10 @@ ^samba4.rpc.lsa.forest.trust #Not fully provided by Samba4 ^samba4.blackbox.kinit\(.*\).kinit with user password for expired password\(.*\) # We need to work out why this fails only during the pw change ^samba4.blackbox.dbcheck\(vampire_dc\).dbcheck\(vampire_dc:local\) # Due to replicating with --domain-critical-only we fail dbcheck on this database +^samba4.blackbox.upgradeprovision.alpha13.ldapcmp_sd\(none\) # Due to something rewriting the NT ACL on DNS objects and not getting the DC ACL right +^samba4.blackbox.upgradeprovision.alpha13.ldapcmp_full_sd\(none\) # Due to something rewriting the NT ACL on DNS objects and not getting the DC ACL right +^samba4.blackbox.upgradeprovision.release-4-0-0.ldapcmp_sd\(none\) # Due to something rewriting the NT ACL on DNS objects and not getting the DC ACL right +^samba4.blackbox.upgradeprovision.release-4-0-0.ldapcmp_full_sd\(none\) # Due to something rewriting the NT ACL on DNS objects and not getting the DC ACL right ^samba3.smb2.create.gentest ^samba3.smb2.create.blob ^samba3.smb2.create.open diff --git a/selftest/tests.py b/selftest/tests.py index 03bedfc..9a59e9d 100644 --- a/selftest/tests.py +++ b/selftest/tests.py @@ -45,8 +45,10 @@ planpythontestsuite("none", "samba.tests.hostconfig") planpythontestsuite("none", "samba.tests.messaging") planpythontestsuite("none", "samba.tests.samba3sam") planpythontestsuite("none", "wafsamba.tests.test_suite", extra_path=[os.path.join(samba4srcdir, "..", "buildtools"), os.path.join(samba4srcdir, "..", "buildtools", "wafadmin")]) -plantestsuite("samba4.blackbox.dbcheck.alpha13", "none" , ["PYTHON=%s" % python, os.path.join(bbdir, "dbcheck-alpha13.sh"), '$PREFIX_ABS/provision', configuration]) -plantestsuite("samba4.blackbox.upgradeprovision.alpha13", "none" , ["PYTHON=%s" % python, os.path.join(bbdir, "upgradeprovision-alpha13.sh"), '$PREFIX_ABS/provision', configuration]) +plantestsuite("samba4.blackbox.dbcheck.alpha13", "none" , ["PYTHON=%s" % python, os.path.join(bbdir, "dbcheck-oldrelease.sh"), '$PREFIX_ABS/provision', 'alpha13', configuration]) +plantestsuite("samba4.blackbox.dbcheck.release-4-0-0", "none" , ["PYTHON=%s" % python, os.path.join(bbdir, "dbcheck-oldrelease.sh"), '$PREFIX_ABS/provision', 'release-4-0-0', configuration]) +plantestsuite("samba4.blackbox.upgradeprovision.alpha13", "none" , ["PYTHON=%s" % python, os.path.join(bbdir, "upgradeprovision-oldrelease.sh"), '$PREFIX_ABS/provision', 'alpha13', configuration]) +plantestsuite("samba4.blackbox.upgradeprovision.release-4-0-0", "none" , ["PYTHON=%s" % python, os.path.join(bbdir, "upgradeprovision-oldrelease.sh"), '$PREFIX_ABS/provision', 'release-4-0-0', configuration]) planpythontestsuite("none", "samba.tests.upgradeprovision") planpythontestsuite("none", "samba.tests.xattr") planpythontestsuite("none", "samba.tests.ntacls") diff --git a/source4/lib/socket/socket_unix.c b/source4/lib/socket/socket_unix.c index 049e570..0774b12 100644 --- a/source4/lib/socket/socket_unix.c +++ b/source4/lib/socket/socket_unix.c @@ -295,7 +295,7 @@ static NTSTATUS unixdom_sendto(struct socket_context *sock, if (setsockopt(sock->fd, SOL_SOCKET, SO_SNDBUF, &bufsize, sizeof(bufsize)) == -1) { - return map_nt_error_from_unix_common(errno); + return map_nt_error_from_unix_common(EMSGSIZE); } len = sendto(sock->fd, blob->data, blob->length, 0, sa, sa_len); } diff --git a/source4/scripting/bin/samba_upgradeprovision b/source4/scripting/bin/samba_upgradeprovision index 25c3ac2..8e7d792 100755 --- a/source4/scripting/bin/samba_upgradeprovision +++ b/source4/scripting/bin/samba_upgradeprovision @@ -75,7 +75,7 @@ from samba.dcerpc.security import ( from samba.ndr import ndr_unpack from samba.upgradehelpers import (dn_sort, get_paths, newprovision, get_ldbs, findprovisionrange, - usn_in_range, identic_rename, get_diff_sddls, + usn_in_range, identic_rename, get_diff_sds, update_secrets, CHANGE, ERROR, SIMPLE, CHANGEALL, GUESS, CHANGESD, PROVISION, updateOEMInfo, getOEMInfo, update_gpo, @@ -149,7 +149,7 @@ hashOverwrittenAtt = { "prefixMap": replace, "systemMayContain": replace, "attributeDisplayNames": replace + add, "versionNumber": add} -dnNotToRecalculate = [] +dnNotToRecalculateFound = False dnToRecalculate = [] backlinked = [] forwardlinked = set() @@ -191,6 +191,8 @@ parser.add_option("--db_backup_only", action="store_true", help="Do the backup of the database in the provision, skip the sysvol / netlogon shares") parser.add_option("--full", action="store_true", help="Perform full upgrade of the samdb (schema, configuration, new objects, ...") +parser.add_option("--very-old-pre-alpha9", action="store_true", + help="Perform additional forced SD resets required for a database from before Samba 4.0.0alpha9.") opts = parser.parse_args()[0] @@ -830,68 +832,6 @@ def handle_links(samdb, att, basedn, dn, value, ref_value, delta): return delta -msg_elt_flag_strs = { - ldb.FLAG_MOD_ADD: "MOD_ADD", - ldb.FLAG_MOD_REPLACE: "MOD_REPLACE", - ldb.FLAG_MOD_DELETE: "MOD_DELETE" } - -def checkKeepAttributeOldMtd(delta, att, reference, current, - basedn, samdb): - """ Check if we should keep the attribute modification or not. - This function didn't use replicationMetadata to take a decision. - - :param delta: A message diff object - :param att: An attribute - :param reference: A message object for the current entry comming from - the reference provision. - :param current: A message object for the current entry commin from - the current provision. - :param basedn: The DN of the partition - :param samdb: A ldb connection to the sam database of the current provision. - - :return: The modified message diff. - """ - # Old school way of handling things for pre alpha12 upgrade - global defSDmodified - isFirst = False - txt = "" - dn = current[0].dn - - for att in list(delta): - msgElt = delta.get(att) - - if att == "nTSecurityDescriptor": - defSDmodified = True - delta.remove(att) - continue - - if att == "dn": - continue - - if not hashOverwrittenAtt.has_key(att): - if msgElt.flags() != FLAG_MOD_ADD: - if not handle_special_case(att, delta, reference, current, - False, basedn, samdb): - if opts.debugchange or opts.debugall: - try: - dump_denied_change(dn, att, - msg_elt_flag_strs[msgElt.flags()], - current[0][att], reference[0][att]) - except KeyError: - dump_denied_change(dn, att, - msg_elt_flag_strs[msgElt.flags()], - current[0][att], None) - delta.remove(att) - continue - else: - if hashOverwrittenAtt.get(att)&2**msgElt.flags() : - continue - elif hashOverwrittenAtt.get(att) == never: - delta.remove(att) - continue - - return delta - def checkKeepAttributeWithMetadata(delta, att, message, reference, current, hash_attr_usn, basedn, usns, samdb): """ Check if we should keep the attribute modification or not @@ -961,12 +901,10 @@ def checkKeepAttributeWithMetadata(delta, att, message, reference, current, if att == "nTSecurityDescriptor": cursd = ndr_unpack(security.descriptor, str(current[0]["nTSecurityDescriptor"])) - cursddl = cursd.as_sddl(names.domainsid) refsd = ndr_unpack(security.descriptor, str(reference[0]["nTSecurityDescriptor"])) - refsddl = refsd.as_sddl(names.domainsid) - diff = get_diff_sddls(refsddl, cursddl) + diff = get_diff_sds(refsd, cursd, names.domainsid) if diff == "": # FIXME find a way to have it only with huge huge verbose mode # message(CHANGE, "%ssd are identical" % txt) @@ -981,9 +919,9 @@ def checkKeepAttributeWithMetadata(delta, att, message, reference, current, message(CHANGESD, "But the SD has been changed by someonelse " "so it's impossible to know if the difference" " cames from the modification or from a previous bug") - dnNotToRecalculate.append(str(dn)) + dnNotToRecalculateFound = True else: - dnToRecalculate.append(str(dn)) + dnToRecalculate.append(dn) continue if attrUSN == -1: -- Samba Shared Repository