The branch, v4-3-test has been updated
via 0befbeb samr4: Use <SID=%s> in GetGroupsForUser
via c7d8aca python/tests: Add more assertions that we get back the
value we expect
via cbb9a2a python/tests: Add tests for 64 bit signed integers
via 215c284 pidl/python: also add a ndr_PyLong_FromLongLong() for
symnetric reasons
via 14930fe pidl/python: Provide static inline helper function
ndr_PyLong_FromUnsignedLongLong
via bd44d03 pidl/python: Calculate maximum integer values using a
lookup table
via d2f2df6 selftest: Add assertion that we actually fix the
replPropertyMetaData sort order
via 47af9ce selftest: Add in steps to re-create this database
via 74043c5 Update release-4-1-0rc3 to include data using schema
modifications
via b5c196a ldb: create a cache of known wellknown objects instead of
continously searching in the db
via e120381 dbcheck: Use set() operations to make dbcheck more efficient
via 9a8165b dbcheck: Try to avoid duplicate searches
via 3c29480 dbcheck: Add additional tests for the attributeID list
via df141f7 dbcheck: Add explict tests for unknown and unsorted
attributeID values
via 429db95 pidl: Assert that python arrays will not overflow the C
array
via 1e432b9 pydsdb: Allow the full range of uint32_t values for
attributeID
via 43c6f8c python/tests: Add tests for integer overflow handling
via c9b80d7 pidl: Change PIDL to correctly use and validate python
integer types
via a779039 python: Use an unsigned integer for buf_size, not -1
from c9ed862 WHATSNEW: Prepare release notes for Samba 4.3.0rc5
https://git.samba.org/?p=samba.git;a=shortlog;h=v4-3-test
- Log -----------------------------------------------------------------
commit 0befbeb33bf3ebb189639598281e989178efbc4a
Author: Volker Lendecke <v...@samba.org>
Date: Tue Sep 1 08:41:04 2015 +0200
samr4: Use <SID=%s> in GetGroupsForUser
This way we avoid quoting problems in user's DNs
Signed-off-by: Volker Lendecke <v...@samba.org>
Reviewed-by: Jeremy Allison <j...@samba.org>
Autobuild-User(master): Jeremy Allison <j...@samba.org>
Autobuild-Date(master): Tue Sep 1 23:49:14 CEST 2015 on sn-devel-104
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11488
(cherry picked from commit 841845dea35089a187fd1626c9752d708989ac7b)
Autobuild-User(v4-3-test): Stefan Metzmacher <me...@samba.org>
Autobuild-Date(v4-3-test): Thu Sep 3 12:14:12 CEST 2015 on sn-devel-104
commit c7d8aca2de88b72e69921558e5cf708177ecd5bc
Author: Andrew Bartlett <abart...@samba.org>
Date: Tue Sep 1 15:00:30 2015 +1200
python/tests: Add more assertions that we get back the value we expect
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11429
Signed-off-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Stefan Metzmacher <me...@samba.org>
Autobuild-User(master): Stefan Metzmacher <me...@samba.org>
Autobuild-Date(master): Tue Sep 1 17:00:53 CEST 2015 on sn-devel-104
(cherry picked from commit 5aefea842528d053b86b50ff2ed9047db1ca4594)
commit cbb9a2a65f900d961a50b9e1c51c20b8208a6b03
Author: Andrew Bartlett <abart...@samba.org>
Date: Tue Sep 1 14:58:20 2015 +1200
python/tests: Add tests for 64 bit signed integers
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11429
Signed-off-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Stefan Metzmacher <me...@samba.org>
(cherry picked from commit 1f50e194517b84ccc8d0208d563e83dabfb2327a)
commit 215c284eaded01e251b07e5db4cd5d04d6812601
Author: Stefan Metzmacher <me...@samba.org>
Date: Tue Sep 1 10:30:49 2015 +0200
pidl/python: also add a ndr_PyLong_FromLongLong() for symnetric reasons
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11429
Signed-off-by: Stefan Metzmacher <me...@samba.org>
Reviewed-by: Stefan Metzmacher <me...@samba.org>
(cherry picked from commit e6fbeb860638ad1113914b9460a618025d950d08)
commit 14930febe33b17ba5abf877a5ad4d7ffc55740b8
Author: Andrew Bartlett <abart...@samba.org>
Date: Tue Sep 1 14:33:35 2015 +1200
pidl/python: Provide static inline helper function
ndr_PyLong_FromUnsignedLongLong
This should isolate any coverity warnings on 64-bit platforms
(where LONG_MAX is larger than any possible 32 bit value) to
a single spot, or possibly eliminate it.
This is needed for the unsigned 64 bit case, and on 32 bit
systems, as PyInt_FromLong is limited to a signed "long" int.
The compiler should be able to eliminate many of these calls
with the embedded type knowlege.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11429
Signed-off-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Stefan Metzmacher <me...@samba.org>
(cherry picked from commit d1416d65a3cc61e4e56d1a43bb634d12f418ba0e)
commit bd44d03159d7ba2048b1fd27e2623fe4bb7ec773
Author: Andrew Bartlett <abart...@samba.org>
Date: Fri Aug 28 11:46:56 2015 +1200
pidl/python: Calculate maximum integer values using a lookup table
This avoids a << of 64 bits in the unused end of the conditional expression.
This was flagged by Coverity and the fix was suggested by metze.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11429
Andrew Bartlett
Signed-off-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Stefan Metzmacher <me...@samba.org>
(cherry picked from commit c2f4e324d9c1ced2e1da859594ef67ae9f645919)
commit d2f2df68471085c0ac01d26be9a406f7fe781a7b
Author: Andrew Bartlett <abart...@samba.org>
Date: Mon Aug 3 13:50:08 2015 +1200
selftest: Add assertion that we actually fix the replPropertyMetaData sort
order
This ensures that the dbcheck rule fixes the sort order (and only fixes the
sort order).
BUG: https://bugzilla.samba.org/show_bug.cgi?id=10973
Signed-off-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Stefan Metzmacher <me...@samba.org>
Autobuild-User(master): Stefan Metzmacher <me...@samba.org>
Autobuild-Date(master): Tue Aug 25 02:45:58 CEST 2015 on sn-devel-104
(cherry picked from commit dcc657a2219498beac8bde6cbf999ee7cf6acae8)
commit 47af9ce98ad852f66c69a054a768429839d8bfdd
Author: Andrew Bartlett <abart...@samba.org>
Date: Mon Aug 3 11:25:02 2015 +1200
selftest: Add in steps to re-create this database
This may assist if this needs to be changed again
BUG: https://bugzilla.samba.org/show_bug.cgi?id=10973
Signed-off-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Stefan Metzmacher <me...@samba.org>
(cherry picked from commit 5504502aa68f4901f52dc2e8f7ee8b3a9c74546e)
commit 74043c5edc19be602b12d54dcf261ea58290d0f6
Author: Andrew Bartlett <abart...@samba.org>
Date: Mon Aug 3 11:24:10 2015 +1200
Update release-4-1-0rc3 to include data using schema modifications
This allows us to know that the previous patches are correct.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=10973
Signed-off-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Stefan Metzmacher <me...@samba.org>
(cherry picked from commit a6957ba5da3130994d2f35328aea6f4cd604147a)
commit b5c196ac277a756159a03aa02c0ba424247491f5
Author: Matthieu Patou <m...@matws.net>
Date: Mon May 25 09:17:55 2015 -0700
ldb: create a cache of known wellknown objects instead of continously
searching in the db
Profiling on dbcheck have shown that we spend 10% of the time looking
for wellknown objects.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=10973
Change-Id: I13ed58e8062d1b7b6179d17b0e7e56f943572c6c
Signed-off-by: Matthieu Patou <m...@matws.net>
Reviewed-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Stefan Metzmacher <me...@samba.org>
(cherry picked from commit 6122acad0f1a7bc23b6f58862c16968e13da979d)
commit e120381231056277a604fb5a73d5847439cd1851
Author: Andrew Bartlett <abart...@samba.org>
Date: Mon Jul 27 15:11:56 2015 +1200
dbcheck: Use set() operations to make dbcheck more efficient
BUG: https://bugzilla.samba.org/show_bug.cgi?id=10973
Signed-off-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Stefan Metzmacher <me...@samba.org>
(cherry picked from commit c049106bf8267b30a9242d2d574661291cced780)
commit 9a8165b31ffaa5283fed3badb822fcca01a702a0
Author: Andrew Bartlett <abart...@samba.org>
Date: Mon Jul 27 15:44:56 2015 +1200
dbcheck: Try to avoid duplicate searches
BUG: https://bugzilla.samba.org/show_bug.cgi?id=10973
Signed-off-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Stefan Metzmacher <me...@samba.org>
(cherry picked from commit fb88f9cbd969267aaffa021724cf34087c653ba8)
commit 3c29480ab7ef9562779278a5064d18ea5c3d2f29
Author: Andrew Bartlett <abart...@samba.org>
Date: Tue Jul 28 16:11:54 2015 +1200
dbcheck: Add additional tests for the attributeID list
BUG: https://bugzilla.samba.org/show_bug.cgi?id=10973
Signed-off-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Stefan Metzmacher <me...@samba.org>
(cherry picked from commit 2ff9b171378922e68205d5f0eefd2231607b5b89)
commit df141f779f94acaea2c3876263d77fc8d8b6e3b7
Author: Andrew Bartlett <abart...@samba.org>
Date: Thu Jul 23 16:01:14 2015 +1200
dbcheck: Add explict tests for unknown and unsorted attributeID values
Unknown attributeID values would cause an exception previously, and
unsorted attributes cause a failure to replicate with Samba 4.2.
In commit 61b978872fe86906611f64430b2608f5e7ea7ad8 we started
to sort these values correctly, but previous versions of Samba
did not sort them correctly (we sorted high-bit-set values as
negative), and then after 9c9df40220234cba973e84b4985d90da1334a1d1
we stoped accepting these.
To ensure we are allowed to make this unusual change to the
replPropertyMetaData, a new OID is allocated and checked
for in repl_meta_data.c
BUG: https://bugzilla.samba.org/show_bug.cgi?id=10973
Signed-off-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Stefan Metzmacher <me...@samba.org>
(cherry picked from commit 2766bad5ef1e1949746c059c29f179ddae476239)
commit 429db9527866fea464f4f1f7b06107c28e2ef30d
Author: Andrew Bartlett <abart...@samba.org>
Date: Thu Jul 30 14:28:48 2015 +1200
pidl: Assert that python arrays will not overflow the C array
We do not write network services in Python, so this is not a security
issue, but would cause
a crash or other odd behaviour if the length was changed
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11430
Signed-off-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Stefan Metzmacher <me...@samba.org>
(cherry picked from commit e3cf25bdc04795f884fe2a4ce0605d5175b45e93)
commit 1e432b9fd4796474888d66c30614ec5a8ae2059b
Author: Andrew Bartlett <abart...@samba.org>
Date: Tue Jul 28 14:29:25 2015 +1200
pydsdb: Allow the full range of uint32_t values for attributeID
The high bit may be set in these integers, so we need an unsigned int to
store it in
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11429
Signed-off-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Stefan Metzmacher <me...@samba.org>
(cherry picked from commit bed29f3c92861fb2b83ad03a5c759061e1c31eba)
commit 43c6f8cf69b4d49824e601480ca51c105fa9932f
Author: Andrew Bartlett <abart...@samba.org>
Date: Thu Jul 30 14:29:54 2015 +1200
python/tests: Add tests for integer overflow handling
This also documents an issue with our python bindings and lists, as changes
to integers in a list
of integers are not preserved
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11429
Signed-off-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Stefan Metzmacher <me...@samba.org>
(cherry picked from commit 336d41155e94303d0e1fa0530874539339439fed)
commit c9b80d79ec2d8f453dabac495b611fe9b04ba318
Author: Andrew Bartlett <abart...@samba.org>
Date: Mon Jul 27 10:57:43 2015 +1200
pidl: Change PIDL to correctly use and validate python integer types
In particular, it is critical that we use unsigned integers of
sufficient size in python for unsigned C integers, and it is
critical that we check for overflow at both the python and C
level.
Otherwise, we may both represent and sort these incorrectly,
in particular when sorting attributeID values from DRSUAPI
which are represented as an signed enum in C and a uint32_t in IDL,
but which often has the high bit set (in schema extensions).
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11429
Signed-off-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Stefan Metzmacher <me...@samba.org>
(cherry picked from commit 5206ccd002b015a16952d21bd2fa441ff2e39eab)
commit a7790399c64cebdaae6f011a13397c877e7b56ce
Author: Andrew Bartlett <abart...@samba.org>
Date: Wed Jul 29 15:25:09 2015 +1200
python: Use an unsigned integer for buf_size, not -1
This will fail once our python bindings correctly check value ranges
BUG: https://bugzilla.samba.org/show_bug.cgi?id=11429
Signed-off-by: Andrew Bartlett <abart...@samba.org>
Reviewed-by: Stefan Metzmacher <me...@samba.org>
(cherry picked from commit 3faa7dc5a7d39cfdfc908a51e38772fda801c1a3)
-----------------------------------------------------------------------
Summary of changes:
pidl/lib/Parse/Pidl/Samba4/Python.pm | 215 +-
python/samba/dbchecker.py | 116 +-
python/samba/samdb.py | 15 +-
python/samba/tests/dcerpc/integer.py | 235 +
python/samba/tests/dcerpc/sam.py | 2 +-
python/samba/tests/dsdb.py | 24 +
selftest/knownfail | 5 +
selftest/tests.py | 1 +
source4/dsdb/pydsdb.c | 4 +-
source4/dsdb/samdb/ldb_modules/repl_meta_data.c | 31 +-
source4/dsdb/samdb/samdb.h | 6 +
source4/rpc_server/samr/dcesrv_samr.c | 10 +-
...xpected-replpropertymetadata-after-dbcheck.ldif | 122 +
...pected-replpropertymetadata-before-dbcheck.ldif | 122 +
.../provisions/release-4-1-0rc3/private/dns.keytab | Bin 1037 -> 1037 bytes
...DRELEASE-4-1-0RC3,DC%3DSAMBA,DC%3DCORP.ldb.dump | 16082 ++++++-------
...DRELEASE-4-1-0RC3,DC%3DSAMBA,DC%3DCORP.ldb.dump | 22532 +++++++++---------
...DRELEASE-4-1-0RC3,DC%3DSAMBA,DC%3DCORP.ldb.dump | 364 +-
...DRELEASE-4-1-0RC3,DC%3DSAMBA,DC%3DCORP.ldb.dump | 198 +-
...DRELEASE-4-1-0RC3,DC%3DSAMBA,DC%3DCORP.ldb.dump | 4 +-
.../private/dns/sam.ldb.d/metadata.tdb.dump | 4 +-
.../release-4-1-0rc3/private/dns/sam.ldb.dump | 4 +-
.../release-4-1-0rc3/private/hklm.ldb.dump | 2 +-
.../release-4-1-0rc3/private/idmap.ldb.dump | 36 +-
.../provisions/release-4-1-0rc3/private/named.conf | 6 +-
.../provisions/release-4-1-0rc3/private/named.txt | 8 +-
.../release-4-1-0rc3/private/privilege.ldb.dump | 2 +-
...DRELEASE-4-1-0RC3,DC%3DSAMBA,DC%3DCORP.ldb.dump | 16154 ++++++-------
...DRELEASE-4-1-0RC3,DC%3DSAMBA,DC%3DCORP.ldb.dump | 22932 +++++++++----------
...DRELEASE-4-1-0RC3,DC%3DSAMBA,DC%3DCORP.ldb.dump | 364 +-
...DRELEASE-4-1-0RC3,DC%3DSAMBA,DC%3DCORP.ldb.dump | 198 +-
...DRELEASE-4-1-0RC3,DC%3DSAMBA,DC%3DCORP.ldb.dump | 2798 +--
.../private/sam.ldb.d/metadata.tdb.dump | 4 +-
.../release-4-1-0rc3/private/sam.ldb.dump | 8 +-
.../release-4-1-0rc3/private/secrets.keytab | Bin 1482 -> 1482 bytes
.../release-4-1-0rc3/private/secrets.ldb.dump | 10 +-
.../release-4-1-0rc3/private/secrets.tdb.dump | 4 +-
.../release-4-1-0rc3/private/share.ldb.dump | 2 +-
.../private/smbd.tmp/msg/names.tdb.dump | 24 +-
.../release-4-1-0rc3/steps-to-reproduce.txt | 30 +
.../provisions/release-4-1-0rc3/sudo.schema.2.ldif | 52 +
.../provisions/release-4-1-0rc3/sudo.schema.ldif | 235 +
.../provisions/release-4-1-0rc3/sudoers-mod.ldif | 4 +
.../provisions/release-4-1-0rc3/sudoers.ldif | 13 +
source4/setup/schema_samba4.ldif | 1 +
testprogs/blackbox/dbcheck-oldrelease.sh | 47 +-
46 files changed, 41773 insertions(+), 41257 deletions(-)
create mode 100644 python/samba/tests/dcerpc/integer.py
create mode 100644
source4/selftest/provisions/release-4-1-0rc3/expected-replpropertymetadata-after-dbcheck.ldif
create mode 100644
source4/selftest/provisions/release-4-1-0rc3/expected-replpropertymetadata-before-dbcheck.ldif
create mode 100644
source4/selftest/provisions/release-4-1-0rc3/steps-to-reproduce.txt
create mode 100644
source4/selftest/provisions/release-4-1-0rc3/sudo.schema.2.ldif
create mode 100644
source4/selftest/provisions/release-4-1-0rc3/sudo.schema.ldif
create mode 100644
source4/selftest/provisions/release-4-1-0rc3/sudoers-mod.ldif
create mode 100644 source4/selftest/provisions/release-4-1-0rc3/sudoers.ldif
Changeset truncated at 500 lines:
diff --git a/pidl/lib/Parse/Pidl/Samba4/Python.pm
b/pidl/lib/Parse/Pidl/Samba4/Python.pm
index ff68e7a..180b6b2 100644
--- a/pidl/lib/Parse/Pidl/Samba4/Python.pm
+++ b/pidl/lib/Parse/Pidl/Samba4/Python.pm
@@ -10,7 +10,7 @@ use Exporter;
use strict;
use Parse::Pidl qw(warning fatal error);
-use Parse::Pidl::Typelist qw(hasType resolveType getType mapTypeName
expandAlias);
+use Parse::Pidl::Typelist qw(hasType resolveType getType mapTypeName
expandAlias bitmap_type_fn enum_type_fn);
use Parse::Pidl::Util qw(has_property ParseExpr unmake_str);
use Parse::Pidl::NDR qw(GetPrevLevel GetNextLevel ContainsDeferred
ContainsPipe is_charset_array);
use Parse::Pidl::CUtil qw(get_value_of get_pointer_to);
@@ -953,14 +953,58 @@ sub ConvertObjectFromPythonData($$$$$$;$)
$actual_ctype = $actual_ctype->{DATA};
}
- if ($actual_ctype->{TYPE} eq "ENUM" or $actual_ctype->{TYPE} eq
"BITMAP") {
+ # We need to cover ENUMs, BITMAPS and SCALAR values here, as
+ # all could otherwise be assigned invalid integer values
+ my $ctype_alias = "";
+ my $uint_max = "";
+ if ($actual_ctype->{TYPE} eq "ENUM") {
+ # Importantly, ENUM values are unsigned in pidl, and
+ # typically map to uint32
+ $ctype_alias = enum_type_fn($actual_ctype);
+ } elsif ($actual_ctype->{TYPE} eq "BITMAP") {
+ $ctype_alias = bitmap_type_fn($actual_ctype);
+ } elsif ($actual_ctype->{TYPE} eq "SCALAR") {
+ $ctype_alias = expandAlias($actual_ctype->{NAME});
+ }
+
+ # This is the unsigned Python Integer -> C integer validation
+ # case. The signed case is below.
+ if ($ctype_alias =~ /^(uint[0-9]*|hyper|udlong|udlongr
+ |NTTIME_hyper|NTTIME|NTTIME_1sec
+ |uid_t|gid_t)$/x) {
+ $self->pidl("{");
+ $self->indent;
+ $self->pidl("const unsigned long long uint_max =
ndr_sizeof2uintmax(sizeof($target));");
$self->pidl("if (PyLong_Check($cvar)) {");
$self->indent;
- $self->pidl("$target = PyLong_AsLongLong($cvar);");
+ $self->pidl("unsigned long long test_var;");
+ $self->pidl("test_var = PyLong_AsUnsignedLongLong($cvar);");
+ $self->pidl("if (PyErr_Occurred() != NULL) {");
+ $self->indent;
+ $self->pidl($fail);
+ $self->deindent;
+ $self->pidl("}");
+ $self->pidl("if (test_var > uint_max) {");
+ $self->indent;
+ $self->pidl("PyErr_Format(PyExc_OverflowError, \"Expected type
%s or %s within range 0 - %llu, got %llu\",\\");
+ $self->pidl(" PyInt_Type.tp_name, PyLong_Type.tp_name,
uint_max, test_var);");
+ $self->pidl($fail);
+ $self->deindent;
+ $self->pidl("}");
+ $self->pidl("$target = test_var;");
$self->deindent;
$self->pidl("} else if (PyInt_Check($cvar)) {");
$self->indent;
- $self->pidl("$target = PyInt_AsLong($cvar);");
+ $self->pidl("long test_var;");
+ $self->pidl("test_var = PyInt_AsLong($cvar);");
+ $self->pidl("if (test_var < 0 || test_var > uint_max) {");
+ $self->indent;
+ $self->pidl("PyErr_Format(PyExc_OverflowError, \"Expected type
%s or %s within range 0 - %llu, got %ld\",\\");
+ $self->pidl(" PyInt_Type.tp_name, PyLong_Type.tp_name,
uint_max, test_var);");
+ $self->pidl($fail);
+ $self->deindent;
+ $self->pidl("}");
+ $self->pidl("$target = test_var;");
$self->deindent;
$self->pidl("} else {");
$self->indent;
@@ -969,34 +1013,61 @@ sub ConvertObjectFromPythonData($$$$$$;$)
$self->pidl($fail);
$self->deindent;
$self->pidl("}");
+ $self->deindent;
+ $self->pidl("}");
return;
}
- if ($actual_ctype->{TYPE} eq "SCALAR" ) {
- if (expandAlias($actual_ctype->{NAME}) =~
/^(u?int64|hyper|dlong|udlong|udlongr
-
|NTTIME_hyper|NTTIME|NTTIME_1sec
- |uid_t|gid_t)$/x) {
- $self->pidl("if (PyLong_Check($cvar)) {");
- $self->indent;
- $self->pidl("$target = PyLong_AsLongLong($cvar);");
- $self->deindent;
- $self->pidl("} else if (PyInt_Check($cvar)) {");
- $self->indent;
- $self->pidl("$target = PyInt_AsLong($cvar);");
- $self->deindent;
- $self->pidl("} else {");
- $self->indent;
- $self->pidl("PyErr_Format(PyExc_TypeError, \"Expected
type %s or %s\",\\");
- $self->pidl(" PyInt_Type.tp_name,
PyLong_Type.tp_name);");
- $self->pidl($fail);
- $self->deindent;
- $self->pidl("}");
- return;
- }
- if (expandAlias($actual_ctype->{NAME}) =~
/^(char|u?int[0-9]*|time_t)$/) {
- $self->pidl("PY_CHECK_TYPE(&PyInt_Type, $cvar,
$fail);");
- $self->pidl("$target = PyInt_AsLong($cvar);");
- return;
- }
+
+ # Confirm the signed python integer fits in the C type
+ # correctly. It is subtly different from the unsigned case
+ # above, so while it looks like a duplicate, it is not
+ # actually a duplicate.
+ if ($ctype_alias =~ /^(dlong|char|int[0-9]*|time_t)$/x) {
+ $self->pidl("{");
+ $self->indent;
+ $self->pidl("const long long int_max =
ndr_sizeof2intmax(sizeof($target));");
+ $self->pidl("const long long int_min = -int_max - 1;");
+ $self->pidl("if (PyLong_Check($cvar)) {");
+ $self->indent;
+ $self->pidl("long long test_var;");
+ $self->pidl("test_var = PyLong_AsLongLong($cvar);");
+ $self->pidl("if (PyErr_Occurred() != NULL) {");
+ $self->indent;
+ $self->pidl($fail);
+ $self->deindent;
+ $self->pidl("}");
+ $self->pidl("if (test_var < int_min || test_var > int_max) {");
+ $self->indent;
+ $self->pidl("PyErr_Format(PyExc_OverflowError, \"Expected type
%s or %s within range %lld - %lld, got %lld\",\\");
+ $self->pidl(" PyInt_Type.tp_name, PyLong_Type.tp_name,
int_min, int_max, test_var);");
+ $self->pidl($fail);
+ $self->deindent;
+ $self->pidl("}");
+ $self->pidl("$target = test_var;");
+ $self->deindent;
+ $self->pidl("} else if (PyInt_Check($cvar)) {");
+ $self->indent;
+ $self->pidl("long test_var;");
+ $self->pidl("test_var = PyInt_AsLong($cvar);");
+ $self->pidl("if (test_var < int_min || test_var > int_max) {");
+ $self->indent;
+ $self->pidl("PyErr_Format(PyExc_OverflowError, \"Expected type
%s or %s within range %lld - %lld, got %ld\",\\");
+ $self->pidl(" PyInt_Type.tp_name, PyLong_Type.tp_name,
int_min, int_max, test_var);");
+ $self->pidl($fail);
+ $self->deindent;
+ $self->pidl("}");
+ $self->pidl("$target = test_var;");
+ $self->deindent;
+ $self->pidl("} else {");
+ $self->indent;
+ $self->pidl("PyErr_Format(PyExc_TypeError, \"Expected type %s
or %s\",\\");
+ $self->pidl(" PyInt_Type.tp_name, PyLong_Type.tp_name);");
+ $self->pidl($fail);
+ $self->deindent;
+ $self->pidl("}");
+ $self->deindent;
+ $self->pidl("}");
+ return;
}
if ($actual_ctype->{TYPE} eq "STRUCT" or $actual_ctype->{TYPE} eq
"INTERFACE") {
@@ -1152,6 +1223,13 @@ sub ConvertObjectFromPythonLevel($$$$$$$$)
$self->pidl("$var_name =
talloc_array_ptrtype($mem_ctx, $var_name, PyList_GET_SIZE($py_var));");
$self->pidl("if (!$var_name) { $fail; }");
$self->pidl("talloc_set_name_const($var_name,
\"ARRAY: $var_name\");");
+ } else {
+ $self->pidl("if (ARRAY_SIZE($var_name) !=
PyList_GET_SIZE($py_var)) {");
+ $self->indent;
+ $self->pidl("PyErr_Format(PyExc_TypeError,
\"Expected list of type %s, length %zu, got %zd\", Py_TYPE($py_var)->tp_name,
ARRAY_SIZE($var_name), PyList_GET_SIZE($py_var));");
+ $self->pidl("$fail");
+ $self->deindent;
+ $self->pidl("}");
}
$self->pidl("for ($counter = 0; $counter <
PyList_GET_SIZE($py_var); $counter++) {");
$self->indent;
@@ -1202,17 +1280,29 @@ sub ConvertScalarToPython($$$)
$ctypename = expandAlias($ctypename);
if ($ctypename =~ /^(int64|dlong)$/) {
- return "PyLong_FromLongLong($cvar)";
+ return "ndr_PyLong_FromLongLong($cvar)";
}
- if ($ctypename =~
/^(uint64|hyper|udlong|udlongr|NTTIME_hyper|NTTIME|NTTIME_1sec|uid_t|gid_t)$/) {
- return "PyLong_FromUnsignedLongLong($cvar)";
+ if ($ctypename =~
/^(uint64|hyper|NTTIME_hyper|NTTIME|NTTIME_1sec|udlong|udlongr|uid_t|gid_t)$/) {
+ return "ndr_PyLong_FromUnsignedLongLong($cvar)";
}
- if ($ctypename =~ /^(char|u?int[0-9]*|time_t)$/) {
+ if ($ctypename =~ /^(char|int|int8|int16|int32|time_t)$/) {
return "PyInt_FromLong($cvar)";
}
+ # Needed to ensure unsigned values in a 32 or 16 bit enum is
+ # cast correctly to a uint32_t, not sign extended to a a
+ # possibly 64 bit unsigned long. (enums are signed in C,
+ # unsigned in NDR)
+ if ($ctypename =~ /^(uint32|uint3264)$/) {
+ return "ndr_PyLong_FromUnsignedLongLong((uint32_t)$cvar)";
+ }
+
+ if ($ctypename =~ /^(uint|uint8|uint16|uint1632)$/) {
+ return "PyInt_FromLong((uint16_t)$cvar)";
+ }
+
if ($ctypename eq "DATA_BLOB") {
return "PyString_FromStringAndSize((char *)($cvar).data,
($cvar).length)";
}
@@ -1408,6 +1498,61 @@ sub Parse($$$$$)
#include \"$hdr\"
#include \"$ndr_hdr\"
+/*
+ * These functions are here to ensure they can be optomised out by
+ * the compiler based on the constant input values
+ */
+
+static inline unsigned long long ndr_sizeof2uintmax(size_t var_size)
+{
+ switch (var_size) {
+ case 8:
+ return UINT64_MAX;
+ case 4:
+ return UINT32_MAX;
+ case 2:
+ return UINT16_MAX;
+ case 1:
+ return UINT8_MAX;
+ }
+
+ return 0;
+}
+
+static inline long long ndr_sizeof2intmax(size_t var_size)
+{
+ switch (var_size) {
+ case 8:
+ return INT64_MAX;
+ case 4:
+ return INT32_MAX;
+ case 2:
+ return INT16_MAX;
+ case 1:
+ return INT8_MAX;
+ }
+
+ return 0;
+}
+
+static inline PyObject *ndr_PyLong_FromLongLong(long long v)
+{
+ if (v > LONG_MAX || v < LONG_MIN) {
+ return PyLong_FromLongLong(v);
+ } else {
+ return PyInt_FromLong(v);
+ }
+}
+
+static inline PyObject *ndr_PyLong_FromUnsignedLongLong(unsigned long long v)
+{
+ if (v > LONG_MAX) {
+ return PyLong_FromUnsignedLongLong(v);
+ } else {
+ return PyInt_FromLong(v);
+ }
+}
+
");
foreach my $x (@$ndr) {
@@ -1487,7 +1632,7 @@ sub Parse($$$$$)
my $py_obj;
my ($ctype, $cvar) = @{$h->{'val'}};
if ($cvar =~ /^[0-9]+$/ or $cvar =~ /^0x[0-9a-fA-F]+$/) {
- $py_obj = "PyInt_FromLong($cvar)";
+ $py_obj = "ndr_PyLong_FromUnsignedLongLong($cvar)";
} elsif ($cvar =~ /^".*"$/) {
$py_obj = "PyString_FromString($cvar)";
} else {
diff --git a/python/samba/dbchecker.py b/python/samba/dbchecker.py
index 74e9678..4fb9d12 100644
--- a/python/samba/dbchecker.py
+++ b/python/samba/dbchecker.py
@@ -24,6 +24,7 @@ from base64 import b64decode
from samba import dsdb
from samba import common
from samba.dcerpc import misc
+from samba.dcerpc import drsuapi
from samba.ndr import ndr_unpack, ndr_pack
from samba.dcerpc import drsblobs
from samba.common import dsdb_Dn
@@ -63,6 +64,7 @@ class dbcheck(object):
self.move_to_lost_and_found = False
self.fix_instancetype = False
self.fix_replmetadata_zero_invocationid = False
+ self.fix_replmetadata_unsorted_attid = False
self.fix_deleted_deleted_objects = False
self.fix_dn = False
self.fix_base64_userparameters = False
@@ -80,6 +82,8 @@ class dbcheck(object):
self.wellknown_sds = get_wellknown_sds(self.samdb)
self.fix_all_missing_objectclass = False
+ self.dn_set = set()
+
self.name_map = {}
try:
res = samdb.search(base="CN=DnsAdmins,CN=Users,%s" %
samdb.domain_dn(), scope=ldb.SCOPE_BASE,
@@ -125,12 +129,12 @@ class dbcheck(object):
def check_database(self, DN=None, scope=ldb.SCOPE_SUBTREE, controls=[],
attrs=['*']):
'''perform a database check, returning the number of errors found'''
-
res = self.samdb.search(base=DN, scope=scope, attrs=['dn'],
controls=controls)
self.report('Checking %u objects' % len(res))
error_count = 0
for object in res:
+ self.dn_set.add(str(object.dn))
error_count += self.check_object(object.dn, attrs=attrs)
if DN is None:
@@ -698,18 +702,21 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn),
str(to_base)))
return 0
def process_metadata(self, val):
- '''Read metadata properties and list attributes in it'''
+ '''Read metadata properties and list attributes in it.
+ raises KeyError if the attid is unknown.'''
- list_att = []
+ set_att = set()
+ list_attid = []
repl = ndr_unpack(drsblobs.replPropertyMetaDataBlob, str(val))
obj = repl.ctr
for o in repl.ctr.array:
att = self.samdb_schema.get_lDAPDisplayName_by_attid(o.attid)
- list_att.append(att.lower())
+ set_att.add(att.lower())
+ list_attid.append(o.attid)
- return list_att
+ return (set_att, list_attid)
def fix_metadata(self, dn, attr):
@@ -989,11 +996,52 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn),
str(to_base)))
nmsg = ldb.Message()
nmsg.dn = dn
nmsg[attr] = ldb.MessageElement(replBlob, ldb.FLAG_MOD_REPLACE,
attr)
- if self.do_modify(nmsg, ["local_oid:1.3.6.1.4.1.7165.4.3.14:0"],
+ if self.do_modify(nmsg, ["local_oid:%s:0" %
dsdb.DSDB_CONTROL_DBCHECK_MODIFY_RO_REPLICA,
+ "local_oid:1.3.6.1.4.1.7165.4.3.14:0"],
"Failed to fix attribute %s" % attr):
self.report("Fixed attribute '%s' of '%s'\n" % (attr, dn))
+ def err_replmetadata_unknown_attid(self, dn, attr, repl_meta_data):
+ repl = ndr_unpack(drsblobs.replPropertyMetaDataBlob,
+ str(repl_meta_data))
+ ctr = repl.ctr
+ for o in ctr.array:
+ # Search for an invalid attid
+ try:
+ att = self.samdb_schema.get_lDAPDisplayName_by_attid(o.attid)
+ except KeyError:
+ self.report('ERROR: attributeID 0X%0X is not known in our
schema, not fixing %s on %s\n' % (o.attid, attr, dn))
+ return
+
+
+ def err_replmetadata_unsorted_attid(self, dn, attr, repl_meta_data):
+ repl = ndr_unpack(drsblobs.replPropertyMetaDataBlob,
+ str(repl_meta_data))
+ ctr = repl.ctr
+ found = False
+
+ self.report('ERROR: unsorted attributeID values in %s on %s\n' %
(attr, dn))
+ if not self.confirm_all('Fix %s on %s by sorting the attribute list?'
+ % (attr, dn),
'fix_replmetadata_unsorted_attid'):
+ self.report('Not fixing %s on %s\n' % (attr, dn))
+ return
+
+ # Sort the array, except for the last element
+ ctr.array[:-1] = sorted(ctr.array[:-1], key=lambda o: o.attid)
+
+ replBlob = ndr_pack(repl)
+
+ nmsg = ldb.Message()
+ nmsg.dn = dn
+ nmsg[attr] = ldb.MessageElement(replBlob, ldb.FLAG_MOD_REPLACE, attr)
+ if self.do_modify(nmsg, ["local_oid:%s:0" %
dsdb.DSDB_CONTROL_DBCHECK_MODIFY_RO_REPLICA,
+ "local_oid:1.3.6.1.4.1.7165.4.3.14:0",
+ "local_oid:1.3.6.1.4.1.7165.4.3.25:0"],
+ "Failed to fix attribute %s" % attr):
+ self.report("Fixed attribute '%s' of '%s'\n" % (attr, dn))
+
+
def is_deleted_deleted_objects(self, obj):
faulty = False
if "description" not in obj:
@@ -1125,8 +1173,8 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn),
str(to_base)))
return 1
obj = res[0]
error_count = 0
- list_attrs_from_md = []
- list_attrs_seen = []
+ set_attrs_from_md = set()
+ set_attrs_seen = set()
got_repl_property_meta_data = False
got_objectclass = False
@@ -1181,7 +1229,31 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn),
str(to_base)))
# We don't continue, as we may also have other fixes for
this attribute
# based on what other attributes we see.
- list_attrs_from_md = self.process_metadata(obj[attrname])
+ try:
+ (set_attrs_from_md, list_attid_from_md) =
self.process_metadata(obj[attrname])
+ except KeyError:
+ error_count += 1
+ self.err_replmetadata_unknown_attid(dn, attrname,
obj[attrname])
+ continue
+
+ if sorted(list_attid_from_md[:-1]) != list_attid_from_md[:-1]:
+ error_count += 1
+ self.err_replmetadata_unsorted_attid(dn, attrname,
obj[attrname])
+ else:
+ # Here we check that the first attid is 0
+ # (objectClass) and that the last on is the RDN
+ # from the DN.
+ rdn_attid =
self.samdb_schema.get_attid_from_lDAPDisplayName(dn.get_rdn_name())
+ if list_attid_from_md[-1] != rdn_attid:
+ error_count += 1
+ self.report("ERROR: Not fixing incorrect final
attributeID in '%s' on '%s', it should match the RDN %s" %
+ (attrname, str(dn), dn.get_rdn_name()))
+
+ if list_attid_from_md[0] != 0:
+ error_count += 1
+ self.report("ERROR: Not fixing incorrect inital
attributeID in '%s' on '%s', it should be objectClass" %
+ (attrname, str(dn)))
+
got_repl_property_meta_data = True
continue
@@ -1274,7 +1346,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn),
str(to_base)))
if (not flag & dsdb.DS_FLAG_ATTR_NOT_REPLICATED
and not flag & dsdb.DS_FLAG_ATTR_IS_CONSTRUCTED
and not
self.samdb_schema.get_linkId_from_lDAPDisplayName(attrname)):
- list_attrs_seen.append(str(attrname).lower())
+ set_attrs_seen.add(str(attrname).lower())
if syntax_oid in [ dsdb.DSDB_SYNTAX_BINARY_DN,
dsdb.DSDB_SYNTAX_OR_NAME,
dsdb.DSDB_SYNTAX_STRING_DN, ldb.SYNTAX_DN ]:
@@ -1345,17 +1417,17 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn),
str(to_base)))
else:
self.report("Not fixing isDeleted
originating_change_time on '%s'" % str(dn))
- for att in list_attrs_seen:
- if not att in list_attrs_from_md:
- if show_dn:
- self.report("On object %s" % dn)
- show_dn = False
- error_count += 1
- self.report("ERROR: Attribute %s not present in
replication metadata" % att)
- if not self.confirm_all("Fix missing replPropertyMetaData
element '%s'" % att, 'fix_all_metadata'):
- self.report("Not fixing missing replPropertyMetaData
element '%s'" % att)
- continue
- self.fix_metadata(dn, att)
+
+ for att in set_attrs_seen.difference(set_attrs_from_md):
+ if show_dn:
+ self.report("On object %s" % dn)
+ show_dn = False
+ error_count += 1
+ self.report("ERROR: Attribute %s not present in replication
metadata" % att)
+ if not self.confirm_all("Fix missing replPropertyMetaData
element '%s'" % att, 'fix_all_metadata'):
+ self.report("Not fixing missing replPropertyMetaData
element '%s'" % att)
+ continue
+ self.fix_metadata(dn, att)
if self.is_fsmo_role(dn):
if "fSMORoleOwner" not in obj and ("*" in attrs or "fsmoroleowner"
in map(str.lower, attrs)):
@@ -1363,7 +1435,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn),
str(to_base)))
error_count += 1
try:
- if dn != self.samdb.get_root_basedn():
+ if dn != self.samdb.get_root_basedn() and str(dn.parent()) not in
self.dn_set:
res = self.samdb.search(base=dn.parent(), scope=ldb.SCOPE_BASE,
controls=["show_recycled:1",
"show_deleted:1"])
except ldb.LdbError, (enum, estr):
diff --git a/python/samba/samdb.py b/python/samba/samdb.py
index e3a6292..817fbdb 100644
--- a/python/samba/samdb.py
+++ b/python/samba/samdb.py
--
Samba Shared Repository
