[SCM] Samba Shared Repository - branch master updated

2019-04-05 Thread Jeremy Allison
The branch, master has been updated
   via  b7028c42462 torture: Add test for talloc size accounting in memcache
   via  9ff5c0bab76 memcache: Increase size of default memcache to 512k
   via  a04ca6f3438 memcache: Properly track the size of talloc objects
   via  7c44f2f76ee memcache: Introduce struct for storing talloc pointer
  from  7a410ccb5f6 netcmd: Fix passwordsettings --max-pwd-age command

https://git.samba.org/?p=samba.git;a=shortlog;h=master


- Log -
commit b7028c42462c34cf86cb949bfdb16ebc7ed0a6c6
Author: Christof Schmitt 
Date:   Thu Mar 28 10:46:43 2019 -0700

torture: Add test for talloc size accounting in memcache

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13865

Signed-off-by: Christof Schmitt 
Reviewed-by: Jeremy Allison 

Autobuild-User(master): Jeremy Allison 
Autobuild-Date(master): Sat Apr  6 06:08:42 UTC 2019 on sn-devel-144

commit 9ff5c0bab76c5d3d7bea1fcb79861d0c9a3b9839
Author: Christof Schmitt 
Date:   Fri Apr 5 15:43:21 2019 -0700

memcache: Increase size of default memcache to 512k

With the fixed accounting of talloc objects, the default cache size
needs to increase. The exact increase required depends on the workloads,
going form 256k to 512k seems like a reasonable guess.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13865

Signed-off-by: Christof Schmitt 
Reviewed-by: Jeremy Allison 

commit a04ca6f3438595ba7e1a110877f53d1cac0f0402
Author: Christof Schmitt 
Date:   Mon Apr 1 16:23:35 2019 -0700

memcache: Properly track the size of talloc objects

With memcache_add_talloc, the talloc object becomes part of the pool and
the memcache_element stores a pointer to the talloc object. The
size of the the talloc object was not used when tracking the used space,
allowing the cache to grow larger than defined in the memcache_init
call.

Fix this by adding the size of the talloc object to the used space.

Also record the initial size of the talloc object for proper adjustment
of the used space in the cache later. This is in case the size of the
talloc object is modified while being owned by the cache (e.g.
allocating talloc child objects). This should never happen, but better
be safe than ending up with a broken cache usage counter.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13865

Signed-off-by: Christof Schmitt 
Reviewed-by: Jeremy Allison 

commit 7c44f2f76eefb9156cb1d170c92b4ff07dd6a3d5
Author: Christof Schmitt 
Date:   Mon Apr 1 15:38:59 2019 -0700

memcache: Introduce struct for storing talloc pointer

This allows extending the additional data stored for talloced objects
later.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13865

Signed-off-by: Christof Schmitt 
Reviewed-by: Jeremy Allison 

---

Summary of changes:
 docs-xml/smbdotconf/filename/maxstatcachesize.xml |  2 +-
 lib/param/loadparm.c  |  2 +-
 lib/util/memcache.c   | 54 -
 source3/param/loadparm.c  |  2 +-
 source3/torture/torture.c | 70 ++-
 5 files changed, 111 insertions(+), 19 deletions(-)


Changeset truncated at 500 lines:

diff --git a/docs-xml/smbdotconf/filename/maxstatcachesize.xml 
b/docs-xml/smbdotconf/filename/maxstatcachesize.xml
index 63f91e70fd2..866d74d6ffd 100644
--- a/docs-xml/smbdotconf/filename/maxstatcachesize.xml
+++ b/docs-xml/smbdotconf/filename/maxstatcachesize.xml
@@ -13,6 +13,6 @@

 
 stat cache
-256
+512
 100
 
diff --git a/lib/param/loadparm.c b/lib/param/loadparm.c
index 7ef2cc7d3f6..ebbccc22b71 100644
--- a/lib/param/loadparm.c
+++ b/lib/param/loadparm.c
@@ -2945,7 +2945,7 @@ struct loadparm_context *loadparm_init(TALLOC_CTX 
*mem_ctx)
 
lpcfg_do_global_parameter(lp_ctx, "durable handles", "yes");
 
-   lpcfg_do_global_parameter(lp_ctx, "max stat cache size", "256");
+   lpcfg_do_global_parameter(lp_ctx, "max stat cache size", "512");
 
lpcfg_do_global_parameter(lp_ctx, "ldap passwd sync", "no");
 
diff --git a/lib/util/memcache.c b/lib/util/memcache.c
index 819ba44b443..1e616bd0e9a 100644
--- a/lib/util/memcache.c
+++ b/lib/util/memcache.c
@@ -27,6 +27,11 @@
 
 static struct memcache *global_cache;
 
+struct memcache_talloc_value {
+   void *ptr;
+   size_t len;
+};
+
 struct memcache_element {
struct rb_node rb_node;
struct memcache_element *prev, *next;
@@ -180,19 +185,19 @@ void *memcache_lookup_talloc(struct memcache *cache, enum 
memcache_number n,
 DATA_BLOB key)
 {
DATA_BLOB value;
-   void *result;
+   struct memcache_talloc_value mtv;
 
if (!memcache_lookup(cache, n, key, &val

autobuild[sn-devel-144]: intermittent test failure detected

2019-04-05 Thread autobuild
The autobuild test system (on sn-devel-144) has detected an intermittent 
failing test in 
the current master tree.

The autobuild log of the failure is available here:

   
https://git.samba.org/autobuild.flakey.sn-devel-144/2019-04-05-1327/flakey.log

The failure seems to be in the "samba-ad-dc-ntvfs" suite, whose build logs are 
available here:

   
https://git.samba.org/autobuild.flakey.sn-devel-144/2019-04-05-1327/samba-ad-dc-ntvfs.stderr
   
https://git.samba.org/autobuild.flakey.sn-devel-144/2019-04-05-1327/samba-ad-dc-ntvfs.stdout
  
The top commit at the time of the failure was:

commit 7a410ccb5f6f2958d56fa6f16d8780c69a3830dd
Author: Tim Beale 
Date:   Wed Apr 3 09:10:55 2019 +1300

netcmd: Fix passwordsettings --max-pwd-age command

The min_pwd_age and max_pwd_age parameters are both optional and default
to None. However, if we just set the max-pwd-age, then the check
'min_pwd_age >= max_pwd_age' will throw a Python exception because it's
trying to compare an int to NoneType (min_pwd_age). This works on Python 2
but is a problem on Python 3.

We could just add a check that min_pwd_age is not None, but that defeats
the point of having the check if you're only setting either the min or
max age indepedently.

This patch gets the current min/max password age from the DB (in ticks).
If either setting is changed, the ticks will be updated. Then at the end
we check the min is still less than the max (to do this, we convert the
ticks back to days in the interests of readability).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13873

Signed-off-by: Tim Beale 
Reviewed-by: Andrew Bartlett 

Autobuild-User(master): Andrew Bartlett 
Autobuild-Date(master): Fri Apr  5 08:03:08 UTC 2019 on sn-devel-144

and the last 50 lines of the stdout log were:

[215(856)/252 at 32m38s] samba4.smb.signing on with -k yes 
--signing=on(ad_dc_ntvfs)
[216(857)/252 at 32m39s] samba4.smb.signing on with -k yes 
--signing=required(ad_dc_ntvfs)
[217(858)/252 at 32m40s] samba4.smb.signing on with -k yes 
--option=gensec:fake_gssapi_krb5=yes --option=gensec:gssapi_krb5=no 
--signing=on(ad_dc_ntvfs)
[218(859)/252 at 32m41s] samba4.smb.signing on with -k yes 
--option=gensec:fake_gssapi_krb5=yes --option=gensec:gssapi_krb5=no 
--signing=required(ad_dc_ntvfs)
[219(860)/252 at 32m43s] samba4.smb.signing --signing=yes anon(ad_dc_ntvfs)
[220(861)/252 at 32m43s] samba4.smb.signing --signing=required anon(ad_dc_ntvfs)
[221(862)/252 at 32m44s] samba.tests.complex_expressions(ad_dc_ntvfs)
[222(884)/252 at 32m58s] samba.tests.samba_tool.gpo(ad_dc_ntvfs:local)
[223(896)/252 at 34m18s] samba.tests.dcerpc.registry(ad_dc_ntvfs:local)
[224(899)/252 at 34m20s] samba.tests.dcerpc.dnsserver(ad_dc_ntvfs)
[225(916)/252 at 34m44s] samba.tests.auth_log(ad_dc_ntvfs:local)
[226(963)/252 at 35m48s] samba.tests.auth_log_pass_change(ad_dc_ntvfs)
[227(971)/252 at 36m3s] samba.tests.auth_log_ncalrpc(ad_dc_ntvfs:local)
[228(973)/252 at 36m4s] samba.tests.auth_log_samlogon(ad_dc_ntvfs:local)
[229(974)/252 at 36m6s] samba.tests.auth_log_netlogon(ad_dc_ntvfs:local)
[230(975)/252 at 36m8s] 
samba.tests.auth_log_netlogon_bad_creds(ad_dc_ntvfs:local)
[231(979)/252 at 36m14s] samba.tests.password_hash_fl2008(ad_dc_ntvfs:local)
[232(983)/252 at 36m18s] samba.tests.krb5_credentials(ad_dc_ntvfs)
[233(986)/252 at 36m22s] samba.tests.py_credentials(ad_dc_ntvfs)
[234(994)/252 at 36m32s] samba.tests.emulate.traffic(ad_dc_ntvfs)
[235(996)/252 at 36m34s] samba.tests.emulate.traffic_packet(ad_dc_ntvfs)
[236(1085)/252 at 39m21s] samba.tests.blackbox.traffic_replay(ad_dc_ntvfs)
[237(1090)/252 at 40m10s] samba.tests.blackbox.traffic_learner(ad_dc_ntvfs)
[238(1092)/252 at 40m12s] samba.tests.blackbox.traffic_summary(ad_dc_ntvfs)
[239(1092)/252 at 40m13s] samba.tests.blackbox.smbcontrol(ad_dc_ntvfs:local)
[240(1094)/252 at 40m17s] 
samba4.urgent_replication.python(ad_dc_ntvfs)(ad_dc_ntvfs:local)
[241(1102)/252 at 40m26s] samba4.ldap.match_rules.python(ad_dc_ntvfs)
[242(1136)/252 at 41m10s] 
samba4.ldap.notification.python(ad_dc_ntvfs)(ad_dc_ntvfs)
UNEXPECTED(failure): 
samba4.ldap.notification.python(ad_dc_ntvfs).__main__.LDAPNotificationTest.test_simple_search(ad_dc_ntvfs)
REASON: Exception: Exception: Traceback (most recent call last):
  File 
"/memdisk/autobuild/fl/b267853/samba-ad-dc-ntvfs/source4/dsdb/tests/python/notification.py",
 line 151, in test_simple_search
self.assertIsNotNone(msg3)
AssertionError: unexpectedly None

FAILED (1 failures, 0 errors and 0 unexpected successes in 0 testsuites)

A summary with detailed information can be found in:
  ./bin/ab/summary
TOP 10 slowest tests
samba.tests.emulate.traffic_packet(ad_dc_ntvfs) -> 168
samba4.raw.notify(ad_dc_ntvfs) -> 119
samba.tests.samba_tool.gpo(ad_dc_ntvfs:local) -> 80
samba4.nbt.winsreplication(ad_dc_ntvfs) -> 75
samba4.raw.oplock(ad_dc_ntvfs) -> 73
samba.tests.auth_log(ad_dc_ntvfs:local) -> 64
samba4.blackbox.kinit

[SCM] Samba Shared Repository - branch master updated

2019-04-05 Thread Andrew Bartlett
The branch, master has been updated
   via  7a410ccb5f6 netcmd: Fix passwordsettings --max-pwd-age command
   via  940306a24a8 netcmd: Add some timestamp conversion helper functions
   via  b43f997f239 netcmd: Use python constant for -0x8000
   via  d247a600845 tests: Add test for setting min/maxPwdAge
  from  2da9d7d130c ldb_kv_search: avoid handling uninitialised dn

https://git.samba.org/?p=samba.git;a=shortlog;h=master


- Log -
commit 7a410ccb5f6f2958d56fa6f16d8780c69a3830dd
Author: Tim Beale 
Date:   Wed Apr 3 09:10:55 2019 +1300

netcmd: Fix passwordsettings --max-pwd-age command

The min_pwd_age and max_pwd_age parameters are both optional and default
to None. However, if we just set the max-pwd-age, then the check
'min_pwd_age >= max_pwd_age' will throw a Python exception because it's
trying to compare an int to NoneType (min_pwd_age). This works on Python 2
but is a problem on Python 3.

We could just add a check that min_pwd_age is not None, but that defeats
the point of having the check if you're only setting either the min or
max age indepedently.

This patch gets the current min/max password age from the DB (in ticks).
If either setting is changed, the ticks will be updated. Then at the end
we check the min is still less than the max (to do this, we convert the
ticks back to days in the interests of readability).

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13873

Signed-off-by: Tim Beale 
Reviewed-by: Andrew Bartlett 

Autobuild-User(master): Andrew Bartlett 
Autobuild-Date(master): Fri Apr  5 08:03:08 UTC 2019 on sn-devel-144

commit 940306a24a8d14fbb8c76c5a60b3d5f2773873a0
Author: Tim Beale 
Date:   Tue Apr 2 11:10:41 2019 +1300

netcmd: Add some timestamp conversion helper functions

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13873

Signed-off-by: Tim Beale 
Reviewed-by: Andrew Bartlett 

commit b43f997f2397771b159c49526a36bd2b3467b0ef
Author: Tim Beale 
Date:   Mon Apr 1 16:42:32 2019 +1300

netcmd: Use python constant for -0x8000

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13873

Signed-off-by: Tim Beale 
Reviewed-by: Andrew Bartlett 

commit d247a600845fdc6bf232496e8db56cd1d95a3022
Author: Tim Beale 
Date:   Mon Apr 1 16:32:27 2019 +1300

tests: Add test for setting min/maxPwdAge

Currently setting maxPwdAge doesn't work at all.

While we're adding a test, we might as well assert that minPwdAge
can't be greater than maxPwdAge as well.

BUG: https://bugzilla.samba.org/show_bug.cgi?id=13873

Signed-off-by: Tim Beale 
Reviewed-by: Andrew Bartlett 

---

Summary of changes:
 python/samba/netcmd/domain.py | 56 +--
 python/samba/tests/samba_tool/passwordsettings.py | 38 +++
 2 files changed, 79 insertions(+), 15 deletions(-)


Changeset truncated at 500 lines:

diff --git a/python/samba/netcmd/domain.py b/python/samba/netcmd/domain.py
index 851e7241d15..b616dba3438 100644
--- a/python/samba/netcmd/domain.py
+++ b/python/samba/netcmd/domain.py
@@ -1255,6 +1255,26 @@ class cmd_domain_level(Command):
 raise CommandError("invalid argument: '%s' (choose from 'show', 
'raise')" % subcommand)
 
 
+# In MS AD, setting a timeout to '(never)' corresponds to this value
+NEVER_TIMESTAMP = int(-0x8000)
+
+
+def timestamp_to_mins(timestamp_str):
+"""Converts a timestamp in -100 nanosecond units to minutes"""
+# treat a timestamp of 'never' the same as zero (this should work OK for
+# most settings, and it displays better than trying to convert
+# -0x8000 to minutes)
+if int(timestamp_str) == NEVER_TIMESTAMP:
+return 0
+else:
+return abs(int(timestamp_str)) / (1e7 * 60)
+
+
+def timestamp_to_days(timestamp_str):
+"""Converts a timestamp in -100 nanosecond units to days"""
+return timestamp_to_mins(timestamp_str) / (60 * 24)
+
+
 class cmd_domain_passwordsettings_show(Command):
 """Display current password settings for the domain."""
 
@@ -1289,18 +1309,14 @@ class cmd_domain_passwordsettings_show(Command):
 pwd_hist_len = int(res[0]["pwdHistoryLength"][0])
 cur_min_pwd_len = int(res[0]["minPwdLength"][0])
 # ticks -> days
-cur_min_pwd_age = int(abs(int(res[0]["minPwdAge"][0])) / (1e7 * 60 
* 60 * 24))
-if int(res[0]["maxPwdAge"][0]) == -0x8000:
-cur_max_pwd_age = 0
-else:
-cur_max_pwd_age = int(abs(int(res[0]["maxPwdAge"][0])) / (1e7 
* 60 * 60 * 24))
+cur_min_pwd_age = timestamp_to_days(res[0]["minPwdAge"][0])
+cur_max_pwd_age = timestamp_to_days(res[0]["maxPw