[SCM] Samba Shared Repository - branch v3-6-test updated
The branch, v3-6-test has been updated
via e28ec90 smbd: fix initial large PAC sess setup response
from d21280f Fix bug 9519 - Samba returns unexpected error on SMB posix
open.
http://gitweb.samba.org/?p=samba.git;a=shortlog;h=v3-6-test
- Log -
commit e28ec902a207655acab665c4cfabb1f2031fb24f
Author: David Disseldorp dd...@samba.org
Date: Tue Feb 12 11:58:06 2013 +0100
smbd: fix initial large PAC sess setup response
An oversize Kerberos security token may be split across multiple Session
Setup AndX requests when authenticating as a user who is a member of
many (~2000) groups.
In such a case the NativeOS, NativeLanMan PrimaryDomain fields must be
sent with the NT_STATUS_MORE_PROCESSING_REQUIRED response. Otherwise
Windows clients may resend the same security token data in subsequent
session setup andX requests, as observed with Windows 7 and Server 2012.
This change fixes the SMB1 server only.
Fix bug #9658 - Session Setup AndX exchange fails with an oversize security
token.
---
Summary of changes:
source3/smbd/sesssetup.c | 14 +-
1 files changed, 13 insertions(+), 1 deletions(-)
Changeset truncated at 500 lines:
diff --git a/source3/smbd/sesssetup.c b/source3/smbd/sesssetup.c
index 4f09db9..75c2a15 100644
--- a/source3/smbd/sesssetup.c
+++ b/source3/smbd/sesssetup.c
@@ -905,6 +905,13 @@ static NTSTATUS check_spnego_blob_complete(struct
smbd_server_connection *sconn,
(unsigned int)copy_len,
(unsigned int)pblob-length ));
+ if (pblob-length pad-needed_len) {
+ DEBUG(2, (subsequent security token data length %u
+ exceeds expected length %u\n,
+ (unsigned int)pblob-length,
+ (unsigned int)pad-needed_len));
+ }
+
tmp_blob = data_blob(NULL,
pad-partial_data.length + copy_len);
@@ -1165,13 +1172,18 @@ static void reply_sesssetup_and_X_spnego(struct
smb_request *req)
status = check_spnego_blob_complete(sconn, smbpid, vuid, blob1);
if (!NT_STATUS_IS_OK(status)) {
+ /*
+* Pack error response, ensuring to fill NativeOS, NativeLanMan
+* PrimaryDomain fields on NT_STATUS_MORE_PROCESSING_REQUIRED
+*/
+ reply_outbuf(req, 4, 0);
+ reply_sesssetup_blob(req, data_blob_null, status);
if (!NT_STATUS_EQUAL(status,
NT_STATUS_MORE_PROCESSING_REQUIRED)) {
/* Real error - kill the intermediate vuid */
invalidate_vuid(sconn, vuid);
}
data_blob_free(blob1);
- reply_nterror(req, nt_status_squash(status));
return;
}
--
Samba Shared Repository
[SCM] Samba Shared Repository - branch v4-0-test updated
The branch, v4-0-test has been updated
via 97c3b5d waf: Fix correct linking of libreplace with
cmdline-credentials.
via db13fac Fallback to the internal resolver on EAI_FAIL.
via 6284031 s3:idmap_autorid: fix freeing of non-talloced memory
(uninitialized pointer) (bug #9653)
from 1c74a7d Fix bug 9519 - Samba returns unexpected error on SMB posix
open.
http://gitweb.samba.org/?p=samba.git;a=shortlog;h=v4-0-test
- Log -
commit 97c3b5db4214336786c1fadde8292fe4d1aba56b
Author: Andreas Schneider a...@samba.org
Date: Thu Feb 14 12:14:33 2013 +0100
waf: Fix correct linking of libreplace with cmdline-credentials.
cmdline-credentials needs rep_getpass from libreplace. The function has
been replaced in master so this issue only exists in 4.0 releases.
Reviewed-by: Andrew Bartlet abart...@samba.org
Signed-off-by: Andreas Schneider a...@samba.org
Fix bug #9664 - libcmdline-credentials.so isn't correctly linked again
libreplace.so.
Autobuild-User(v4-0-test): Karolin Seeger ksee...@samba.org
Autobuild-Date(v4-0-test): Mon Feb 18 11:55:59 CET 2013 on sn-devel-104
commit db13facf3397d6043ae4324b78daf5346fed20ef
Author: Landon Fuller land...@bikemonkey.org
Date: Sat Feb 16 22:57:40 2013 -0500
Fallback to the internal resolver on EAI_FAIL.
On Linux, non-RFC 1034-complaint names (such as gc._msdsc.example.org)
will result in the resolver returning the non-POSIX EAI_NODATA. In that
case, the case statement here would fall back on the internal resolver,
allowing resolution to complete successfully.
On FreeBSD, the libc resolver uses the same validation code, but
the POSIX result of EAI_FAIL is returned instead of EAI_NODATA. Since
there was no case for this error code, no fallback to the internal
resolver would occur. This led to replication failing on FreeBSD.
Reviewed-by: Andrew Bartlett abart...@samba.org
Autobuild-User(master): Andrew Bartlett abart...@samba.org
Autobuild-Date(master): Sun Feb 17 07:06:36 CET 2013 on sn-devel-104
(cherry picked from commit 6dfb35f3ff7ad2d2089c0a3e5eab342384e45e4c)
Fix bug #9656 - [patch] Work around FreeBSD's getaddrinfo() underscore
issue.
commit 62840319080908e234ba78affdbad25f53214441
Author: Michael Adam ob...@samba.org
Date: Mon Feb 11 22:52:55 2013 +0100
s3:idmap_autorid: fix freeing of non-talloced memory (uninitialized
pointer) (bug #9653)
Signed-off-by: Michael Adam ob...@samba.org
Reviewed-by: Stefan Metzmacher me...@samba.org
Autobuild-User(master): Stefan Metzmacher me...@samba.org
Autobuild-Date(master): Wed Feb 13 09:51:53 CET 2013 on sn-devel-104
(cherry picked from commit 19c68f80251f443016e505c5cf87f697fb552e8c)
---
Summary of changes:
source3/winbindd/idmap_autorid.c |2 +-
source4/lib/cmdline/wscript_build |1 +
source4/libcli/resolve/dns_ex.c |4 +++-
3 files changed, 5 insertions(+), 2 deletions(-)
Changeset truncated at 500 lines:
diff --git a/source3/winbindd/idmap_autorid.c b/source3/winbindd/idmap_autorid.c
index 621cae9..306b176 100644
--- a/source3/winbindd/idmap_autorid.c
+++ b/source3/winbindd/idmap_autorid.c
@@ -244,7 +244,7 @@ static NTSTATUS idmap_autorid_id_to_sid(struct
autorid_global_config *cfg,
struct id_map *map)
{
uint32_t range;
- TDB_DATA data;
+ TDB_DATA data = tdb_null;
char *keystr;
struct dom_sid sid;
NTSTATUS status;
diff --git a/source4/lib/cmdline/wscript_build
b/source4/lib/cmdline/wscript_build
index 9b6f6e5..eec92d4 100644
--- a/source4/lib/cmdline/wscript_build
+++ b/source4/lib/cmdline/wscript_build
@@ -4,6 +4,7 @@ bld.SAMBA_LIBRARY('cmdline-credentials',
source='credentials.c',
autoproto='credentials.h',
public_deps='samba-credentials popt',
+ deps='replace',
private_library=True)
bld.SAMBA_SUBSYSTEM('POPT_SAMBA',
diff --git a/source4/libcli/resolve/dns_ex.c b/source4/libcli/resolve/dns_ex.c
index bc64e8d..1226ed6 100644
--- a/source4/libcli/resolve/dns_ex.c
+++ b/source4/libcli/resolve/dns_ex.c
@@ -400,8 +400,10 @@ static void run_child_getaddrinfo(struct dns_ex_state
*state, int fd)
#ifdef EAI_NODATA
case EAI_NODATA:
#endif
+ case EAI_FAIL:
+ /* Linux returns EAI_NODATA on non-RFC1034-compliant
names. FreeBSD returns EAI_FAIL */
case EAI_NONAME:
- /* getaddrinfo() doesn't handle CNAME records */
+ /* getaddrinfo() doesn't handle CNAME or non-RFC1034
compatible records */
run_child_dns_lookup(state, fd);
return;
[SCM] Samba Shared Repository - branch master updated
The branch, master has been updated via 8adbd1c srv_epmapper.c: Fix typo. via 240df6c wb_samba3_cmd.c: Fix typo in comment. via b22b22d brlock_tdb.c: Fix typo in comment. via 75ca814 srv_netlog_nt.c: Fix typo in comment. via 6eb59eb brlock.c: Fix typo in comment. via 38cb141 vfs_gpfs: Fix typos in comments. via 2ed035b fault.c: Fix typo in comment. via 68b2e30 docs: Fix typo. from dcc94f0 s4-nbt: Ensure source4/ nbt client and server honour 'disable netbios' http://gitweb.samba.org/?p=samba.git;a=shortlog;h=master - Log - commit 8adbd1cf75492869f7fd1935eb211a070ef924cc Author: Karolin Seeger ksee...@samba.org Date: Mon Feb 18 10:25:09 2013 +0100 srv_epmapper.c: Fix typo. priviledge - privilege Signed-off-by: Karolin Seeger ksee...@samba.org Autobuild-User(master): Andrew Bartlett abart...@samba.org Autobuild-Date(master): Mon Feb 18 13:57:40 CET 2013 on sn-devel-104 commit 240df6c7b05e3c5c7be80d7607824147f360d64e Author: Karolin Seeger ksee...@samba.org Date: Mon Feb 18 10:05:23 2013 +0100 wb_samba3_cmd.c: Fix typo in comment. redundent - redundant Signed-off-by: Karolin Seeger ksee...@samba.org commit b22b22dccac6d3bdf7f02d9fe037a472df7956fd Author: Karolin Seeger ksee...@samba.org Date: Mon Feb 18 10:04:42 2013 +0100 brlock_tdb.c: Fix typo in comment. redundent - redundant Signed-off-by: Karolin Seeger ksee...@samba.org commit 75ca814f1efe435da018f7604865fda7ac59f712 Author: Karolin Seeger ksee...@samba.org Date: Mon Feb 18 10:03:51 2013 +0100 srv_netlog_nt.c: Fix typo in comment. redundent - redundant Signed-off-by: Karolin Seeger ksee...@samba.org commit 6eb59eb388ac7b98f7f7812e45ad4c8d333f03e8 Author: Karolin Seeger ksee...@samba.org Date: Mon Feb 18 10:02:51 2013 +0100 brlock.c: Fix typo in comment. redundent - redundant Signed-off-by: Karolin Seeger ksee...@samba.org commit 38cb1410f5107f42ddea9cbf9555adb273b35b18 Author: Karolin Seeger ksee...@samba.org Date: Mon Feb 18 10:01:21 2013 +0100 vfs_gpfs: Fix typos in comments. Signed-off-by: Karolin Seeger ksee...@samba.org commit 2ed035b5a064e21f57c89adc9c947ffa7721c600 Author: Karolin Seeger ksee...@samba.org Date: Mon Feb 18 09:59:52 2013 +0100 fault.c: Fix typo in comment. redundent - redundant Signed-off-by: Karolin Seeger ksee...@samba.org commit 68b2e30ae62d8a563cb7ee35e10c45fe0266c612 Author: Karolin Seeger ksee...@samba.org Date: Wed Feb 6 09:08:15 2013 +0100 docs: Fix typo. Signed-off-by: Karolin Seeger ksee...@samba.org --- Summary of changes: .../smbdotconf/printing/showaddprinterwizard.xml |2 +- lib/util/fault.c |2 +- prog_guide4.txt|6 +++--- source3/locking/brlock.c |2 +- source3/modules/vfs_gpfs.c |4 ++-- source3/rpc_server/epmapper/srv_epmapper.c | 10 +- source3/rpc_server/netlogon/srv_netlog_nt.c|2 +- source4/ntvfs/common/brlock_tdb.c |2 +- source4/winbind/wb_samba3_cmd.c|2 +- 9 files changed, 16 insertions(+), 16 deletions(-) Changeset truncated at 500 lines: diff --git a/docs-xml/smbdotconf/printing/showaddprinterwizard.xml b/docs-xml/smbdotconf/printing/showaddprinterwizard.xml index f6c1b90..f24bdb5 100644 --- a/docs-xml/smbdotconf/printing/showaddprinterwizard.xml +++ b/docs-xml/smbdotconf/printing/showaddprinterwizard.xml @@ -14,7 +14,7 @@ paraUnder normal circumstances, the Windows NT/2000 client will open a handle on the printer server with OpenPrinterEx() asking for Administrator privileges. If the user does not have administrative -access on the print server (i.e is not root or the priviledge +access on the print server (i.e is not root or has granted the SePrintOperatorPrivilege), the OpenPrinterEx() call fails and the client makes another open call with a request for a lower privilege level. This should succeed, however the APW diff --git a/lib/util/fault.c b/lib/util/fault.c index 4f8e8db..13d29db 100644 --- a/lib/util/fault.c +++ b/lib/util/fault.c @@ -76,7 +76,7 @@ static void fault_report(int sig) smb_panic(internal error); - /* smb_panic() never returns, so this is really redundent */ + /* smb_panic() never returns, so this is really redundant */ exit(1); } diff --git a/prog_guide4.txt b/prog_guide4.txt index c8c91c4..0a33284 100644 --- a/prog_guide4.txt +++ b/prog_guide4.txt @@ -267,7 +267,7 @@ parser where to find the following four variables, but they should In Samba3 there were unwritten rules about which
[SCM] Samba Shared Repository - branch master updated
The branch, master has been updated via cfebce3 s3:smbd: add debugging to close code (regarding disconnect of a durable) via 2954442 s4:torture: fix segfault in test_durable_open_open2_oplock() via 2f8a033 s4:torture:smb2: fix segfault on error condition in the durable-open.delete_on_close2 test via 1d3bd45 s4:torture:smb2: fix segfault on error condition in the durable-open.delete_on_close1 test via 98b0e90 s4:torture:smb2: fix segfault on error condition in the durable-open.reopen4 test via 10fcbc6 s4:torture:smb2: fix segfault on error condition in the durable-open.reopen3 test via cb9b897 s4:torture:smb2: fix segfault on error condition in the durable-open.reopen2a test via 6385f75 s4:torture:smb2: fix segfault on error condition in durable-open.reopen2 test via 6240a7d s4:torture:smb2:durable: make test functions static via 42bf98d s4:torture: add a durable-open-disconnect test (suite) via f0e6a9b s3:smbd: use smbXsrv_open_close() instead of smbXsrv_open_update() from 8adbd1c srv_epmapper.c: Fix typo. http://gitweb.samba.org/?p=samba.git;a=shortlog;h=master - Log - commit cfebce3c56474ac914474b57ed94f93418b0564b Author: Michael Adam ob...@samba.org Date: Tue Feb 12 17:44:51 2013 +0100 s3:smbd: add debugging to close code (regarding disconnect of a durable) Signed-off-by: Michael Adam ob...@samba.org Reviewed-by: Stefan Metzmacher me...@samba.org Autobuild-User(master): Stefan Metzmacher me...@samba.org Autobuild-Date(master): Mon Feb 18 17:42:45 CET 2013 on sn-devel-104 commit 295444266d33863e3a8b7c8ffa5d193123db6132 Author: Stefan Metzmacher me...@samba.org Date: Wed Feb 13 14:11:57 2013 +0100 s4:torture: fix segfault in test_durable_open_open2_oplock() Signed-off-by: Stefan Metzmacher me...@samba.org Reviewed-by: Michael Adam ob...@samba.org commit 2f8a033bf2563d547e42be5603074223078595f9 Author: Michael Adam ob...@samba.org Date: Wed Feb 13 15:05:40 2013 +0100 s4:torture:smb2: fix segfault on error condition in the durable-open.delete_on_close2 test Signed-off-by: Michael Adam ob...@samba.org Reviewed-by: Stefan Metzmacher me...@samba.org commit 1d3bd45d6177c8af653dbacac934eb2061acf35e Author: Michael Adam ob...@samba.org Date: Wed Feb 13 15:04:10 2013 +0100 s4:torture:smb2: fix segfault on error condition in the durable-open.delete_on_close1 test Signed-off-by: Michael Adam ob...@samba.org Reviewed-by: Stefan Metzmacher me...@samba.org commit 98b0e909b7b65bee694743617476d4d36fe595a3 Author: Michael Adam ob...@samba.org Date: Wed Feb 13 15:03:00 2013 +0100 s4:torture:smb2: fix segfault on error condition in the durable-open.reopen4 test Signed-off-by: Michael Adam ob...@samba.org Reviewed-by: Stefan Metzmacher me...@samba.org commit 10fcbc6869dc2803e1c2dd3183c6781f4233550d Author: Michael Adam ob...@samba.org Date: Wed Feb 13 15:01:47 2013 +0100 s4:torture:smb2: fix segfault on error condition in the durable-open.reopen3 test Signed-off-by: Michael Adam ob...@samba.org Reviewed-by: Stefan Metzmacher me...@samba.org commit cb9b8975e5a9c032f7c4227c3eafb1ed6ada5e19 Author: Michael Adam ob...@samba.org Date: Wed Feb 13 15:00:26 2013 +0100 s4:torture:smb2: fix segfault on error condition in the durable-open.reopen2a test Signed-off-by: Michael Adam ob...@samba.org Reviewed-by: Stefan Metzmacher me...@samba.org commit 6385f750f19a00a6ba16a0aec9bb91ab1cfcb2be Author: Michael Adam ob...@samba.org Date: Wed Feb 13 14:58:29 2013 +0100 s4:torture:smb2: fix segfault on error condition in durable-open.reopen2 test Signed-off-by: Michael Adam ob...@samba.org Reviewed-by: Stefan Metzmacher me...@samba.org commit 6240a7d11e910027b42ac1d31ca13264e90d2266 Author: Michael Adam ob...@samba.org Date: Tue Feb 12 21:51:06 2013 +0100 s4:torture:smb2:durable: make test functions static Signed-off-by: Michael Adam ob...@samba.org Reviewed-by: Stefan Metzmacher me...@samba.org commit 42bf98d54c36abd5f532a4510cdd4066fe82143d Author: Michael Adam ob...@samba.org Date: Tue Feb 12 17:45:23 2013 +0100 s4:torture: add a durable-open-disconnect test (suite) this opens a durable, disconnects it and exits Signed-off-by: Michael Adam ob...@samba.org Reviewed-by: Stefan Metzmacher me...@samba.org commit f0e6a9be00e441e50f0087c543e1b7c9012d126f Author: Stefan Metzmacher me...@samba.org Date: Wed Feb 13 08:26:43 2013 -0500 s3:smbd: use smbXsrv_open_close() instead of smbXsrv_open_update() This makes sure we store the correct disconnect_time for disconnected durable handles. Signed-off-by: Stefan Metzmacher me...@samba.org Reviewed-by: Michael Adam ob...@samba.org
autobuild: intermittent test failure detected
The autobuild test system has detected an intermittent failing test in the current master tree. The autobuild log of the failure is available here: http://git.samba.org/autobuild.flakey/2013-02-19-0210/flakey.log The samba3 build logs are available here: http://git.samba.org/autobuild.flakey/2013-02-19-0210/samba3.stderr http://git.samba.org/autobuild.flakey/2013-02-19-0210/samba3.stdout The source4 build logs are available here: http://git.samba.org/autobuild.flakey/2013-02-19-0210/samba.stderr http://git.samba.org/autobuild.flakey/2013-02-19-0210/samba.stdout The top commit at the time of the failure was: commit cfebce3c56474ac914474b57ed94f93418b0564b Author: Michael Adam ob...@samba.org Date: Tue Feb 12 17:44:51 2013 +0100 s3:smbd: add debugging to close code (regarding disconnect of a durable) Signed-off-by: Michael Adam ob...@samba.org Reviewed-by: Stefan Metzmacher me...@samba.org Autobuild-User(master): Stefan Metzmacher me...@samba.org Autobuild-Date(master): Mon Feb 18 17:42:45 CET 2013 on sn-devel-104
[SCM] Samba Shared Repository - branch master updated
The branch, master has been updated via 06780ae samba_upgradeprovision: Remove options to fix FS ACLs from cfebce3 s3:smbd: add debugging to close code (regarding disconnect of a durable) http://gitweb.samba.org/?p=samba.git;a=shortlog;h=master - Log - commit 06780ae82281fb62a08d0c3604d2e679976756c2 Author: Andrew Bartlett abart...@samba.org Date: Sat Feb 16 08:51:51 2013 +1100 samba_upgradeprovision: Remove options to fix FS ACLs samba-tool ntacl sysvolreset handles this better, and makes this tool much less confusing internally. Andrew Bartlett Reviewed-by: Matthieu Patou m...@samba.org Autobuild-User(master): Andrew Bartlett abart...@samba.org Autobuild-Date(master): Tue Feb 19 06:06:41 CET 2013 on sn-devel-104 --- Summary of changes: source4/scripting/bin/samba_upgradeprovision | 427 +++--- source4/scripting/python/samba/upgradehelpers.py | 49 +--- 2 files changed, 205 insertions(+), 271 deletions(-) Changeset truncated at 500 lines: diff --git a/source4/scripting/bin/samba_upgradeprovision b/source4/scripting/bin/samba_upgradeprovision index e2c57f2..570f783 100755 --- a/source4/scripting/bin/samba_upgradeprovision +++ b/source4/scripting/bin/samba_upgradeprovision @@ -186,12 +186,6 @@ parser.add_option(--debugchangesd, action=store_true, help=Print security descriptor differences) parser.add_option(--debugall, action=store_true, help=Print all available information (very verbose)) -parser.add_option(--resetfileacl, action=store_true, - help=Force a reset on filesystem acls in sysvol / netlogon share) -parser.add_option(--nontaclfix, action=store_true, - help=In full upgrade mode do not try to upgrade sysvol / netlogon acls) -parser.add_option(--fixntacl, action=store_true, - help=Only fix NT ACLs in sysvol / netlogon share) parser.add_option(--db_backup_only, action=store_true, help=Do the backup of the database in the provision, skip the sysvol / netlogon shares) parser.add_option(--full, action=store_true, @@ -1726,8 +1720,6 @@ if __name__ == '__main__': global defSDmodified defSDmodified = False -if opts.nontaclfix and opts.fixntacl: -message(SIMPLE, nontaclfix and fixntacl are mutally exclusive) # From here start the big steps of the program # 1) First get files paths paths = get_paths(param, smbconf=smbconf) @@ -1787,225 +1779,214 @@ if __name__ == '__main__': adm_session = admin_session(lp, str(names.domainsid)) # So we reget handle on objects # ldbs = get_ldbs(paths, creds, adm_session, lp) -if not opts.fixntacl: -if not sanitychecks(ldbs.sam, names): -message(SIMPLE, Sanity checks for the upgrade have failed. -Check the messages and correct the errors -before rerunning upgradeprovision) -ldbs.groupedRollback() -sys.exit(1) -# Let's see provision parameters -print_provision_key_parameters(names) - -# 5) With all this information let's create a fresh new provision used as -# reference -message(SIMPLE, Creating a reference provision) -provisiondir = tempfile.mkdtemp(dir=paths.private_dir, -prefix=referenceprovision) -result = newprovision(names, creds, session, smbconf, provisiondir, -provision_logger) -result.report_logger(provision_logger) - -# TODO -# 6) and 7) -# We need to get a list of object which SD is directly computed from -# defaultSecurityDescriptor. -# This will allow us to know which object we can rebuild the SD in case -# of change of the parent's SD or of the defaultSD. -# Get file paths of this new provision -newpaths = get_paths(param, targetdir=provisiondir) -new_ldbs = get_ldbs(newpaths, creds, session, lp) -new_ldbs.startTransactions() - -populateNotReplicated(new_ldbs.sam, names.schemadn) -# 8) Populate some associative array to ease the update process -# List of attribute which are link and backlink -populate_links(new_ldbs.sam, names.schemadn) -# List of attribute with ASN DN synthax) -populate_dnsyntax(new_ldbs.sam, names.schemadn) -# 9) -update_privilege(newpaths.private_dir, paths.private_dir) -# 10) -oem = getOEMInfo(ldbs.sam, str(names.rootdn)) -# Do some modification on sam.ldb -ldbs.groupedCommit() -new_ldbs.groupedCommit() -
[SCM] CTDB repository - branch master updated - ctdb-2.1-26-gd788bc8
The branch, master has been updated
via d788bc8f7212b7dc1587ae592242dc8c876f4053 (commit)
from 855ab348901edb3ec1327499a43f509d279b8182 (commit)
http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=master
- Log -
commit d788bc8f7212b7dc1587ae592242dc8c876f4053
Author: Amitay Isaacs ami...@gmail.com
Date: Fri Jan 18 10:42:14 2013 +1100
common/io: Rewrite socket handling code to read all available data
This improves the processing of packets considerably. It has been
observed that there can be as many as 10 packets in the socket buffer and
the current code of reading a single packet from a socket at a time is
not very optimal. This change reads all the bytes from socket buffer and
then parses to extract multiple packets. If there are multiple packets,
set up a timed event to process next packet.
Signed-off-by: Amitay Isaacs ami...@gmail.com
---
Summary of changes:
common/ctdb_io.c | 160 +++---
1 files changed, 92 insertions(+), 68 deletions(-)
Changeset truncated at 500 lines:
diff --git a/common/ctdb_io.c b/common/ctdb_io.c
index 3ac1b63..b4224c4 100644
--- a/common/ctdb_io.c
+++ b/common/ctdb_io.c
@@ -30,9 +30,10 @@
#include stdarg.h
/* structures for packet queueing - see common/ctdb_io.c */
-struct ctdb_partial {
+struct ctdb_buffer {
uint8_t *data;
uint32_t length;
+ uint32_t size;
};
struct ctdb_queue_pkt {
@@ -44,7 +45,7 @@ struct ctdb_queue_pkt {
struct ctdb_queue {
struct ctdb_context *ctdb;
- struct ctdb_partial partial; /* partial input packet */
+ struct ctdb_buffer buffer; /* input buffer */
struct ctdb_queue_pkt *out_queue, *out_queue_tail;
uint32_t out_queue_length;
struct fd_event *fde;
@@ -63,6 +64,75 @@ int ctdb_queue_length(struct ctdb_queue *queue)
return queue-out_queue_length;
}
+static void queue_process(struct ctdb_queue *queue);
+
+static void queue_process_event(struct event_context *ev, struct timed_event
*te,
+ struct timeval t, void *private_data)
+{
+ struct ctdb_queue *queue = talloc_get_type(private_data, struct
ctdb_queue);
+
+ queue_process(queue);
+}
+
+/*
+ * This function is used to process data in queue buffer.
+ *
+ * Queue callback function can end up freeing the queue, there should not be a
+ * loop processing packets from queue buffer. Instead set up a timed event for
+ * immediate run to process remaining packets from buffer.
+ */
+static void queue_process(struct ctdb_queue *queue)
+{
+ uint32_t pkt_size;
+ uint8_t *data;
+
+ if (queue-buffer.length sizeof(pkt_size)) {
+ return;
+ }
+
+ pkt_size = *(uint32_t *)queue-buffer.data;
+ if (pkt_size == 0) {
+ DEBUG(DEBUG_CRIT, (Invalid packet of length 0\n));
+ goto failed;
+ }
+
+ if (queue-buffer.length pkt_size) {
+ DEBUG(DEBUG_DEBUG, (Partial packet data read\n));
+ return;
+ }
+
+ /* Extract complete packet */
+ data = talloc_size(queue, pkt_size);
+ if (data == NULL) {
+ DEBUG(DEBUG_ERR, (read error alloc failed for %u\n,
pkt_size));
+ return;
+ }
+ memcpy(data, queue-buffer.data, pkt_size);
+
+ /* Shift packet out from buffer */
+ if (queue-buffer.length pkt_size) {
+ memmove(queue-buffer.data,
+ queue-buffer.data + pkt_size,
+ queue-buffer.length - pkt_size);
+ }
+ queue-buffer.length -= pkt_size;
+
+ if (queue-buffer.length 0) {
+ /* There is more data to be processed, setup timed event */
+ event_add_timed(queue-ctdb-ev, queue, timeval_zero(),
+ queue_process_event, queue);
+ }
+
+ /* It is the responsibility of the callback to free 'data' */
+ queue-callback(data, pkt_size, queue-private_data);
+ return;
+
+failed:
+ queue-callback(NULL, 0, queue-private_data);
+
+}
+
+
/*
called when an incoming connection is readable
This function MUST be safe for reentry via the queue callback!
@@ -70,10 +140,6 @@ int ctdb_queue_length(struct ctdb_queue *queue)
static void queue_io_read(struct ctdb_queue *queue)
{
int num_ready = 0;
- uint32_t sz_bytes_req;
- uint32_t pkt_size;
- uint32_t pkt_bytes_remaining;
- uint32_t to_read;
ssize_t nread;
uint8_t *data;
@@ -91,77 +157,33 @@ static void queue_io_read(struct ctdb_queue *queue)
goto failed;
}
- if (queue-partial.data == NULL) {
- /* starting fresh, allocate buf for size bytes */
- sz_bytes_req = sizeof(pkt_size);
-
[SCM] Samba Shared Repository - branch master updated
The branch, master has been updated via 2cf83f7 samba_upgradeprovision: Use tdb_util.tdb_copy not shutil.copy2 via 3c51e18 samba_upgradeprovision: Do not update privileges.ldb any more (unchanged since 2009) via 396df64 scripting: Make tdb_copy a common util function in samba.tdb_util via 2c2759e scripting: Make tdb_copy use the python subprocess module from 06780ae samba_upgradeprovision: Remove options to fix FS ACLs http://gitweb.samba.org/?p=samba.git;a=shortlog;h=master - Log - commit 2cf83f7c645e4b216cf6f23857fd72ec0e6ca7a6 Author: Andrew Bartlett abart...@samba.org Date: Sun Feb 17 18:15:52 2013 +1100 samba_upgradeprovision: Use tdb_util.tdb_copy not shutil.copy2 This is really important, because copying a file will both ignore locks held by another process and break any locks we hold (due to POSIX brain-damage regarding multiple fds on one file in a process). By leaving this to tdbbackup in a child, both of these issues are avoided. Andrew Bartlett Reviewed-by: Matthieu Patou m...@samba.org Autobuild-User(master): Andrew Bartlett abart...@samba.org Autobuild-Date(master): Tue Feb 19 07:48:18 CET 2013 on sn-devel-104 commit 3c51e18a0cd1cb4b54cd29e312abd7cc2c0fbc98 Author: Andrew Bartlett abart...@samba.org Date: Sun Feb 17 18:41:00 2013 +1100 samba_upgradeprovision: Do not update privileges.ldb any more (unchanged since 2009) This update was only a total oblitoration of the existing database and not a merge, and the shutil.copy would both disregard and break locks on the database that are held at this point. Andrew Bartlett Reviewed-by: Matthieu Patou m...@samba.org commit 396df64ef6f2c66c35989ecda3e564d5578fe9f3 Author: Andrew Bartlett abart...@samba.org Date: Sun Feb 17 18:14:06 2013 +1100 scripting: Make tdb_copy a common util function in samba.tdb_util This will allow samba_upgradeprovision to also call it. Andrew Bartlett Reviewed-by: Matthieu Patou m...@samba.org commit 2c2759e408d9c45c2aee0c2578f45edd246afec3 Author: Andrew Bartlett abart...@samba.org Date: Sun Feb 17 17:57:42 2013 +1100 scripting: Make tdb_copy use the python subprocess module This makes the code more robust to spaces in the file names (etc). Andrew Bartlett Reviewed-by: Matthieu Patou m...@samba.org --- Summary of changes: source4/scripting/bin/samba_upgradeprovision | 51 --- .../scripting/python/samba/provision/sambadns.py | 23 + source4/scripting/python/samba/tdb_util.py | 41 3 files changed, 66 insertions(+), 49 deletions(-) create mode 100644 source4/scripting/python/samba/tdb_util.py Changeset truncated at 500 lines: diff --git a/source4/scripting/bin/samba_upgradeprovision b/source4/scripting/bin/samba_upgradeprovision index 570f783..25c3ac2 100755 --- a/source4/scripting/bin/samba_upgradeprovision +++ b/source4/scripting/bin/samba_upgradeprovision @@ -40,6 +40,7 @@ import samba.getopt as options from base64 import b64encode from samba.credentials import DONT_USE_KERBEROS from samba.auth import system_session, admin_session +from samba import tdb_util from ldb import (SCOPE_SUBTREE, SCOPE_BASE, FLAG_MOD_REPLACE, FLAG_MOD_ADD, FLAG_MOD_DELETE, MessageElement, Message, Dn, LdbError) @@ -1470,7 +1471,7 @@ def simple_update_basesamdb(newpaths, paths, names): :param names: List of key provision parameters message(SIMPLE, Copy samdb) -shutil.copy(newpaths.samdb, paths.samdb) +tdb_util.tdb_copy(newpaths.samdb, paths.samdb) message(SIMPLE, Update partitions filename if needed) schemaldb = os.path.join(paths.private_dir, schema.ldb) @@ -1482,31 +1483,19 @@ def simple_update_basesamdb(newpaths, paths, names): os.mkdir(samldbdir) os.chmod(samldbdir, 0700) if os.path.isfile(schemaldb): -shutil.copy(schemaldb, os.path.join(samldbdir, +tdb_util.tdb_copy(schemaldb, os.path.join(samldbdir, %s.ldb%str(names.schemadn).upper())) os.remove(schemaldb) if os.path.isfile(usersldb): -shutil.copy(usersldb, os.path.join(samldbdir, +tdb_util.tdb_copy(usersldb, os.path.join(samldbdir, %s.ldb%str(names.rootdn).upper())) os.remove(usersldb) if os.path.isfile(configldb): -shutil.copy(configldb, os.path.join(samldbdir, +tdb_util.tdb_copy(configldb, os.path.join(samldbdir, %s.ldb%str(names.configdn).upper())) os.remove(configldb) -def update_privilege(ref_private_path, cur_private_path): -Update the privilege
[SCM] CTDB repository - branch 1.2.40 updated - ctdb-1.2.57-7-g4455822
The branch, 1.2.40 has been updated via 44558223c2f83cafbe4ee63b4ce3d508dc7f0a02 (commit) via 9d8338fc14cfa5f4d8236eccb90ffdd4cdc11343 (commit) via c6e1b84595039edb5c49a5851b440710dc0e2ac1 (commit) via aae948c3c2ce1d2678ce57b4219674bafda5008e (commit) via 4697a83866a04357d386473a50685f677a730daf (commit) via 1cbdb44d604461509676b903a9e9515e7cab2598 (commit) via 537bed11c339a10faf45a6ede778f7c8e26e4f91 (commit) from 584e8c25971219614f3f2bfd5be233b04de7e0af (commit) http://gitweb.samba.org/?p=ctdb.git;a=shortlog;h=1.2.40 - Log - commit 44558223c2f83cafbe4ee63b4ce3d508dc7f0a02 Author: Amitay Isaacs ami...@gmail.com Date: Tue Feb 19 18:09:05 2013 +1100 New Version 1.2.58 Signed-off-by: Amitay Isaacs ami...@gmail.com commit 9d8338fc14cfa5f4d8236eccb90ffdd4cdc11343 Author: Ronnie Sahlberg ronniesahlb...@gmail.com Date: Fri May 25 12:27:59 2012 +1000 RECOVER: When we pull databases during recovery, we used to reallocate the databuffer for each entry added. This would normally not be an issue, but for cases where memory is fragmented, this could start to cost significant cpu if we need to reallocate and move to a different region. Change this to instead preallocate , by default, 10MByte chunks to the data buffer. This significantly reduces the number of potential reallocate and move operations that may be required. Create a tunable to override/change how much preallocation should be used. Conflicts: include/ctdb_private.h server/ctdb_tunables.c Cherry-pick-from: 1f262deaad0818f159f9c68330f7fec121679023 Also, make sure the preallocation size is 10MB and not 100MB. Signed-off-by: Amitay Isaacs ami...@gmail.com commit c6e1b84595039edb5c49a5851b440710dc0e2ac1 Author: Martin Schwenke mar...@meltin.net Date: Tue Feb 5 12:09:36 2013 +1100 Logging: Free the ringbuffer in child processes created with ctdb_fork() At the moment the log ringbuffer is duplicated in every child process. Althought it is copy-on-write we want to see if it is contributing to out-of-memory situations when there are a lot of children. The ringbuffer isn't accessible from any of the children anyway... Signed-off-by: Martin Schwenke mar...@meltin.net Conflicts: common/ctdb_fork.c Cherry-pick-from: a82d3ec12f0fda16d6bfa8442a07595de897c10e commit aae948c3c2ce1d2678ce57b4219674bafda5008e Author: Martin Schwenke mar...@meltin.net Date: Tue Feb 5 12:08:11 2013 +1100 Logging: New function ctdb_log_ringbuffer_free() Signed-off-by: Martin Schwenke mar...@meltin.net Cherry-pick-from: a4f622e85168f59417c11705f1734e0352e1d44a commit 4697a83866a04357d386473a50685f677a730daf Author: Amitay Isaacs ami...@gmail.com Date: Mon Feb 11 11:25:49 2013 +1100 eventscripts: Remove calls to smbstatus -np for samba cleanup This is an artifact from older versions of Samba. In the newer versions of Samba, smbstatus -np command does not do anything useful, but causes a traverse in CTDB which is expensive and causes CPU utilization to shoot up. Signed-off-by: Amitay Isaacs ami...@gmail.com Conflicts: config/events.d/50.samba Cherry-pick-from: 053b89c6dbce47001505524606889334559d2ec4 commit 1cbdb44d604461509676b903a9e9515e7cab2598 Author: Amitay Isaacs ami...@gmail.com Date: Fri Jan 18 10:42:14 2013 +1100 common/io: Rewrite socket handling code to read all available data This improves the processing of packets considerably. It has been observed that there can be as many as 10 packets in the socket buffer and the current code of reading a single packet from a socket at a time is not very optimal. This change reads all the bytes from socket buffer and then parses to extract multiple packets. If there are multiple packets, set up a timed event to process next packet. Signed-off-by: Amitay Isaacs ami...@gmail.com Cherry-pick-from: d788bc8f7212b7dc1587ae592242dc8c876f4053 Conflicts: common/ctdb_io.c commit 537bed11c339a10faf45a6ede778f7c8e26e4f91 Author: Amitay Isaacs ami...@gmail.com Date: Tue Jan 22 13:27:20 2013 +1100 daemon: Make sure all the traverse children are terminated if traverse times out When traverse times out, callback function is called with key and data set to tdb_null. This is also the way to signal end of traverse. So if the traverse times out, callback function treats it as traverse ended and frees state without calling the destructor. Keep track if the traverse timed out, so callback function can take appropriate action for traverse timeout and traverse end. Signed-off-by: Amitay Isaacs ami...@gmail.com Cherry-pick-from:
[SCM] CTDB repository - annotated tag ctdb-1.2.58 created - ctdb-1.2.58
The annotated tag, ctdb-1.2.58 has been created at 347a504424435225d8829edb9e04d24349bfee77 (tag) tagging 44558223c2f83cafbe4ee63b4ce3d508dc7f0a02 (commit) replaces ctdb-1.2.57 tagged by Amitay Isaacs on Tue Feb 19 18:10:12 2013 +1100 - Log - new version 1.2.58 Amitay Isaacs (4): daemon: Make sure all the traverse children are terminated if traverse times out common/io: Rewrite socket handling code to read all available data eventscripts: Remove calls to smbstatus -np for samba cleanup New Version 1.2.58 Martin Schwenke (2): Logging: New function ctdb_log_ringbuffer_free() Logging: Free the ringbuffer in child processes created with ctdb_fork() Ronnie Sahlberg (1): RECOVER: When we pull databases during recovery, we used to reallocate the databuffer for each entry added. This would normally not be an issue, but for cases where memory is fragmented, this could start to cost significant cpu if we need to reallocate and move to a different region. --- -- CTDB repository
