The branch, master has been updated via d6f1215 KCC: avoid (so far harmless) variable name clash via 5aa3fca KCC: colour dot graph vertices as well as edges via 2e9fefc KCC: use connect_table.values(), not .items() where that's best via 68bb12b KCC: adjust update_rodc_connection to cope with multiple connections via b01100b KCC: use a table of bridgeheads to simplify and clarify via 9ad8a32 KCC: remove unused enumeration variable via c7bf297 KCC: a "should not happen" vanishing DSA is worth a warning via dc8aa6f KCC: index DSAs by guid and dn_str when loading sites via ce707ae kcc: Change create_connection to properly query existing connections via 0e77967 KCC: samba_kcc --test-all-reps-from tries to graph repsfrom via 8877716 KCC: sort vertices by ndr_packed_guid instead of string guid via 6c24a7b KCC: changes in debugging messages via 948da94 KCC: warn when we are ignoring non-IP transports via 6153cf5 KCC: waste less time assigning transport a NULL guid via e278722 KCC: use a set to track kept connections via 4d0f892 KCC: Comment about schedule types via 71045b3 KCC: add options to forget local and intersite links via cf362ab KCC: kcc.my_site_guid was not really used via 73fe7ed KCC: use more efficient, less polluting dictionary operations via 9b7d11b KCC: truth in documentation via b1c9039 KCC: a whole lot of debugging messages via 8ff6093 KCC: correctly update flags when repsFrom schedule is changed via 59af1b5 KCC: fix is_bridgehead_failed() according to documentation via bb945df KCC: set site topo_generator in RO case, for debug clarity via e299fbc KCC: colour dsa dot graphs, and plot site edges via 1b681ee KCC: don't try to connect to a nonexistent bridgehead via 68ef17f KCC: Don't form self connections via 6146a8a KCC: reuse convert_schedule_to_repltimes() for Sitelinks via 3aad85f KCC: fix NTDSConnection.convert_schedule_to_repltimes via a8a1724 KCC: InternalEdge should know its sitelink via 2bb2480 KCC: use rep.is_default rather than non-existent part.is_default via 92a9bce KCC: debugging changes via fee5717 KCC: add colouring for DSA link graphs via 66116d5 KCC: Use fake guids in readonly mode via 660e125 KCC: beginning of --test-all-reps-from, exposing brokenness via 35f2b2b KCC: kcc.list_dsas() returns a list rather than prints it via 8a8ba28 KCC: intrasite doesn't need a ring with < 3 nodes via 5fbdf6e KCC: intrasite graph only uses local DSAs via 49e13f8 KCC: post-Kruskal graph test only needs multi-edge-forest via 4ad04a0 KCC: minor formatting changes via 5c72b13 KCC: move console colours from kcc_utils to graph_utils via c33761d KCC: pep8 for ldif_utils.py via 070c9f7 KCC: move ldif import/export functions into their own module via e753d11 KCC: rearrange samba_kcc entry point via 00e8d92 KCC: move import to top of file via e66a86b KCC: add an option to list valid DSA dn_strs via edd4aa2 KCC graph_utils: rename KCCGraphError, GraphError via 222b822 KCC: try graphing edges in colour for partition type via 195b9f4 KCC: shift graph plotting and verification into a separate module via d383cd6 KCC: improved documentation for verify_graph_directed_double_ring via a2d3b31 KCC: more graph verification: repsFrom via 7e59faa KCC: the commit that mops up stray comments via d80cdcd KCC: add --attempt-live-connections to verify real life links via 0331c53 KCC: make --verify graph errors fatal via d683877 KCC: attempt to use correct verification tests for graphs via c35ba6f KCC: debug nc_type by name rather than enum number via 59e9ae6 KCC: sitelink graph is undirected via a4cdd90 KCC: use ndr_pack for connection GUID comparisons via eb498a1 KCC: consistently use ndr_pack() for DSA GUID comparisons via 7e4dfb7 KCC: Use ndrpack(GUID) sorting for all graph vertex operations via abfb702 kcc: reduce brokenness of --import-lidf via 259d122 KCC: skip looking for extra edges if we have enough via 5246a13 KCC: do not skip unrelated edges when rodc topology is found via 736ecf0 KCC: A whole lot of debugging and other non-functional changes via b73212c kcc: sort guids by ndr_pack representation via b47efc0 KCC: rewrite random intrasite connection addition for clarity via 333c8ad KCC: intrasite_post_ntdscon is directed via 66c1759 KCC: add NC identifiers to dot file names via 159db2a KCC: stub for check of 1 or 2 node or double ring directed graph via 7501313 kcc: Interpret msDS-hasMasterNCs as hasMasterNCs via 8b6f3e6 KCC: Fix original script for inverted logic via 0b17932 KCC: rename "fully_connected" check, "complete" via 2a01bce KCC: write_dot_graph prints filename in --debug mode via 1899109 KCC: Set option --readonly to default to False via c8944ae KCC: add directed double ring verification via 7c39344 KCC: add an option to list the graph verification options via d474bfa KCC: comment noting the progression of edge limit calculation via 071ef32 KCC: add a wider range of colour DEBUG messages via 4889770 KCC: add graph verification at all dot file points via e19330e KCC: add an option to verify graph properties from 591f71f s3-libsmbclient: change vnum to 0.2.2
https://git.samba.org/?p=samba.git;a=shortlog;h=master - Log ----------------------------------------------------------------- commit d6f12159d111fd850ae77e6e6b70b988c983b91d Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Fri Mar 27 15:38:04 2015 +1300 KCC: avoid (so far harmless) variable name clash Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> Autobuild-User(master): Andrew Bartlett <abart...@samba.org> Autobuild-Date(master): Fri May 29 09:45:23 CEST 2015 on sn-devel-104 commit 5aa3fca44d490e76f8dbebb7c433c20413bab50a Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Fri Mar 27 15:37:17 2015 +1300 KCC: colour dot graph vertices as well as edges It's quicker to see who is RODC. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 2e9fefc738dd8b15be0a51aff820e2af33b9cf45 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Fri Mar 27 15:18:38 2015 +1300 KCC: use connect_table.values(), not .items() where that's best The keys in dsa.connect_table objects are fairly arbitrary strings (containing guids) and are of no use except when we're deleting connections. Looping over the .items() clutters up the namespace and potentially confuses or clobbers, given the hugeness of some of these functions. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 68bb12bef1a34cde8dd4ffb81e2a3fa0d1af09cf Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Fri Mar 27 15:02:37 2015 +1300 KCC: adjust update_rodc_connection to cope with multiple connections Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit b01100bf949d8fff0de6b5b8a789c618a3af10d0 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Fri Mar 27 12:02:37 2015 +1300 KCC: use a table of bridgeheads to simplify and clarify Looping over lists to find membership has bitten us enough times. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 9ad8a321dc48780b49000cd5fe4b59345f8b39e1 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Fri Mar 27 12:00:38 2015 +1300 KCC: remove unused enumeration variable Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit c7bf297e004e1077f1db0097dd65ca2e37ce0c6a Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Fri Mar 27 12:00:00 2015 +1300 KCC: a "should not happen" vanishing DSA is worth a warning Not just a debug message. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit dc8aa6f00638889d88476a2e97c0022c8eea98fd Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Fri Mar 27 11:58:48 2015 +1300 KCC: index DSAs by guid and dn_str when loading sites Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit ce707aec0cd96f399d8486be43821464dba07878 Author: Garming Sam <garm...@catalyst.net.nz> Date: Thu Mar 26 18:13:43 2015 +1300 kcc: Change create_connection to properly query existing connections Originally it didn't correctly detect the right 'from server' continuing with any connection. Signed-off-by: Garming Sam <garm...@catalyst.net.nz> Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 0e77967c315057f51a8ce1e84a7220f035736db1 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 26 17:50:42 2015 +1300 KCC: samba_kcc --test-all-reps-from tries to graph repsfrom In future it may try to verify them too. Right now it reveals that KCC is broken. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 8877716d980e255743844e43f0763717498698a9 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 26 12:17:06 2015 +1300 KCC: sort vertices by ndr_packed_guid instead of string guid ndr_packed guids seems to be what windows uses. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 6c24a7b95679035812d69ee9192f1e8b0bf3f069 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 26 12:16:02 2015 +1300 KCC: changes in debugging messages Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 948da9474df41e856d58410c12b563138d2e1e95 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 26 12:14:44 2015 +1300 KCC: warn when we are ignoring non-IP transports Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 6153cf5cea1fe4ad562c6f4d390832928166a482 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 26 12:05:05 2015 +1300 KCC: waste less time assigning transport a NULL guid Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit e278722635b223835cf2a1214c103fe4c0660781 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 26 12:02:50 2015 +1300 KCC: use a set to track kept connections Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 4d0f892a1ab13e8a374ff988fb440b22c775de65 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Apr 23 17:08:34 2015 +1200 KCC: Comment about schedule types Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 71045b33255ae9695bc76afb071ad91408b4032f Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed Mar 25 17:47:59 2015 +1300 KCC: add options to forget local and intersite links The graphs formed using these options are likely to be different from those you would normally get, which is OK because this is only for testing. Also add a helper function to draw a graph from DSA tables. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit cf362abe4bba078c875ac4ff6c2309d66274317f Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed Mar 25 17:47:13 2015 +1300 KCC: kcc.my_site_guid was not really used Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 73fe7ed8ecd9412df3bca2d7f510c607632fcfa0 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Fri May 1 16:54:05 2015 +1200 KCC: use more efficient, less polluting dictionary operations `in d` is more efficient and idiomatic than `in d.keys()`. `for v in d.values()` is better than `for k, v in d.items()` when `k` is not used. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 9b7d11bf5873d8658c7d2671562628b3f867fb1a Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed Mar 25 12:12:53 2015 +1300 KCC: truth in documentation Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit b1c9039d51aeaf61d84bff8fdf596d828ac28ab5 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed Mar 25 11:55:03 2015 +1300 KCC: a whole lot of debugging messages Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 8ff60938ce37b7b5cea54ba5ebdc0966f4407024 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed Mar 25 11:53:26 2015 +1300 KCC: correctly update flags when repsFrom schedule is changed Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 59af1b51755086d12aed19737a4e84f86fd3b889 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Fri Mar 20 16:40:05 2015 +1300 KCC: fix is_bridgehead_failed() according to documentation Throughout the KCC specification `detectFailedDCs` is documented along the lines of "true to detect failed DCs", and it gets passed down to this function. And what do we see here? It is used as a default value when a stale link is not detected. That is entirely different. So who is right -- the comments or the pseudo-code? This commit follows the comments. It works! Documentation 1, Pseudo-code 0 See [MS-ADTS] — v20140502, section 6.2.2.3.4.4, page 569. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit bb945df0ef3cb9ee73cbba58e2a5ac9267073872 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Fri Mar 20 16:02:58 2015 +1300 KCC: set site topo_generator in RO case, for debug clarity This has no actual effect except that it makes the Site object print with the site_topo_generator that is actually used (because RODCs are always their own topology generators and this variable doesn't affect that). Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit e299fbcc40ac5bbbafa16069f390b1e188d1ba6a Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Fri Mar 20 15:57:23 2015 +1300 KCC: colour dsa dot graphs, and plot site edges The red links in the dsa graphs are RODC topology. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 1b681ee1c9dff14424a9723ba661e60c8cbac5b0 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Fri Mar 20 15:39:28 2015 +1300 KCC: don't try to connect to a nonexistent bridgehead If self.get_bridgehead returns None, there are no eligible bridgeheads over there to connect to. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 68ef17f8ee8a103c95551e37c657fc39429ce47e Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Fri Mar 20 15:31:57 2015 +1300 KCC: Don't form self connections This was a bug found in the original script. We have found a remote bridgehead to connect to, but we were accidentally connection to ourselves instead. Oops. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 6146a8a3f8e0c9e2aac932c270887b9fc62f923a Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed May 6 16:43:20 2015 +1200 KCC: reuse convert_schedule_to_repltimes() for Sitelinks This means wrenching it out of NTDSConnection, where it didn't really need to be anyway. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 3aad85f2a2d0f309a920328a60ee078fb3ae5bf0 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed May 6 16:41:09 2015 +1200 KCC: fix NTDSConnection.convert_schedule_to_repltimes There were two bugs: the wrong nibbles were being added, and they weren't actually being saved. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit a8a172471face6e986571a4895864ffc62b5145d Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed May 6 16:34:58 2015 +1200 KCC: InternalEdge should know its sitelink Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 2bb248076574c2b518fc9bbe792dd207a9084ad6 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 19 16:50:11 2015 +1300 KCC: use rep.is_default rather than non-existent part.is_default Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 92a9bce47506a0d6ae5632d8f7139ef8f0671069 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 19 16:48:25 2015 +1300 KCC: debugging changes Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit fee5717cad6c1f5535db53d7a9f534cc8631c6ff Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 19 16:47:42 2015 +1300 KCC: add colouring for DSA link graphs Red for RODC, blue otherwise. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 66116d5b732d6bac20b4078b67591ae9e5cb3fad Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 19 16:05:24 2015 +1300 KCC: Use fake guids in readonly mode This is necessary because the connections need to look valid so they are found when unnecesary connections are removed. The usual way is to query the database for the guid. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 660e1258c6e25007bfbda9a200f86bdd9789ba72 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 19 14:26:16 2015 +1300 KCC: beginning of --test-all-reps-from, exposing brokenness This should produce a list of repsFrom links from all DSAs, which together should produce a nice graph. samba_kcc --test-all-reps-from won't actually run, but the problems are things it found rather than caused. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 35f2b2b264cedad6fbaa93321a8453ee45e99b5e Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 19 14:22:42 2015 +1300 KCC: kcc.list_dsas() returns a list rather than prints it This will be useful for the --test-all-reps-from option. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 8a8ba28f400958592d5c1dddd4a2db6442963337 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 19 12:07:29 2015 +1300 KCC: intrasite doesn't need a ring with < 3 nodes Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 5fbdf6e8cb72f4c636135c14b5959f70725a8131 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 19 12:06:36 2015 +1300 KCC: intrasite graph only uses local DSAs In the specs this is implied by the special definition of 'implies'. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 49e13f8310d409e869ec1ec24cc51fff43cf1a1c Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 19 12:04:40 2015 +1300 KCC: post-Kruskal graph test only needs multi-edge-forest Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 4ad04a0f6f5be9f1cef750e821605fde4e3c241c Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 19 12:03:53 2015 +1300 KCC: minor formatting changes Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 5c72b13887ff6642ecefde6fd42a16993c788992 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 19 12:03:08 2015 +1300 KCC: move console colours from kcc_utils to graph_utils Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit c33761d9be052d8389606e3b887cff917b8b708a Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Apr 9 14:46:05 2015 +1200 KCC: pep8 for ldif_utils.py Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 070c9f7094a7840f89d45039e7f17b731ec89ead Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 19 10:46:47 2015 +1300 KCC: move ldif import/export functions into their own module They might be of use elsewhere, and they are easily separable from the KCC core. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit e753d11e4d22815c4991b09b89b0d289c4b4fc0b Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 19 10:42:37 2015 +1300 KCC: rearrange samba_kcc entry point For locality, so that it becomes clearer which variables are used where. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 00e8d929ba418707be6c2cbc93f009064d3c2d2c Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 19 09:29:00 2015 +1300 KCC: move import to top of file Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit e66a86b45c7531bbd122a570fae607fd89e430e3 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed Mar 18 18:24:52 2015 +1300 KCC: add an option to list valid DSA dn_strs This makes testing a bit easier. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit edd4aa2d4703f9a7b24c5263829ac42f95f7a825 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed Mar 18 18:23:06 2015 +1300 KCC graph_utils: rename KCCGraphError, GraphError In case it gets used elsewhere. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 222b8221d3c9cc44c1e0e14a5dbe8d19b6a2a846 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed Mar 18 18:22:14 2015 +1300 KCC: try graphing edges in colour for partition type It doesn't add much. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 195b9f4c79240534f88f089deccf46c3be6058e4 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed Mar 18 17:17:02 2015 +1300 KCC: shift graph plotting and verification into a separate module These might possibly be useful outside the KCC context, and the don't rely on the rest of kcc_utils. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit d383cd6f5e106280cedbab281cd341acef02a57a Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed Mar 18 17:00:07 2015 +1300 KCC: improved documentation for verify_graph_directed_double_ring The actual function is still somewhat broken. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit a2d3b3117c5e95cec8cccd06adc79c0ba704ced5 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed Mar 18 16:58:29 2015 +1300 KCC: more graph verification: repsFrom Some of these graphs are wrong headed and broken Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 7e59faa10fde5f8beb295ce62b8c023ef277f1c0 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed Mar 18 14:27:50 2015 +1300 KCC: the commit that mops up stray comments Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit d80cdcd609c0d5855c58c4e9516bb8bd7146acf7 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed Mar 18 14:24:07 2015 +1300 KCC: add --attempt-live-connections to verify real life links The spec makes these connections in a couple of places. That makes no sense in our main testing environment which is a samba database pretending to be a network. The default should be changed to True when failed link detection is implemented. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 0331c53b4cf30c299fadb24892ddb511a9c62659 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed Mar 18 14:23:21 2015 +1300 KCC: make --verify graph errors fatal Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit d68387774dd25c38dd1cf4a8a03fb8f5032178b4 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed Mar 18 11:33:53 2015 +1300 KCC: attempt to use correct verification tests for graphs Before they were liberally assigned because the failures were as interesting as the passes. Also add a stub for the forest_of_rings test. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit c35ba6f349bcca38b051cfb4529a909a2f5ca9ca Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed Mar 18 11:30:14 2015 +1300 KCC: debug nc_type by name rather than enum number Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 59e9ae64d863542f40ad50903f731652787d7392 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed May 20 12:19:02 2015 +1200 KCC: sitelink graph is undirected Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit a4cdd907b58d3dcf22d86be197e0eb6adc9be141 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Tue Mar 17 11:58:00 2015 +1300 KCC: use ndr_pack for connection GUID comparisons Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit eb498a157533e3f3324f36066a7a1f0300048438 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Tue Mar 17 11:57:22 2015 +1300 KCC: consistently use ndr_pack() for DSA GUID comparisons Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 7e4dfb7e7508fc5bf1af54ad22f2d850030a96ce Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Tue Mar 17 11:50:02 2015 +1300 KCC: Use ndrpack(GUID) sorting for all graph vertex operations For intrasite rings, DSA GUIDs are sorted by their ndr_pack representation, not their string/as-if-128-bit-int representation. Supposing some consistency across KCC, the graph vertex guids might be compared the same way. But we don't yet know for sure. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit abfb70272c87c62a3d71115588e1d43908749a89 Author: Andrew Bartlett <abart...@samba.org> Date: Wed May 6 16:11:35 2015 +1200 kcc: reduce brokenness of --import-lidf It still doesn't combine well with --forced-local-dsa, due it seems to pervasive fragile cross-dependencies within the ldb system. Signed-off-by: Andrew Bartlett <abart...@samba.org> Reviewed-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 259d122b48fc47aa8bca2a2b44fe852d2baf619b Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Fri Mar 13 13:56:55 2015 +1300 KCC: skip looking for extra edges if we have enough Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 5246a139887456c8ac70aac1ac1414bb258cb05c Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Fri Mar 13 13:53:51 2015 +1300 KCC: do not skip unrelated edges when rodc topology is found Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 736ecf024423085828e4dbb0476c18bcb090cd48 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Fri Mar 13 16:40:11 2015 +1300 KCC: A whole lot of debugging and other non-functional changes Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit b73212c33772732f5d0e9dd72742063ccd7fb6fd Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Fri Mar 13 16:39:00 2015 +1300 kcc: sort guids by ndr_pack representation This gives the correct rings in intrasite. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit b47efc0f4f23bd709831d82f90147557b8fe4888 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 12 17:08:55 2015 +1300 KCC: rewrite random intrasite connection addition for clarity And add debug messages. No change in results though. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 333c8ad4342ad4d113a823c3f63c61196ca77a9c Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 12 15:56:30 2015 +1300 KCC: intrasite_post_ntdscon is directed Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 66c17599ecdf611b6d9148039253e585e4a643da Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 12 14:47:51 2015 +1300 KCC: add NC identifiers to dot file names Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 159db2a054c55d081ed68bcef342c941354d09aa Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 12 14:44:54 2015 +1300 KCC: stub for check of 1 or 2 node or double ring directed graph 1 node graphs might have no directed edges. 2 node graphs might only have one pair. I think I have seen this behaviour but I can't remember where. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 750131395ba451c3f5bf63baccc269142f5ff452 Author: Garming Sam <garm...@catalyst.net.nz> Date: Thu Mar 12 14:44:34 2015 +1300 kcc: Interpret msDS-hasMasterNCs as hasMasterNCs Signed-off-by: Garming Sam <garm...@catalyst.net.nz> Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 8b6f3e61148ab1a681b30471ea1c0003b89e5f56 Author: Garming Sam <garm...@catalyst.net.nz> Date: Thu Mar 12 12:13:36 2015 +1300 KCC: Fix original script for inverted logic Signed-off-by: Garming Sam <garm...@catalyst.net.nz> Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 0b17932682b30819701d0db65e2891420ae10eda Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 12 12:00:03 2015 +1300 KCC: rename "fully_connected" check, "complete" We also change some of the graphs to do only the relevent validations. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 2a01bcec9087083d31055976c8e476e8a4af57b6 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 12 11:56:59 2015 +1300 KCC: write_dot_graph prints filename in --debug mode Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 18991094ded58c20ad0d91d0c8eaefb1a19bcfa2 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 12 10:25:03 2015 +1300 KCC: Set option --readonly to default to False Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit c8944ae38a4071175d47cd3af25ec42df5a93121 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 12 10:21:42 2015 +1300 KCC: add directed double ring verification Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 7c39344767b816023e7474653c138983d6c1d160 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Thu Mar 12 10:19:51 2015 +1300 KCC: add an option to list the graph verification options Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit d474bfa3c31db0f755f4cde3e8fa50cac381e6c6 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed Mar 11 16:29:16 2015 +1300 KCC: comment noting the progression of edge limit calculation Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 071ef322d760e457a8687b391c66d97bbcf7fca4 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed Mar 11 16:23:55 2015 +1300 KCC: add a wider range of colour DEBUG messages and use them in a couple of places, for no good reason. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit 4889770f9850bd839a226958ccb5c7768d6cf532 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed Mar 11 16:22:43 2015 +1300 KCC: add graph verification at all dot file points The tests are not yet relevant. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> commit e19330e579797ec4762b1b026787bac3f3c35323 Author: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Date: Wed Mar 11 13:53:38 2015 +1300 KCC: add an option to verify graph properties Any function with a name in the form "verify_graph_FOO" will be available as a graph verification option (with the property name "FOO"). The signature is "verify_graph_FOO(edges, vertices, edge_vertices)", where edge_vertices is the set of vertices found on the edges, while vertices is the set of vertices given to verify_graph (or a copy of edge_vertices if no such set was given). This makes it easier to add new tests without making the function too unwieldy. Signed-off-by: Douglas Bagnall <douglas.bagn...@catalyst.net.nz> Reviewed-by: Garming Sam <garm...@catalyst.net.nz> Reviewed-by: Andrew Bartlett <abart...@samba.org> ----------------------------------------------------------------------- Summary of changes: python/samba/graph_utils.py | 325 +++++++++++ python/samba/kcc_utils.py | 180 +++--- python/samba/ldif_utils.py | 402 +++++++++++++ source4/scripting/bin/samba_kcc | 1213 ++++++++++++++++++++------------------- 4 files changed, 1462 insertions(+), 658 deletions(-) create mode 100644 python/samba/graph_utils.py create mode 100644 python/samba/ldif_utils.py Changeset truncated at 500 lines: diff --git a/python/samba/graph_utils.py b/python/samba/graph_utils.py new file mode 100644 index 0000000..9e97c62 --- /dev/null +++ b/python/samba/graph_utils.py @@ -0,0 +1,325 @@ +# Graph topology utilities, used by KCC +# +# Copyright (C) Andrew Bartlett 2015 +# +# Copyright goes to Andrew Bartlett, but the actual work was performed +# by Douglas Bagnall and Garming Sam. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. + +import sys +import itertools + +#colours for prettier logs +C_NORMAL = "\033[00m" +DARK_RED = "\033[00;31m" +RED = "\033[01;31m" +DARK_GREEN = "\033[00;32m" +GREEN = "\033[01;32m" +YELLOW = "\033[01;33m" +DARK_YELLOW = "\033[00;33m" +DARK_BLUE = "\033[00;34m" +BLUE = "\033[01;34m" +PURPLE = "\033[00;35m" +MAGENTA = "\033[01;35m" +DARK_CYAN = "\033[00;36m" +CYAN = "\033[01;36m" +GREY = "\033[00;37m" +WHITE = "\033[01;37m" +REV_RED = "\033[01;41m" + + +def write_dot_file(basename, edge_list, vertices=None, label=None, destdir=None, + reformat_labels=True, directed=False, debug=None, edge_colors=None, + edge_labels=None, vertex_colors=None): + from tempfile import NamedTemporaryFile + if label: + basename += '_' + label.translate(None, ', ') #fix DN, guid labels + f = NamedTemporaryFile(suffix='.dot', prefix=basename + '_', delete=False, dir=destdir) + if debug is not None: + debug(f.name) + graphname = ''.join(x for x in basename if x.isalnum()) + print >>f, '%s %s {' % ('digraph' if directed else 'graph', graphname) + print >>f, 'label="%s";\nfontsize=20;' % (label or graphname) + if vertices: + for i, v in enumerate(vertices): + if reformat_labels: + v = v.replace(',', '\\n') + vc = ('color="%s"' % vertex_colors[i]) if vertex_colors else '' + print >>f, '"%s" [%s];' % (v, vc) + + for i, edge in enumerate(edge_list): + a, b = edge + if reformat_labels: + a = a.replace(',', '\\n') + b = b.replace(',', '\\n') + line = '->' if directed else '--' + el = ('label="%s"' % edge_labels[i]) if edge_labels else '' + ec = ('color="%s"' % edge_colors[i]) if edge_colors else '' + print >>f, '"%s" %s "%s" [%s %s];' % (a, line, b, el, ec) + print >>f, '}' + f.close() + + + +class GraphError(Exception): + pass + +def verify_graph_complete(edges, vertices, edge_vertices): + """The graph is complete, which is to say there is an edge between + every pair of nodes.""" + for v in vertices: + remotes = set() + for a, b in edges: + if a == v: + remotes.add(b) + elif b == v: + remotes.add(a) + if len(remotes) + 1 != len(vertices): + raise GraphError("graph is not fully connected") + + +def verify_graph_connected(edges, vertices, edge_vertices): + """There is a path between any two nodes.""" + if not edges: + if len(vertices) <= 1: + return + raise GraphError("disconnected vertices were found:\nvertices: %s\n edges: %s" % + (sorted(vertices), sorted(edges))) + + remaining_edges = list(edges) + reached = set(remaining_edges.pop()) + while True: + doomed = [] + for i, e in enumerate(remaining_edges): + a, b = e + if a in reached: + reached.add(b) + doomed.append(i) + elif b in reached: + reached.add(a) + doomed.append(i) + if not doomed: + break + for i in reversed(doomed): + del remaining_edges[i] + + if remaining_edges or reached != vertices: + raise GraphError("graph is not connected:\nvertices: %s\n edges: %s" % + (sorted(vertices), sorted(edges))) + + + +def verify_graph_forest(edges, vertices, edge_vertices): + """The graph contains no loops. A forest that is also connected is a + tree.""" + trees = [set(e) for e in edges] + while True: + for a, b in itertools.combinations(trees, 2): + intersection = a & b + if intersection: + if len(intersection) == 1: + a |= b + trees.remove(b) + break + else: + raise GraphError("there is a loop in the graph") + else: + # no break in itertools.combinations loop means no + # further mergers, so we're done. + # + # XXX here we also know whether it is a tree or a + # forest by len(trees) but the connected test already + # tells us that. + return + +def verify_graph_multi_edge_forest(edges, vertices, edge_vertices): + """This allows a forest with duplicate edges. That is if multiple + edges go between the same two vertices, they are treated as a + single edge by this test. + + e.g.: + o + pass: o-o=o o=o (|) fail: o-o + `o o `o' + """ + unique_edges = set(edges) + trees = [set(e) for e in unique_edges] + while True: + for a, b in itertools.combinations(trees, 2): + intersection = a & b + if intersection: + if len(intersection) == 1: + a |= b + trees.remove(b) + break + else: + raise GraphError("there is a loop in the graph") + else: + return + + +def verify_graph_forest_of_rings(edges, vertices, edge_vertices): + """The graph should consist of clusters of node connected in rings, + with the connections between the clusters forming a forest.""" + pass + + +def verify_graph_no_lonely_vertices(edges, vertices, edge_vertices): + """There are no vertices without edges.""" + lonely = vertices - edge_vertices + if lonely: + raise GraphError("some vertices are not connected:\n%s" % '\n'.join(sorted(lonely))) + + +def verify_graph_no_unknown_vertices(edges, vertices, edge_vertices): + """The edge endpoints contain no vertices that are otherwise unknown.""" + unknown = edge_vertices - vertices + if unknown: + raise GraphError("some edge vertices are seemingly unknown:\n%s" % '\n'.join(sorted(unknown))) + + +def verify_graph_directed_double_ring(edges, vertices, edge_vertices): + """Each node has at least two directed edges leaving it, and two + arriving. The edges work in pairs that have the same end points + but point in opposite directions. The pairs form a path that + touches every vertex and form a loop. + + There might be other connections that *aren't* part of the ring. + """ + #XXX 1 and 2 vertex cases are special cases. + if not edges: + return + if len(edges) < 2 * len(vertices): + raise GraphError("directed double ring requires at least twice as many edges as vertices") + + exits = {} + for start, end in edges: + s = exits.setdefault(start, []) + s.append(end) + + try: + #follow both paths at once -- they should be the same length + #XXX there is probably a simpler way. + forwards, backwards = exits[start] + fprev, bprev = (start, start) + f_path = [start] + b_path = [start] + for i in range(len(vertices)): + a, b = exits[forwards] + if a == fprev: + fnext = b + else: + fnext = a + f_path.append(forwards) + fprev = forwards + forwards = fnext + + a, b = exits[backwards] + if a == bprev: + bnext = b + else: + bnext = a + b_path.append(backwards) + bprev = backwards + backwards = bnext + + except ValueError, e: + raise GraphError("wrong number of exits '%s'" % e) + + f_set = set(f_path) + b_set = set(b_path) + + if (f_path != list(reversed(b_path)) or + len(f_path) != len(f_set) + 1 or + len(f_set) != len(vertices)): + raise GraphError("doesn't seem like a double ring to me!") + + +def verify_graph_directed_double_ring_or_small(edges, vertices, edge_vertices): + if len(vertices) < 3: + return + return verify_graph_directed_double_ring(edges, vertices, edge_vertices) + + + +def verify_graph(title, edges, vertices=None, directed=False, properties=(), fatal=True, + debug=None): + errors = [] + if debug is None: + def debug(*args): pass + + debug("%sStarting verify_graph for %s%s%s" % (PURPLE, MAGENTA, title, C_NORMAL)) + + properties = [x.replace(' ', '_') for x in properties] + + edge_vertices = set() + for a, b in edges: + edge_vertices.add(a) + edge_vertices.add(b) + + if vertices is None: + vertices = edge_vertices + else: + vertices = set(vertices) + if vertices != edge_vertices: + debug("vertices in edges don't match given vertices:\n %s != %s" % + (sorted(edge_vertices), sorted(vertices))) + + for p in properties: + fn = 'verify_graph_%s' % p + try: + f = globals()[fn] + except KeyError: + errors.append((p, "There is no verification check for '%s'" % p)) + try: + f(edges, vertices, edge_vertices) + debug(" %s%18s:%s verified!" % (DARK_GREEN, p, C_NORMAL)) + except GraphError, e: + errors.append((p, e)) + + if errors: + if fatal: + raise GraphError("The '%s' graph lacks the following properties:\n%s" % + (title, '\n'.join('%s: %s' % x for x in errors))) + debug(("%s%s%s FAILED:" % (MAGENTA, title, RED))) + for p, e in errors: + debug(" %18s: %s%s%s" %(p, DARK_YELLOW, e, RED)) + debug(C_NORMAL) + + + +def verify_and_dot(basename, edges, vertices=None, label=None, destdir=None, + reformat_labels=True, directed=False, properties=(), fatal=True, + debug=None, verify=True, dot_files=False, edge_colors=None, + edge_labels=None, vertex_colors=None): + + title = '%s %s' % (basename, label or '') + if verify: + verify_graph(title, edges, vertices, properties=properties, fatal=fatal, + debug=debug) + if dot_files: + write_dot_file(basename, edges, vertices=vertices, label=label, destdir=destdir, + reformat_labels=reformat_labels, directed=directed, debug=debug, + edge_colors=edge_colors, edge_labels=edge_labels, + vertex_colors=vertex_colors) + +def list_verify_tests(): + for k, v in sorted(globals().items()): + if k.startswith('verify_graph_'): + print k.replace('verify_graph_', '') + if v.__doc__: + print ' %s%s%s' %(GREY, v.__doc__, C_NORMAL) + else: + print diff --git a/python/samba/kcc_utils.py b/python/samba/kcc_utils.py index d392033..210bcdd 100644 --- a/python/samba/kcc_utils.py +++ b/python/samba/kcc_utils.py @@ -2,6 +2,10 @@ # # Copyright (C) Dave Craft 2011 # Copyright (C) Jelmer Vernooij 2011 +# Copyright (C) Andrew Bartlett 2015 +# +# Andrew Bartlett's alleged work performed by his underlings Douglas +# Bagnall and Garming Sam. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -19,6 +23,8 @@ import ldb import uuid import time +import sys +import itertools from samba import dsdb, unix2nttime from samba.dcerpc import ( @@ -29,30 +35,14 @@ from samba.dcerpc import ( from samba.common import dsdb_Dn from samba.ndr import (ndr_unpack, ndr_pack) -#colours for prettier logs -C_NORMAL = "\033[00m" -DARK_RED = "\033[00;31m" -RED = "\033[01;31m" -DARK_GREEN = "\033[00;32m" -GREEN = "\033[01;32m" -YELLOW = "\033[01;33m" -DARK_YELLOW = "\033[00;33m" -DARK_BLUE = "\033[00;34m" -BLUE = "\033[01;34m" -PURPLE = "\033[00;35m" -MAGENTA = "\033[01;35m" -DARK_CYAN = "\033[00;36m" -CYAN = "\033[01;36m" -GREY = "\033[00;37m" -WHITE = "\033[01;37m" -REV_RED = "\033[01;41m" - class KCCError(Exception): pass class NCType(object): (unknown, schema, domain, config, application) = range(0, 5) +# map the NCType enum to strings for debugging +nctype_lut = {v:k for k, v in NCType.__dict__.items() if k[:2] != '__'} class NamingContext(object): """Base class for a naming context. @@ -73,7 +63,7 @@ class NamingContext(object): def __str__(self): '''Debug dump string output of class''' - text = "%s%s%s:" % (CYAN, self.__class__.__name__, C_NORMAL) + text = "%s:" % (self.__class__.__name__,) text = text + "\n\tnc_dnstr=%s" % self.nc_dnstr text = text + "\n\tnc_guid=%s" % str(self.nc_guid) @@ -82,7 +72,7 @@ class NamingContext(object): else: text = text + "\n\tnc_sid=<present>" - text = text + "\n\tnc_type=%s" % self.nc_type + text = text + "\n\tnc_type=%s (%s)" % (nctype_lut[self.nc_type], self.nc_type) return text def load_nc(self, samdb): @@ -279,6 +269,7 @@ class NCReplica(NamingContext): # Not RODC elif attr == "msDS-hasMasterNCs": + self.rep_present_criteria_one = True self.rep_ro = False # Now use this DSA attribute to identify the naming @@ -707,7 +698,7 @@ class DirectoryServiceAgent(object): dnstr = str(msg.dn) # already loaded - if dnstr in self.connect_table.keys(): + if dnstr in self.connect_table: continue connect = NTDSConnection(dnstr) @@ -742,8 +733,11 @@ class DirectoryServiceAgent(object): for dnstr in delconn: del self.connect_table[dnstr] + + + def add_connection(self, dnstr, connect): - assert dnstr not in self.connect_table.keys() + assert dnstr not in self.connect_table self.connect_table[dnstr] = connect def get_connection_by_from_dnstr(self, from_dnstr): @@ -753,7 +747,8 @@ class DirectoryServiceAgent(object): :param from_dnstr: search for this from server entry """ - for dnstr, connect in self.connect_table.items(): + #XXX is this connection always unique? + for connect in self.connect_table.values(): if connect.get_from_dnstr() == from_dnstr: return connect return None @@ -776,6 +771,10 @@ class DirectoryServiceAgent(object): connect_table and will be marked as "to be added" pending a call to commit_connections() """ + print("new_connection: between %s and %s" % + (self.dsa_dnstr, from_dnstr)) + + dnstr = "CN=%s," % str(uuid.uuid4()) + self.dsa_dnstr connect = NTDSConnection(dnstr) @@ -1124,6 +1123,8 @@ class NTDSConnection(object): """Returns True if our schedule includes at least one replication interval within the week. False otherwise """ + # replinfo schedule is None means "always", while + # NTDSConnection schedule is None means "never". if self.schedule is None or self.schedule.dataArray[0] is None: return False @@ -1164,30 +1165,6 @@ class NTDSConnection(object): return False return True - def convert_schedule_to_repltimes(self): - """Convert NTDS Connection schedule to replTime schedule. - - NTDS Connection schedule slots are double the size of - the replTime slots but the top portion of the NTDS - Connection schedule slot (4 most significant bits in - uchar) are unused. The 4 least significant bits have - the same (15 minute interval) bit positions as replTimes. - We thus pack two elements of the NTDS Connection schedule - slots into one element of the replTimes slot - If no schedule appears in NTDS Connection then a default - of 0x11 is set in each replTimes slot as per behaviour - noted in a Windows DC. That default would cause replication - within the last 15 minutes of each hour. - """ - times = [0x11] * 84 - - for i, slot in enumerate(times): - if self.schedule is not None and \ - self.schedule.dataArray[0] is not None: - slot = (self.schedule.dataArray[0].slots[i*2] & 0xF) << 4 | \ - (self.schedule.dataArray[0].slots[i*2] & 0xF) - return times - def is_rodc_topology(self): """Returns True if NTDS Connection specifies RODC topology only @@ -1490,7 +1467,7 @@ class Site(object): :return: None if DSA doesn't exist -- Samba Shared Repository