Re: [Samba] ClassicUpgrade = EpicFail
- Original Message - From: Andrew Bartlett abart...@samba.org To: Jon Detert jdet...@infinityhealthcare.com Cc: samba@lists.samba.org Sent: Sunday, April 7, 2013 4:16:30 AM Subject: Re: [Samba] ClassicUpgrade = EpicFail On Fri, 2013-04-05 at 14:47 -0500, Jon Detert wrote: ClassicUpgrade of my samba3 data to samba4 fails, with this error: ERROR(class 'passdb.error'): uncaught exception - Unable to get id for sid Full log of the classicupgrade is at the end of this email. Project member on this list, Andrew Barlett, wrote that the issue is probably that my Samba 3 passdb was passable in an NT 4 DC mode, but is actually 'invalid' : I should have been clearer: I make no statement as to that validity of your database, but note that this tool has much stricter requirements than we enforced on passdb databases in the past. Understood. I think you were clear. My problem is that I have no idea how to proceed. -- snip -- In any case, from here the next debugging step would be to run with git master or v4-0-test, as I included some idmap patches there that didn't make 4.0.4. I already tried the git master (as of March 18th) as well as the v4-0-test (as of March 4th). Are you saying I should try a more recent snapshot of those git projects? Eventually, we will either to improve the import of the DB for your particular issue, either to accept it (possibly fixing it along the way) or more clearly rejecting it with a proper explanation. That would be great. In the mean-time, is there nothing for me to do but wait? Can someone give a list of common data problems to look for and fix? I.e. I've already resolved user/group name overlaps. You listed 2 other common probs (duplicate SIDs; accounts flagged as both user and machine accounts). Any tips on how to detect those problems? In other words, it might be faster for me to resolve my data problems than to wait for updated code. Thanks, Jon -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] ClassicUpgrade = EpicFail
On Mon, 2013-04-08 at 13:21 -0500, Jon Detert wrote: - Original Message - From: Andrew Bartlett abart...@samba.org To: Jon Detert jdet...@infinityhealthcare.com Cc: samba@lists.samba.org Sent: Sunday, April 7, 2013 4:16:30 AM Subject: Re: [Samba] ClassicUpgrade = EpicFail On Fri, 2013-04-05 at 14:47 -0500, Jon Detert wrote: ClassicUpgrade of my samba3 data to samba4 fails, with this error: ERROR(class 'passdb.error'): uncaught exception - Unable to get id for sid Full log of the classicupgrade is at the end of this email. Project member on this list, Andrew Barlett, wrote that the issue is probably that my Samba 3 passdb was passable in an NT 4 DC mode, but is actually 'invalid' : I should have been clearer: I make no statement as to that validity of your database, but note that this tool has much stricter requirements than we enforced on passdb databases in the past. Understood. I think you were clear. My problem is that I have no idea how to proceed. -- snip -- In any case, from here the next debugging step would be to run with git master or v4-0-test, as I included some idmap patches there that didn't make 4.0.4. I already tried the git master (as of March 18th) as well as the v4-0-test (as of March 4th). Are you saying I should try a more recent snapshot of those git projects? Probably not, but if you have nothing else to loose, please try current master. Eventually, we will either to improve the import of the DB for your particular issue, either to accept it (possibly fixing it along the way) or more clearly rejecting it with a proper explanation. That would be great. In the mean-time, is there nothing for me to do but wait? Can someone give a list of common data problems to look for and fix? I.e. I've already resolved user/group name overlaps. You listed 2 other common probs (duplicate SIDs; accounts flagged as both user and machine accounts). Any tips on how to detect those problems? In other words, it might be faster for me to resolve my data problems than to wait for updated code. If those problems were present, then it would have failed much earlier than this. At this stage we need to work out which SID is failing to convert, and then look at the uidNumber or gidNumber records on that record. Inserting some print statements into the python scripts would be the best place to start, if you are comfortable with that. Andrew Bartlett -- Andrew Bartletthttp://samba.org/~abartlet/ Authentication Developer, Samba Team http://samba.org -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] ClassicUpgrade = EpicFail
On Fri, 2013-04-05 at 14:47 -0500, Jon Detert wrote: ClassicUpgrade of my samba3 data to samba4 fails, with this error: ERROR(class 'passdb.error'): uncaught exception - Unable to get id for sid Full log of the classicupgrade is at the end of this email. Project member on this list, Andrew Barlett, wrote that the issue is probably that my Samba 3 passdb was passable in an NT 4 DC mode, but is actually 'invalid' : I should have been clearer: I make no statement as to that validity of your database, but note that this tool has much stricter requirements than we enforced on passdb databases in the past. We never clearly specified nor enforced those requirements in the past, but our new AD DC is much stricter, following the rules Microsoft has always enforced in both NT4 and AD. Databases created purely with our tools and with matching /etc/passwd or (for ldap backends) LDAP-based posixAccount entires are normally not an issue, but for example, we have seen: - Duplicate SIDs - Names of users and groups overlapping - Accounts flagged as both normal users and machine accounts In any case, from here the next debugging step would be to run with git master or v4-0-test, as I included some idmap patches there that didn't make 4.0.4. Eventually, we will either to improve the import of the DB for your particular issue, either to accept it (possibly fixing it along the way) or more clearly rejecting it with a proper explanation. Andrew Bartlett -- Andrew Bartletthttp://samba.org/~abartlet/ Authentication Developer, Samba Team http://samba.org -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] ClassicUpgrade = EpicFail
2013-04-05 21:47 keltezéssel, Jon Detert írta:
ClassicUpgrade of my samba3 data to samba4 fails, with this error:
ERROR(class 'passdb.error'): uncaught exception - Unable to get id
for sid
Full log of the classicupgrade is at the end of this email.
Project member on this list, Andrew Barlett, wrote that the issue is probably
that my Samba 3 passdb was passable in an NT 4 DC mode, but is actually
'invalid' :
The big issue here is that passdb has never had a 'fsck', and Samba
operates quite well as a 'classic' DC with an almost totally invalid
database!
As to what has happened in your particular instance, could you please
post me the output of ldbdump private/idmap.ldb?
I did post that, and will do so again, at the end of this email.
Assuming that the problem is my samba3 passdb.tdb data, what can I do to get on
with the upgrade?
My passdb is small-ish: 927 keys, according to this command, using samba3
binaries:
tdbtool passdb.db keys | wc -l
Is it feasible for me to manually 'fsck' my passdb.db?
Just looking at the output of tdbtool, it appears that there are 3 different
kinds of keys:
1) RID_8 character hex code; e.g. RID_0c54
2) USER_machine name; e.g. USER_mailserver$
3) USER_username; e.g. USER_jdoe
There are 463 RID_ keys, and 463 USER_ keys.
That makes me think that there's supposed to be a RID_ key for each USER_ key.
On that assumption, I did this to compare:
1) get sorted list of names appearing to be associated to RID_ keys:
tdbtool passdb.tdb dump | perl -ne 'if (/^(RID_\S+)/) { $rid=$1; $count =0;} else { $count++; if
($count == 2 /^\[\w+\]\s+(\w\w\s\s*)+(\w{3,}.*)$/) { $name = $2; $name =~ s/\s//g; print
$name\n;}}' | sort RID-names
2) get sorted list of names from USER_ keys:
tdbtool passdb.tdb keys | grep USER | sed 's/USER_//' | sort USER-names
3) compare the 2 lists:
diff USER-names RID-names
6c6
a758b$
---
a758$
147d146
foo-0m1onzr8h2a$
175,176d173
is-conference$
is-contractor$
244a242
kstachowiak$
270d267
lwilcott$
421a419
termservbill$
424a423
termservdev$
450d448
tthomas
There are diffs. I.e. There is a USER_ key for machine a758b, but no
associated RID_ key.
There are RID_ keys for 4 machine accounts (a758$, kstachowiak$, termservbill$,
termservdev$) that have no USER_ keys. Etc.
Are these diffs indicative of problems that would cause the Classic Upgrade to
fail? If so, can I use pdbedit to remove these problems from my samba3
passdb.tdb?
Thanks,
Jon
p.s. The full classic upgrade log, with log level set to 3:
classicUpgradeLog
Reading smb.conf
Processing section [netlogon]
Processing section [homes]
Processing section [hr]
Processing section [is]
Processing section [billing]
Processing section [names]
Processing section [changed]
Processing section [to]
Processing section [protect]
Processing section [the]
Processing section [innocent]
Processing section [is_helpdesk]
Processing section [ISContractsAndLicenses]
Processing section [unsecure]
Processing section [names]
Processing section [changed]
Processing section [spaceplan]
Processing section [dr]
Processing section [to]
Processing section [hr_scan]
Processing section [ar]
Processing section [minutes]
Processing section [meeting_08_05]
Processing section [meeting_08_18]
Processing section [hr_analyst]
Processing section [hr_payroll]
Processing section [protect]
Processing section [financial_systems]
Processing section [is_files]
Processing section [valuation_model]
Processing section [the]
Processing section [innocent]
Processing section [bla]
Processing section [is_technical_services]
Processing section [bla bla]
Processing section [bla bla bla]
Processing section [bla bla bla bla]
Processing section [is_billing_files]
Processing section [lawson_project]
Processing section [jklsdfjklsdf]
Processing section [sdfsdfa]
Processing section [fax]
Processing section [werwer]
Processing section [anesth_coding]
Processing section [is_crystal_reports]
Processing section [7iiio]
Processing section [uiui]
Processing section [asdasdasd]
Provisioning
Exporting account policy
Exporting groups
Exporting users
snip
I omitted a whole bunch of lines from this output like the following, in order
to remove sensitive names.
/snip
Ignoring group memberships of 'helpstar-phone$'
S-1-5-21-4219228698-1431711829-1578001372-2776: Unable to enumerate group
memberships, (-1073741724,No such user)
Demoting BDC account trust for mobius, this DC must be elevated to an AD DC
using 'samba-tool domain promote'
Ignoring group memberships of 'mrad$'
S-1-5-21-4219228698-1431711829-1578001372-2952: Unable to enumerate group
memberships, (-1073741724,No such user)
Next rid = 3689
Exporting posix attributes
Reading WINS database
Cannot open wins database, Ignoring: [Errno 2] No such file or directory:
'/usr/local/mobius/var/wins.dat'
lpcfg_load: refreshing parameters from /usr/local/samba/etc/smb.conf
params.c:pm_process() - Processing configuration file
/usr/local/samba/etc/smb.conf
[Samba] ClassicUpgrade = EpicFail
ClassicUpgrade of my samba3 data to samba4 fails, with this error:
ERROR(class 'passdb.error'): uncaught exception - Unable to get id for
sid
Full log of the classicupgrade is at the end of this email.
Project member on this list, Andrew Barlett, wrote that the issue is probably
that my Samba 3 passdb was passable in an NT 4 DC mode, but is actually
'invalid' :
The big issue here is that passdb has never had a 'fsck', and Samba
operates quite well as a 'classic' DC with an almost totally invalid
database!
As to what has happened in your particular instance, could you please
post me the output of ldbdump private/idmap.ldb?
I did post that, and will do so again, at the end of this email.
Assuming that the problem is my samba3 passdb.tdb data, what can I do to get on
with the upgrade?
My passdb is small-ish: 927 keys, according to this command, using samba3
binaries:
tdbtool passdb.db keys | wc -l
Is it feasible for me to manually 'fsck' my passdb.db?
Just looking at the output of tdbtool, it appears that there are 3 different
kinds of keys:
1) RID_8 character hex code; e.g. RID_0c54
2) USER_machine name; e.g. USER_mailserver$
3) USER_username; e.g. USER_jdoe
There are 463 RID_ keys, and 463 USER_ keys.
That makes me think that there's supposed to be a RID_ key for each USER_ key.
On that assumption, I did this to compare:
1) get sorted list of names appearing to be associated to RID_ keys:
tdbtool passdb.tdb dump | perl -ne 'if (/^(RID_\S+)/) { $rid=$1; $count =0;}
else { $count++; if ($count == 2 /^\[\w+\]\s+(\w\w\s\s*)+(\w{3,}.*)$/) {
$name = $2; $name =~ s/\s//g; print $name\n;}}' | sort RID-names
2) get sorted list of names from USER_ keys:
tdbtool passdb.tdb keys | grep USER | sed 's/USER_//' | sort USER-names
3) compare the 2 lists:
diff USER-names RID-names
6c6
a758b$
---
a758$
147d146
foo-0m1onzr8h2a$
175,176d173
is-conference$
is-contractor$
244a242
kstachowiak$
270d267
lwilcott$
421a419
termservbill$
424a423
termservdev$
450d448
tthomas
There are diffs. I.e. There is a USER_ key for machine a758b, but no
associated RID_ key.
There are RID_ keys for 4 machine accounts (a758$, kstachowiak$, termservbill$,
termservdev$) that have no USER_ keys. Etc.
Are these diffs indicative of problems that would cause the Classic Upgrade to
fail? If so, can I use pdbedit to remove these problems from my samba3
passdb.tdb?
Thanks,
Jon
p.s. The full classic upgrade log, with log level set to 3:
classicUpgradeLog
Reading smb.conf
Processing section [netlogon]
Processing section [homes]
Processing section [hr]
Processing section [is]
Processing section [billing]
Processing section [names]
Processing section [changed]
Processing section [to]
Processing section [protect]
Processing section [the]
Processing section [innocent]
Processing section [is_helpdesk]
Processing section [ISContractsAndLicenses]
Processing section [unsecure]
Processing section [names]
Processing section [changed]
Processing section [spaceplan]
Processing section [dr]
Processing section [to]
Processing section [hr_scan]
Processing section [ar]
Processing section [minutes]
Processing section [meeting_08_05]
Processing section [meeting_08_18]
Processing section [hr_analyst]
Processing section [hr_payroll]
Processing section [protect]
Processing section [financial_systems]
Processing section [is_files]
Processing section [valuation_model]
Processing section [the]
Processing section [innocent]
Processing section [bla]
Processing section [is_technical_services]
Processing section [bla bla]
Processing section [bla bla bla]
Processing section [bla bla bla bla]
Processing section [is_billing_files]
Processing section [lawson_project]
Processing section [jklsdfjklsdf]
Processing section [sdfsdfa]
Processing section [fax]
Processing section [werwer]
Processing section [anesth_coding]
Processing section [is_crystal_reports]
Processing section [7iiio]
Processing section [uiui]
Processing section [asdasdasd]
Provisioning
Exporting account policy
Exporting groups
Exporting users
snip
I omitted a whole bunch of lines from this output like the following, in order
to remove sensitive names.
/snip
Ignoring group memberships of 'helpstar-phone$'
S-1-5-21-4219228698-1431711829-1578001372-2776: Unable to enumerate group
memberships, (-1073741724,No such user)
Demoting BDC account trust for mobius, this DC must be elevated to an AD DC
using 'samba-tool domain promote'
Ignoring group memberships of 'mrad$'
S-1-5-21-4219228698-1431711829-1578001372-2952: Unable to enumerate group
memberships, (-1073741724,No such user)
Next rid = 3689
Exporting posix attributes
Reading WINS database
Cannot open wins database, Ignoring: [Errno 2] No such file or directory:
'/usr/local/mobius/var/wins.dat'
lpcfg_load: refreshing parameters from /usr/local/samba/etc/smb.conf
params.c:pm_process() - Processing configuration file
/usr/local/samba/etc/smb.conf
Looking up IPv4 addresses
Looking up IPv6 addresses
No
