Re: NSS DB migration problem
Wolfgang Rosenauer wrote: Hi, Hans Petter Jansson schrieb: This database only fails to migrate if the target database was not already created by another, successful merge, though. I think you're saying that the failures only occur if the "target" (cert9) DB doesn't already exist when your program is run, but does succeed if the target is already a legit cert9 DB. Is that what you're saying? That would not surprise me. I think that the function you're using, NSS_InitWithMerge, does assume that the target DB already exists. Almost. I'm saying that if the database fails to migrate the first time, and leaves an empty database, then subsequent merges to that database also fail. However, if the first merge is successful (creating the database and populating it), then subsequent merges always work. I have a small sample size, though, since I don't have (nor want, really) direct access to other users' databases. It looks like Wolfgang found the problem, or at least a problem, in the "Firefox' password manager" thread, so I'm going to test using a normal NSS_Init() to create the database if it doesn't already exist, and see if that works. Thanks, Wolfgang :) I _think_ the problem when creating a new database (if one doesn't already exist) using NSS_Init() could be that the internal token (SDR) is created and people will never be able to read their passwords again even after a merge? I think what should be done is to mimic exactly what "certutil -L -X -d sql:." is doing for the initial creation of the database? In the single cert8/key3 update to cert9/key4 (as opposed to a merge update where you are merging from cert8/key3 to a cert9/key4 in a shared location) case, which is what we are talking about here, there is a 2 step update. The second step requires a password. The code is supposed to know if the update has only half completed (I had some problem with early version of the shared DB code), where the databases are created, but the tables have not been. When it comes time to do the actual migrate, the tables are then created. If this is the case that's failing, we can simulate it with certutil as follows: Start it a dbm database with a password. certutil -L -X -d sql:. This command does not require a password, so the databases will only partially update (databases are created, but not populated). certutil -K -x -d sql:. This should successfully update the database. I've just tried this and it failed. So this looks like a good test to reproduce the problem. I'll attach it to the bug. bob Wolfgang ___ dev-tech-crypto mailing list dev-tech-crypto@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-tech-crypto smime.p7s Description: S/MIME Cryptographic Signature ___ dev-tech-crypto mailing list dev-tech-crypto@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-tech-crypto
Re: NSS DB migration problem
Hi, Hans Petter Jansson schrieb: >>> This database only fails to migrate if the target database was not >>> already created by another, successful merge, though. > >> I think you're saying that the failures only occur if the "target" (cert9) >> DB doesn't already exist when your program is run, but does succeed if >> the target is already a legit cert9 DB. Is that what you're saying? >> That would not surprise me. I think that the function you're using, >> NSS_InitWithMerge, does assume that the target DB already exists. > > Almost. I'm saying that if the database fails to migrate the first time, > and leaves an empty database, then subsequent merges to that database > also fail. However, if the first merge is successful (creating the > database and populating it), then subsequent merges always work. > > I have a small sample size, though, since I don't have (nor want, > really) direct access to other users' databases. > > It looks like Wolfgang found the problem, or at least a problem, in the > "Firefox' password manager" thread, so I'm going to test using a normal > NSS_Init() to create the database if it doesn't already exist, and see > if that works. > > Thanks, Wolfgang :) I _think_ the problem when creating a new database (if one doesn't already exist) using NSS_Init() could be that the internal token (SDR) is created and people will never be able to read their passwords again even after a merge? I think what should be done is to mimic exactly what "certutil -L -X -d sql:." is doing for the initial creation of the database? Wolfgang ___ dev-tech-crypto mailing list dev-tech-crypto@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-tech-crypto
Re: NSS DB migration problem
On Sat, 2008-11-15 at 19:06 -0800, Nelson B Bolyard wrote: > Hans Petter Jansson wrote, On 2008-11-15 17:57: > > It's on separate workstations, but in some cases one database migrates > > successfully while another fails on the same system. > Is is possible that more than one version of the NSS shared libraries is > installed, and that the two users have different LD_LIBRARY_PATHs, so > that the two users are getting different versions of the NSS when they > attempt this? It's highly unlikely. > > This database only fails to migrate if the target database was not > > already created by another, successful merge, though. > I think you're saying that the failures only occur if the "target" (cert9) > DB doesn't already exist when your program is run, but does succeed if > the target is already a legit cert9 DB. Is that what you're saying? > That would not surprise me. I think that the function you're using, > NSS_InitWithMerge, does assume that the target DB already exists. Almost. I'm saying that if the database fails to migrate the first time, and leaves an empty database, then subsequent merges to that database also fail. However, if the first merge is successful (creating the database and populating it), then subsequent merges always work. I have a small sample size, though, since I don't have (nor want, really) direct access to other users' databases. It looks like Wolfgang found the problem, or at least a problem, in the "Firefox' password manager" thread, so I'm going to test using a normal NSS_Init() to create the database if it doesn't already exist, and see if that works. Thanks, Wolfgang :) -- Hans Petter Jansson <[EMAIL PROTECTED]> ___ dev-tech-crypto mailing list dev-tech-crypto@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-tech-crypto
Re: NSS DB migration problem
Hans Petter Jansson wrote: This works for some databases, but not others. It doesn't seem to matter which application created the database (I've tried with databases from Firefox and Evolution) - e.g. one user's database may fail while another user's database may migrate properly. When it fails, it's always on the first PK11_Authenticate () call (step 3). The code above produces the following output: *** Auth call failed: 4294959104. That is, 0xe000. If I set up an auth callback, it never gets called. Do you have any suggestions as to what I'm doing wrong here? Do these databases have passwords set on them? Also, does certutil --merge on these databases? bob smime.p7s Description: S/MIME Cryptographic Signature ___ dev-tech-crypto mailing list dev-tech-crypto@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-tech-crypto
Re: NSS DB migration problem
Hans Petter Jansson wrote, On 2008-11-15 17:57: > On Fri, 2008-11-14 at 22:56 -0800, Nelson B Bolyard wrote: >> Hans Petter Jansson wrote, On 2008-11-14 21:54: > >>> This works for some databases, but not others. > It's on separate workstations, but in some cases one database migrates > successfully while another fails on the same system. Is is possible that more than one version of the NSS shared libraries is installed, and that the two users have different LD_LIBRARY_PATHs, so that the two users are getting different versions of the NSS when they attempt this? >> Let's start with basics. >> Might some of them have corrupt cert8 DBs? > > I doubt it - this affects many users on different systems, and the old > databases are working in their original contexts. Could there be an > entry in the database that cannot be migrated, or that confuses the > migration code for some reason? Perhaps. The migration tries to copy ALL the old DB entries. It's possible to have a bogus entry that never gets used by the browser in normal circumstances, and so causes no problem for the browser, but is a problem for the migration software. The fact that you can get DB listings with certutil is evidence that suggests that the DB is not corrupted, but I wouldn't call it disproof. > This database only fails to migrate if the target database was not > already created by another, successful merge, though. I think you're saying that the failures only occur if the "target" (cert9) DB doesn't already exist when your program is run, but does succeed if the target is already a legit cert9 DB. Is that what you're saying? That would not surprise me. I think that the function you're using, NSS_InitWithMerge, does assume that the target DB already exists. >> What version of NSS are you using? > > 3.12.0. At this point, I think the problem probably lies in one of these classes of problems (in no particular order or probability): a) The DB is corrupt b) Using a "wrong" version of the NSS shared libs c) a bug in NSS 3.12.0 d) a bug in your program I can suggest a couple more tests that might help eliminate one or more of those classes. With the browser not running, and the cert8 DBs not in use, and the DBs in a directory that has no cert9.db or key4.db, use certutil from NSS 3.12.x to do this command: certutil -L -X -d sql:DBDIR where "sql:" is that literal prefix (that's lower case SQL), and DBDIR is the path name of the directory that holds the cert8.db file. If that succeeds, it should generate cert9 and key4.db files in the directory that held the cert8. You can then try the command certutil -L -d sql:DBDIR (no -X this time) and if that also lists the certs, then the conversion was probably successful. That would pretty strongly disprove a DB corruption. OTOH, if that also fails, it suggests there is a DB problem. Note that this test won't be using NSS_InitWithMerge. It's using ordinary NSS_Init. It won't merge two DBs together. It only upgrades the old DB. However, knowing if that works, or not, will help us narrow down the problem, and decide what to do next. > Yeah... I think I can provide you with a problem DB in private that you > can look at if you've got the time. Let's try some other things first. /Nelson ___ dev-tech-crypto mailing list dev-tech-crypto@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-tech-crypto
Re: NSS DB migration problem
On Fri, 2008-11-14 at 22:56 -0800, Nelson B Bolyard wrote: > Hans Petter Jansson wrote, On 2008-11-14 21:54: > > This works for some databases, but not others. It doesn't seem to matter > > which application created the database (I've tried with databases from > > Firefox and Evolution) - e.g. one user's database may fail while another > > user's database may migrate properly. > Do these separate users have separate systems, which separate installations > of the software? Or do they share a single multi-user system? > > I'm wondering if perhaps differences in their installations can explain it. It's on separate workstations, but in some cases one database migrates successfully while another fails on the same system. In one case, I have a Firefox database that migrates successfully (resulting in a 150KB migrated database, slightly smaller than the source) and a small Evolution database that does not migrate, for the same user on the same workstation with the same software versions. > which as a signed number is -8192, which is SEC_ERROR_IO. Ah yes, sorry. I was using %d initially, then changed it to %u while looking for the problem and subsequently forgot about it :) > Let's start with basics. > Might some of them have corrupt cert8 DBs? I doubt it - this affects many users on different systems, and the old databases are working in their original contexts. Could there be an entry in the database that cannot be migrated, or that confuses the migration code for some reason? > Can you get a listing of their cert DBs with certutil? Here's one: [EMAIL PROTECTED]:~/.mozilla/firefox/lim69e3n.default> certutil -L -d . Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI VeriSign Class 3 Secure Server CA,, Go Daddy Secure Certification Authority ,, Cybertrust Educational CA,, Deutsche Telekom CA 5,, campusonline.uni-freiburg.de ,, NewMedia-NET GmbH,, DPWN Root CA R2 PS ,, GlobalSign ,, XRamp Security Services GS CA,, GlobalSign RootSign Partners CA ,, VeriSign, Inc. ,, VeriSign Class 3 Extended Validation SSL SGC CA ,, VeriSign Class 3 Public Primary Certification Authority - G5 ,, imtek.uni-freiburg.de,, I.T. Telecom Global CA ,, DPWN SSL CA I2 PS,, GlobalSign Extended Validation CA,, And another: [EMAIL PROTECTED]:~/.mozilla/firefox/lim69e3n.default> certutil -L -d . Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI Microsoft Secure Server Authority,, VeriSign Class 3 Extended Validation SSL CA ,, GlobalSign ,, Deutsche Telekom CA 5,, www.asensys.de ,, EssentialSSL CA ,, Microsoft Internet Authority ,, VeriSign Class 3 Extended Validation SSL SGC CA ,, VeriSign, Inc. ,, WebSpace-Forum Server CA ,, UTN-USERFirst-Hardware ,, Akamai Subordinate CA 3 ,, Starfield Secure Certification Authority ,, GlobalSign Extended Validation CA,, GlobalSign RootSign Partners CA ,, Thawte SGC CA,, VeriSign Class 3 Secure Server CA,, Go Daddy Secure Certification Authority ,, cfc-comm.cfc-int ,, DigiCert Global CA ,, > Does modutil show any unloaded PKCS#11 modules in their secmod DB? No. They tend to look like this: [EMAIL PROTECTED]:~/.mozilla/firefox/lim69e3n.default> modutil -dbdir . -list Listing of PKCS #11 Modules --- 1. NSS Internal PKCS #11 Module slots: 2 slots attached status: loaded slot: NSS Internal Cryptographic Services token: NSS Generic Crypto Services slot: NSS User Private Key and Certificate Services token: NSS Certif
Re: NSS DB migration problem
Hans Petter Jansson wrote, On 2008-11-14 21:54: > I've been looking in NSS docs and ML archives for a solution to the > following problem, with no luck. Asking around on IRC, I was pointed to > this ML. Hopefully it's the appropriate forum. > > I'm trying to migrate existing NSS DBs from the old Berkeley DB format > to the new, SQLite-based format that can be shared by multiple > applications, using a snipped of C code from your migration guide. [code snipped] > This works for some databases, but not others. It doesn't seem to matter > which application created the database (I've tried with databases from > Firefox and Evolution) - e.g. one user's database may fail while another > user's database may migrate properly. Do these separate users have separate systems, which separate installations of the software? Or do they share a single multi-user system? I'm wondering if perhaps differences in their installations can explain it. > When it fails, it's always on the first PK11_Authenticate () call (step > 3). The code above produces the following output: > > *** Auth call failed: 4294959104. > > That is, 0xe000. which as a signed number is -8192, which is SEC_ERROR_IO. http://www.mozilla.org/projects/security/pki/nss/ref/ssl/sslerr.html#1039257 > If I set up an auth callback, it never gets called. SEC_ERROR_IO was originally intended to have a specific meaning, which was something like "I wanted to ask you for your password, but I couldn't open the file or device (terminal) with which to get it, so I gave up." That meaning would be consistent with a failure in PK11_Authenticate. However, it has been overloaded with another meaning which is much more vague, basically "We tried to do something with a PKCS#11 token and it failed". For one example of that, see https://bugzilla.mozilla.org/show_bug.cgi?id=464411 > Do you have any suggestions as to what I'm doing wrong here? Let's start with basics. Might some of them have corrupt cert8 DBs? Can you get a listing of their cert DBs with certutil? Does modutil show any unloaded PKCS#11 modules in their secmod DB? What version of NSS are you using? This sounds somewhat like https://bugzilla.mozilla.org/show_bug.cgi?id=397122 but that was fixed a year ago. ___ dev-tech-crypto mailing list dev-tech-crypto@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-tech-crypto
Re: NSS DB migration problem
On Fri, Nov 14, 2008 at 9:54 PM, Hans Petter Jansson <[EMAIL PROTECTED]> wrote:
>
>/* Step 3: Authenticate to the token */
>
>rv = PK11_Authenticate (slot, PR_FALSE, &pwdata);
>if (rv != SECSuccess)
>{
> fprintf (stderr, "*** Auth call failed: %u.\n", PORT_GetError ());
> return rv;
>}
> }
>
> This works for some databases, but not others. It doesn't seem to matter
> which application created the database (I've tried with databases from
> Firefox and Evolution) - e.g. one user's database may fail while another
> user's database may migrate properly. When it fails, it's always on the
> first PK11_Authenticate () call (step 3). The code above produces the
> following output:
>
> *** Auth call failed: 4294959104.
>
> That is, 0xe000. If I set up an auth callback, it never gets called.
> Do you have any suggestions as to what I'm doing wrong here?
Bob Relyea is the best person to answer your question. I just
wanted to suggest a change to your code. PORT_GetError()
returns 'int', so you should print it with the %d format.
The error code you get is -8192, which is SEC_ERROR_IO:
http://www.mozilla.org/projects/security/pki/nss/ref/ssl/sslerr.html
The fact that PK11_Authenticate failed, the error code is
SEC_ERROR_IO, and your auth callback never gets called implies
that the PK11_NeedUserInit call in PK11_DoPassword failed:
http://mxr.mozilla.org/security/source/security/nss/lib/pk11wrap/pk11auth.c#553
553 if (PK11_NeedUserInit(slot)) {
554 PORT_SetError(SEC_ERROR_IO);
555 return SECFailure;
556 }
This is all I can fine out...
Bob, SEC_ERROR_IO seems like a strange error code for reporting
that PK11_NeedUserInit(slot) failed.
Wan-Teh
___
dev-tech-crypto mailing list
dev-tech-crypto@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-tech-crypto
NSS DB migration problem
I've been looking in NSS docs and ML archives for a solution to the
following problem, with no luck. Asking around on IRC, I was pointed to
this ML. Hopefully it's the appropriate forum.
I'm trying to migrate existing NSS DBs from the old Berkeley DB format
to the new, SQLite-based format that can be shared by multiple
applications, using a snipped of C code from your migration guide. The
relevant code looks like this (abbreviated for clarity):
/* STEP 1: Signal that update/merge may be needed */
rv = NSS_InitWithMerge ("sql:/home/user/new_db",
"", "", "secmod.db",
"/home/user/old_db", "", "",
"myapp", "myapp",
flags);
if (rv == SECFailure)
{
/* Fallback code */
return rv;
}
slot = PK11_GetInternalKeySlot ();
/* Step 2: Determine if update/merge is needed. */
if (PK11_IsRemovable (slot))
{
/* need to update/Merge the database */
/* Step 3: Authenticate to the token */
rv = PK11_Authenticate (slot, PR_FALSE, &pwdata);
if (rv != SECSuccess)
{
fprintf (stderr, "*** Auth call failed: %u.\n", PORT_GetError ());
return rv;
}
}
This works for some databases, but not others. It doesn't seem to matter
which application created the database (I've tried with databases from
Firefox and Evolution) - e.g. one user's database may fail while another
user's database may migrate properly. When it fails, it's always on the
first PK11_Authenticate () call (step 3). The code above produces the
following output:
*** Auth call failed: 4294959104.
That is, 0xe000. If I set up an auth callback, it never gets called.
Do you have any suggestions as to what I'm doing wrong here?
--
Hans Petter Jansson <[EMAIL PROTECTED]>
___
dev-tech-crypto mailing list
dev-tech-crypto@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-tech-crypto
