Re: NSS DB migration problem

2008-11-20 Thread Wolfgang Rosenauer
Hi,

Hans Petter Jansson schrieb:
 This database only fails to migrate if the target database was not
 already created by another, successful merge, though.
 
 I think you're saying that the failures only occur if the target (cert9)
 DB doesn't already exist when your program is run, but does succeed if
 the target is already a legit cert9 DB.  Is that what you're saying?
 That would not surprise me.  I think that the function you're using,
 NSS_InitWithMerge, does assume that the target DB already exists.
 
 Almost. I'm saying that if the database fails to migrate the first time,
 and leaves an empty database, then subsequent merges to that database
 also fail. However, if the first merge is successful (creating the
 database and populating it), then subsequent merges always work.
 
 I have a small sample size, though, since I don't have (nor want,
 really) direct access to other users' databases.
 
 It looks like Wolfgang found the problem, or at least a problem, in the
 Firefox' password manager thread, so I'm going to test using a normal
 NSS_Init() to create the database if it doesn't already exist, and see
 if that works.
 
 Thanks, Wolfgang :)

I _think_ the problem when creating a new database (if one doesn't
already exist) using NSS_Init() could be that the internal token (SDR)
is created and people will never be able to read their passwords again
even after a merge? I think what should be done is to mimic exactly what
certutil -L -X -d sql:. is doing for the initial creation of the database?

Wolfgang
___
dev-tech-crypto mailing list
dev-tech-crypto@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-tech-crypto


Re: NSS DB migration problem

2008-11-20 Thread Robert Relyea

Wolfgang Rosenauer wrote:

Hi,

Hans Petter Jansson schrieb:
  

This database only fails to migrate if the target database was not
already created by another, successful merge, though.


I think you're saying that the failures only occur if the target (cert9)
DB doesn't already exist when your program is run, but does succeed if
the target is already a legit cert9 DB.  Is that what you're saying?
That would not surprise me.  I think that the function you're using,
NSS_InitWithMerge, does assume that the target DB already exists.
  

Almost. I'm saying that if the database fails to migrate the first time,
and leaves an empty database, then subsequent merges to that database
also fail. However, if the first merge is successful (creating the
database and populating it), then subsequent merges always work.

I have a small sample size, though, since I don't have (nor want,
really) direct access to other users' databases.

It looks like Wolfgang found the problem, or at least a problem, in the
Firefox' password manager thread, so I'm going to test using a normal
NSS_Init() to create the database if it doesn't already exist, and see
if that works.

Thanks, Wolfgang :)



I _think_ the problem when creating a new database (if one doesn't
already exist) using NSS_Init() could be that the internal token (SDR)
is created and people will never be able to read their passwords again
even after a merge? I think what should be done is to mimic exactly what
certutil -L -X -d sql:. is doing for the initial creation of the database?
  


In the single cert8/key3 update to cert9/key4 (as opposed to a merge 
update where you are merging from cert8/key3 to a cert9/key4 in a shared 
location) case, which is what we are talking about here, there is a 2 
step update. The second step requires a password. The code is supposed 
to know if the update has only half completed (I had some problem with 
early version of the shared DB code), where the databases are created, 
but the tables have not been. When it comes time to do the actual 
migrate, the tables are then created. If this is the case that's 
failing, we can simulate it with certutil as follows:


Start it a dbm database with a password.

certutil -L -X -d sql:.

This command does not require a password, so the databases will only 
partially update (databases are created, but not populated).


certutil -K -x -d sql:.
supply the password

This should successfully update the database.
I've just tried this and it failed. So this looks like a good test to 
reproduce the problem. I'll attach it to the bug.


bob


Wolfgang
___
dev-tech-crypto mailing list
dev-tech-crypto@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-tech-crypto
  




smime.p7s
Description: S/MIME Cryptographic Signature
___
dev-tech-crypto mailing list
dev-tech-crypto@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-tech-crypto


Re: NSS DB migration problem

2008-11-19 Thread Hans Petter Jansson
On Sat, 2008-11-15 at 19:06 -0800, Nelson B Bolyard wrote:
 Hans Petter Jansson wrote, On 2008-11-15 17:57:

  It's on separate workstations, but in some cases one database migrates
  successfully while another fails on the same system. 

 Is is possible that more than one version of the NSS shared libraries is
 installed, and that the two users have different LD_LIBRARY_PATHs, so
 that the two users are getting different versions of the NSS when they
 attempt this?

It's highly unlikely.

  This database only fails to migrate if the target database was not
  already created by another, successful merge, though.

 I think you're saying that the failures only occur if the target (cert9)
 DB doesn't already exist when your program is run, but does succeed if
 the target is already a legit cert9 DB.  Is that what you're saying?
 That would not surprise me.  I think that the function you're using,
 NSS_InitWithMerge, does assume that the target DB already exists.

Almost. I'm saying that if the database fails to migrate the first time,
and leaves an empty database, then subsequent merges to that database
also fail. However, if the first merge is successful (creating the
database and populating it), then subsequent merges always work.

I have a small sample size, though, since I don't have (nor want,
really) direct access to other users' databases.

It looks like Wolfgang found the problem, or at least a problem, in the
Firefox' password manager thread, so I'm going to test using a normal
NSS_Init() to create the database if it doesn't already exist, and see
if that works.

Thanks, Wolfgang :)

-- 
Hans Petter Jansson [EMAIL PROTECTED]

___
dev-tech-crypto mailing list
dev-tech-crypto@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-tech-crypto


Re: NSS DB migration problem

2008-11-17 Thread Robert Relyea

Hans Petter Jansson wrote:


This works for some databases, but not others. It doesn't seem to matter
which application created the database (I've tried with databases from
Firefox and Evolution) - e.g. one user's database may fail while another
user's database may migrate properly. When it fails, it's always on the
first PK11_Authenticate () call (step 3). The code above produces the
following output:

*** Auth call failed: 4294959104.

That is, 0xe000. If I set up an auth callback, it never gets called.
Do you have any suggestions as to what I'm doing wrong here?
  

Do these databases have passwords set on them?

Also, does certutil --merge on these databases?

bob



smime.p7s
Description: S/MIME Cryptographic Signature
___
dev-tech-crypto mailing list
dev-tech-crypto@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-tech-crypto


Re: NSS DB migration problem

2008-11-15 Thread Hans Petter Jansson
On Fri, 2008-11-14 at 22:56 -0800, Nelson B Bolyard wrote:
 Hans Petter Jansson wrote, On 2008-11-14 21:54:

  This works for some databases, but not others. It doesn't seem to matter
  which application created the database (I've tried with databases from
  Firefox and Evolution) - e.g. one user's database may fail while another
  user's database may migrate properly. 

 Do these separate users have separate systems, which separate installations
 of the software?  Or do they share a single multi-user system?
 
 I'm wondering if perhaps differences in their installations can explain it.

It's on separate workstations, but in some cases one database migrates
successfully while another fails on the same system. In one case, I have
a Firefox database that migrates successfully (resulting in a 150KB
migrated database, slightly smaller than the source) and a small
Evolution database that does not migrate, for the same user on the same
workstation with the same software versions.

 which as a signed number is -8192, which is SEC_ERROR_IO.

Ah yes, sorry. I was using %d initially, then changed it to %u while
looking for the problem and subsequently forgot about it :)

 Let's start with basics.
 Might some of them have corrupt cert8 DBs?

I doubt it - this affects many users on different systems, and the old
databases are working in their original contexts. Could there be an
entry in the database that cannot be migrated, or that confuses the
migration code for some reason?

 Can you get a listing of their cert DBs with certutil?

Here's one:

[EMAIL PROTECTED]:~/.mozilla/firefox/lim69e3n.default certutil -L -d .

Certificate Nickname Trust Attributes
 SSL,S/MIME,JAR/XPI

VeriSign Class 3 Secure Server CA,,
Go Daddy Secure Certification Authority  ,,
Cybertrust Educational CA,,
Deutsche Telekom CA 5,,
campusonline.uni-freiburg.de ,,
NewMedia-NET GmbH,,
DPWN Root CA R2 PS   ,,
GlobalSign   ,,
XRamp Security Services GS CA,,
GlobalSign RootSign Partners CA  ,,
VeriSign, Inc.   ,,
VeriSign Class 3 Extended Validation SSL SGC CA  ,,
VeriSign Class 3 Public Primary Certification Authority - G5 ,,
imtek.uni-freiburg.de,,
I.T. Telecom Global CA   ,,
DPWN SSL CA I2 PS,,
GlobalSign Extended Validation CA,,

And another:

[EMAIL PROTECTED]:~/.mozilla/firefox/lim69e3n.default certutil -L -d . 
 

Certificate Nickname Trust Attributes
 SSL,S/MIME,JAR/XPI

Microsoft Secure Server Authority,,   
VeriSign Class 3 Extended Validation SSL CA  ,,   
GlobalSign   ,,   
Deutsche Telekom CA 5,,   
www.asensys.de   ,,   
EssentialSSL CA  ,,
Microsoft Internet Authority ,,
VeriSign Class 3 Extended Validation SSL SGC CA  ,,
VeriSign, Inc.   ,,
WebSpace-Forum Server CA ,,
UTN-USERFirst-Hardware   ,,
Akamai Subordinate CA 3  ,,
Starfield Secure Certification Authority ,,
GlobalSign Extended Validation CA,,
GlobalSign RootSign Partners CA  ,,
Thawte SGC CA,,
VeriSign Class 3 Secure Server CA,,
Go Daddy Secure Certification Authority  ,,
cfc-comm.cfc-int ,,
DigiCert Global CA   ,,

 Does modutil show any unloaded PKCS#11 modules in their secmod DB?

No. They tend to look like this:

[EMAIL PROTECTED]:~/.mozilla/firefox/lim69e3n.default modutil -dbdir . -list

Listing of PKCS #11 Modules
---
  1. NSS Internal PKCS #11 Module
 slots: 2 slots attached
status: loaded

 slot: NSS Internal Cryptographic Services
token: NSS Generic Crypto Services

 slot: NSS User Private Key and Certificate Services
token: NSS Certificate DB

Re: NSS DB migration problem

2008-11-15 Thread Nelson B Bolyard
Hans Petter Jansson wrote, On 2008-11-15 17:57:
 On Fri, 2008-11-14 at 22:56 -0800, Nelson B Bolyard wrote:
 Hans Petter Jansson wrote, On 2008-11-14 21:54:
 
 This works for some databases, but not others. 

 It's on separate workstations, but in some cases one database migrates
 successfully while another fails on the same system. 

Is is possible that more than one version of the NSS shared libraries is
installed, and that the two users have different LD_LIBRARY_PATHs, so
that the two users are getting different versions of the NSS when they
attempt this?

 Let's start with basics.
 Might some of them have corrupt cert8 DBs?
 
 I doubt it - this affects many users on different systems, and the old
 databases are working in their original contexts. Could there be an
 entry in the database that cannot be migrated, or that confuses the
 migration code for some reason?

Perhaps.  The migration tries to copy ALL the old DB entries.  It's possible
to have a bogus entry that never gets used by the browser in
normal circumstances, and so causes no problem for the browser, but is
a problem for the migration software.

The fact that you can get DB listings with certutil is evidence that
suggests that the DB is not corrupted, but I wouldn't call it disproof.


 This database only fails to migrate if the target database was not
 already created by another, successful merge, though.

I think you're saying that the failures only occur if the target (cert9)
DB doesn't already exist when your program is run, but does succeed if
the target is already a legit cert9 DB.  Is that what you're saying?
That would not surprise me.  I think that the function you're using,
NSS_InitWithMerge, does assume that the target DB already exists.

 What version of NSS are you using?
 
 3.12.0.

At this point, I think the problem probably lies in one of these classes
of problems (in no particular order or probability):

a) The DB is corrupt
b) Using a wrong version of the NSS shared libs
c) a bug in NSS 3.12.0
d) a bug in your program

I can suggest a couple more tests that might help eliminate one or more
of those classes.

With the browser not running, and the cert8 DBs not in use, and the DBs
in a directory that has no cert9.db or key4.db, use certutil from NSS 3.12.x
to do this command:
   certutil -L -X -d sql:DBDIR
where sql: is that literal prefix (that's lower case SQL), and
DBDIR is the path name of the directory that holds the cert8.db file.
If that succeeds, it should generate cert9 and key4.db files in the
directory that held the cert8.  You can then try the command
   certutil -L -d sql:DBDIR   (no -X this time)
and if that also lists the certs, then the conversion was probably
successful.  That would pretty strongly disprove a DB corruption.
OTOH, if that also fails, it suggests there is a DB problem.

Note that this test won't be using NSS_InitWithMerge.  It's using
ordinary NSS_Init.  It won't merge two DBs together.  It only upgrades
the old DB.  However, knowing if that works, or not, will help us
narrow down the problem, and decide what to do next.

 Yeah... I think I can provide you with a problem DB in private that you
 can look at if you've got the time.

Let's try some other things first.

/Nelson
___
dev-tech-crypto mailing list
dev-tech-crypto@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-tech-crypto


NSS DB migration problem

2008-11-14 Thread Hans Petter Jansson
I've been looking in NSS docs and ML archives for a solution to the
following problem, with no luck. Asking around on IRC, I was pointed to
this ML. Hopefully it's the appropriate forum.

I'm trying to migrate existing NSS DBs from the old Berkeley DB format
to the new, SQLite-based format that can be shared by multiple
applications, using a snipped of C code from your migration guide. The
relevant code looks like this (abbreviated for clarity):

  /* STEP 1: Signal that update/merge may be needed  */

  rv = NSS_InitWithMerge (sql:/home/user/new_db,
  , , secmod.db,
  /home/user/old_db, , ,
  myapp, myapp,
  flags);

  if (rv == SECFailure)
  {
/* Fallback code */
return rv;
  }

  slot = PK11_GetInternalKeySlot ();

  /* Step 2: Determine if update/merge is needed. */

  if (PK11_IsRemovable (slot))
  {
/* need to update/Merge the database */

/* Step 3: Authenticate to the token */

rv = PK11_Authenticate (slot, PR_FALSE, pwdata);
if (rv != SECSuccess)
{
  fprintf (stderr, *** Auth call failed: %u.\n, PORT_GetError ());
  return rv;
}
  }

This works for some databases, but not others. It doesn't seem to matter
which application created the database (I've tried with databases from
Firefox and Evolution) - e.g. one user's database may fail while another
user's database may migrate properly. When it fails, it's always on the
first PK11_Authenticate () call (step 3). The code above produces the
following output:

*** Auth call failed: 4294959104.

That is, 0xe000. If I set up an auth callback, it never gets called.
Do you have any suggestions as to what I'm doing wrong here?

-- 
Hans Petter Jansson [EMAIL PROTECTED]

___
dev-tech-crypto mailing list
dev-tech-crypto@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-tech-crypto


Re: NSS DB migration problem

2008-11-14 Thread Wan-Teh Chang
On Fri, Nov 14, 2008 at 9:54 PM, Hans Petter Jansson [EMAIL PROTECTED] wrote:

/* Step 3: Authenticate to the token */

rv = PK11_Authenticate (slot, PR_FALSE, pwdata);
if (rv != SECSuccess)
{
  fprintf (stderr, *** Auth call failed: %u.\n, PORT_GetError ());
  return rv;
}
  }

 This works for some databases, but not others. It doesn't seem to matter
 which application created the database (I've tried with databases from
 Firefox and Evolution) - e.g. one user's database may fail while another
 user's database may migrate properly. When it fails, it's always on the
 first PK11_Authenticate () call (step 3). The code above produces the
 following output:

 *** Auth call failed: 4294959104.

 That is, 0xe000. If I set up an auth callback, it never gets called.
 Do you have any suggestions as to what I'm doing wrong here?

Bob Relyea is the best person to answer your question.  I just
wanted to suggest a change to your code.  PORT_GetError()
returns 'int', so you should print it with the %d format.

The error code you get is -8192, which is SEC_ERROR_IO:
http://www.mozilla.org/projects/security/pki/nss/ref/ssl/sslerr.html

The fact that PK11_Authenticate failed, the error code is
SEC_ERROR_IO, and your auth callback never gets called implies
that the PK11_NeedUserInit call in PK11_DoPassword failed:
http://mxr.mozilla.org/security/source/security/nss/lib/pk11wrap/pk11auth.c#553

553 if (PK11_NeedUserInit(slot)) {
554 PORT_SetError(SEC_ERROR_IO);
555 return SECFailure;
556 }

This is all I can fine out...

Bob, SEC_ERROR_IO seems like a strange error code for reporting
that PK11_NeedUserInit(slot) failed.

Wan-Teh
___
dev-tech-crypto mailing list
dev-tech-crypto@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-tech-crypto


Re: NSS DB migration problem

2008-11-14 Thread Nelson B Bolyard
Hans Petter Jansson wrote, On 2008-11-14 21:54:
 I've been looking in NSS docs and ML archives for a solution to the
 following problem, with no luck. Asking around on IRC, I was pointed to
 this ML. Hopefully it's the appropriate forum.
 
 I'm trying to migrate existing NSS DBs from the old Berkeley DB format
 to the new, SQLite-based format that can be shared by multiple
 applications, using a snipped of C code from your migration guide. 

[code snipped]

 This works for some databases, but not others. It doesn't seem to matter
 which application created the database (I've tried with databases from
 Firefox and Evolution) - e.g. one user's database may fail while another
 user's database may migrate properly. 

Do these separate users have separate systems, which separate installations
of the software?  Or do they share a single multi-user system?

I'm wondering if perhaps differences in their installations can explain it.

 When it fails, it's always on the first PK11_Authenticate () call (step
 3). The code above produces the following output:
 
 *** Auth call failed: 4294959104.
 
 That is, 0xe000. 

which as a signed number is -8192, which is SEC_ERROR_IO.

http://www.mozilla.org/projects/security/pki/nss/ref/ssl/sslerr.html#1039257

 If I set up an auth callback, it never gets called.

SEC_ERROR_IO was originally intended to have a specific meaning, which
was something like I wanted to ask you for your password, but I couldn't
open the file or device (terminal) with which to get it, so I gave up.
That meaning would be consistent with a failure in PK11_Authenticate.

However, it has been overloaded with another meaning which is much more
vague, basically We tried to do something with a PKCS#11 token and it
failed.  For one example of that, see
https://bugzilla.mozilla.org/show_bug.cgi?id=464411

 Do you have any suggestions as to what I'm doing wrong here?

Let's start with basics.
Might some of them have corrupt cert8 DBs?
Can you get a listing of their cert DBs with certutil?

Does modutil show any unloaded PKCS#11 modules in their secmod DB?

What version of NSS are you using?
This sounds somewhat like
https://bugzilla.mozilla.org/show_bug.cgi?id=397122
but that was fixed a year ago.
___
dev-tech-crypto mailing list
dev-tech-crypto@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-tech-crypto