Re: [Freeipa-devel] [PATCH] 1085 cert-find command

2013-02-19 Thread Petr Viktorin

On 02/18/2013 08:39 PM, Rob Crittenden wrote:

Rob Crittenden wrote:

Petr Viktorin wrote:

On 02/15/2013 10:43 PM, Rob Crittenden wrote:

Petr Viktorin wrote:

On 02/06/2013 07:23 PM, Rob Crittenden wrote:

Petr Viktorin wrote:

On 02/06/2013 12:44 AM, Rob Crittenden wrote:

This adds a cert-find command for the dogtag backend.

Searches can be done by serial number, by subject, revocation
reason,
issue date, notbefore, notafter and revocation dates.

I added some basic tests for this. I made it a separate test file
because the cert plugin tests do not use the declarative format and
rely
on the selfsign backend by default.

rob


Thanks! The code works well, but I found a few issues.


These tests don't work when the full test suite is run: test_cert
adds
and revokes additional certs that throw the code off.
Perhaps have the tests only query valid certs? I don't see that
option
but I think it would be helpful to support.


I added some rather nasty hacks to the test to make things pass. I
limit
the search to 10 certificates, which is the number start with by
default. There is an open dogtag ticket to return only VALID
certificates so we should be able to drop this eventually.

I had to go further on one of the revocation tests, limiting it to a
sizelimit of 1. The count changes every time the suite runs so
this is
the safest for now. It also means that one test will fail if this is
the
only part of the suite executed.


This gets rid of most of the failures, but it still fails the certs
for
this IPA server/short name tests if the cert from ./make-cert is
present (creating it is part of `make test`).
If make-cert is run more times, it'll revoke the previous cert, so the
test for revocation reason 4 fails as well.

It looks like when using sizelimit Dogtag will always discard *newer*
certs, ones with higher serials. Is it documented behavior or does
Dogtag just happen to do that?


It isn't documented anywhere I could find, it is just what dogtag
returns


I wonder how other people run their tests. This solution looks like it
could break easily if people do something differently :(

I'm not sure how to solve this properly. Perhaps not using
Declarative,
and checking by hand that the wanted certs are in the response and
the
unwanted ones are not, would work better.


I ended up switching the test class. It is not a very fine-grained set
of tests, mostly searching with limits and verifying that we fall
within
a reasonable range, but it is better than nothing.

rob


This works much better, thanks! Just two nitpicks now.

The patch doesn't apply well, there's a conflict in VERSION and some
added trailing whitespace,

AFAIK this would be the only (first?) test that relies on Nose's
ordering of test modules -- tests 0011 and 0030 rely on the other cert
tests running first. Please at least mention that in a comment. Or
better, move class test_cert_find to test_cert.py




Rebased the patch and removed whitespace.

I went ahead and combined this with the existing test_cert file.
Originally test_cert was only tested against lite-server but since it
works against a live dogtag server too it makes sense to combine them.

I improved the set up documentation a little bit and tried to handle all
the different configurations that one might see so that this should be
runnable against either a live server or the lite-server for both the
selfsign and dogtag backends.

This relies on the user configuring ~/.ipa/default.conf to match the
remote server. There is no way from a client to know what kind of CA
backend a server is running.

rob


And the patch.

rob



ACK, thank you!

--
Petr³

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 1085 cert-find command

2013-02-19 Thread Rob Crittenden

Petr Viktorin wrote:

On 02/18/2013 08:39 PM, Rob Crittenden wrote:

Rob Crittenden wrote:

Petr Viktorin wrote:

On 02/15/2013 10:43 PM, Rob Crittenden wrote:

Petr Viktorin wrote:

On 02/06/2013 07:23 PM, Rob Crittenden wrote:

Petr Viktorin wrote:

On 02/06/2013 12:44 AM, Rob Crittenden wrote:

This adds a cert-find command for the dogtag backend.

Searches can be done by serial number, by subject, revocation
reason,
issue date, notbefore, notafter and revocation dates.

I added some basic tests for this. I made it a separate test file
because the cert plugin tests do not use the declarative format
and
rely
on the selfsign backend by default.

rob


Thanks! The code works well, but I found a few issues.


These tests don't work when the full test suite is run: test_cert
adds
and revokes additional certs that throw the code off.
Perhaps have the tests only query valid certs? I don't see that
option
but I think it would be helpful to support.


I added some rather nasty hacks to the test to make things pass. I
limit
the search to 10 certificates, which is the number start with by
default. There is an open dogtag ticket to return only VALID
certificates so we should be able to drop this eventually.

I had to go further on one of the revocation tests, limiting it to a
sizelimit of 1. The count changes every time the suite runs so
this is
the safest for now. It also means that one test will fail if this is
the
only part of the suite executed.


This gets rid of most of the failures, but it still fails the certs
for
this IPA server/short name tests if the cert from ./make-cert is
present (creating it is part of `make test`).
If make-cert is run more times, it'll revoke the previous cert, so
the
test for revocation reason 4 fails as well.

It looks like when using sizelimit Dogtag will always discard *newer*
certs, ones with higher serials. Is it documented behavior or does
Dogtag just happen to do that?


It isn't documented anywhere I could find, it is just what dogtag
returns


I wonder how other people run their tests. This solution looks
like it
could break easily if people do something differently :(

I'm not sure how to solve this properly. Perhaps not using
Declarative,
and checking by hand that the wanted certs are in the response and
the
unwanted ones are not, would work better.


I ended up switching the test class. It is not a very fine-grained set
of tests, mostly searching with limits and verifying that we fall
within
a reasonable range, but it is better than nothing.

rob


This works much better, thanks! Just two nitpicks now.

The patch doesn't apply well, there's a conflict in VERSION and some
added trailing whitespace,

AFAIK this would be the only (first?) test that relies on Nose's
ordering of test modules -- tests 0011 and 0030 rely on the other cert
tests running first. Please at least mention that in a comment. Or
better, move class test_cert_find to test_cert.py




Rebased the patch and removed whitespace.

I went ahead and combined this with the existing test_cert file.
Originally test_cert was only tested against lite-server but since it
works against a live dogtag server too it makes sense to combine them.

I improved the set up documentation a little bit and tried to handle all
the different configurations that one might see so that this should be
runnable against either a live server or the lite-server for both the
selfsign and dogtag backends.

This relies on the user configuring ~/.ipa/default.conf to match the
remote server. There is no way from a client to know what kind of CA
backend a server is running.

rob


And the patch.

rob



ACK, thank you!



pushed to master and ipa-3-1

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 1085 cert-find command

2013-02-18 Thread Rob Crittenden

Petr Viktorin wrote:

On 02/15/2013 10:43 PM, Rob Crittenden wrote:

Petr Viktorin wrote:

On 02/06/2013 07:23 PM, Rob Crittenden wrote:

Petr Viktorin wrote:

On 02/06/2013 12:44 AM, Rob Crittenden wrote:

This adds a cert-find command for the dogtag backend.

Searches can be done by serial number, by subject, revocation reason,
issue date, notbefore, notafter and revocation dates.

I added some basic tests for this. I made it a separate test file
because the cert plugin tests do not use the declarative format and
rely
on the selfsign backend by default.

rob


Thanks! The code works well, but I found a few issues.


These tests don't work when the full test suite is run: test_cert adds
and revokes additional certs that throw the code off.
Perhaps have the tests only query valid certs? I don't see that option
but I think it would be helpful to support.


I added some rather nasty hacks to the test to make things pass. I
limit
the search to 10 certificates, which is the number start with by
default. There is an open dogtag ticket to return only VALID
certificates so we should be able to drop this eventually.

I had to go further on one of the revocation tests, limiting it to a
sizelimit of 1. The count changes every time the suite runs so this is
the safest for now. It also means that one test will fail if this is
the
only part of the suite executed.


This gets rid of most of the failures, but it still fails the certs for
this IPA server/short name tests if the cert from ./make-cert is
present (creating it is part of `make test`).
If make-cert is run more times, it'll revoke the previous cert, so the
test for revocation reason 4 fails as well.

It looks like when using sizelimit Dogtag will always discard *newer*
certs, ones with higher serials. Is it documented behavior or does
Dogtag just happen to do that?


It isn't documented anywhere I could find, it is just what dogtag returns


I wonder how other people run their tests. This solution looks like it
could break easily if people do something differently :(

I'm not sure how to solve this properly. Perhaps not using Declarative,
and checking by hand that the wanted certs are in the response and the
unwanted ones are not, would work better.


I ended up switching the test class. It is not a very fine-grained set
of tests, mostly searching with limits and verifying that we fall within
a reasonable range, but it is better than nothing.

rob


This works much better, thanks! Just two nitpicks now.

The patch doesn't apply well, there's a conflict in VERSION and some
added trailing whitespace,

AFAIK this would be the only (first?) test that relies on Nose's
ordering of test modules -- tests 0011 and 0030 rely on the other cert
tests running first. Please at least mention that in a comment. Or
better, move class test_cert_find to test_cert.py




Rebased the patch and removed whitespace.

I went ahead and combined this with the existing test_cert file. 
Originally test_cert was only tested against lite-server but since it 
works against a live dogtag server too it makes sense to combine them.


I improved the set up documentation a little bit and tried to handle all 
the different configurations that one might see so that this should be 
runnable against either a live server or the lite-server for both the 
selfsign and dogtag backends.


This relies on the user configuring ~/.ipa/default.conf to match the 
remote server. There is no way from a client to know what kind of CA 
backend a server is running.


rob

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 1085 cert-find command

2013-02-18 Thread Rob Crittenden

Rob Crittenden wrote:

Petr Viktorin wrote:

On 02/15/2013 10:43 PM, Rob Crittenden wrote:

Petr Viktorin wrote:

On 02/06/2013 07:23 PM, Rob Crittenden wrote:

Petr Viktorin wrote:

On 02/06/2013 12:44 AM, Rob Crittenden wrote:

This adds a cert-find command for the dogtag backend.

Searches can be done by serial number, by subject, revocation
reason,
issue date, notbefore, notafter and revocation dates.

I added some basic tests for this. I made it a separate test file
because the cert plugin tests do not use the declarative format and
rely
on the selfsign backend by default.

rob


Thanks! The code works well, but I found a few issues.


These tests don't work when the full test suite is run: test_cert
adds
and revokes additional certs that throw the code off.
Perhaps have the tests only query valid certs? I don't see that
option
but I think it would be helpful to support.


I added some rather nasty hacks to the test to make things pass. I
limit
the search to 10 certificates, which is the number start with by
default. There is an open dogtag ticket to return only VALID
certificates so we should be able to drop this eventually.

I had to go further on one of the revocation tests, limiting it to a
sizelimit of 1. The count changes every time the suite runs so this is
the safest for now. It also means that one test will fail if this is
the
only part of the suite executed.


This gets rid of most of the failures, but it still fails the certs
for
this IPA server/short name tests if the cert from ./make-cert is
present (creating it is part of `make test`).
If make-cert is run more times, it'll revoke the previous cert, so the
test for revocation reason 4 fails as well.

It looks like when using sizelimit Dogtag will always discard *newer*
certs, ones with higher serials. Is it documented behavior or does
Dogtag just happen to do that?


It isn't documented anywhere I could find, it is just what dogtag
returns


I wonder how other people run their tests. This solution looks like it
could break easily if people do something differently :(

I'm not sure how to solve this properly. Perhaps not using Declarative,
and checking by hand that the wanted certs are in the response and
the
unwanted ones are not, would work better.


I ended up switching the test class. It is not a very fine-grained set
of tests, mostly searching with limits and verifying that we fall within
a reasonable range, but it is better than nothing.

rob


This works much better, thanks! Just two nitpicks now.

The patch doesn't apply well, there's a conflict in VERSION and some
added trailing whitespace,

AFAIK this would be the only (first?) test that relies on Nose's
ordering of test modules -- tests 0011 and 0030 rely on the other cert
tests running first. Please at least mention that in a comment. Or
better, move class test_cert_find to test_cert.py




Rebased the patch and removed whitespace.

I went ahead and combined this with the existing test_cert file.
Originally test_cert was only tested against lite-server but since it
works against a live dogtag server too it makes sense to combine them.

I improved the set up documentation a little bit and tried to handle all
the different configurations that one might see so that this should be
runnable against either a live server or the lite-server for both the
selfsign and dogtag backends.

This relies on the user configuring ~/.ipa/default.conf to match the
remote server. There is no way from a client to know what kind of CA
backend a server is running.

rob


And the patch.

rob

From c06cb0899e4b0a02feadd1dbfb4bfcbe28109e9a Mon Sep 17 00:00:00 2001
From: Rob Crittenden rcrit...@redhat.com
Date: Thu, 15 Nov 2012 10:55:33 -0500
Subject: [PATCH] Implement the cert-find command for the dogtag CA backend.

Use a new RESTful API provided by dogtag 10+. Construct an XML document
representing the search request. The output is limited to whatever dogtag
sends us, there is no way to request additional attributes other than
to read each certificate individually.

dogtag uses a boolean for each search term to indicate that it is used.
Presense of the search item is not enough, both need to be set.

The search operation is unauthenticated

Design page: http://freeipa.org/page/V3/Cert_find

https://fedorahosted.org/freeipa/ticket/2528
---
 API.txt|  23 +++
 VERSION|   2 +-
 ipalib/plugins/cert.py | 137 +-
 ipaserver/plugins/dogtag.py| 138 ++
 ipaserver/plugins/rabase.py|   8 ++
 tests/test_xmlrpc/test_cert.py | 312 ++---
 6 files changed, 598 insertions(+), 22 deletions(-)

diff --git a/API.txt b/API.txt
index 5219c51be62862c43ebe9396147ff220b33605c7..e39936974c7216c4e0d9266bbf56d0f3ba2b3f01 100644
--- a/API.txt
+++ b/API.txt
@@ -425,6 +425,29 @@ args: 1,0,2
 arg: Any('methods*')
 output: Output('count', type 'int', None)
 output: Output('results', (type 'list', type 

Re: [Freeipa-devel] [PATCH] 1085 cert-find command

2013-02-15 Thread Rob Crittenden

Petr Viktorin wrote:

On 02/06/2013 07:23 PM, Rob Crittenden wrote:

Petr Viktorin wrote:

On 02/06/2013 12:44 AM, Rob Crittenden wrote:

This adds a cert-find command for the dogtag backend.

Searches can be done by serial number, by subject, revocation reason,
issue date, notbefore, notafter and revocation dates.

I added some basic tests for this. I made it a separate test file
because the cert plugin tests do not use the declarative format and
rely
on the selfsign backend by default.

rob


Thanks! The code works well, but I found a few issues.


These tests don't work when the full test suite is run: test_cert adds
and revokes additional certs that throw the code off.
Perhaps have the tests only query valid certs? I don't see that option
but I think it would be helpful to support.


I added some rather nasty hacks to the test to make things pass. I limit
the search to 10 certificates, which is the number start with by
default. There is an open dogtag ticket to return only VALID
certificates so we should be able to drop this eventually.

I had to go further on one of the revocation tests, limiting it to a
sizelimit of 1. The count changes every time the suite runs so this is
the safest for now. It also means that one test will fail if this is the
only part of the suite executed.


This gets rid of most of the failures, but it still fails the certs for
this IPA server/short name tests if the cert from ./make-cert is
present (creating it is part of `make test`).
If make-cert is run more times, it'll revoke the previous cert, so the
test for revocation reason 4 fails as well.

It looks like when using sizelimit Dogtag will always discard *newer*
certs, ones with higher serials. Is it documented behavior or does
Dogtag just happen to do that?


It isn't documented anywhere I could find, it is just what dogtag returns


I wonder how other people run their tests. This solution looks like it
could break easily if people do something differently :(

I'm not sure how to solve this properly. Perhaps not using Declarative,
and checking by hand that the wanted certs are in the response and the
unwanted ones are not, would work better.


I ended up switching the test class. It is not a very fine-grained set 
of tests, mostly searching with limits and verifying that we fall within 
a reasonable range, but it is better than nothing.


rob
From e7972a96abc62d41a90f9bd3e816c827627b7120 Mon Sep 17 00:00:00 2001
From: Rob Crittenden rcrit...@redhat.com
Date: Thu, 15 Nov 2012 10:55:33 -0500
Subject: [PATCH] Implement the cert-find command for the dogtag CA backend.

Use a new RESTful API provided by dogtag 10+. Construct an XML document
representing the search request. The output is limited to whatever dogtag
sends us, there is no way to request additional attributes other than
to read each certificate individually.

dogtag uses a boolean for each search term to indicate that it is used.
Presense of the search item is not enough, both need to be set.

The search operation is unauthenticated

Design page: http://freeipa.org/page/V3/Cert_find

https://fedorahosted.org/freeipa/ticket/2528
---
 API.txt |  23 +++
 VERSION |   2 +-
 ipalib/plugins/cert.py  | 137 -
 ipaserver/plugins/dogtag.py | 138 +
 ipaserver/plugins/rabase.py |   8 +
 tests/test_xmlrpc/test_cert_find.py | 289 
 6 files changed, 594 insertions(+), 3 deletions(-)
 create mode 100644 tests/test_xmlrpc/test_cert_find.py

diff --git a/API.txt b/API.txt
index 8fbfe6f5d8da44e991b8d1a36725fc6ace1f0616..6b997f37b455366c66b34fd2df11c2acaa79d739 100644
--- a/API.txt
+++ b/API.txt
@@ -425,6 +425,29 @@ args: 1,0,2
 arg: Any('methods*')
 output: Output('count', type 'int', None)
 output: Output('results', (type 'list', type 'tuple'), None)
+command: cert_find
+args: 0,17,4
+option: Flag('all', autofill=True, cli_name='all', default=False, exclude='webui')
+option: Flag('exactly?', autofill=True, default=False)
+option: Str('issuedon_from?', autofill=False)
+option: Str('issuedon_to?', autofill=False)
+option: Int('max_serial_number?', autofill=False, maxvalue=2147483647)
+option: Int('min_serial_number?', autofill=False, minvalue=0)
+option: Flag('raw', autofill=True, cli_name='raw', default=False, exclude='webui')
+option: Int('revocation_reason?', autofill=False, maxvalue=10, minvalue=0)
+option: Str('revokedon_from?', autofill=False)
+option: Str('revokedon_to?', autofill=False)
+option: Int('sizelimit?', default=100, minvalue=0)
+option: Str('subject?', autofill=False)
+option: Str('validnotafter_from?', autofill=False)
+option: Str('validnotafter_to?', autofill=False)
+option: Str('validnotbefore_from?', autofill=False)
+option: Str('validnotbefore_to?', autofill=False)
+option: Str('version?', exclude='webui')
+output: Output('count', type 'int', None)
+output: ListOfEntries('result', (type 'list', type 'tuple'), 

Re: [Freeipa-devel] [PATCH] 1085 cert-find command

2013-02-14 Thread Petr Vobornik

On 02/07/2013 03:08 PM, Rob Crittenden wrote:

Petr Vobornik wrote:

On 02/06/2013 12:44 AM, Rob Crittenden wrote:

This adds a cert-find command for the dogtag backend.

Searches can be done by serial number, by subject, revocation reason,
issue date, notbefore, notafter and revocation dates.

I added some basic tests for this. I made it a separate test file
because the cert plugin tests do not use the declarative format and rely
on the selfsign backend by default.

rob



Should I create Web UI in scope of this ticket or a new one?

I was also thinking if it's time to implement #191 'Web UI: specify
fields to search on' [1]. Maybe in Pilsner.

[1] https://fedorahosted.org/freeipa/ticket/191


I'm going to open a UI ticket once the API is finalized. I didn't want
to give you a moving target to work against.

rob



I see that the search requires to specify options for attributes to 
search on. There is no general CRITERIA positional argument as in other 
find commands or am I mistaken?


Is it possible to add the CRITERIA argument? Is the no 'OR' search an 
obstacle for it?


If so we would really need to push the ticket #191 because UI doesn't 
support search by only specifying specific attributes yet.

--
Petr Vobornik

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 1085 cert-find command

2013-02-14 Thread Rob Crittenden

Petr Vobornik wrote:

On 02/07/2013 03:08 PM, Rob Crittenden wrote:

Petr Vobornik wrote:

On 02/06/2013 12:44 AM, Rob Crittenden wrote:

This adds a cert-find command for the dogtag backend.

Searches can be done by serial number, by subject, revocation reason,
issue date, notbefore, notafter and revocation dates.

I added some basic tests for this. I made it a separate test file
because the cert plugin tests do not use the declarative format and
rely
on the selfsign backend by default.

rob



Should I create Web UI in scope of this ticket or a new one?

I was also thinking if it's time to implement #191 'Web UI: specify
fields to search on' [1]. Maybe in Pilsner.

[1] https://fedorahosted.org/freeipa/ticket/191


I'm going to open a UI ticket once the API is finalized. I didn't want
to give you a moving target to work against.

rob



I see that the search requires to specify options for attributes to
search on. There is no general CRITERIA positional argument as in other
find commands or am I mistaken?

Is it possible to add the CRITERIA argument? Is the no 'OR' search an
obstacle for it?

If so we would really need to push the ticket #191 because UI doesn't
support search by only specifying specific attributes yet.


Your analysis is correct.

It may be considered a hack but what if I treat subject as the CRITERIA 
argument?


rob

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 1085 cert-find command

2013-02-14 Thread Petr Vobornik

On 02/14/2013 03:34 PM, Rob Crittenden wrote:

Petr Vobornik wrote:

On 02/07/2013 03:08 PM, Rob Crittenden wrote:

Petr Vobornik wrote:

On 02/06/2013 12:44 AM, Rob Crittenden wrote:

This adds a cert-find command for the dogtag backend.

Searches can be done by serial number, by subject, revocation reason,
issue date, notbefore, notafter and revocation dates.

I added some basic tests for this. I made it a separate test file
because the cert plugin tests do not use the declarative format and
rely
on the selfsign backend by default.

rob



Should I create Web UI in scope of this ticket or a new one?

I was also thinking if it's time to implement #191 'Web UI: specify
fields to search on' [1]. Maybe in Pilsner.

[1] https://fedorahosted.org/freeipa/ticket/191


I'm going to open a UI ticket once the API is finalized. I didn't want
to give you a moving target to work against.

rob



I see that the search requires to specify options for attributes to
search on. There is no general CRITERIA positional argument as in other
find commands or am I mistaken?

Is it possible to add the CRITERIA argument? Is the no 'OR' search an
obstacle for it?

If so we would really need to push the ticket #191 because UI doesn't
support search by only specifying specific attributes yet.


Your analysis is correct.

It may be considered a hack but what if I treat subject as the CRITERIA
argument?

rob


Better that than nothing.

Just a confirmation: when user does not set any option, it will return 
all certificates? Or it will return nothing?


I see Web UI implementation this way:
 1) implement simple search with the hack now
 2) if there will be time before release (after the refactoring and 
other tickets) implement #191 (will require UXD input) to implement this 
the proper way. We can move #191 to triage to decide it.

--
Petr Vobornik

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 1085 cert-find command

2013-02-14 Thread Petr Vobornik

On 02/14/2013 05:27 PM, Rob Crittenden wrote:

Petr Vobornik wrote:

On 02/14/2013 03:34 PM, Rob Crittenden wrote:

Petr Vobornik wrote:

On 02/07/2013 03:08 PM, Rob Crittenden wrote:

Petr Vobornik wrote:

On 02/06/2013 12:44 AM, Rob Crittenden wrote:

This adds a cert-find command for the dogtag backend.

Searches can be done by serial number, by subject, revocation
reason,
issue date, notbefore, notafter and revocation dates.

I added some basic tests for this. I made it a separate test file
because the cert plugin tests do not use the declarative format and
rely
on the selfsign backend by default.

rob



Should I create Web UI in scope of this ticket or a new one?

I was also thinking if it's time to implement #191 'Web UI: specify
fields to search on' [1]. Maybe in Pilsner.

[1] https://fedorahosted.org/freeipa/ticket/191


I'm going to open a UI ticket once the API is finalized. I didn't want
to give you a moving target to work against.

rob



I see that the search requires to specify options for attributes to
search on. There is no general CRITERIA positional argument as in other
find commands or am I mistaken?

Is it possible to add the CRITERIA argument? Is the no 'OR' search an
obstacle for it?

If so we would really need to push the ticket #191 because UI doesn't
support search by only specifying specific attributes yet.


Your analysis is correct.

It may be considered a hack but what if I treat subject as the CRITERIA
argument?

rob


Better that than nothing.

Just a confirmation: when user does not set any option, it will return
all certificates? Or it will return nothing?

I see Web UI implementation this way:
  1) implement simple search with the hack now
  2) if there will be time before release (after the refactoring and
other tickets) implement #191 (will require UXD input) to implement this
the proper way. We can move #191 to triage to decide it.


The thing is we have to live with whatever API choice we decide on.

It is more correct to have no positional args and stick with options, to
mimik what the remote API provides.


It's not a problem to use the --subject option, so we don't have to 
implement the positional argument when it has the same meaning. What I 
meant by use hack is 'filter by subject or nothing'.


-- This patch is OK from UI perspective (considering the limitations).



If you search on nothing you get everything, up to the default sizelimit.

rob




--
Petr Vobornik

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 1085 cert-find command

2013-02-13 Thread Petr Viktorin

On 02/06/2013 07:23 PM, Rob Crittenden wrote:

Petr Viktorin wrote:

On 02/06/2013 12:44 AM, Rob Crittenden wrote:

This adds a cert-find command for the dogtag backend.

Searches can be done by serial number, by subject, revocation reason,
issue date, notbefore, notafter and revocation dates.

I added some basic tests for this. I made it a separate test file
because the cert plugin tests do not use the declarative format and rely
on the selfsign backend by default.

rob


Thanks! The code works well, but I found a few issues.


These tests don't work when the full test suite is run: test_cert adds
and revokes additional certs that throw the code off.
Perhaps have the tests only query valid certs? I don't see that option
but I think it would be helpful to support.


I added some rather nasty hacks to the test to make things pass. I limit
the search to 10 certificates, which is the number start with by
default. There is an open dogtag ticket to return only VALID
certificates so we should be able to drop this eventually.

I had to go further on one of the revocation tests, limiting it to a
sizelimit of 1. The count changes every time the suite runs so this is
the safest for now. It also means that one test will fail if this is the
only part of the suite executed.


This gets rid of most of the failures, but it still fails the certs for 
this IPA server/short name tests if the cert from ./make-cert is 
present (creating it is part of `make test`).
If make-cert is run more times, it'll revoke the previous cert, so the 
test for revocation reason 4 fails as well.


It looks like when using sizelimit Dogtag will always discard *newer* 
certs, ones with higher serials. Is it documented behavior or does 
Dogtag just happen to do that?


I wonder how other people run their tests. This solution looks like it 
could break easily if people do something differently :(


I'm not sure how to solve this properly. Perhaps not using Declarative, 
and checking by hand that the wanted certs are in the response and the 
unwanted ones are not, would work better.







The API.txt check fails:
Option sizelimit? of command cert_find in ipalib, not in API file:


Int('sizelimit?', default=100, minvalue=0)


Ouch. I thought I had fixed that, obviously not. Done now.



What are --all and --raw for? Is the plan to implement --all if/when
Dogtag supports requesting additional data?


Correct, they don't do anything at the moment. I have an RFE open to
return additional data from certs. Once that is done then all will make
sense. I don't know that raw will ever do anything interesting here but
it comes with all commands.



The format of --validnotbefore-to and friends should be mentioned in
--help text; the following is confusing:
$ ipa cert-show 1
[...]
   Not Before: Wed Feb 06 09:32:17 2013 UTC
[...]
$ ipa cert-find -h
[...]
   --validnotbefore-to=STR
 Valid not before to this date
[...]
$ ipa cert-find --validnotbefore-to='Wed Feb 06 09:32:17 2013 UTC'
ipa: ERROR: invalid 'validnotbefore_to': time data u'Wed Feb 06 09:32:17
2013 UTC' does not match format '%Y-%m-%d'


It was listed in the top block but I added it to the usage help as well
for clarity.


Could you make the help text for --exactly more specific?


Done.



Please remove the extra whitespace at the end of dogtag.py

I'd welcome a link to the design page in the commit message.



both done

rob





--
Petr³

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 1085 cert-find command

2013-02-08 Thread Rob Crittenden

Jan Cholasta wrote:

Hi,

On 6.2.2013 00:44, Rob Crittenden wrote:

This adds a cert-find command for the dogtag backend.

Searches can be done by serial number, by subject, revocation reason,
issue date, notbefore, notafter and revocation dates.

I added some basic tests for this. I made it a separate test file
because the cert plugin tests do not use the declarative format and rely
on the selfsign backend by default.

rob



I have one design question: why do you emulate object interface with
Command plugins? Wouldn't it be better to add an actual Object plugin
and Method plugins? That way you would not have to duplicate the Object
bits for certs and as a result, the code would be cleaner and consistent
with the rest of our plugins.

Honza



I forget the details of the reasoning but IIRC it is because these 
commands aren't backed by LDAP. So the normal things we get out of 
Object don't apply.


rob

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 1085 cert-find command

2013-02-07 Thread Petr Vobornik

On 02/06/2013 12:44 AM, Rob Crittenden wrote:

This adds a cert-find command for the dogtag backend.

Searches can be done by serial number, by subject, revocation reason,
issue date, notbefore, notafter and revocation dates.

I added some basic tests for this. I made it a separate test file
because the cert plugin tests do not use the declarative format and rely
on the selfsign backend by default.

rob



Should I create Web UI in scope of this ticket or a new one?

I was also thinking if it's time to implement #191 'Web UI: specify 
fields to search on' [1]. Maybe in Pilsner.


[1] https://fedorahosted.org/freeipa/ticket/191
--
Petr Vobornik

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 1085 cert-find command

2013-02-07 Thread Rob Crittenden

Petr Vobornik wrote:

On 02/06/2013 12:44 AM, Rob Crittenden wrote:

This adds a cert-find command for the dogtag backend.

Searches can be done by serial number, by subject, revocation reason,
issue date, notbefore, notafter and revocation dates.

I added some basic tests for this. I made it a separate test file
because the cert plugin tests do not use the declarative format and rely
on the selfsign backend by default.

rob



Should I create Web UI in scope of this ticket or a new one?

I was also thinking if it's time to implement #191 'Web UI: specify
fields to search on' [1]. Maybe in Pilsner.

[1] https://fedorahosted.org/freeipa/ticket/191


I'm going to open a UI ticket once the API is finalized. I didn't want 
to give you a moving target to work against.


rob

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 1085 cert-find command

2013-02-07 Thread Jan Cholasta

Hi,

On 6.2.2013 00:44, Rob Crittenden wrote:

This adds a cert-find command for the dogtag backend.

Searches can be done by serial number, by subject, revocation reason,
issue date, notbefore, notafter and revocation dates.

I added some basic tests for this. I made it a separate test file
because the cert plugin tests do not use the declarative format and rely
on the selfsign backend by default.

rob



I have one design question: why do you emulate object interface with 
Command plugins? Wouldn't it be better to add an actual Object plugin 
and Method plugins? That way you would not have to duplicate the Object 
bits for certs and as a result, the code would be cleaner and consistent 
with the rest of our plugins.


Honza

--
Jan Cholasta

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 1085 cert-find command

2013-02-06 Thread Petr Viktorin

On 02/06/2013 12:44 AM, Rob Crittenden wrote:

This adds a cert-find command for the dogtag backend.

Searches can be done by serial number, by subject, revocation reason,
issue date, notbefore, notafter and revocation dates.

I added some basic tests for this. I made it a separate test file
because the cert plugin tests do not use the declarative format and rely
on the selfsign backend by default.

rob


Thanks! The code works well, but I found a few issues.


These tests don't work when the full test suite is run: test_cert adds 
and revokes additional certs that throw the code off.
Perhaps have the tests only query valid certs? I don't see that option 
but I think it would be helpful to support.



The API.txt check fails:
Option sizelimit? of command cert_find in ipalib, not in API file: 




Int('sizelimit?', default=100, minvalue=0) 






What are --all and --raw for? Is the plan to implement --all if/when 
Dogtag supports requesting additional data?



The format of --validnotbefore-to and friends should be mentioned in 
--help text; the following is confusing:

$ ipa cert-show 1
[...]
  Not Before: Wed Feb 06 09:32:17 2013 UTC
[...]
$ ipa cert-find -h
[...]
  --validnotbefore-to=STR
Valid not before to this date
[...]
$ ipa cert-find --validnotbefore-to='Wed Feb 06 09:32:17 2013 UTC'
ipa: ERROR: invalid 'validnotbefore_to': time data u'Wed Feb 06 09:32:17 
2013 UTC' does not match format '%Y-%m-%d'


Could you make the help text for --exactly more specific?


Please remove the extra whitespace at the end of dogtag.py

I'd welcome a link to the design page in the commit message.

--
Petr³

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 1085 cert-find command

2013-02-06 Thread Rob Crittenden

Petr Viktorin wrote:

On 02/06/2013 12:44 AM, Rob Crittenden wrote:

This adds a cert-find command for the dogtag backend.

Searches can be done by serial number, by subject, revocation reason,
issue date, notbefore, notafter and revocation dates.

I added some basic tests for this. I made it a separate test file
because the cert plugin tests do not use the declarative format and rely
on the selfsign backend by default.

rob


Thanks! The code works well, but I found a few issues.


These tests don't work when the full test suite is run: test_cert adds
and revokes additional certs that throw the code off.
Perhaps have the tests only query valid certs? I don't see that option
but I think it would be helpful to support.


I added some rather nasty hacks to the test to make things pass. I limit 
the search to 10 certificates, which is the number start with by 
default. There is an open dogtag ticket to return only VALID 
certificates so we should be able to drop this eventually.


I had to go further on one of the revocation tests, limiting it to a 
sizelimit of 1. The count changes every time the suite runs so this is 
the safest for now. It also means that one test will fail if this is the 
only part of the suite executed.





The API.txt check fails:
Option sizelimit? of command cert_find in ipalib, not in API file:


Int('sizelimit?', default=100, minvalue=0)


Ouch. I thought I had fixed that, obviously not. Done now.



What are --all and --raw for? Is the plan to implement --all if/when
Dogtag supports requesting additional data?


Correct, they don't do anything at the moment. I have an RFE open to 
return additional data from certs. Once that is done then all will make 
sense. I don't know that raw will ever do anything interesting here but 
it comes with all commands.




The format of --validnotbefore-to and friends should be mentioned in
--help text; the following is confusing:
$ ipa cert-show 1
[...]
   Not Before: Wed Feb 06 09:32:17 2013 UTC
[...]
$ ipa cert-find -h
[...]
   --validnotbefore-to=STR
 Valid not before to this date
[...]
$ ipa cert-find --validnotbefore-to='Wed Feb 06 09:32:17 2013 UTC'
ipa: ERROR: invalid 'validnotbefore_to': time data u'Wed Feb 06 09:32:17
2013 UTC' does not match format '%Y-%m-%d'


It was listed in the top block but I added it to the usage help as well 
for clarity.



Could you make the help text for --exactly more specific?


Done.



Please remove the extra whitespace at the end of dogtag.py

I'd welcome a link to the design page in the commit message.



both done

rob
From b8994fd0530ef357d79605fb4b74c6ff0eb2e536 Mon Sep 17 00:00:00 2001
From: Rob Crittenden rcrit...@redhat.com
Date: Thu, 15 Nov 2012 10:55:33 -0500
Subject: [PATCH] Implement the cert-find command for the dogtag CA backend.

Use a new RESTful API provided by dogtag 10+. Construct an XML document
representing the search request. The output is limited to whatever dogtag
sends us, there is no way to request additional attributes other than
to read each certificate individually.

dogtag uses a boolean for each search term to indicate that it is used.
Presense of the search item is not enough, both need to be set.

The search operation is unauthenticated

Design page: http://freeipa.org/page/V3/Cert_find

https://fedorahosted.org/freeipa/ticket/2528
---
 API.txt |  23 ++
 VERSION |   2 +-
 ipalib/plugins/cert.py  | 137 +-
 ipaserver/plugins/dogtag.py | 138 ++
 ipaserver/plugins/rabase.py |   8 +
 tests/test_xmlrpc/test_cert_find.py | 531 
 6 files changed, 836 insertions(+), 3 deletions(-)
 create mode 100644 tests/test_xmlrpc/test_cert_find.py

diff --git a/API.txt b/API.txt
index 8fbfe6f5d8da44e991b8d1a36725fc6ace1f0616..6b997f37b455366c66b34fd2df11c2acaa79d739 100644
--- a/API.txt
+++ b/API.txt
@@ -425,6 +425,29 @@ args: 1,0,2
 arg: Any('methods*')
 output: Output('count', type 'int', None)
 output: Output('results', (type 'list', type 'tuple'), None)
+command: cert_find
+args: 0,17,4
+option: Flag('all', autofill=True, cli_name='all', default=False, exclude='webui')
+option: Flag('exactly?', autofill=True, default=False)
+option: Str('issuedon_from?', autofill=False)
+option: Str('issuedon_to?', autofill=False)
+option: Int('max_serial_number?', autofill=False, maxvalue=2147483647)
+option: Int('min_serial_number?', autofill=False, minvalue=0)
+option: Flag('raw', autofill=True, cli_name='raw', default=False, exclude='webui')
+option: Int('revocation_reason?', autofill=False, maxvalue=10, minvalue=0)
+option: Str('revokedon_from?', autofill=False)
+option: Str('revokedon_to?', autofill=False)
+option: Int('sizelimit?', default=100, minvalue=0)
+option: Str('subject?', autofill=False)
+option: Str('validnotafter_from?', autofill=False)
+option: Str('validnotafter_to?', autofill=False)
+option: Str('validnotbefore_from?',