Re: [Freeipa-devel] [PATCH] 1085 cert-find command
Petr Viktorin wrote: On 02/18/2013 08:39 PM, Rob Crittenden wrote: Rob Crittenden wrote: Petr Viktorin wrote: On 02/15/2013 10:43 PM, Rob Crittenden wrote: Petr Viktorin wrote: On 02/06/2013 07:23 PM, Rob Crittenden wrote: Petr Viktorin wrote: On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Thanks! The code works well, but I found a few issues. These tests don't work when the full test suite is run: test_cert adds and revokes additional certs that throw the code off. Perhaps have the tests only query valid certs? I don't see that option but I think it would be helpful to support. I added some rather nasty hacks to the test to make things pass. I limit the search to 10 certificates, which is the number start with by default. There is an open dogtag ticket to return only VALID certificates so we should be able to drop this eventually. I had to go further on one of the revocation tests, limiting it to a sizelimit of 1. The count changes every time the suite runs so this is the safest for now. It also means that one test will fail if this is the only part of the suite executed. This gets rid of most of the failures, but it still fails the "certs for this IPA server/short name" tests if the cert from ./make-cert is present (creating it is part of `make test`). If make-cert is run more times, it'll revoke the previous cert, so the test for revocation reason 4 fails as well. It looks like when using sizelimit Dogtag will always discard *newer* certs, ones with higher serials. Is it documented behavior or does Dogtag just happen to do that? It isn't documented anywhere I could find, it is just what dogtag returns I wonder how other people run their tests. This solution looks like it could break easily if people do something differently :( I'm not sure how to solve this properly. Perhaps not using Declarative, and checking "by hand" that the wanted certs are in the response and the unwanted ones are not, would work better. I ended up switching the test class. It is not a very fine-grained set of tests, mostly searching with limits and verifying that we fall within a reasonable range, but it is better than nothing. rob This works much better, thanks! Just two nitpicks now. The patch doesn't apply well, there's a conflict in VERSION and some added trailing whitespace, AFAIK this would be the only (first?) test that relies on Nose's ordering of test modules -- tests 0011 and 0030 rely on the other cert tests running first. Please at least mention that in a comment. Or better, move class test_cert_find to test_cert.py Rebased the patch and removed whitespace. I went ahead and combined this with the existing test_cert file. Originally test_cert was only tested against lite-server but since it works against a live dogtag server too it makes sense to combine them. I improved the set up documentation a little bit and tried to handle all the different configurations that one might see so that this should be runnable against either a live server or the lite-server for both the selfsign and dogtag backends. This relies on the user configuring ~/.ipa/default.conf to match the remote server. There is no way from a client to know what kind of CA backend a server is running. rob And the patch. rob ACK, thank you! pushed to master and ipa-3-1 ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
On 02/18/2013 08:39 PM, Rob Crittenden wrote: Rob Crittenden wrote: Petr Viktorin wrote: On 02/15/2013 10:43 PM, Rob Crittenden wrote: Petr Viktorin wrote: On 02/06/2013 07:23 PM, Rob Crittenden wrote: Petr Viktorin wrote: On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Thanks! The code works well, but I found a few issues. These tests don't work when the full test suite is run: test_cert adds and revokes additional certs that throw the code off. Perhaps have the tests only query valid certs? I don't see that option but I think it would be helpful to support. I added some rather nasty hacks to the test to make things pass. I limit the search to 10 certificates, which is the number start with by default. There is an open dogtag ticket to return only VALID certificates so we should be able to drop this eventually. I had to go further on one of the revocation tests, limiting it to a sizelimit of 1. The count changes every time the suite runs so this is the safest for now. It also means that one test will fail if this is the only part of the suite executed. This gets rid of most of the failures, but it still fails the "certs for this IPA server/short name" tests if the cert from ./make-cert is present (creating it is part of `make test`). If make-cert is run more times, it'll revoke the previous cert, so the test for revocation reason 4 fails as well. It looks like when using sizelimit Dogtag will always discard *newer* certs, ones with higher serials. Is it documented behavior or does Dogtag just happen to do that? It isn't documented anywhere I could find, it is just what dogtag returns I wonder how other people run their tests. This solution looks like it could break easily if people do something differently :( I'm not sure how to solve this properly. Perhaps not using Declarative, and checking "by hand" that the wanted certs are in the response and the unwanted ones are not, would work better. I ended up switching the test class. It is not a very fine-grained set of tests, mostly searching with limits and verifying that we fall within a reasonable range, but it is better than nothing. rob This works much better, thanks! Just two nitpicks now. The patch doesn't apply well, there's a conflict in VERSION and some added trailing whitespace, AFAIK this would be the only (first?) test that relies on Nose's ordering of test modules -- tests 0011 and 0030 rely on the other cert tests running first. Please at least mention that in a comment. Or better, move class test_cert_find to test_cert.py Rebased the patch and removed whitespace. I went ahead and combined this with the existing test_cert file. Originally test_cert was only tested against lite-server but since it works against a live dogtag server too it makes sense to combine them. I improved the set up documentation a little bit and tried to handle all the different configurations that one might see so that this should be runnable against either a live server or the lite-server for both the selfsign and dogtag backends. This relies on the user configuring ~/.ipa/default.conf to match the remote server. There is no way from a client to know what kind of CA backend a server is running. rob And the patch. rob ACK, thank you! -- Petr³ ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
Rob Crittenden wrote: Petr Viktorin wrote: On 02/15/2013 10:43 PM, Rob Crittenden wrote: Petr Viktorin wrote: On 02/06/2013 07:23 PM, Rob Crittenden wrote: Petr Viktorin wrote: On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Thanks! The code works well, but I found a few issues. These tests don't work when the full test suite is run: test_cert adds and revokes additional certs that throw the code off. Perhaps have the tests only query valid certs? I don't see that option but I think it would be helpful to support. I added some rather nasty hacks to the test to make things pass. I limit the search to 10 certificates, which is the number start with by default. There is an open dogtag ticket to return only VALID certificates so we should be able to drop this eventually. I had to go further on one of the revocation tests, limiting it to a sizelimit of 1. The count changes every time the suite runs so this is the safest for now. It also means that one test will fail if this is the only part of the suite executed. This gets rid of most of the failures, but it still fails the "certs for this IPA server/short name" tests if the cert from ./make-cert is present (creating it is part of `make test`). If make-cert is run more times, it'll revoke the previous cert, so the test for revocation reason 4 fails as well. It looks like when using sizelimit Dogtag will always discard *newer* certs, ones with higher serials. Is it documented behavior or does Dogtag just happen to do that? It isn't documented anywhere I could find, it is just what dogtag returns I wonder how other people run their tests. This solution looks like it could break easily if people do something differently :( I'm not sure how to solve this properly. Perhaps not using Declarative, and checking "by hand" that the wanted certs are in the response and the unwanted ones are not, would work better. I ended up switching the test class. It is not a very fine-grained set of tests, mostly searching with limits and verifying that we fall within a reasonable range, but it is better than nothing. rob This works much better, thanks! Just two nitpicks now. The patch doesn't apply well, there's a conflict in VERSION and some added trailing whitespace, AFAIK this would be the only (first?) test that relies on Nose's ordering of test modules -- tests 0011 and 0030 rely on the other cert tests running first. Please at least mention that in a comment. Or better, move class test_cert_find to test_cert.py Rebased the patch and removed whitespace. I went ahead and combined this with the existing test_cert file. Originally test_cert was only tested against lite-server but since it works against a live dogtag server too it makes sense to combine them. I improved the set up documentation a little bit and tried to handle all the different configurations that one might see so that this should be runnable against either a live server or the lite-server for both the selfsign and dogtag backends. This relies on the user configuring ~/.ipa/default.conf to match the remote server. There is no way from a client to know what kind of CA backend a server is running. rob And the patch. rob >From c06cb0899e4b0a02feadd1dbfb4bfcbe28109e9a Mon Sep 17 00:00:00 2001 From: Rob Crittenden Date: Thu, 15 Nov 2012 10:55:33 -0500 Subject: [PATCH] Implement the cert-find command for the dogtag CA backend. Use a new RESTful API provided by dogtag 10+. Construct an XML document representing the search request. The output is limited to whatever dogtag sends us, there is no way to request additional attributes other than to read each certificate individually. dogtag uses a boolean for each search term to indicate that it is used. Presense of the search item is not enough, both need to be set. The search operation is unauthenticated Design page: http://freeipa.org/page/V3/Cert_find https://fedorahosted.org/freeipa/ticket/2528 --- API.txt| 23 +++ VERSION| 2 +- ipalib/plugins/cert.py | 137 +- ipaserver/plugins/dogtag.py| 138 ++ ipaserver/plugins/rabase.py| 8 ++ tests/test_xmlrpc/test_cert.py | 312 ++--- 6 files changed, 598 insertions(+), 22 deletions(-) diff --git a/API.txt b/API.txt index 5219c51be62862c43ebe9396147ff220b33605c7..e39936974c7216c4e0d9266bbf56d0f3ba2b3f01 100644 --- a/API.txt +++ b/API.txt @@ -425,6 +425,29 @@ args: 1,0,2 arg: Any('methods*') output: Output('count', , None) output: Output('results', (, ), None) +command: cert_find +args: 0,17,4 +o
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
Petr Viktorin wrote: On 02/15/2013 10:43 PM, Rob Crittenden wrote: Petr Viktorin wrote: On 02/06/2013 07:23 PM, Rob Crittenden wrote: Petr Viktorin wrote: On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Thanks! The code works well, but I found a few issues. These tests don't work when the full test suite is run: test_cert adds and revokes additional certs that throw the code off. Perhaps have the tests only query valid certs? I don't see that option but I think it would be helpful to support. I added some rather nasty hacks to the test to make things pass. I limit the search to 10 certificates, which is the number start with by default. There is an open dogtag ticket to return only VALID certificates so we should be able to drop this eventually. I had to go further on one of the revocation tests, limiting it to a sizelimit of 1. The count changes every time the suite runs so this is the safest for now. It also means that one test will fail if this is the only part of the suite executed. This gets rid of most of the failures, but it still fails the "certs for this IPA server/short name" tests if the cert from ./make-cert is present (creating it is part of `make test`). If make-cert is run more times, it'll revoke the previous cert, so the test for revocation reason 4 fails as well. It looks like when using sizelimit Dogtag will always discard *newer* certs, ones with higher serials. Is it documented behavior or does Dogtag just happen to do that? It isn't documented anywhere I could find, it is just what dogtag returns I wonder how other people run their tests. This solution looks like it could break easily if people do something differently :( I'm not sure how to solve this properly. Perhaps not using Declarative, and checking "by hand" that the wanted certs are in the response and the unwanted ones are not, would work better. I ended up switching the test class. It is not a very fine-grained set of tests, mostly searching with limits and verifying that we fall within a reasonable range, but it is better than nothing. rob This works much better, thanks! Just two nitpicks now. The patch doesn't apply well, there's a conflict in VERSION and some added trailing whitespace, AFAIK this would be the only (first?) test that relies on Nose's ordering of test modules -- tests 0011 and 0030 rely on the other cert tests running first. Please at least mention that in a comment. Or better, move class test_cert_find to test_cert.py Rebased the patch and removed whitespace. I went ahead and combined this with the existing test_cert file. Originally test_cert was only tested against lite-server but since it works against a live dogtag server too it makes sense to combine them. I improved the set up documentation a little bit and tried to handle all the different configurations that one might see so that this should be runnable against either a live server or the lite-server for both the selfsign and dogtag backends. This relies on the user configuring ~/.ipa/default.conf to match the remote server. There is no way from a client to know what kind of CA backend a server is running. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
On 02/15/2013 10:43 PM, Rob Crittenden wrote: Petr Viktorin wrote: On 02/06/2013 07:23 PM, Rob Crittenden wrote: Petr Viktorin wrote: On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Thanks! The code works well, but I found a few issues. These tests don't work when the full test suite is run: test_cert adds and revokes additional certs that throw the code off. Perhaps have the tests only query valid certs? I don't see that option but I think it would be helpful to support. I added some rather nasty hacks to the test to make things pass. I limit the search to 10 certificates, which is the number start with by default. There is an open dogtag ticket to return only VALID certificates so we should be able to drop this eventually. I had to go further on one of the revocation tests, limiting it to a sizelimit of 1. The count changes every time the suite runs so this is the safest for now. It also means that one test will fail if this is the only part of the suite executed. This gets rid of most of the failures, but it still fails the "certs for this IPA server/short name" tests if the cert from ./make-cert is present (creating it is part of `make test`). If make-cert is run more times, it'll revoke the previous cert, so the test for revocation reason 4 fails as well. It looks like when using sizelimit Dogtag will always discard *newer* certs, ones with higher serials. Is it documented behavior or does Dogtag just happen to do that? It isn't documented anywhere I could find, it is just what dogtag returns I wonder how other people run their tests. This solution looks like it could break easily if people do something differently :( I'm not sure how to solve this properly. Perhaps not using Declarative, and checking "by hand" that the wanted certs are in the response and the unwanted ones are not, would work better. I ended up switching the test class. It is not a very fine-grained set of tests, mostly searching with limits and verifying that we fall within a reasonable range, but it is better than nothing. rob This works much better, thanks! Just two nitpicks now. The patch doesn't apply well, there's a conflict in VERSION and some added trailing whitespace, AFAIK this would be the only (first?) test that relies on Nose's ordering of test modules -- tests 0011 and 0030 rely on the other cert tests running first. Please at least mention that in a comment. Or better, move class test_cert_find to test_cert.py -- Petr³ ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
Petr Viktorin wrote: On 02/06/2013 07:23 PM, Rob Crittenden wrote: Petr Viktorin wrote: On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Thanks! The code works well, but I found a few issues. These tests don't work when the full test suite is run: test_cert adds and revokes additional certs that throw the code off. Perhaps have the tests only query valid certs? I don't see that option but I think it would be helpful to support. I added some rather nasty hacks to the test to make things pass. I limit the search to 10 certificates, which is the number start with by default. There is an open dogtag ticket to return only VALID certificates so we should be able to drop this eventually. I had to go further on one of the revocation tests, limiting it to a sizelimit of 1. The count changes every time the suite runs so this is the safest for now. It also means that one test will fail if this is the only part of the suite executed. This gets rid of most of the failures, but it still fails the "certs for this IPA server/short name" tests if the cert from ./make-cert is present (creating it is part of `make test`). If make-cert is run more times, it'll revoke the previous cert, so the test for revocation reason 4 fails as well. It looks like when using sizelimit Dogtag will always discard *newer* certs, ones with higher serials. Is it documented behavior or does Dogtag just happen to do that? It isn't documented anywhere I could find, it is just what dogtag returns I wonder how other people run their tests. This solution looks like it could break easily if people do something differently :( I'm not sure how to solve this properly. Perhaps not using Declarative, and checking "by hand" that the wanted certs are in the response and the unwanted ones are not, would work better. I ended up switching the test class. It is not a very fine-grained set of tests, mostly searching with limits and verifying that we fall within a reasonable range, but it is better than nothing. rob >From e7972a96abc62d41a90f9bd3e816c827627b7120 Mon Sep 17 00:00:00 2001 From: Rob Crittenden Date: Thu, 15 Nov 2012 10:55:33 -0500 Subject: [PATCH] Implement the cert-find command for the dogtag CA backend. Use a new RESTful API provided by dogtag 10+. Construct an XML document representing the search request. The output is limited to whatever dogtag sends us, there is no way to request additional attributes other than to read each certificate individually. dogtag uses a boolean for each search term to indicate that it is used. Presense of the search item is not enough, both need to be set. The search operation is unauthenticated Design page: http://freeipa.org/page/V3/Cert_find https://fedorahosted.org/freeipa/ticket/2528 --- API.txt | 23 +++ VERSION | 2 +- ipalib/plugins/cert.py | 137 - ipaserver/plugins/dogtag.py | 138 + ipaserver/plugins/rabase.py | 8 + tests/test_xmlrpc/test_cert_find.py | 289 6 files changed, 594 insertions(+), 3 deletions(-) create mode 100644 tests/test_xmlrpc/test_cert_find.py diff --git a/API.txt b/API.txt index 8fbfe6f5d8da44e991b8d1a36725fc6ace1f0616..6b997f37b455366c66b34fd2df11c2acaa79d739 100644 --- a/API.txt +++ b/API.txt @@ -425,6 +425,29 @@ args: 1,0,2 arg: Any('methods*') output: Output('count', , None) output: Output('results', (, ), None) +command: cert_find +args: 0,17,4 +option: Flag('all', autofill=True, cli_name='all', default=False, exclude='webui') +option: Flag('exactly?', autofill=True, default=False) +option: Str('issuedon_from?', autofill=False) +option: Str('issuedon_to?', autofill=False) +option: Int('max_serial_number?', autofill=False, maxvalue=2147483647) +option: Int('min_serial_number?', autofill=False, minvalue=0) +option: Flag('raw', autofill=True, cli_name='raw', default=False, exclude='webui') +option: Int('revocation_reason?', autofill=False, maxvalue=10, minvalue=0) +option: Str('revokedon_from?', autofill=False) +option: Str('revokedon_to?', autofill=False) +option: Int('sizelimit?', default=100, minvalue=0) +option: Str('subject?', autofill=False) +option: Str('validnotafter_from?', autofill=False) +option: Str('validnotafter_to?', autofill=False) +option: Str('validnotbefore_from?', autofill=False) +option: Str('validnotbefore_to?', autofill=False) +option: Str('version?', exclude='webui') +output: Output('count', , None) +output: ListOfEntries('result', (, ), Gettext('A list of LDAP entries', domain='ipa', localedir=None)) +output: Output('
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
On 02/14/2013 05:27 PM, Rob Crittenden wrote: Petr Vobornik wrote: On 02/14/2013 03:34 PM, Rob Crittenden wrote: Petr Vobornik wrote: On 02/07/2013 03:08 PM, Rob Crittenden wrote: Petr Vobornik wrote: On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Should I create Web UI in scope of this ticket or a new one? I was also thinking if it's time to implement #191 'Web UI: specify fields to search on' [1]. Maybe in Pilsner. [1] https://fedorahosted.org/freeipa/ticket/191 I'm going to open a UI ticket once the API is finalized. I didn't want to give you a moving target to work against. rob I see that the search requires to specify options for attributes to search on. There is no general CRITERIA positional argument as in other find commands or am I mistaken? Is it possible to add the CRITERIA argument? Is the no 'OR' search an obstacle for it? If so we would really need to push the ticket #191 because UI doesn't support search by only specifying specific attributes yet. Your analysis is correct. It may be considered a hack but what if I treat subject as the CRITERIA argument? rob Better that than nothing. Just a confirmation: when user does not set any option, it will return all certificates? Or it will return nothing? I see Web UI implementation this way: 1) implement simple search with the hack now 2) if there will be time before release (after the refactoring and other tickets) implement #191 (will require UXD input) to implement this the proper way. We can move #191 to triage to decide it. The thing is we have to live with whatever API choice we decide on. It is more correct to have no positional args and stick with options, to mimik what the remote API provides. It's not a problem to use the --subject option, so we don't have to implement the positional argument when it has the same meaning. What I meant by use hack is 'filter by subject or nothing'. --> This patch is OK from UI perspective (considering the limitations). If you search on nothing you get everything, up to the default sizelimit. rob -- Petr Vobornik ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
Petr Vobornik wrote: On 02/14/2013 03:34 PM, Rob Crittenden wrote: Petr Vobornik wrote: On 02/07/2013 03:08 PM, Rob Crittenden wrote: Petr Vobornik wrote: On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Should I create Web UI in scope of this ticket or a new one? I was also thinking if it's time to implement #191 'Web UI: specify fields to search on' [1]. Maybe in Pilsner. [1] https://fedorahosted.org/freeipa/ticket/191 I'm going to open a UI ticket once the API is finalized. I didn't want to give you a moving target to work against. rob I see that the search requires to specify options for attributes to search on. There is no general CRITERIA positional argument as in other find commands or am I mistaken? Is it possible to add the CRITERIA argument? Is the no 'OR' search an obstacle for it? If so we would really need to push the ticket #191 because UI doesn't support search by only specifying specific attributes yet. Your analysis is correct. It may be considered a hack but what if I treat subject as the CRITERIA argument? rob Better that than nothing. Just a confirmation: when user does not set any option, it will return all certificates? Or it will return nothing? I see Web UI implementation this way: 1) implement simple search with the hack now 2) if there will be time before release (after the refactoring and other tickets) implement #191 (will require UXD input) to implement this the proper way. We can move #191 to triage to decide it. The thing is we have to live with whatever API choice we decide on. It is more correct to have no positional args and stick with options, to mimik what the remote API provides. If you search on nothing you get everything, up to the default sizelimit. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
On 02/14/2013 03:34 PM, Rob Crittenden wrote: Petr Vobornik wrote: On 02/07/2013 03:08 PM, Rob Crittenden wrote: Petr Vobornik wrote: On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Should I create Web UI in scope of this ticket or a new one? I was also thinking if it's time to implement #191 'Web UI: specify fields to search on' [1]. Maybe in Pilsner. [1] https://fedorahosted.org/freeipa/ticket/191 I'm going to open a UI ticket once the API is finalized. I didn't want to give you a moving target to work against. rob I see that the search requires to specify options for attributes to search on. There is no general CRITERIA positional argument as in other find commands or am I mistaken? Is it possible to add the CRITERIA argument? Is the no 'OR' search an obstacle for it? If so we would really need to push the ticket #191 because UI doesn't support search by only specifying specific attributes yet. Your analysis is correct. It may be considered a hack but what if I treat subject as the CRITERIA argument? rob Better that than nothing. Just a confirmation: when user does not set any option, it will return all certificates? Or it will return nothing? I see Web UI implementation this way: 1) implement simple search with the hack now 2) if there will be time before release (after the refactoring and other tickets) implement #191 (will require UXD input) to implement this the proper way. We can move #191 to triage to decide it. -- Petr Vobornik ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
Petr Vobornik wrote: On 02/07/2013 03:08 PM, Rob Crittenden wrote: Petr Vobornik wrote: On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Should I create Web UI in scope of this ticket or a new one? I was also thinking if it's time to implement #191 'Web UI: specify fields to search on' [1]. Maybe in Pilsner. [1] https://fedorahosted.org/freeipa/ticket/191 I'm going to open a UI ticket once the API is finalized. I didn't want to give you a moving target to work against. rob I see that the search requires to specify options for attributes to search on. There is no general CRITERIA positional argument as in other find commands or am I mistaken? Is it possible to add the CRITERIA argument? Is the no 'OR' search an obstacle for it? If so we would really need to push the ticket #191 because UI doesn't support search by only specifying specific attributes yet. Your analysis is correct. It may be considered a hack but what if I treat subject as the CRITERIA argument? rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
On 02/07/2013 03:08 PM, Rob Crittenden wrote: Petr Vobornik wrote: On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Should I create Web UI in scope of this ticket or a new one? I was also thinking if it's time to implement #191 'Web UI: specify fields to search on' [1]. Maybe in Pilsner. [1] https://fedorahosted.org/freeipa/ticket/191 I'm going to open a UI ticket once the API is finalized. I didn't want to give you a moving target to work against. rob I see that the search requires to specify options for attributes to search on. There is no general CRITERIA positional argument as in other find commands or am I mistaken? Is it possible to add the CRITERIA argument? Is the no 'OR' search an obstacle for it? If so we would really need to push the ticket #191 because UI doesn't support search by only specifying specific attributes yet. -- Petr Vobornik ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
On 02/06/2013 07:23 PM, Rob Crittenden wrote: Petr Viktorin wrote: On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Thanks! The code works well, but I found a few issues. These tests don't work when the full test suite is run: test_cert adds and revokes additional certs that throw the code off. Perhaps have the tests only query valid certs? I don't see that option but I think it would be helpful to support. I added some rather nasty hacks to the test to make things pass. I limit the search to 10 certificates, which is the number start with by default. There is an open dogtag ticket to return only VALID certificates so we should be able to drop this eventually. I had to go further on one of the revocation tests, limiting it to a sizelimit of 1. The count changes every time the suite runs so this is the safest for now. It also means that one test will fail if this is the only part of the suite executed. This gets rid of most of the failures, but it still fails the "certs for this IPA server/short name" tests if the cert from ./make-cert is present (creating it is part of `make test`). If make-cert is run more times, it'll revoke the previous cert, so the test for revocation reason 4 fails as well. It looks like when using sizelimit Dogtag will always discard *newer* certs, ones with higher serials. Is it documented behavior or does Dogtag just happen to do that? I wonder how other people run their tests. This solution looks like it could break easily if people do something differently :( I'm not sure how to solve this properly. Perhaps not using Declarative, and checking "by hand" that the wanted certs are in the response and the unwanted ones are not, would work better. The API.txt check fails: Option sizelimit? of command cert_find in ipalib, not in API file: Int('sizelimit?', default=100, minvalue=0) Ouch. I thought I had fixed that, obviously not. Done now. What are --all and --raw for? Is the plan to implement --all if/when Dogtag supports requesting additional data? Correct, they don't do anything at the moment. I have an RFE open to return additional data from certs. Once that is done then all will make sense. I don't know that raw will ever do anything interesting here but it comes with all commands. The format of --validnotbefore-to and friends should be mentioned in --help text; the following is confusing: $ ipa cert-show 1 [...] Not Before: Wed Feb 06 09:32:17 2013 UTC [...] $ ipa cert-find -h [...] --validnotbefore-to=STR Valid not before to this date [...] $ ipa cert-find --validnotbefore-to='Wed Feb 06 09:32:17 2013 UTC' ipa: ERROR: invalid 'validnotbefore_to': time data u'Wed Feb 06 09:32:17 2013 UTC' does not match format '%Y-%m-%d' It was listed in the top block but I added it to the usage help as well for clarity. Could you make the help text for --exactly more specific? Done. Please remove the extra whitespace at the end of dogtag.py I'd welcome a link to the design page in the commit message. both done rob -- Petr³ ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
Jan Cholasta wrote: Hi, On 6.2.2013 00:44, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob I have one design question: why do you emulate object interface with Command plugins? Wouldn't it be better to add an actual Object plugin and Method plugins? That way you would not have to duplicate the Object bits for certs and as a result, the code would be cleaner and consistent with the rest of our plugins. Honza I forget the details of the reasoning but IIRC it is because these commands aren't backed by LDAP. So the normal things we get out of Object don't apply. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
Hi, On 6.2.2013 00:44, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob I have one design question: why do you emulate object interface with Command plugins? Wouldn't it be better to add an actual Object plugin and Method plugins? That way you would not have to duplicate the Object bits for certs and as a result, the code would be cleaner and consistent with the rest of our plugins. Honza -- Jan Cholasta ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
Petr Vobornik wrote: On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Should I create Web UI in scope of this ticket or a new one? I was also thinking if it's time to implement #191 'Web UI: specify fields to search on' [1]. Maybe in Pilsner. [1] https://fedorahosted.org/freeipa/ticket/191 I'm going to open a UI ticket once the API is finalized. I didn't want to give you a moving target to work against. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Should I create Web UI in scope of this ticket or a new one? I was also thinking if it's time to implement #191 'Web UI: specify fields to search on' [1]. Maybe in Pilsner. [1] https://fedorahosted.org/freeipa/ticket/191 -- Petr Vobornik ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
Petr Viktorin wrote: On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Thanks! The code works well, but I found a few issues. These tests don't work when the full test suite is run: test_cert adds and revokes additional certs that throw the code off. Perhaps have the tests only query valid certs? I don't see that option but I think it would be helpful to support. I added some rather nasty hacks to the test to make things pass. I limit the search to 10 certificates, which is the number start with by default. There is an open dogtag ticket to return only VALID certificates so we should be able to drop this eventually. I had to go further on one of the revocation tests, limiting it to a sizelimit of 1. The count changes every time the suite runs so this is the safest for now. It also means that one test will fail if this is the only part of the suite executed. The API.txt check fails: Option sizelimit? of command cert_find in ipalib, not in API file: Int('sizelimit?', default=100, minvalue=0) Ouch. I thought I had fixed that, obviously not. Done now. What are --all and --raw for? Is the plan to implement --all if/when Dogtag supports requesting additional data? Correct, they don't do anything at the moment. I have an RFE open to return additional data from certs. Once that is done then all will make sense. I don't know that raw will ever do anything interesting here but it comes with all commands. The format of --validnotbefore-to and friends should be mentioned in --help text; the following is confusing: $ ipa cert-show 1 [...] Not Before: Wed Feb 06 09:32:17 2013 UTC [...] $ ipa cert-find -h [...] --validnotbefore-to=STR Valid not before to this date [...] $ ipa cert-find --validnotbefore-to='Wed Feb 06 09:32:17 2013 UTC' ipa: ERROR: invalid 'validnotbefore_to': time data u'Wed Feb 06 09:32:17 2013 UTC' does not match format '%Y-%m-%d' It was listed in the top block but I added it to the usage help as well for clarity. Could you make the help text for --exactly more specific? Done. Please remove the extra whitespace at the end of dogtag.py I'd welcome a link to the design page in the commit message. both done rob >From b8994fd0530ef357d79605fb4b74c6ff0eb2e536 Mon Sep 17 00:00:00 2001 From: Rob Crittenden Date: Thu, 15 Nov 2012 10:55:33 -0500 Subject: [PATCH] Implement the cert-find command for the dogtag CA backend. Use a new RESTful API provided by dogtag 10+. Construct an XML document representing the search request. The output is limited to whatever dogtag sends us, there is no way to request additional attributes other than to read each certificate individually. dogtag uses a boolean for each search term to indicate that it is used. Presense of the search item is not enough, both need to be set. The search operation is unauthenticated Design page: http://freeipa.org/page/V3/Cert_find https://fedorahosted.org/freeipa/ticket/2528 --- API.txt | 23 ++ VERSION | 2 +- ipalib/plugins/cert.py | 137 +- ipaserver/plugins/dogtag.py | 138 ++ ipaserver/plugins/rabase.py | 8 + tests/test_xmlrpc/test_cert_find.py | 531 6 files changed, 836 insertions(+), 3 deletions(-) create mode 100644 tests/test_xmlrpc/test_cert_find.py diff --git a/API.txt b/API.txt index 8fbfe6f5d8da44e991b8d1a36725fc6ace1f0616..6b997f37b455366c66b34fd2df11c2acaa79d739 100644 --- a/API.txt +++ b/API.txt @@ -425,6 +425,29 @@ args: 1,0,2 arg: Any('methods*') output: Output('count', , None) output: Output('results', (, ), None) +command: cert_find +args: 0,17,4 +option: Flag('all', autofill=True, cli_name='all', default=False, exclude='webui') +option: Flag('exactly?', autofill=True, default=False) +option: Str('issuedon_from?', autofill=False) +option: Str('issuedon_to?', autofill=False) +option: Int('max_serial_number?', autofill=False, maxvalue=2147483647) +option: Int('min_serial_number?', autofill=False, minvalue=0) +option: Flag('raw', autofill=True, cli_name='raw', default=False, exclude='webui') +option: Int('revocation_reason?', autofill=False, maxvalue=10, minvalue=0) +option: Str('revokedon_from?', autofill=False) +option: Str('revokedon_to?', autofill=False) +option: Int('sizelimit?', default=100, minvalue=0) +option: Str('subject?', autofill=False) +option: Str('validnotafter_from?', autofill=False) +option: Str('validnotafter_to?', autofill=False) +option: Str('validnotbefore_from?', autofill=False) +option: Str('validnotbefore_to?', autofill=Fa
Re: [Freeipa-devel] [PATCH] 1085 cert-find command
On 02/06/2013 12:44 AM, Rob Crittenden wrote: This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob Thanks! The code works well, but I found a few issues. These tests don't work when the full test suite is run: test_cert adds and revokes additional certs that throw the code off. Perhaps have the tests only query valid certs? I don't see that option but I think it would be helpful to support. The API.txt check fails: Option sizelimit? of command cert_find in ipalib, not in API file: Int('sizelimit?', default=100, minvalue=0) What are --all and --raw for? Is the plan to implement --all if/when Dogtag supports requesting additional data? The format of --validnotbefore-to and friends should be mentioned in --help text; the following is confusing: $ ipa cert-show 1 [...] Not Before: Wed Feb 06 09:32:17 2013 UTC [...] $ ipa cert-find -h [...] --validnotbefore-to=STR Valid not before to this date [...] $ ipa cert-find --validnotbefore-to='Wed Feb 06 09:32:17 2013 UTC' ipa: ERROR: invalid 'validnotbefore_to': time data u'Wed Feb 06 09:32:17 2013 UTC' does not match format '%Y-%m-%d' Could you make the help text for --exactly more specific? Please remove the extra whitespace at the end of dogtag.py I'd welcome a link to the design page in the commit message. -- Petr³ ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
[Freeipa-devel] [PATCH] 1085 cert-find command
This adds a cert-find command for the dogtag backend. Searches can be done by serial number, by subject, revocation reason, issue date, notbefore, notafter and revocation dates. I added some basic tests for this. I made it a separate test file because the cert plugin tests do not use the declarative format and rely on the selfsign backend by default. rob >From f02d024abd77650b8ecf45de5215350b1602664f Mon Sep 17 00:00:00 2001 From: Rob Crittenden Date: Thu, 15 Nov 2012 10:55:33 -0500 Subject: [PATCH] Implement the cert-find command for the dogtag CA backend. Use a new RESTful API provided by dogtag 10+. Construct an XML document representing the search request. The output is limited to whatever dogtag sends us, there is no way to request additional attributes other than to read each certificate individually. dogtag uses a boolean for each search term to indicate that it is used. Presense of the search item is not enough, both need to be set. The search operation is unauthenticated https://fedorahosted.org/freeipa/ticket/2528 --- API.txt | 22 ++ VERSION | 2 +- ipalib/plugins/cert.py | 137 +- ipaserver/plugins/dogtag.py | 138 ++ ipaserver/plugins/rabase.py | 8 + tests/test_xmlrpc/test_cert_find.py | 503 6 files changed, 807 insertions(+), 3 deletions(-) create mode 100644 tests/test_xmlrpc/test_cert_find.py diff --git a/API.txt b/API.txt index 8fbfe6f5d8da44e991b8d1a36725fc6ace1f0616..a9ee9edd841ab0862f0554b752c33d738fb82c67 100644 --- a/API.txt +++ b/API.txt @@ -425,6 +425,28 @@ args: 1,0,2 arg: Any('methods*') output: Output('count', , None) output: Output('results', (, ), None) +command: cert_find +args: 0,16,4 +option: Flag('all', autofill=True, cli_name='all', default=False, exclude='webui') +option: Flag('exactly?', autofill=True, default=False) +option: Str('issuedon_from?', autofill=False) +option: Str('issuedon_to?', autofill=False) +option: Int('max_serial_number?', autofill=False, maxvalue=2147483647) +option: Int('min_serial_number?', autofill=False, minvalue=0) +option: Flag('raw', autofill=True, cli_name='raw', default=False, exclude='webui') +option: Int('revocation_reason?', autofill=False, maxvalue=10, minvalue=0) +option: Str('revokedon_from?', autofill=False) +option: Str('revokedon_to?', autofill=False) +option: Str('subject?', autofill=False) +option: Str('validnotafter_from?', autofill=False) +option: Str('validnotafter_to?', autofill=False) +option: Str('validnotbefore_from?', autofill=False) +option: Str('validnotbefore_to?', autofill=False) +option: Str('version?', exclude='webui') +output: Output('count', , None) +output: ListOfEntries('result', (, ), Gettext('A list of LDAP entries', domain='ipa', localedir=None)) +output: Output('summary', (, ), None) +output: Output('truncated', , None) command: cert_remove_hold args: 1,0,1 arg: Str('serial_number') diff --git a/VERSION b/VERSION index 61f578dbfc9415f6f94a6612f198218c5a5e0c9a..37af5ef73b74500e0cd7397fb2c109332c049bc6 100644 --- a/VERSION +++ b/VERSION @@ -89,4 +89,4 @@ IPA_DATA_VERSION=2010061412 # # IPA_API_VERSION_MAJOR=2 -IPA_API_VERSION_MINOR=47 +IPA_API_VERSION_MINOR=48 diff --git a/ipalib/plugins/cert.py b/ipalib/plugins/cert.py index 3aa01621dbb519a2f0f671a8df2489c03faa6f34..4d7abd9aa3feabe4b677b93a3933260b218fecb2 100644 --- a/ipalib/plugins/cert.py +++ b/ipalib/plugins/cert.py @@ -24,18 +24,20 @@ if api.env.enable_ra is not True: # In this case, abort loading this plugin module... raise SkipPluginModule(reason='env.enable_ra is not True') import os +import time from ipalib import Command, Str, Int, Bytes, Flag, File from ipalib import errors from ipalib import pkcs10 from ipalib import x509 from ipalib import util +from ipalib import ngettext from ipalib.plugins.virtual import * from ipalib.plugins.service import split_principal import base64 import traceback from ipalib.text import _ from ipalib.request import context -from ipalib.output import Output +from ipalib import output from ipalib.plugins.service import validate_principal import nss.nss as nss from nss.error import NSPRError @@ -60,6 +62,18 @@ In order to request a certificate: * The host must exist * The service must exist (or you use the --add option to automatically add it) +SEARCHING: + +Certificates may be searched on by certificate subject, serial number, +revocation reason, validity dates and the issued date. + +When searching on dates the _from date does a >= search and the _to date +does a <= search. When combined these are done as an AND. + +Dates are treated as GMT to match the dates in the certificates. + +The date format is -mm-dd. + EXAMPLES: Request a new certificate and add the principal: @@ -77,6 +91,15 @@ EXAMPLES: Check