[Freeipa-devel] [freeipa PR#359][comment] dogtag: search past the first 100 certificates
URL: https://github.com/freeipa/freeipa/pull/359 Title: #359: dogtag: search past the first 100 certificates HonzaCholasta commented: """ Fixed upstream master: https://fedorahosted.org/freeipa/changeset/d84edc43e55c2f7c30614a4a5268aeb58e33a087 https://fedorahosted.org/freeipa/changeset/85834abad655c6aed54c0253bc194ece81d78774 """ See the full comment at https://github.com/freeipa/freeipa/pull/359#issuecomment-274794236 -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
[Freeipa-devel] [freeipa PR#359][comment] dogtag: search past the first 100 certificates
URL: https://github.com/freeipa/freeipa/pull/359 Title: #359: dogtag: search past the first 100 certificates tomaskrizek commented: """ We examined the WebUI side and it behaves as expected - the size limit is respected when viewing certificates. """ See the full comment at https://github.com/freeipa/freeipa/pull/359#issuecomment-274779025 -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
[Freeipa-devel] [freeipa PR#359][comment] dogtag: search past the first 100 certificates
URL: https://github.com/freeipa/freeipa/pull/359 Title: #359: dogtag: search past the first 100 certificates tomaskrizek commented: """ The behavior of the command seems to be correct now, but I'm also not sure about the WebUI. There seems to be a limit of 20 items when displayed in WebUI (with pagination). I'm not sure if it's possible to configure that. @pvomacka Were there any recent changes in the WebUI pagination? Is it possible to configure how many items should be displayed? """ See the full comment at https://github.com/freeipa/freeipa/pull/359#issuecomment-274504322 -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
[Freeipa-devel] [freeipa PR#359][comment] dogtag: search past the first 100 certificates
URL: https://github.com/freeipa/freeipa/pull/359 Title: #359: dogtag: search past the first 100 certificates HonzaCholasta commented: """ I have identified some issues in search limit handling in `cert-find` and fixed them in an additional commit. See commit message for details. """ See the full comment at https://github.com/freeipa/freeipa/pull/359#issuecomment-273166075 -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
[Freeipa-devel] [freeipa PR#359][comment] dogtag: search past the first 100 certificates
URL: https://github.com/freeipa/freeipa/pull/359 Title: #359: dogtag: search past the first 100 certificates frasertweedale commented: """ @stlaz as I see it, the `_ldap_search` can potentially search all objects of a particular type (user/service/host), which have `(userCertificate=*)`. The result is then used to filter or add to the result, depending on whether the result is "key complete" or not (indicated by the variable `complete`). Anyhow I leave to Honza to comment further; he probably understands the code better than me :) """ See the full comment at https://github.com/freeipa/freeipa/pull/359#issuecomment-270534943 -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
[Freeipa-devel] [freeipa PR#359][comment] dogtag: search past the first 100 certificates
URL: https://github.com/freeipa/freeipa/pull/359 Title: #359: dogtag: search past the first 100 certificates stlaz commented: """ @frasertweedale if `_ldap_search` is performed with correct filters, `sizelimit=0` is not the correct solution at least for CLI which should either follow the `sizelimit` argument if set or the records size limit in ipa config. It is only correct for WebUI which I believe should be setting `sizelimit=0` and if it's not, I'd be looking for the bug there. I tried to briefly go through the cert plugin code but it's a bit messy so my only hope is that the correct filter is indeed used there. On the way through it, though, I found something that seems like another size limit bug: https://github.com/freeipa/freeipa/blob/master/ipaserver/plugins/cert.py#L1306 -> which will not set our "unlimited" if `sizelimit` is set to 0. Also from there, if `sizelimit` is not set, we should go with ipa config sizelimit rather than having the magic do its trick somewhere else, right? Then the proposed value in options.get() could go away (be set in the cert.py module instead). """ See the full comment at https://github.com/freeipa/freeipa/pull/359#issuecomment-270328738 -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
[Freeipa-devel] [freeipa PR#359][comment] dogtag: search past the first 100 certificates
URL: https://github.com/freeipa/freeipa/pull/359 Title: #359: dogtag: search past the first 100 certificates frasertweedale commented: """ @tomaskrizek @HonzaCholasta it looks like the problem is: 1. subsearches are conducted in order: 1. `_cert_search` (if `'certificate' in options` add key to result and "seal" it) 2. `_ca_search` (actually perform the search against Dogtag, via `ra.find`) 3. `_ldap_search` (look for local entries that have given cert in their `userCertificate` attr. 2. if no explicit `sizelimit` is requested, and if there are > 100 entries with `(userCertificate=*)`, `_ldap_search` will be truncated, and this result is carried across to the final result. The cert search from Dogtag is not truncated, but the search for entries to use to filter the result may have been truncated. The simplest way to resolve this is (I think) to forcibly execute `_ldap_search` with `sizelimit=0`. IMO `_ldap_search` should also be avoided or short-circuited if none of the owner-flitering options to `cert-find` are given. (edit to note: this will not find certs that are in IPA LDAP but not in Dogtag, which is guess is the wrong behaviour..? So I think we just have to have sizelimit=0. I am concerned about performance impact of cert-find with many principals with certs set... but that is a separate issue). """ See the full comment at https://github.com/freeipa/freeipa/pull/359#issuecomment-270283124 -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
[Freeipa-devel] [freeipa PR#359][comment] dogtag: search past the first 100 certificates
URL: https://github.com/freeipa/freeipa/pull/359 Title: #359: dogtag: search past the first 100 certificates frasertweedale commented: """ @tomaskrizek @HonzaCholasta it looks like the problem is: 1. subsearches are conducted in order: 1. `_cert_search` (if `'certificate' in options` add key to result and "seal" it) 2. `_ca_search` (actually perform the search against Dogtag, via `ra.find`) 3. `_ldap_search` (look for local entries that have given cert in their `userCertificate` attr. 2. Due to raising of search limit internally within `ra.find`, `_ca_search` will return `sub_complete = True` always. 3. ~line 1477: ```python if sub_complete: sizelimit = None ... ``` This causes the next sub-search (`_ldap_search`) to be carried out with the *default* size limit (100). 4. If there are > 100 entries with the `(userCertificate=*)`, this search will be truncated, and this result is carried across to the final result. The cert search from Dogtag is not truncated, but the search for entries to use to filter the result may have been truncated. The simplest way to resolve this is (I think) to forcibly execute `_ldap_search` with `sizelimit=0`. IMO `_ldap_search` should also be avoided or short-circuited if none of the owner-flitering options to `cert-find` are given. (edit to note: this will not find certs that are in IPA LDAP but not in Dogtag, which is guess is the wrong behaviour..? So I think we just have to have sizelimit=0. I am concerned about performance impact of cert-find with many principals with certs set... but that is a separate issue). """ See the full comment at https://github.com/freeipa/freeipa/pull/359#issuecomment-270283124 -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
[Freeipa-devel] [freeipa PR#359][comment] dogtag: search past the first 100 certificates
URL: https://github.com/freeipa/freeipa/pull/359 Title: #359: dogtag: search past the first 100 certificates frasertweedale commented: """ @tomaskrizek @HonzaCholasta it looks like the problem is: 1. subsearches are conducted in order: 1. `_cert_search` (if `'certificate' in options` add key to result and "seal" it) 2. `_ca_search` (actually perform the search against Dogtag, via `ra.find`) 3. `_ldap_search` (look for local entries that have given cert in their `userCertificate` attr. 2. Due to raising of search limit internally within `ra.find`, `_ca_search` will return `sub_complete = True` always. 3. ~line 1477: ```python if sub_complete: sizelimit = None ... ``` This causes the next sub-search (`_ldap_search`) to be carried out with the *default* size limit (100). 4. If there are > 100 entries with the `(userCertificate=*)`, this search will be truncated, and this result is carried across to the final result. The cert search from Dogtag is not truncated, but the search for entries to use to filter the result may have been truncated. The simplest way to resolve this is (I think) to forcibly execute `_ldap_search` with `sizelimit=0`. IMO `_ldap_search` should also be avoided or short-circuited if none of the owner-flitering options to `cert-find` are given. """ See the full comment at https://github.com/freeipa/freeipa/pull/359#issuecomment-270283124 -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
[Freeipa-devel] [freeipa PR#359][comment] dogtag: search past the first 100 certificates
URL: https://github.com/freeipa/freeipa/pull/359 Title: #359: dogtag: search past the first 100 certificates frasertweedale commented: """ @tomaskrizek @HonzaCholasta it looks like the problem is: 1. subsearches are conducted in order: 1. `_cert_search` (if `'certificate' in options` add key to result and "seal" it) 2. `_ca_search` (actually perform the search against Dogtag, via `ra.find`) 3. `_ldap_search` (look for local entries that have given cert in their `userCertificate` attr. 2. Due to raising of search limit internally within `ra.find`, for this sub-search, `sub_complete = True` always. 3. ~line 1477: ```python if sub_complete: sizelimit = None ... ``` This causes the next sub-search (`_ldap_search`) to be carried out with the *default* size limit (100). 4. If there are > 100 entries with the `(userCertificate=*)`, this search will be truncated, and this result is carried across to the final result. The cert search from Dogtag is not truncated, but the search for entries to use to filter the result may have been truncated. The simplest way to resolve this is (I think) to forcibly execute `_ldap_search` with `sizelimit=0`. IMO `_ldap_search` should also be avoided or short-circuited if none of the owner-flitering options to `cert-find` are given. """ See the full comment at https://github.com/freeipa/freeipa/pull/359#issuecomment-270283124 -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
[Freeipa-devel] [freeipa PR#359][comment] dogtag: search past the first 100 certificates
URL: https://github.com/freeipa/freeipa/pull/359 Title: #359: dogtag: search past the first 100 certificates frasertweedale commented: """ @tomaskrizek yes, I can reproduce with your steps. """ See the full comment at https://github.com/freeipa/freeipa/pull/359#issuecomment-270274050 -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
[Freeipa-devel] [freeipa PR#359][comment] dogtag: search past the first 100 certificates
URL: https://github.com/freeipa/freeipa/pull/359
Title: #359: dogtag: search past the first 100 certificates
tomaskrizek commented:
"""
@frasertweedale
I ran into this issue when I created 100 users with different user certificates:
```
for i in {300..400}; do
ipa user-add "test$i" --first T --last T;
openssl req -new -newkey rsa:1024 -days 365 -nodes -keyout "private$i.key" -out
"cert$i.csr" -subj "/CN=test$i";
ipa cert-request "cert$i.csr" --principal "test$i" ;
done
```
Please let me know if you are able to reproduce the issue in this way. It might
be possible some unrelated issues may be the cause here.
"""
See the full comment at
https://github.com/freeipa/freeipa/pull/359#issuecomment-270068629
--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
[Freeipa-devel] [freeipa PR#359][comment] dogtag: search past the first 100 certificates
URL: https://github.com/freeipa/freeipa/pull/359 Title: #359: dogtag: search past the first 100 certificates frasertweedale commented: """ This change is working for me, including having the expected behaviour for WebUI. @tomaskrizek please provide steps to reproduce your WebUI behaviour. """ See the full comment at https://github.com/freeipa/freeipa/pull/359#issuecomment-268710308 -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
[Freeipa-devel] [freeipa PR#359][comment] dogtag: search past the first 100 certificates
URL: https://github.com/freeipa/freeipa/pull/359 Title: #359: dogtag: search past the first 100 certificates tomaskrizek commented: """ With this fix, more than 100 certificates are displayed and click-able from WebUI overview. However, I'm still getting an error message pop up saying ``` Search result has been truncated: Configured size limit exceeded ``` And there is also this message at the bottom of the page: ``` Query returned more results than the configured size limit. Displaying the first 110 results. ``` """ See the full comment at https://github.com/freeipa/freeipa/pull/359#issuecomment-268535538 -- Manage your subscription for the Freeipa-devel mailing list: https://www.redhat.com/mailman/listinfo/freeipa-devel Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code
