Re: [Freeipa-devel] Client-side command in the IPA framework
On Mon, 2014-03-03 at 20:12 -0500, Dmitri Pal wrote: On 03/01/2014 10:07 PM, Adam Young wrote: On 02/28/2014 10:21 AM, Petr Viktorin wrote: On 02/28/2014 04:15 PM, Alexander Bokovoy wrote: On Fri, 28 Feb 2014, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 16:43 +0200, Alexander Bokovoy wrote: On Fri, 28 Feb 2014, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 10:47 +0100, Petr Vobornik wrote: On 28.2.2014 04:02, Rob Crittenden wrote: Alexander Bokovoy wrote: On Thu, 27 Feb 2014, Nathaniel McCallum wrote: So the recent discussion on importing tokens led me to write a script to parse RFC 6030 xml files into IPA token data. This all works well. But now I need to integrate it into the IPA framework. This command will parse one or more xml files, creating a set of tokens to be added. Given that we already have otptoken-add on the server-side, it seems to me that all work needs to be done on the client-side. How do I create a new client-side command that calls existing server-side API? subclass from frontend.Local, override run() or forward() method and perform batch operation of otptoken_add from there. See cli.help, for example. If you do an override, do forward() for cli-specific work. But you should do as little as possible for reasons you already stated: the UI. Anything you do in forward Petr will need to implement in the UI. Unfortunately we don't yet have a nice way to handle files. We have tickets open at https://fedorahosted.org/freeipa/ticket/1225 and https://fedorahosted.org/freeipa/ticket/2933 If this file is something that would be pasted into a big text field then you can probably handle it in a similarly clumsy way that we do CSRs in the cert plugin. rob +1 for parsing it on server. Otherwise every client, not just CLI or Web UI, would have to reimplement the same logic - having it on server will support better integration with third party products. Parsing on client would be understandable if there was some middle step which would require some action from user, i.e, pick only some tokens to import. If we parse on the server side, how do we handle the long-running operation? Think of the case of importing hundreds or thousands of tokens... Why then to do it as a IPA CLI command at all? This is an administrative task which can be done with a separate ipa-otp-import command, designated to run on IPA masters. Agreed. 1. Is there a framework for this? Or should it just be an independent script? We don't really have a framework for administrative tools. You may start with install/tools/ipa-adtrust-install, it is main part is relatively independent of the task (which is in ipaserver/install/adtrustinstance.py) The framework is there, new tools use it, and there's a ticket to convert old ones: https://fedorahosted.org/freeipa/ticket/2652 (it's low priority in Future Releases, so not much progress is there...) Also see http://www.freeipa.org/page/V3/Logging_and_output The RESTful approach would be: 1. Upload a file to a specific URL (not JSON RPC) 2. Receive back a 202 Accepted HTTP Request, to include an URL to poll for status Not certain the right response from the URL in step 2 would be, but I am assuming it would be 200 with the body of the message stating: processing or completed. It would be really nice if the Batch command could be handled this way as well. The response back could be the partial responses until processing is complete. It might also be nice to supply an email address for notification of completed processing instead of polling, if it is going to be a really long running task. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel Yes I think that: 1) We should not limit it to server side operation only 2) Upload the whole file and then process it. 3) We should already have code to upload files, we did it for entitlements and were supposed to use for certs. 4) Make sure we have a generic upload mechanism that reads a chunk of a configurable size and asks for more (pagination by 65K might be a good default). Regarding token files specifically: they can be big but not super huge. 10-20K tokens makes sense but probably not more. More than that would be a real corner case becuase it is hard to deploy that amount of tokens at the same time. It can take months and you do not want token file to contain many tokens that would sit on the shelf. Tokens expire so it is inefficient to buy huge chunks and let them sit unused. UI you allow uploading file too and then would process it locally. The processing of the file should generate a log or report. It
Re: [Freeipa-devel] Client-side command in the IPA framework
On 03/04/2014 02:03 PM, Nathaniel McCallum wrote: On Mon, 2014-03-03 at 20:12 -0500, Dmitri Pal wrote: On 03/01/2014 10:07 PM, Adam Young wrote: On 02/28/2014 10:21 AM, Petr Viktorin wrote: On 02/28/2014 04:15 PM, Alexander Bokovoy wrote: On Fri, 28 Feb 2014, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 16:43 +0200, Alexander Bokovoy wrote: On Fri, 28 Feb 2014, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 10:47 +0100, Petr Vobornik wrote: On 28.2.2014 04:02, Rob Crittenden wrote: Alexander Bokovoy wrote: On Thu, 27 Feb 2014, Nathaniel McCallum wrote: So the recent discussion on importing tokens led me to write a script to parse RFC 6030 xml files into IPA token data. This all works well. But now I need to integrate it into the IPA framework. This command will parse one or more xml files, creating a set of tokens to be added. Given that we already have otptoken-add on the server-side, it seems to me that all work needs to be done on the client-side. How do I create a new client-side command that calls existing server-side API? subclass from frontend.Local, override run() or forward() method and perform batch operation of otptoken_add from there. See cli.help, for example. If you do an override, do forward() for cli-specific work. But you should do as little as possible for reasons you already stated: the UI. Anything you do in forward Petr will need to implement in the UI. Unfortunately we don't yet have a nice way to handle files. We have tickets open at https://fedorahosted.org/freeipa/ticket/1225 and https://fedorahosted.org/freeipa/ticket/2933 If this file is something that would be pasted into a big text field then you can probably handle it in a similarly clumsy way that we do CSRs in the cert plugin. rob +1 for parsing it on server. Otherwise every client, not just CLI or Web UI, would have to reimplement the same logic - having it on server will support better integration with third party products. Parsing on client would be understandable if there was some middle step which would require some action from user, i.e, pick only some tokens to import. If we parse on the server side, how do we handle the long-running operation? Think of the case of importing hundreds or thousands of tokens... Why then to do it as a IPA CLI command at all? This is an administrative task which can be done with a separate ipa-otp-import command, designated to run on IPA masters. Agreed. 1. Is there a framework for this? Or should it just be an independent script? We don't really have a framework for administrative tools. You may start with install/tools/ipa-adtrust-install, it is main part is relatively independent of the task (which is in ipaserver/install/adtrustinstance.py) The framework is there, new tools use it, and there's a ticket to convert old ones: https://fedorahosted.org/freeipa/ticket/2652 (it's low priority in Future Releases, so not much progress is there...) Also see http://www.freeipa.org/page/V3/Logging_and_output The RESTful approach would be: 1. Upload a file to a specific URL (not JSON RPC) 2. Receive back a 202 Accepted HTTP Request, to include an URL to poll for status Not certain the right response from the URL in step 2 would be, but I am assuming it would be 200 with the body of the message stating: processing or completed. It would be really nice if the Batch command could be handled this way as well. The response back could be the partial responses until processing is complete. It might also be nice to supply an email address for notification of completed processing instead of polling, if it is going to be a really long running task. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel Yes I think that: 1) We should not limit it to server side operation only 2) Upload the whole file and then process it. 3) We should already have code to upload files, we did it for entitlements and were supposed to use for certs. 4) Make sure we have a generic upload mechanism that reads a chunk of a configurable size and asks for more (pagination by 65K might be a good default). Regarding token files specifically: they can be big but not super huge. 10-20K tokens makes sense but probably not more. More than that would be a real corner case becuase it is hard to deploy that amount of tokens at the same time. It can take months and you do not want token file to contain many tokens that would sit on the shelf. Tokens expire so it is inefficient to buy huge chunks and let them sit unused. UI you allow uploading file too and then would process it locally. The processing of the file should generate a log or report. It would be nice to get indication from the server that it is still working so may be upload protocol should be something like: client: Initialize the transfer server: ready client: here is the chunk of data server: ack
Re: [Freeipa-devel] Client-side command in the IPA framework
On Tue, 2014-03-04 at 14:11 -0500, Dmitri Pal wrote: On 03/04/2014 02:03 PM, Nathaniel McCallum wrote: On Mon, 2014-03-03 at 20:12 -0500, Dmitri Pal wrote: On 03/01/2014 10:07 PM, Adam Young wrote: On 02/28/2014 10:21 AM, Petr Viktorin wrote: On 02/28/2014 04:15 PM, Alexander Bokovoy wrote: On Fri, 28 Feb 2014, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 16:43 +0200, Alexander Bokovoy wrote: On Fri, 28 Feb 2014, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 10:47 +0100, Petr Vobornik wrote: On 28.2.2014 04:02, Rob Crittenden wrote: Alexander Bokovoy wrote: On Thu, 27 Feb 2014, Nathaniel McCallum wrote: So the recent discussion on importing tokens led me to write a script to parse RFC 6030 xml files into IPA token data. This all works well. But now I need to integrate it into the IPA framework. This command will parse one or more xml files, creating a set of tokens to be added. Given that we already have otptoken-add on the server-side, it seems to me that all work needs to be done on the client-side. How do I create a new client-side command that calls existing server-side API? subclass from frontend.Local, override run() or forward() method and perform batch operation of otptoken_add from there. See cli.help, for example. If you do an override, do forward() for cli-specific work. But you should do as little as possible for reasons you already stated: the UI. Anything you do in forward Petr will need to implement in the UI. Unfortunately we don't yet have a nice way to handle files. We have tickets open at https://fedorahosted.org/freeipa/ticket/1225 and https://fedorahosted.org/freeipa/ticket/2933 If this file is something that would be pasted into a big text field then you can probably handle it in a similarly clumsy way that we do CSRs in the cert plugin. rob +1 for parsing it on server. Otherwise every client, not just CLI or Web UI, would have to reimplement the same logic - having it on server will support better integration with third party products. Parsing on client would be understandable if there was some middle step which would require some action from user, i.e, pick only some tokens to import. If we parse on the server side, how do we handle the long-running operation? Think of the case of importing hundreds or thousands of tokens... Why then to do it as a IPA CLI command at all? This is an administrative task which can be done with a separate ipa-otp-import command, designated to run on IPA masters. Agreed. 1. Is there a framework for this? Or should it just be an independent script? We don't really have a framework for administrative tools. You may start with install/tools/ipa-adtrust-install, it is main part is relatively independent of the task (which is in ipaserver/install/adtrustinstance.py) The framework is there, new tools use it, and there's a ticket to convert old ones: https://fedorahosted.org/freeipa/ticket/2652 (it's low priority in Future Releases, so not much progress is there...) Also see http://www.freeipa.org/page/V3/Logging_and_output The RESTful approach would be: 1. Upload a file to a specific URL (not JSON RPC) 2. Receive back a 202 Accepted HTTP Request, to include an URL to poll for status Not certain the right response from the URL in step 2 would be, but I am assuming it would be 200 with the body of the message stating: processing or completed. It would be really nice if the Batch command could be handled this way as well. The response back could be the partial responses until processing is complete. It might also be nice to supply an email address for notification of completed processing instead of polling, if it is going to be a really long running task. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel Yes I think that: 1) We should not limit it to server side operation only 2) Upload the whole file and then process it. 3) We should already have code to upload files, we did it for entitlements and were supposed to use for certs. 4) Make sure we have a generic upload mechanism that reads a chunk of a configurable size and asks for more (pagination by 65K might be a good default). Regarding token files specifically: they can be big but not super huge. 10-20K tokens makes sense but probably not more. More than that would be a real corner case becuase it is hard to deploy that amount of tokens at the same time. It can take months and you do not want token file to contain many tokens that would sit on the shelf. Tokens expire so it is inefficient to buy huge chunks and let them sit unused. UI you allow uploading file too and then would process it locally. The processing of the file should generate a log
Re: [Freeipa-devel] Client-side command in the IPA framework
On 03/04/2014 02:27 PM, Nathaniel McCallum wrote: On Tue, 2014-03-04 at 14:11 -0500, Dmitri Pal wrote: On 03/04/2014 02:03 PM, Nathaniel McCallum wrote: On Mon, 2014-03-03 at 20:12 -0500, Dmitri Pal wrote: On 03/01/2014 10:07 PM, Adam Young wrote: On 02/28/2014 10:21 AM, Petr Viktorin wrote: On 02/28/2014 04:15 PM, Alexander Bokovoy wrote: On Fri, 28 Feb 2014, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 16:43 +0200, Alexander Bokovoy wrote: On Fri, 28 Feb 2014, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 10:47 +0100, Petr Vobornik wrote: On 28.2.2014 04:02, Rob Crittenden wrote: Alexander Bokovoy wrote: On Thu, 27 Feb 2014, Nathaniel McCallum wrote: So the recent discussion on importing tokens led me to write a script to parse RFC 6030 xml files into IPA token data. This all works well. But now I need to integrate it into the IPA framework. This command will parse one or more xml files, creating a set of tokens to be added. Given that we already have otptoken-add on the server-side, it seems to me that all work needs to be done on the client-side. How do I create a new client-side command that calls existing server-side API? subclass from frontend.Local, override run() or forward() method and perform batch operation of otptoken_add from there. See cli.help, for example. If you do an override, do forward() for cli-specific work. But you should do as little as possible for reasons you already stated: the UI. Anything you do in forward Petr will need to implement in the UI. Unfortunately we don't yet have a nice way to handle files. We have tickets open at https://fedorahosted.org/freeipa/ticket/1225 and https://fedorahosted.org/freeipa/ticket/2933 If this file is something that would be pasted into a big text field then you can probably handle it in a similarly clumsy way that we do CSRs in the cert plugin. rob +1 for parsing it on server. Otherwise every client, not just CLI or Web UI, would have to reimplement the same logic - having it on server will support better integration with third party products. Parsing on client would be understandable if there was some middle step which would require some action from user, i.e, pick only some tokens to import. If we parse on the server side, how do we handle the long-running operation? Think of the case of importing hundreds or thousands of tokens... Why then to do it as a IPA CLI command at all? This is an administrative task which can be done with a separate ipa-otp-import command, designated to run on IPA masters. Agreed. 1. Is there a framework for this? Or should it just be an independent script? We don't really have a framework for administrative tools. You may start with install/tools/ipa-adtrust-install, it is main part is relatively independent of the task (which is in ipaserver/install/adtrustinstance.py) The framework is there, new tools use it, and there's a ticket to convert old ones: https://fedorahosted.org/freeipa/ticket/2652 (it's low priority in Future Releases, so not much progress is there...) Also see http://www.freeipa.org/page/V3/Logging_and_output The RESTful approach would be: 1. Upload a file to a specific URL (not JSON RPC) 2. Receive back a 202 Accepted HTTP Request, to include an URL to poll for status Not certain the right response from the URL in step 2 would be, but I am assuming it would be 200 with the body of the message stating: processing or completed. It would be really nice if the Batch command could be handled this way as well. The response back could be the partial responses until processing is complete. It might also be nice to supply an email address for notification of completed processing instead of polling, if it is going to be a really long running task. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel Yes I think that: 1) We should not limit it to server side operation only 2) Upload the whole file and then process it. 3) We should already have code to upload files, we did it for entitlements and were supposed to use for certs. 4) Make sure we have a generic upload mechanism that reads a chunk of a configurable size and asks for more (pagination by 65K might be a good default). Regarding token files specifically: they can be big but not super huge. 10-20K tokens makes sense but probably not more. More than that would be a real corner case becuase it is hard to deploy that amount of tokens at the same time. It can take months and you do not want token file to contain many tokens that would sit on the shelf. Tokens expire so it is inefficient to buy huge chunks and let them sit unused. UI you allow uploading file too and then would process it locally. The processing of the file should generate a log or report. It would be nice to get indication from the server that it is still working so may be upload protocol should be
Re: [Freeipa-devel] Client-side command in the IPA framework
On 03/01/2014 10:07 PM, Adam Young wrote: On 02/28/2014 10:21 AM, Petr Viktorin wrote: On 02/28/2014 04:15 PM, Alexander Bokovoy wrote: On Fri, 28 Feb 2014, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 16:43 +0200, Alexander Bokovoy wrote: On Fri, 28 Feb 2014, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 10:47 +0100, Petr Vobornik wrote: On 28.2.2014 04:02, Rob Crittenden wrote: Alexander Bokovoy wrote: On Thu, 27 Feb 2014, Nathaniel McCallum wrote: So the recent discussion on importing tokens led me to write a script to parse RFC 6030 xml files into IPA token data. This all works well. But now I need to integrate it into the IPA framework. This command will parse one or more xml files, creating a set of tokens to be added. Given that we already have otptoken-add on the server-side, it seems to me that all work needs to be done on the client-side. How do I create a new client-side command that calls existing server-side API? subclass from frontend.Local, override run() or forward() method and perform batch operation of otptoken_add from there. See cli.help, for example. If you do an override, do forward() for cli-specific work. But you should do as little as possible for reasons you already stated: the UI. Anything you do in forward Petr will need to implement in the UI. Unfortunately we don't yet have a nice way to handle files. We have tickets open at https://fedorahosted.org/freeipa/ticket/1225 and https://fedorahosted.org/freeipa/ticket/2933 If this file is something that would be pasted into a big text field then you can probably handle it in a similarly clumsy way that we do CSRs in the cert plugin. rob +1 for parsing it on server. Otherwise every client, not just CLI or Web UI, would have to reimplement the same logic - having it on server will support better integration with third party products. Parsing on client would be understandable if there was some middle step which would require some action from user, i.e, pick only some tokens to import. If we parse on the server side, how do we handle the long-running operation? Think of the case of importing hundreds or thousands of tokens... Why then to do it as a IPA CLI command at all? This is an administrative task which can be done with a separate ipa-otp-import command, designated to run on IPA masters. Agreed. 1. Is there a framework for this? Or should it just be an independent script? We don't really have a framework for administrative tools. You may start with install/tools/ipa-adtrust-install, it is main part is relatively independent of the task (which is in ipaserver/install/adtrustinstance.py) The framework is there, new tools use it, and there's a ticket to convert old ones: https://fedorahosted.org/freeipa/ticket/2652 (it's low priority in Future Releases, so not much progress is there...) Also see http://www.freeipa.org/page/V3/Logging_and_output The RESTful approach would be: 1. Upload a file to a specific URL (not JSON RPC) 2. Receive back a 202 Accepted HTTP Request, to include an URL to poll for status Not certain the right response from the URL in step 2 would be, but I am assuming it would be 200 with the body of the message stating: processing or completed. It would be really nice if the Batch command could be handled this way as well. The response back could be the partial responses until processing is complete. It might also be nice to supply an email address for notification of completed processing instead of polling, if it is going to be a really long running task. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel Yes I think that: 1) We should not limit it to server side operation only 2) Upload the whole file and then process it. 3) We should already have code to upload files, we did it for entitlements and were supposed to use for certs. 4) Make sure we have a generic upload mechanism that reads a chunk of a configurable size and asks for more (pagination by 65K might be a good default). Regarding token files specifically: they can be big but not super huge. 10-20K tokens makes sense but probably not more. More than that would be a real corner case becuase it is hard to deploy that amount of tokens at the same time. It can take months and you do not want token file to contain many tokens that would sit on the shelf. Tokens expire so it is inefficient to buy huge chunks and let them sit unused. UI you allow uploading file too and then would process it locally. The processing of the file should generate a log or report. It would be nice to get indication from the server that it is still working so may be upload protocol should be something like: client: Initialize the transfer server: ready client: here is the chunk of data server: ack ... client: here is the last chunk of data
Re: [Freeipa-devel] Client-side command in the IPA framework
On 02/28/2014 10:21 AM, Petr Viktorin wrote: On 02/28/2014 04:15 PM, Alexander Bokovoy wrote: On Fri, 28 Feb 2014, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 16:43 +0200, Alexander Bokovoy wrote: On Fri, 28 Feb 2014, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 10:47 +0100, Petr Vobornik wrote: On 28.2.2014 04:02, Rob Crittenden wrote: Alexander Bokovoy wrote: On Thu, 27 Feb 2014, Nathaniel McCallum wrote: So the recent discussion on importing tokens led me to write a script to parse RFC 6030 xml files into IPA token data. This all works well. But now I need to integrate it into the IPA framework. This command will parse one or more xml files, creating a set of tokens to be added. Given that we already have otptoken-add on the server-side, it seems to me that all work needs to be done on the client-side. How do I create a new client-side command that calls existing server-side API? subclass from frontend.Local, override run() or forward() method and perform batch operation of otptoken_add from there. See cli.help, for example. If you do an override, do forward() for cli-specific work. But you should do as little as possible for reasons you already stated: the UI. Anything you do in forward Petr will need to implement in the UI. Unfortunately we don't yet have a nice way to handle files. We have tickets open at https://fedorahosted.org/freeipa/ticket/1225 and https://fedorahosted.org/freeipa/ticket/2933 If this file is something that would be pasted into a big text field then you can probably handle it in a similarly clumsy way that we do CSRs in the cert plugin. rob +1 for parsing it on server. Otherwise every client, not just CLI or Web UI, would have to reimplement the same logic - having it on server will support better integration with third party products. Parsing on client would be understandable if there was some middle step which would require some action from user, i.e, pick only some tokens to import. If we parse on the server side, how do we handle the long-running operation? Think of the case of importing hundreds or thousands of tokens... Why then to do it as a IPA CLI command at all? This is an administrative task which can be done with a separate ipa-otp-import command, designated to run on IPA masters. Agreed. 1. Is there a framework for this? Or should it just be an independent script? We don't really have a framework for administrative tools. You may start with install/tools/ipa-adtrust-install, it is main part is relatively independent of the task (which is in ipaserver/install/adtrustinstance.py) The framework is there, new tools use it, and there's a ticket to convert old ones: https://fedorahosted.org/freeipa/ticket/2652 (it's low priority in Future Releases, so not much progress is there...) Also see http://www.freeipa.org/page/V3/Logging_and_output The RESTful approach would be: 1. Upload a file to a specific URL (not JSON RPC) 2. Receive back a 202 Accepted HTTP Request, to include an URL to poll for status Not certain the right response from the URL in step 2 would be, but I am assuming it would be 200 with the body of the message stating: processing or completed. It would be really nice if the Batch command could be handled this way as well. The response back could be the partial responses until processing is complete. It might also be nice to supply an email address for notification of completed processing instead of polling, if it is going to be a really long running task. ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Client-side command in the IPA framework
On 28.2.2014 04:02, Rob Crittenden wrote: Alexander Bokovoy wrote: On Thu, 27 Feb 2014, Nathaniel McCallum wrote: So the recent discussion on importing tokens led me to write a script to parse RFC 6030 xml files into IPA token data. This all works well. But now I need to integrate it into the IPA framework. This command will parse one or more xml files, creating a set of tokens to be added. Given that we already have otptoken-add on the server-side, it seems to me that all work needs to be done on the client-side. How do I create a new client-side command that calls existing server-side API? subclass from frontend.Local, override run() or forward() method and perform batch operation of otptoken_add from there. See cli.help, for example. If you do an override, do forward() for cli-specific work. But you should do as little as possible for reasons you already stated: the UI. Anything you do in forward Petr will need to implement in the UI. Unfortunately we don't yet have a nice way to handle files. We have tickets open at https://fedorahosted.org/freeipa/ticket/1225 and https://fedorahosted.org/freeipa/ticket/2933 If this file is something that would be pasted into a big text field then you can probably handle it in a similarly clumsy way that we do CSRs in the cert plugin. rob +1 for parsing it on server. Otherwise every client, not just CLI or Web UI, would have to reimplement the same logic - having it on server will support better integration with third party products. Parsing on client would be understandable if there was some middle step which would require some action from user, i.e, pick only some tokens to import. -- Petr Vobornik ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Client-side command in the IPA framework
On Fri, 2014-02-28 at 10:47 +0100, Petr Vobornik wrote: On 28.2.2014 04:02, Rob Crittenden wrote: Alexander Bokovoy wrote: On Thu, 27 Feb 2014, Nathaniel McCallum wrote: So the recent discussion on importing tokens led me to write a script to parse RFC 6030 xml files into IPA token data. This all works well. But now I need to integrate it into the IPA framework. This command will parse one or more xml files, creating a set of tokens to be added. Given that we already have otptoken-add on the server-side, it seems to me that all work needs to be done on the client-side. How do I create a new client-side command that calls existing server-side API? subclass from frontend.Local, override run() or forward() method and perform batch operation of otptoken_add from there. See cli.help, for example. If you do an override, do forward() for cli-specific work. But you should do as little as possible for reasons you already stated: the UI. Anything you do in forward Petr will need to implement in the UI. Unfortunately we don't yet have a nice way to handle files. We have tickets open at https://fedorahosted.org/freeipa/ticket/1225 and https://fedorahosted.org/freeipa/ticket/2933 If this file is something that would be pasted into a big text field then you can probably handle it in a similarly clumsy way that we do CSRs in the cert plugin. rob +1 for parsing it on server. Otherwise every client, not just CLI or Web UI, would have to reimplement the same logic - having it on server will support better integration with third party products. Parsing on client would be understandable if there was some middle step which would require some action from user, i.e, pick only some tokens to import. If we parse on the server side, how do we handle the long-running operation? Think of the case of importing hundreds or thousands of tokens... Nathaniel ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Client-side command in the IPA framework
On 28.2.2014 15:25, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 10:47 +0100, Petr Vobornik wrote: On 28.2.2014 04:02, Rob Crittenden wrote: Alexander Bokovoy wrote: On Thu, 27 Feb 2014, Nathaniel McCallum wrote: So the recent discussion on importing tokens led me to write a script to parse RFC 6030 xml files into IPA token data. This all works well. But now I need to integrate it into the IPA framework. This command will parse one or more xml files, creating a set of tokens to be added. Given that we already have otptoken-add on the server-side, it seems to me that all work needs to be done on the client-side. How do I create a new client-side command that calls existing server-side API? subclass from frontend.Local, override run() or forward() method and perform batch operation of otptoken_add from there. See cli.help, for example. If you do an override, do forward() for cli-specific work. But you should do as little as possible for reasons you already stated: the UI. Anything you do in forward Petr will need to implement in the UI. Unfortunately we don't yet have a nice way to handle files. We have tickets open at https://fedorahosted.org/freeipa/ticket/1225 and https://fedorahosted.org/freeipa/ticket/2933 If this file is something that would be pasted into a big text field then you can probably handle it in a similarly clumsy way that we do CSRs in the cert plugin. rob +1 for parsing it on server. Otherwise every client, not just CLI or Web UI, would have to reimplement the same logic - having it on server will support better integration with third party products. Parsing on client would be understandable if there was some middle step which would require some action from user, i.e, pick only some tokens to import. If we parse on the server side, how do we handle the long-running operation? Think of the case of importing hundreds or thousands of tokens... My experience is that operation on server side can run for (at least) few minutes without a problem. I haven't try longer periods but we can check that. -- Petr^2 Spacek ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Client-side command in the IPA framework
On Fri, 28 Feb 2014, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 10:47 +0100, Petr Vobornik wrote: On 28.2.2014 04:02, Rob Crittenden wrote: Alexander Bokovoy wrote: On Thu, 27 Feb 2014, Nathaniel McCallum wrote: So the recent discussion on importing tokens led me to write a script to parse RFC 6030 xml files into IPA token data. This all works well. But now I need to integrate it into the IPA framework. This command will parse one or more xml files, creating a set of tokens to be added. Given that we already have otptoken-add on the server-side, it seems to me that all work needs to be done on the client-side. How do I create a new client-side command that calls existing server-side API? subclass from frontend.Local, override run() or forward() method and perform batch operation of otptoken_add from there. See cli.help, for example. If you do an override, do forward() for cli-specific work. But you should do as little as possible for reasons you already stated: the UI. Anything you do in forward Petr will need to implement in the UI. Unfortunately we don't yet have a nice way to handle files. We have tickets open at https://fedorahosted.org/freeipa/ticket/1225 and https://fedorahosted.org/freeipa/ticket/2933 If this file is something that would be pasted into a big text field then you can probably handle it in a similarly clumsy way that we do CSRs in the cert plugin. rob +1 for parsing it on server. Otherwise every client, not just CLI or Web UI, would have to reimplement the same logic - having it on server will support better integration with third party products. Parsing on client would be understandable if there was some middle step which would require some action from user, i.e, pick only some tokens to import. If we parse on the server side, how do we handle the long-running operation? Think of the case of importing hundreds or thousands of tokens... Why then to do it as a IPA CLI command at all? This is an administrative task which can be done with a separate ipa-otp-import command, designated to run on IPA masters. -- / Alexander Bokovoy ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Client-side command in the IPA framework
Petr Spacek wrote: On 28.2.2014 15:25, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 10:47 +0100, Petr Vobornik wrote: On 28.2.2014 04:02, Rob Crittenden wrote: Alexander Bokovoy wrote: On Thu, 27 Feb 2014, Nathaniel McCallum wrote: So the recent discussion on importing tokens led me to write a script to parse RFC 6030 xml files into IPA token data. This all works well. But now I need to integrate it into the IPA framework. This command will parse one or more xml files, creating a set of tokens to be added. Given that we already have otptoken-add on the server-side, it seems to me that all work needs to be done on the client-side. How do I create a new client-side command that calls existing server-side API? subclass from frontend.Local, override run() or forward() method and perform batch operation of otptoken_add from there. See cli.help, for example. If you do an override, do forward() for cli-specific work. But you should do as little as possible for reasons you already stated: the UI. Anything you do in forward Petr will need to implement in the UI. Unfortunately we don't yet have a nice way to handle files. We have tickets open at https://fedorahosted.org/freeipa/ticket/1225 and https://fedorahosted.org/freeipa/ticket/2933 If this file is something that would be pasted into a big text field then you can probably handle it in a similarly clumsy way that we do CSRs in the cert plugin. rob +1 for parsing it on server. Otherwise every client, not just CLI or Web UI, would have to reimplement the same logic - having it on server will support better integration with third party products. Parsing on client would be understandable if there was some middle step which would require some action from user, i.e, pick only some tokens to import. If we parse on the server side, how do we handle the long-running operation? Think of the case of importing hundreds or thousands of tokens... My experience is that operation on server side can run for (at least) few minutes without a problem. I haven't try longer periods but we can check that. It can run for hours. Migration performance in IPA used to be rather pitiful and migrating several thousand users could easily take 5+ hours. IIRC sometimes the client would time out but the server side would still complete, you just got no feedback. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Client-side command in the IPA framework
On Fri, 2014-02-28 at 16:43 +0200, Alexander Bokovoy wrote: On Fri, 28 Feb 2014, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 10:47 +0100, Petr Vobornik wrote: On 28.2.2014 04:02, Rob Crittenden wrote: Alexander Bokovoy wrote: On Thu, 27 Feb 2014, Nathaniel McCallum wrote: So the recent discussion on importing tokens led me to write a script to parse RFC 6030 xml files into IPA token data. This all works well. But now I need to integrate it into the IPA framework. This command will parse one or more xml files, creating a set of tokens to be added. Given that we already have otptoken-add on the server-side, it seems to me that all work needs to be done on the client-side. How do I create a new client-side command that calls existing server-side API? subclass from frontend.Local, override run() or forward() method and perform batch operation of otptoken_add from there. See cli.help, for example. If you do an override, do forward() for cli-specific work. But you should do as little as possible for reasons you already stated: the UI. Anything you do in forward Petr will need to implement in the UI. Unfortunately we don't yet have a nice way to handle files. We have tickets open at https://fedorahosted.org/freeipa/ticket/1225 and https://fedorahosted.org/freeipa/ticket/2933 If this file is something that would be pasted into a big text field then you can probably handle it in a similarly clumsy way that we do CSRs in the cert plugin. rob +1 for parsing it on server. Otherwise every client, not just CLI or Web UI, would have to reimplement the same logic - having it on server will support better integration with third party products. Parsing on client would be understandable if there was some middle step which would require some action from user, i.e, pick only some tokens to import. If we parse on the server side, how do we handle the long-running operation? Think of the case of importing hundreds or thousands of tokens... Why then to do it as a IPA CLI command at all? This is an administrative task which can be done with a separate ipa-otp-import command, designated to run on IPA masters. Agreed. 1. Is there a framework for this? Or should it just be an independent script? 2. How can I use the ipalib API? Specifically, I'd like to call otptoken-add and pass the --key parameter with a value (not possible from the command line). Nathaniel ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Client-side command in the IPA framework
On Fri, 28 Feb 2014, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 16:43 +0200, Alexander Bokovoy wrote: On Fri, 28 Feb 2014, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 10:47 +0100, Petr Vobornik wrote: On 28.2.2014 04:02, Rob Crittenden wrote: Alexander Bokovoy wrote: On Thu, 27 Feb 2014, Nathaniel McCallum wrote: So the recent discussion on importing tokens led me to write a script to parse RFC 6030 xml files into IPA token data. This all works well. But now I need to integrate it into the IPA framework. This command will parse one or more xml files, creating a set of tokens to be added. Given that we already have otptoken-add on the server-side, it seems to me that all work needs to be done on the client-side. How do I create a new client-side command that calls existing server-side API? subclass from frontend.Local, override run() or forward() method and perform batch operation of otptoken_add from there. See cli.help, for example. If you do an override, do forward() for cli-specific work. But you should do as little as possible for reasons you already stated: the UI. Anything you do in forward Petr will need to implement in the UI. Unfortunately we don't yet have a nice way to handle files. We have tickets open at https://fedorahosted.org/freeipa/ticket/1225 and https://fedorahosted.org/freeipa/ticket/2933 If this file is something that would be pasted into a big text field then you can probably handle it in a similarly clumsy way that we do CSRs in the cert plugin. rob +1 for parsing it on server. Otherwise every client, not just CLI or Web UI, would have to reimplement the same logic - having it on server will support better integration with third party products. Parsing on client would be understandable if there was some middle step which would require some action from user, i.e, pick only some tokens to import. If we parse on the server side, how do we handle the long-running operation? Think of the case of importing hundreds or thousands of tokens... Why then to do it as a IPA CLI command at all? This is an administrative task which can be done with a separate ipa-otp-import command, designated to run on IPA masters. Agreed. 1. Is there a framework for this? Or should it just be an independent script? We don't really have a framework for administrative tools. You may start with install/tools/ipa-adtrust-install, it is main part is relatively independent of the task (which is in ipaserver/install/adtrustinstance.py) 2. How can I use the ipalib API? Specifically, I'd like to call otptoken-add and pass the --key parameter with a value (not possible from the command line). Look in ipa-adtrust-install's main(): # Initialize the ipalib api cfg = dict( in_server=True, debug=options.debug, ) api.bootstrap(**cfg) api.finalize() ... try: ctx = krbV.default_context() ccache = ctx.default_ccache() principal = ccache.principal() except krbV.Krb5Error, e: sys.exit(Must have Kerberos credentials to setup AD trusts on server) try: api.Backend.ldap2.connect(ccache) except errors.ACIError, e: sys.exit(Outdated Kerberos credentials. Use kdestroy and kinit to update your ticket) except errors.DatabaseError, e: sys.exit(Cannot connect to the LDAP database. Please check if IPA is running) try: user = api.Command.user_show(unicode(principal[0]))['result'] group = api.Command.group_show(u'admins')['result'] if not (user['uid'][0] in group['member_user'] and group['cn'][0] in user['memberof_group']): raise errors.RequirementError(name='admins group membership') except errors.RequirementError, e: sys.exit(Must have administrative privileges to setup AD trusts on server) except Exception, e: sys.exit(Unrecognized error during check of admin rights: %s % (str(e))) and so on. -- / Alexander Bokovoy ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Client-side command in the IPA framework
On Fri, 28 Feb 2014, Petr Viktorin wrote: On 02/28/2014 04:02 PM, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 16:43 +0200, Alexander Bokovoy wrote: [...] Why then to do it as a IPA CLI command at all? This is an administrative task which can be done with a separate ipa-otp-import command, designated to run on IPA masters. Agreed. 1. Is there a framework for this? Or should it just be an independent script? There is: ipapython.admintool. Look at ipa-server-certinstall for a simple-ish example, ask if you have any questions. Right, forgot about that one. 2. How can I use the ipalib API? Specifically, I'd like to call otptoken-add and pass the --key parameter with a value (not possible from the command line). Finalize the API (see ipaserver.install.ServerCertInstall.run), and then call api.Command['otptoken-add'](key=...) Or you might think about moving the otptoken-add functionality into a function that you can call directly. I'd still prefer to do token addition through the common interface, i.e. not directly talking to ldap but rather using the CLI commands, maybe batched. -- / Alexander Bokovoy ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Client-side command in the IPA framework
On 02/28/2014 04:15 PM, Alexander Bokovoy wrote: On Fri, 28 Feb 2014, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 16:43 +0200, Alexander Bokovoy wrote: On Fri, 28 Feb 2014, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 10:47 +0100, Petr Vobornik wrote: On 28.2.2014 04:02, Rob Crittenden wrote: Alexander Bokovoy wrote: On Thu, 27 Feb 2014, Nathaniel McCallum wrote: So the recent discussion on importing tokens led me to write a script to parse RFC 6030 xml files into IPA token data. This all works well. But now I need to integrate it into the IPA framework. This command will parse one or more xml files, creating a set of tokens to be added. Given that we already have otptoken-add on the server-side, it seems to me that all work needs to be done on the client-side. How do I create a new client-side command that calls existing server-side API? subclass from frontend.Local, override run() or forward() method and perform batch operation of otptoken_add from there. See cli.help, for example. If you do an override, do forward() for cli-specific work. But you should do as little as possible for reasons you already stated: the UI. Anything you do in forward Petr will need to implement in the UI. Unfortunately we don't yet have a nice way to handle files. We have tickets open at https://fedorahosted.org/freeipa/ticket/1225 and https://fedorahosted.org/freeipa/ticket/2933 If this file is something that would be pasted into a big text field then you can probably handle it in a similarly clumsy way that we do CSRs in the cert plugin. rob +1 for parsing it on server. Otherwise every client, not just CLI or Web UI, would have to reimplement the same logic - having it on server will support better integration with third party products. Parsing on client would be understandable if there was some middle step which would require some action from user, i.e, pick only some tokens to import. If we parse on the server side, how do we handle the long-running operation? Think of the case of importing hundreds or thousands of tokens... Why then to do it as a IPA CLI command at all? This is an administrative task which can be done with a separate ipa-otp-import command, designated to run on IPA masters. Agreed. 1. Is there a framework for this? Or should it just be an independent script? We don't really have a framework for administrative tools. You may start with install/tools/ipa-adtrust-install, it is main part is relatively independent of the task (which is in ipaserver/install/adtrustinstance.py) The framework is there, new tools use it, and there's a ticket to convert old ones: https://fedorahosted.org/freeipa/ticket/2652 (it's low priority in Future Releases, so not much progress is there...) Also see http://www.freeipa.org/page/V3/Logging_and_output -- Petr³ ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Client-side command in the IPA framework
On 02/28/2014 04:02 PM, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 16:43 +0200, Alexander Bokovoy wrote: [...] Why then to do it as a IPA CLI command at all? This is an administrative task which can be done with a separate ipa-otp-import command, designated to run on IPA masters. Agreed. 1. Is there a framework for this? Or should it just be an independent script? There is: ipapython.admintool. Look at ipa-server-certinstall for a simple-ish example, ask if you have any questions. 2. How can I use the ipalib API? Specifically, I'd like to call otptoken-add and pass the --key parameter with a value (not possible from the command line). Finalize the API (see ipaserver.install.ServerCertInstall.run), and then call api.Command['otptoken-add'](key=...) Or you might think about moving the otptoken-add functionality into a function that you can call directly. -- Petr³ ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Client-side command in the IPA framework
On Fri, 2014-02-28 at 10:01 -0500, Rob Crittenden wrote: Petr Spacek wrote: On 28.2.2014 15:25, Nathaniel McCallum wrote: On Fri, 2014-02-28 at 10:47 +0100, Petr Vobornik wrote: On 28.2.2014 04:02, Rob Crittenden wrote: Alexander Bokovoy wrote: On Thu, 27 Feb 2014, Nathaniel McCallum wrote: So the recent discussion on importing tokens led me to write a script to parse RFC 6030 xml files into IPA token data. This all works well. But now I need to integrate it into the IPA framework. This command will parse one or more xml files, creating a set of tokens to be added. Given that we already have otptoken-add on the server-side, it seems to me that all work needs to be done on the client-side. How do I create a new client-side command that calls existing server-side API? subclass from frontend.Local, override run() or forward() method and perform batch operation of otptoken_add from there. See cli.help, for example. If you do an override, do forward() for cli-specific work. But you should do as little as possible for reasons you already stated: the UI. Anything you do in forward Petr will need to implement in the UI. Unfortunately we don't yet have a nice way to handle files. We have tickets open at https://fedorahosted.org/freeipa/ticket/1225 and https://fedorahosted.org/freeipa/ticket/2933 If this file is something that would be pasted into a big text field then you can probably handle it in a similarly clumsy way that we do CSRs in the cert plugin. rob +1 for parsing it on server. Otherwise every client, not just CLI or Web UI, would have to reimplement the same logic - having it on server will support better integration with third party products. Parsing on client would be understandable if there was some middle step which would require some action from user, i.e, pick only some tokens to import. If we parse on the server side, how do we handle the long-running operation? Think of the case of importing hundreds or thousands of tokens... My experience is that operation on server side can run for (at least) few minutes without a problem. I haven't try longer periods but we can check that. It can run for hours. Migration performance in IPA used to be rather pitiful and migrating several thousand users could easily take 5+ hours. IIRC sometimes the client would time out but the server side would still complete, you just got no feedback. In this case, feedback is pretty crucial. We will validate all the tokens before writing any of them, so this feedback could be pretty quick. However, if an error occurs during writing, we need to continue adding all the tokens and give an error report at the end of all the tokens that weren't added. Ideally, this report should be in the same import xml format that was provided. Nathaniel ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Client-side command in the IPA framework
On Thu, 2014-02-27 at 17:44 -0500, Nathaniel McCallum wrote: So the recent discussion on importing tokens led me to write a script to parse RFC 6030 xml files into IPA token data. This all works well. But now I need to integrate it into the IPA framework. This command will parse one or more xml files, creating a set of tokens to be added. Given that we already have otptoken-add on the server-side, it seems to me that all work needs to be done on the client-side. How do I create a new client-side command that calls existing server-side API? Also, would it be better to send the whole file to the server and do the parsing on the server side? Would this simplify UI implementation? Would we run into trouble with large files? Nathaniel ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Client-side command in the IPA framework
On Thu, 27 Feb 2014, Nathaniel McCallum wrote: So the recent discussion on importing tokens led me to write a script to parse RFC 6030 xml files into IPA token data. This all works well. But now I need to integrate it into the IPA framework. This command will parse one or more xml files, creating a set of tokens to be added. Given that we already have otptoken-add on the server-side, it seems to me that all work needs to be done on the client-side. How do I create a new client-side command that calls existing server-side API? subclass from frontend.Local, override run() or forward() method and perform batch operation of otptoken_add from there. See cli.help, for example. -- / Alexander Bokovoy ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
Re: [Freeipa-devel] Client-side command in the IPA framework
Alexander Bokovoy wrote: On Thu, 27 Feb 2014, Nathaniel McCallum wrote: So the recent discussion on importing tokens led me to write a script to parse RFC 6030 xml files into IPA token data. This all works well. But now I need to integrate it into the IPA framework. This command will parse one or more xml files, creating a set of tokens to be added. Given that we already have otptoken-add on the server-side, it seems to me that all work needs to be done on the client-side. How do I create a new client-side command that calls existing server-side API? subclass from frontend.Local, override run() or forward() method and perform batch operation of otptoken_add from there. See cli.help, for example. If you do an override, do forward() for cli-specific work. But you should do as little as possible for reasons you already stated: the UI. Anything you do in forward Petr will need to implement in the UI. Unfortunately we don't yet have a nice way to handle files. We have tickets open at https://fedorahosted.org/freeipa/ticket/1225 and https://fedorahosted.org/freeipa/ticket/2933 If this file is something that would be pasted into a big text field then you can probably handle it in a similarly clumsy way that we do CSRs in the cert plugin. rob ___ Freeipa-devel mailing list Freeipa-devel@redhat.com https://www.redhat.com/mailman/listinfo/freeipa-devel
