On 03/04/2014 02:27 PM, Nathaniel McCallum wrote:
On Tue, 2014-03-04 at 14:11 -0500, Dmitri Pal wrote:
On 03/04/2014 02:03 PM, Nathaniel McCallum wrote:
On Mon, 2014-03-03 at 20:12 -0500, Dmitri Pal wrote:
On 03/01/2014 10:07 PM, Adam Young wrote:
On 02/28/2014 10:21 AM, Petr Viktorin wrote:
On 02/28/2014 04:15 PM, Alexander Bokovoy wrote:
On Fri, 28 Feb 2014, Nathaniel McCallum wrote:
On Fri, 2014-02-28 at 16:43 +0200, Alexander Bokovoy wrote:
On Fri, 28 Feb 2014, Nathaniel McCallum wrote:
On Fri, 2014-02-28 at 10:47 +0100, Petr Vobornik wrote:
On 28.2.2014 04:02, Rob Crittenden wrote:
Alexander Bokovoy wrote:
On Thu, 27 Feb 2014, Nathaniel McCallum wrote:
So the recent discussion on importing tokens led me to write a
script to
parse RFC 6030 xml files into IPA token data. This all works
well. But
now I need to integrate it into the IPA framework.

This command will parse one or more xml files, creating a set
of tokens
to be added. Given that we already have otptoken-add on the
it seems to me that all work needs to be done on the
client-side. How do
I create a new client-side command that calls existing
server-side API?
subclass from frontend.Local, override run() or forward()
method and
perform batch
operation of otptoken_add from there.

See cli.help, for example.
If you do an override, do forward() for cli-specific work.

But you should do as little as possible for reasons you already
the UI. Anything you do in forward Petr will need to implement
in the UI.
Unfortunately we don't yet have a nice way to handle files.
We have
tickets open at https://fedorahosted.org/freeipa/ticket/1225 and

If this file is something that would be pasted into a big text
then you can probably handle it in a similarly clumsy way that
we do
CSRs in the cert plugin.

+1 for parsing it on server. Otherwise every client, not just CLI
or Web
UI, would have to reimplement the same logic - having it on server
support better integration with third party products.

Parsing on client would be understandable if there was some middle
which would require some action from user, i.e, pick only some
tokens to
If we parse on the server side, how do we handle the long-running
operation? Think of the case of importing hundreds or thousands of
Why then to do it as a IPA CLI command at all?
This is an administrative task which can be done with a separate
ipa-otp-import command, designated to run on IPA masters.

1. Is there a framework for this? Or should it just be an independent
We don't really have a framework for administrative tools. You may
with install/tools/ipa-adtrust-install, it is main part is relatively
independent of the task (which is in

The framework is there, new tools use it, and there's a ticket to
convert old ones: https://fedorahosted.org/freeipa/ticket/2652 (it's
low priority in Future Releases, so not much progress is there...)
Also see http://www.freeipa.org/page/V3/Logging_and_output

The RESTful approach would be:

1. Upload a file to a specific URL (not JSON RPC)
2.  Receive back a 202 Accepted  HTTP Request, to include an URL to
poll for status

Not certain the right response from the URL in step 2 would be, but I
am assuming it would be 200 with the body of the message stating:
processing or completed.

It would be really nice if the Batch command could be handled this way
as well.  The response back could be the partial responses until
processing is complete.

It might also be nice to supply an email address for notification of
completed processing instead of polling, if it is going to be a really
long running task.

Freeipa-devel mailing list
Yes I think that:
1) We should not limit it to server side operation only
2) Upload the whole file and then process it.
3) We should already have code to upload files, we did it for
entitlements and were supposed to use for certs.
4) Make sure we have a generic upload mechanism that reads a chunk of a
configurable size and asks for more (pagination by 65K might be a good

Regarding token files specifically: they can be big but not super huge.
10-20K tokens makes sense but probably not more. More than that would be
a real corner case becuase it is hard to deploy that amount of tokens at
the same time. It can take months and you do not want token file to
contain many tokens that would sit on the shelf. Tokens expire so it is
inefficient to buy huge chunks and let them sit unused.

UI you allow uploading file too and then would process it locally.
The processing of the file should generate a log or report. It would be
nice to get indication from the server that it is still working so may
be upload protocol should be something like:

client: Initialize the transfer
server: ready
client: here is the chunk of data
server: ack
client: here is the last chunk of data
server: ack, (forks the file processing method that updates shared
status data) come back in x seconds
client: how are things?
server: working, here is current status, come back in x seconds
client: how are things?
server: done, here is current status, have errors in a file
client: start download
server: here is the chunk

I think we can short socket the command for now to fail if it is not
local on the server and then build the upload mechanism but separate
command as proposed in this thread would lock us in a local approach
The problem is that we have no infrastructure for any of this. It would
all have to be built just for the import command. It is also a fairly
rare, admin-only operation. For this reason, I am leaning towards
implementing it as a simple script to be executed on the FreeIPA master.
The main drawback of this approach is that you don't get import
functionality in the UI.

I also disagree that doing it this way now necessitates we do it this
way forever. In fact, it is more likely that if we design a fully
featured server implementation now, we'll get details wrong and be stuck
with it. Doing it on the master using the existing API frees us to add a
server-side API in the future.


We already have use cases for upload but we defer it again and again for
a better time.
We need to upload and download files for other use cases:

a) Keytabs (we have a ticket to do it in the UI)
b) Reports
c) Import logs (for tokens or users)

a) Token file
b) SSH public keys
c) Register user certificate in IPA

We need this mechanisms anyways.
We have been deferring this for too long.
I agree.

I would rather defer support of the hardware tokens till next version
and do an upload in the cli and UI than cut the corner and create yet
one off mechanism.

IMO we are going the wrong way about it.
So the problem isn't hardware tokens, the problem is that users can
modify tokens they shouldn't (these are most commonly hardware, but not
necessarily so). Support for this can be added trivially once I get some
of the more pressing issues off my plate. I don't see any reason to
delay this. Let's not conflate issues.

Imports are a different story. We could land an independent import
script (IIS) in 4.0. This functionality could be moved to a framework
script (FS) later. If backwards compatibility for this is a concern, the
IIS could just point to the FS. If we desire not to support the IIS at
all, we could ship it as contrib or distribute it independently.

But once we have infrastructure for uploads/downloads/long-running task
state, the code from the IIS should move to the server side with little
modification. So it isn't lost effort.

Having *some* form of import is highly desirable for 4.0.


I bet it will be a complete rewrite but I would not argue.

Thank you,
Dmitri Pal

Sr. Engineering Manager for IdM portfolio
Red Hat Inc.

Looking to carve out IT costs?

Freeipa-devel mailing list

Reply via email to