Re: [Freeipa-devel] [PATCHSET] Replica promotion patches

2015-10-16 Thread Endi Sukma Dewata

On 10/15/2015 9:54 AM, Simo Sorce wrote:

3) ipa-ca-install fails with:

Traceback (most recent call last):
   File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py",
line 445, in start_creation
 run_step(full_msg, method)
   File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py",
line 435, in run_step
 method()
   File
"/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line
631, in __spawn_instance
 DogtagInstance.spawn_instance(self, cfg_file)
   File
"/usr/lib/python2.7/site-packages/ipaserver/install/dogtaginstance.py",
line 185, in spawn_instance
 self.handle_setup_error(e)
   File
"/usr/lib/python2.7/site-packages/ipaserver/install/dogtaginstance.py",
line 448, in handle_setup_error
 raise RuntimeError("%s configuration failed." % self.subsystem)
RuntimeError: CA configuration failed.

I guess I'm hitting the authentication bug in Dogtag. It is supposed to
be fixed in pki-core-10.2.6-10, but is it fixed in pki-core-10.2.7-0.2?
We might need a new 10.2.7 build.


I am not sure which version has it fixed, Endi ?


PKI ticket #1580 was fixed in pki-core-10.2.6-10 for F23 and F24. We 
never released a pki-core-10.2.7. I suppose that is a custom build?


--
Endi S. Dewata

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] [PATCHSET] Replica promotion patches

2015-10-07 Thread Endi Sukma Dewata

On 10/5/2015 9:33 AM, Endi Sukma Dewata wrote:

On 10/5/2015 8:47 AM, Simo Sorce wrote:

2. The second attempt after re-enrolling client resulted in the error of
CA installation:


This is due to the known bug with authentication in Dogtag. Endy fixed
it upstream.

Endy,
do you know when the bug will be released in a package we can use for
testing ?


Here is the bug: https://fedorahosted.org/pki/ticket/1580

I don't think we're ready for a Dogtag 10.3 build, so we may need to
cherry-pick it to 10.2.x. I'll check with Matt.



The fix is now available in the following build:
http://koji.fedoraproject.org/koji/buildinfo?buildID=689985

Please also provide a feedback:
https://bodhi.fedoraproject.org/updates/FEDORA-2015-cea85c052a

Thanks!

--
Endi S. Dewata

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] [PATCHSET] Replica promotion patches

2015-10-05 Thread Endi Sukma Dewata

On 10/5/2015 8:47 AM, Simo Sorce wrote:

2. The second attempt after re-enrolling client resulted in the error of
CA installation:

Starting replication, please wait until this has completed.
Update in progress, 7 seconds elapsed
Update succeeded

   [4/24]: creating installation admin user
   [5/24]: setting up certificate server
ipa.ipaserver.install.cainstance.CAInstance: CRITICAL Failed to
configure CA instance: Command ''/usr/sbin/pkispawn' '-s' 'CA' '-f'
'/tmp/tmpHAJVFG'' returned non-zero exit status 1
ipa.ipaserver.install.cainstance.CAInstance: CRITICAL See the
installation logs and the following files/directories for more
information:
ipa.ipaserver.install.cainstance.CAInstance: CRITICAL
/var/log/pki-ca-install.log
ipa.ipaserver.install.cainstance.CAInstance: CRITICAL
/var/log/pki/pki-tomcat
   [error] RuntimeError: CA configuration failed.
Your system may be partly configured.
Run /usr/sbin/ipa-server-install --uninstall to clean up.

ipa.ipapython.install.cli.install_tool(Replica): ERRORCA
configuration failed.


This is due to the known bug with authentication in Dogtag. Endy fixed
it upstream.

Endy,
do you know when the bug will be released in a package we can use for
testing ?


Here is the bug: https://fedorahosted.org/pki/ticket/1580

I don't think we're ready for a Dogtag 10.3 build, so we may need to 
cherry-pick it to 10.2.x. I'll check with Matt.


--
Endi S. Dewata

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] [PATCH] 377 Using LDAPI to setup CA and KRA agents.

2015-09-04 Thread Endi Sukma Dewata

On 9/4/2015 6:35 AM, Martin Basti wrote:



On 09/02/2015 06:42 AM, Endi Sukma Dewata wrote:

On 9/1/2015 1:52 AM, Martin Basti wrote:

The CA and KRA installation code has been modified to use LDAPI
to create the CA and KRA agents directly in the CA and KRA
database. This way it's no longer necessary to use the Directory
Manager password or CA and KRA admin certificate.

https://fedorahosted.org/freeipa/ticket/5257





Thank you.

1) Can you use following code instead of direct call of
ldap2.ldap2()?

if not api.Backend.ldap2.is_connected():
 api.Backend.ldap2.connect(autobind=True)

conn = api.Backend.ldap2


Why would you want to do that? The original code is fine, except the
connection check is not necessary (it is a new instance of ldap2, so
.isconnected() will always return False).



It's actually isconnected() instead of is_connected(), but even so,
the
proposed code doesn't work:

ipa.ipapython.install.cli.install_tool(Server): DEBUGThe
ipa-server-install command failed, exception: TypeError: 'ldap2'
object
is not callable
ipa.ipapython.install.cli.install_tool(Server): ERROR 'ldap2' object
is not callable


2) Patch needs rebase to master branch.


The original patch does apply cleanly to master. Did you see a
conflict?

Sorry my bad.

Martin^2



3)
+user_dn = DN(('uid', "ipara"), ('ou', 'People'),
self.basedn)
+conn.create(
+dn=user_dn,

can you use add entry() instead of create()? We don't use native
python-ldap, but rather ipaldap methods


It's actually calling the ldap2.create() defined in
ipaserver/plugins/ldap2.py, which calls add_entry().


NACK. We don't use ldap2.create(). Use add_entry().



So my original patch still stands.


New patch attached.


ACK, but IMO that comments is not necessary and I would like to push the
patch without it.

Martin^2


It is necessary if we don't want people to use it. Otherwise someone 
could make the same mistake. Or better yet, just remove the method.


--
Endi S. Dewata

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] [PATCH] 377 Using LDAPI to setup CA and KRA agents.

2015-09-01 Thread Endi Sukma Dewata

On 9/1/2015 1:52 AM, Martin Basti wrote:

The CA and KRA installation code has been modified to use LDAPI
to create the CA and KRA agents directly in the CA and KRA
database. This way it's no longer necessary to use the Directory
Manager password or CA and KRA admin certificate.

https://fedorahosted.org/freeipa/ticket/5257





Thank you.

1) Can you use following code instead of direct call of ldap2.ldap2()?

if not api.Backend.ldap2.is_connected():
 api.Backend.ldap2.connect(autobind=True)

conn = api.Backend.ldap2


Why would you want to do that? The original code is fine, except the
connection check is not necessary (it is a new instance of ldap2, so
.isconnected() will always return False).



It's actually isconnected() instead of is_connected(), but even so, the
proposed code doesn't work:

ipa.ipapython.install.cli.install_tool(Server): DEBUGThe
ipa-server-install command failed, exception: TypeError: 'ldap2' object
is not callable
ipa.ipapython.install.cli.install_tool(Server): ERROR'ldap2' object
is not callable


2) Patch needs rebase to master branch.


The original patch does apply cleanly to master. Did you see a conflict?

Sorry my bad.

Martin^2



3)
+user_dn = DN(('uid', "ipara"), ('ou', 'People'), self.basedn)
+conn.create(
+dn=user_dn,

can you use add entry() instead of create()? We don't use native
python-ldap, but rather ipaldap methods


It's actually calling the ldap2.create() defined in
ipaserver/plugins/ldap2.py, which calls add_entry().


NACK. We don't use ldap2.create(). Use add_entry().



So my original patch still stands.


New patch attached.

--
Endi S. Dewata
>From e03882e89d2acdb23fe289d59dd2db04662bf051 Mon Sep 17 00:00:00 2001
From: "Endi S. Dewata" 
Date: Thu, 27 Aug 2015 06:44:29 +0200
Subject: [PATCH] Using LDAPI to setup CA and KRA agents.

The CA and KRA installation code has been modified to use LDAPI
to create the CA and KRA agents directly in the CA and KRA
database. This way it's no longer necessary to use the Directory
Manager password or CA and KRA admin certificate.

https://fedorahosted.org/freeipa/ticket/5257
---
 ipaplatform/base/paths.py|   2 -
 ipaserver/install/cainstance.py  |  49 ++---
 ipaserver/install/krainstance.py | 113 +++
 ipaserver/plugins/ldap2.py   |   2 +
 4 files changed, 74 insertions(+), 92 deletions(-)

diff --git a/ipaplatform/base/paths.py b/ipaplatform/base/paths.py
index 
5c8f25d6ef85fab2b9b30a660cd1c0360dbe9931..0dd3c7fda3020264a1ace8f2d13557cfddf18c2d
 100644
--- a/ipaplatform/base/paths.py
+++ b/ipaplatform/base/paths.py
@@ -343,8 +343,6 @@ class BasePathNamespace(object):
 SLAPD_INSTANCE_SOCKET_TEMPLATE = "/var/run/slapd-%s.socket"
 ALL_SLAPD_INSTANCE_SOCKETS = "/var/run/slapd-*.socket"
 ADMIN_CERT_PATH = '/root/.dogtag/pki-tomcat/ca_admin.cert'
-KRA_NSSDB_PASSWORD_FILE = "/root/.dogtag/pki-tomcat/kra/password.conf"
-KRA_PKCS12_PASSWORD_FILE = 
"/root/.dogtag/pki-tomcat/kra/pkcs12_password.conf"
 ENTROPY_AVAIL = '/proc/sys/kernel/random/entropy_avail'
 LDIF2DB = '/usr/sbin/ldif2db'
 DB2LDIF = '/usr/sbin/db2ldif'
diff --git a/ipaserver/install/cainstance.py b/ipaserver/install/cainstance.py
index 
6f565dd14a1ce3f22bf1d033eed364dc4b87281b..85ce6cba59442fe934cc96f4983b21c16d9ea8de
 100644
--- a/ipaserver/install/cainstance.py
+++ b/ipaserver/install/cainstance.py
@@ -466,7 +466,7 @@ class CAInstance(DogtagInstance):
 self.step("restarting certificate server", 
self.restart_instance)
 self.step("requesting RA certificate from CA", 
self.__request_ra_certificate)
 self.step("issuing RA agent certificate", self.__issue_ra_cert)
-self.step("adding RA agent as a trusted user", 
self.__configure_ra)
+self.step("adding RA agent as a trusted user", 
self.__create_ca_agent)
 self.step("authorizing RA to modify profiles", 
self.__configure_profiles_acl)
 self.step("configure certmonger for renewals", 
self.configure_certmonger_renewal)
 self.step("configure certificate renewals", self.configure_renewal)
@@ -905,18 +905,26 @@ class CAInstance(DogtagInstance):
 
 self.configure_agent_renewal()
 
-def __configure_ra(self):
-# Create an RA user in the CA LDAP server and add that user to
-# the appropriate groups so it can issue certificates without
-# manual intervention.
-conn = ipaldap.IPAdmin(self.fqdn, self.ds_port)
-conn.do_simple_bind(DN(('cn', 'Directory Manager')), self.dm_password)
+def __create_ca_agent(self):
+"""
+Create CA agent, assign a certificate, and add the user to
+the appropriate groups for accessing CA services.
+"""
 
-decoded = base64.b64decode(self.ra_cert)
+# get ipaCert certificate
+cert_data = base64.b64decode(self.ra_cert)
+

Re: [Freeipa-devel] [PATCH] 916 vault: add vault container commands

2015-09-01 Thread Endi Sukma Dewata

On 9/1/2015 10:22 AM, Simo Sorce wrote:

On Tue, 2015-09-01 at 17:15 +0200, Petr Vobornik wrote:

On 09/01/2015 04:39 PM, Jan Cholasta wrote:

On 1.9.2015 16:26, Jan Cholasta wrote:

On 26.8.2015 13:22, Petr Vobornik wrote:

On 08/25/2015 08:04 PM, Petr Vobornik wrote:

adds commands:
* vaultcontainer-show [--service |--user  ]
* vaultcontainer-add-owner
   [--service |--user  ]
   [--users ]  [--groups ] [--services ]
* vaultcontainer-remove-owner
   [--service |--user  ]
   [--users ]  [--groups ] [--services ]

https://fedorahosted.org/freeipa/ticket/5250

Use cases:
1. When user/service is deleted, associated vault container looses
owner. There was no API command to set the owner.
2. Change owner of container by admin to manage access.

Show command was added to show current owners.

Find command was not added, should it be?




There is also a design for vault container ownership handling created by
Endi - it's for future Vault 2.0.

http://www.freeipa.org/page/V4/Password_Vault_2.0#Adding_container_owner

This patch has a different API than the proposed - different way of
specifying the container. The design page uses path e.g. /users/foobar.
This patch uses the same way as vaults e.g. --user=foobar. This means
that the implementation in this patch cannot manage ownership of parent
vault containers e.g. cn=users,cn=vaults,cn=kra,$SUFFIX.

Do we want to go with this approach in 4.2?

Attaching also new path which removes setting of owner which doesn't
exist so that integrity is OK and that it is consistent with removing of
user.

Updated patch attached - output fix.


We had a long discussion about this with Petr and we think the best
approach is as follows:

* Add new "Vault administrators" privilege. Vault administrators will
have unrestricted access to vaults and vault containers, including the
power to add/remove owners of vaults and vault containers.

* Remove the ability of vault owners to add/remove other vault
owners. If vault owner needs to be changed, vault administrator has to
do it. Note that vault owners will still have the ability to add/remove
vault members.

* When adding new vault container, set owner to the current user. If
vault container owner needs to be changed, vault administrator has to do
it.

* Allow adding vaults and vault containers only if the owner is set
to the current user.

* Introduce commands to modify vault container owner and to delete
vault container, so the administrator has a choice between assigning
ownership or deleting an unowned container.


Also:

* Control access to vault data using an ipaProtectedOperation ACI.
Users which have read access to "ipaProtectedOperation;accessKRA" on a
vault can retrieve data from the vault and users which have write access
to "ipaProtectedOperation;accessKRA" on a vault can archive data in the
vault.

Honza



+1

CCing Simo and Endi to check the proposal.

And Scott (related to #5216, #5215)


Sounds reasonable to me.
I can see that allowing owners to hand over vaults w/o admin
intervention may have some appeal in some use cases, but I also see it
can bring downsides with it, so all in all I think I agree with the
above points.

Simo.



Not a total objection, but if many people in unrelated groups are using 
vaults, and they are sharing the vaults only with members of each group, 
having to ask a Vault Administrator for each ownership change sounds a 
bit cumbersome. Since the Vault Adminstrator will have access to all 
vaults in all groups, only a small number of people can be trusted to 
hold that role. If there are many ownership changes the Vault 
Administrator will have to handle all those requests, and the vault 
users may have to wait until the change is completed.


If owners are allowed to add others as owners, the vaults will be pretty 
much maintenance free to the admin.


Regardless, please update the wiki page to describe the new behavior 
when it's implemented:

http://www.freeipa.org/page/V4/Password_Vault_1.1

--
Endi S. Dewata

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] [PATCH] 377 Using LDAPI to setup CA and KRA agents.

2015-08-31 Thread Endi Sukma Dewata

On 8/31/2015 6:18 AM, Martin Basti wrote:



On 08/27/2015 09:41 PM, Endi Sukma Dewata wrote:

The CA and KRA installation code has been modified to use LDAPI
to create the CA and KRA agents directly in the CA and KRA
database. This way it's no longer necessary to use the Directory
Manager password or CA and KRA admin certificate.

https://fedorahosted.org/freeipa/ticket/5257





Thank you.

1) Can you use following code instead of direct call of ldap2.ldap2()?

if not api.Backend.ldap2.is_connected():
 api.Backend.ldap2.connect(autobind=True)

conn = api.Backend.ldap2


It's actually isconnected() instead of is_connected(), but even so, the 
proposed code doesn't work:


ipa.ipapython.install.cli.install_tool(Server): DEBUGThe 
ipa-server-install command failed, exception: TypeError: 'ldap2' object 
is not callable
ipa.ipapython.install.cli.install_tool(Server): ERROR'ldap2' object 
is not callable



2) Patch needs rebase to master branch.


The original patch does apply cleanly to master. Did you see a conflict?


3)
+user_dn = DN(('uid', "ipara"), ('ou', 'People'), self.basedn)
+conn.create(
+dn=user_dn,

can you use add entry() instead of create()? We don't use native
python-ldap, but rather ipaldap methods


It's actually calling the ldap2.create() defined in 
ipaserver/plugins/ldap2.py, which calls add_entry().


So my original patch still stands.

--
Endi S. Dewata

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


[Freeipa-devel] [PATCH] 377 Using LDAPI to setup CA and KRA agents.

2015-08-27 Thread Endi Sukma Dewata

The CA and KRA installation code has been modified to use LDAPI
to create the CA and KRA agents directly in the CA and KRA
database. This way it's no longer necessary to use the Directory
Manager password or CA and KRA admin certificate.

https://fedorahosted.org/freeipa/ticket/5257

--
Endi S. Dewata
From 45af6d4f9a8ebc9bbd2856d7bf3af48520996dad Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Thu, 27 Aug 2015 06:44:29 +0200
Subject: [PATCH] Using LDAPI to setup CA and KRA agents.

The CA and KRA installation code has been modified to use LDAPI
to create the CA and KRA agents directly in the CA and KRA
database. This way it's no longer necessary to use the Directory
Manager password or CA and KRA admin certificate.

https://fedorahosted.org/freeipa/ticket/5257
---
 ipaplatform/base/paths.py|   2 -
 ipaserver/install/cainstance.py  |  49 ++---
 ipaserver/install/krainstance.py | 113 +++
 3 files changed, 72 insertions(+), 92 deletions(-)

diff --git a/ipaplatform/base/paths.py b/ipaplatform/base/paths.py
index 
5c8f25d6ef85fab2b9b30a660cd1c0360dbe9931..0dd3c7fda3020264a1ace8f2d13557cfddf18c2d
 100644
--- a/ipaplatform/base/paths.py
+++ b/ipaplatform/base/paths.py
@@ -343,8 +343,6 @@ class BasePathNamespace(object):
 SLAPD_INSTANCE_SOCKET_TEMPLATE = /var/run/slapd-%s.socket
 ALL_SLAPD_INSTANCE_SOCKETS = /var/run/slapd-*.socket
 ADMIN_CERT_PATH = '/root/.dogtag/pki-tomcat/ca_admin.cert'
-KRA_NSSDB_PASSWORD_FILE = /root/.dogtag/pki-tomcat/kra/password.conf
-KRA_PKCS12_PASSWORD_FILE = 
/root/.dogtag/pki-tomcat/kra/pkcs12_password.conf
 ENTROPY_AVAIL = '/proc/sys/kernel/random/entropy_avail'
 LDIF2DB = '/usr/sbin/ldif2db'
 DB2LDIF = '/usr/sbin/db2ldif'
diff --git a/ipaserver/install/cainstance.py b/ipaserver/install/cainstance.py
index 
ecd9300036353426097d929918be974cbbb5c69d..bec39419363f1ade0130465d3b70e1c5540b6006
 100644
--- a/ipaserver/install/cainstance.py
+++ b/ipaserver/install/cainstance.py
@@ -904,17 +904,26 @@ class CAInstance(DogtagInstance):
 self.configure_agent_renewal()
 
 def __configure_ra(self):
-# Create an RA user in the CA LDAP server and add that user to
-# the appropriate groups so it can issue certificates without
-# manual intervention.
-conn = ipaldap.IPAdmin(self.fqdn, self.ds_port)
-conn.do_simple_bind(DN(('cn', 'Directory Manager')), self.dm_password)
+
+Create CA agent, assign a certificate, and add the user to
+the appropriate groups for accessing CA services.
+
 
-decoded = base64.b64decode(self.ra_cert)
+# get ipaCert certificate
+cert_data = base64.b64decode(self.ra_cert)
+cert = x509.load_certificate(cert_data, x509.DER)
 
-entry_dn = DN(('uid', ipara), ('ou', 'People'), self.basedn)
-entry = conn.make_entry(
-entry_dn,
+# connect to CA database
+server_id = installutils.realm_to_serverid(api.env.realm)
+dogtag_uri = 'ldapi://%%2fvar%%2frun%%2fslapd-%s.socket' % server_id
+conn = ldap2.ldap2(api, ldap_uri=dogtag_uri)
+if not conn.isconnected():
+conn.connect(autobind=True)
+
+# create ipara user with ipaCert certificate
+user_dn = DN(('uid', ipara), ('ou', 'People'), self.basedn)
+conn.create(
+dn=user_dn,
 objectClass=['top', 'person', 'organizationalPerson',
  'inetOrgPerson', 'cmsuser'],
 uid=[ipara],
@@ -922,23 +931,23 @@ class CAInstance(DogtagInstance):
 cn=[ipara],
 usertype=[agentType],
 userstate=[1],
-userCertificate=[decoded],
+userCertificate=[cert_data],
 description=['2;%s;%s;%s' % (
-str(self.requestId),
+cert.serial_number,
 DN(('CN', 'Certificate Authority'), self.subject_base),
 DN(('CN', 'IPA RA'), self.subject_base))])
 
-conn.add_entry(entry)
+# add ipara user to Certificate Manager Agents group
+group_dn = DN(('cn', 'Certificate Manager Agents'), ('ou', 'groups'),
+self.basedn)
+conn.add_entry_to_group(user_dn, group_dn, 'uniqueMember')
 
-dn = DN(('cn', 'Certificate Manager Agents'), ('ou', 'groups'), 
self.basedn)
-modlist = [(0, 'uniqueMember', '%s' % entry_dn)]
-conn.modify_s(dn, modlist)
+# add ipara user to Registration Manager Agents group
+group_dn = DN(('cn', 'Registration Manager Agents'), ('ou', 'groups'),
+self.basedn)
+conn.add_entry_to_group(user_dn, group_dn, 'uniqueMember')
 
-dn = DN(('cn', 'Registration Manager Agents'), ('ou', 'groups'), 
self.basedn)
-modlist = [(0, 'uniqueMember', '%s' % entry_dn)]
-conn.modify_s(dn, modlist)
-
-conn.unbind()
+conn.disconnect()
 
 def 

Re: [Freeipa-devel] [PATCH] 375 Added mechanism to copy vault secrets.

2015-08-26 Thread Endi Sukma Dewata

On 8/20/2015 2:08 AM, Endi Sukma Dewata wrote:

On 8/19/2015 4:20 AM, Martin Basti wrote:

On 08/16/2015 05:29 PM, Endi Sukma Dewata wrote:

The vault-add and vault-archive commands have been modified to
optionally retrieve a secret from a source vault, then re-archive
the secret into the new/existing target vault.

https://fedorahosted.org/freeipa/ticket/5223




I cannot apply this patch.


Rebased. It depends on patch #371-2.


Rebased due to other changes in vault.

--
Endi S. Dewata
From 676b2043a390e6e68772837cf46e222aeda9da78 Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Sat, 15 Aug 2015 16:17:47 +0200
Subject: [PATCH] Added mechanism to copy vault secrets.

The vault-add and vault-archive commands have been modified to
optionally retrieve a secret from a source vault, then re-archive
the secret into the new/existing target vault.

https://fedorahosted.org/freeipa/ticket/5223
---
 API.txt   |  20 ++-
 VERSION   |   4 +-
 ipalib/plugins/vault.py   | 213 --
 ipatests/test_xmlrpc/test_vault_plugin.py | 143 
 4 files changed, 306 insertions(+), 74 deletions(-)

diff --git a/API.txt b/API.txt
index 
afd5017bee2bc1eed54497ccd504b92619ff7a58..c883271af4ff84f82c623208567f114265c3ce60
 100644
--- a/API.txt
+++ b/API.txt
@@ -5405,7 +5405,7 @@ output: Output('result', type 'bool', None)
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
 command: vault_add
-args: 1,14,3
+args: 1,22,3
 arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
 option: Str('addattr*', cli_name='addattr', exclude='webui')
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
@@ -5419,6 +5419,14 @@ option: Flag('raw', autofill=True, cli_name='raw', 
default=False, exclude='webui
 option: Str('service?')
 option: Str('setattr*', cli_name='setattr', exclude='webui')
 option: Flag('shared?', autofill=True, default=False)
+option: Str('source_password?')
+option: Str('source_password_file?')
+option: Bytes('source_private_key?')
+option: Str('source_private_key_file?')
+option: Str('source_service?')
+option: Flag('source_shared?', autofill=True, default=False)
+option: Str('source_user?')
+option: Str('source_vault?')
 option: Str('username?', cli_name='user')
 option: Str('version?', exclude='webui')
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
@@ -5474,7 +5482,7 @@ output: Output('completed', type 'int', None)
 output: Output('failed', type 'dict', None)
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 command: vault_archive
-args: 1,11,3
+args: 1,19,3
 arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
 option: Bytes('data?')
@@ -5485,6 +5493,14 @@ option: Str('password_file?', cli_name='password_file')
 option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
 option: Str('service?')
 option: Flag('shared?', autofill=True, default=False)
+option: Str('source_password?')
+option: Str('source_password_file?')
+option: Bytes('source_private_key?')
+option: Str('source_private_key_file?')
+option: Str('source_service?')
+option: Flag('source_shared?', autofill=True, default=False)
+option: Str('source_user?')
+option: Str('source_vault?')
 option: Str('username?', cli_name='user')
 option: Str('version?', exclude='webui')
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
diff --git a/VERSION b/VERSION
index 
d3073e52ee022cc08b74953222a5040929ded60f..e3cfaa91f03fc6f4d9f5084809a8f74af333c8ef
 100644
--- a/VERSION
+++ b/VERSION
@@ -90,5 +90,5 @@ IPA_DATA_VERSION=2010061412
 #  #
 
 IPA_API_VERSION_MAJOR=2
-IPA_API_VERSION_MINOR=154
-# Last change: pvoborni - change default vault type to 'symmetric'
+IPA_API_VERSION_MINOR=155
+# Last change: edewata - Added mechanism to copy vault secrets.
diff --git a/ipalib/plugins/vault.py b/ipalib/plugins/vault.py
index 
ff6c22c646e9784b2fa1a6464f0749cb1ce86b50..a625746ab067d915e71504e971eefb0d0222ff77
 100644
--- a/ipalib/plugins/vault.py
+++ b/ipalib/plugins/vault.py
@@ -255,6 +255,78 @@ vault_options = (
 ),
 )
 
+source_vault_options = (
+Str(
+'source_vault?',
+doc=_('Name of the source service vault'),
+),
+Str(
+'source_service?',
+doc=_('Service name of the source

[Freeipa-devel] Adding client-side functionality in Vault

2015-08-24 Thread Endi Sukma Dewata

Hi,

Recently I posted the following patches which are still pending review:
* 371-2: Added support for changing vault encryption.
* 375-1: Added mechanism to copy vault secrets.

Here are the tickets:
* https://fedorahosted.org/freeipa/ticket/5176
* https://fedorahosted.org/freeipa/ticket/5223

These patches add new functionality to the following commands:
* vault-mod: changing vault encryption
* vault-archive: copying a secret from a vault into an existing vault
* vault-add: copying a secret from a vault into a new vault

The changes are quite similar. In order to change the vault encryption 
or to copy the vault secret, the old secret has to be retrieved with the 
old encryption parameters, then the secret will be rearchived with the 
new encryption parameters.


The thing is these operations have to be done on the client side since 
the encryption/decryption is done using a key only known to the client. 
This also means that even if the server is upgraded, someone using an 
old client will not be able to utilize the new functionality unless the 
client is upgraded too. Also, the old vault-mod actually has a bug 
because it will update the vault encryption attributes without 
rearchiving the secret.


Should we require old clients to upgrade? Or should we continue to 
accept old clients, but the buggy operation will now be rejected? Is 
this considered breaking backward compatibility?


Thanks.

--
Endi S. Dewata

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] [PATCH] 375 Added mechanism to copy vault secrets.

2015-08-20 Thread Endi Sukma Dewata

On 8/19/2015 4:20 AM, Martin Basti wrote:

On 08/16/2015 05:29 PM, Endi Sukma Dewata wrote:

The vault-add and vault-archive commands have been modified to
optionally retrieve a secret from a source vault, then re-archive
the secret into the new/existing target vault.

https://fedorahosted.org/freeipa/ticket/5223




I cannot apply this patch.


Rebased. It depends on patch #371-2.

--
Endi S. Dewata
From 20ee7fa94d28239b7b0512db53c6b899baf8a62f Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Sat, 15 Aug 2015 16:17:47 +0200
Subject: [PATCH] Added mechanism to copy vault secrets.

The vault-add and vault-archive commands have been modified to
optionally retrieve a secret from a source vault, then re-archive
the secret into the new/existing target vault.

https://fedorahosted.org/freeipa/ticket/5223
---
 API.txt   |  20 +++-
 VERSION   |   4 +-
 ipalib/plugins/vault.py   | 193 --
 ipatests/test_xmlrpc/test_vault_plugin.py | 143 ++
 4 files changed, 292 insertions(+), 68 deletions(-)

diff --git a/API.txt b/API.txt
index 
749aa41d5cab60e4f2acf7486135ad066db7a8a6..3007bf31934d0b71dd104774ddbd1638cd12e8a3
 100644
--- a/API.txt
+++ b/API.txt
@@ -5397,7 +5397,7 @@ output: Output('result', type 'bool', None)
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
 command: vault_add
-args: 1,14,3
+args: 1,22,3
 arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
 option: Str('addattr*', cli_name='addattr', exclude='webui')
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
@@ -5411,6 +5411,14 @@ option: Flag('raw', autofill=True, cli_name='raw', 
default=False, exclude='webui
 option: Str('service?')
 option: Str('setattr*', cli_name='setattr', exclude='webui')
 option: Flag('shared?', autofill=True, default=False)
+option: Str('source_password?')
+option: Str('source_password_file?')
+option: Bytes('source_private_key?')
+option: Str('source_private_key_file?')
+option: Str('source_service?')
+option: Flag('source_shared?', autofill=True, default=False)
+option: Str('source_user?')
+option: Str('source_vault?')
 option: Str('username?', cli_name='user')
 option: Str('version?', exclude='webui')
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
@@ -5466,7 +5474,7 @@ output: Output('completed', type 'int', None)
 output: Output('failed', type 'dict', None)
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 command: vault_archive
-args: 1,11,3
+args: 1,19,3
 arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
 option: Bytes('data?')
@@ -5477,6 +5485,14 @@ option: Str('password_file?', cli_name='password_file')
 option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
 option: Str('service?')
 option: Flag('shared?', autofill=True, default=False)
+option: Str('source_password?')
+option: Str('source_password_file?')
+option: Bytes('source_private_key?')
+option: Str('source_private_key_file?')
+option: Str('source_service?')
+option: Flag('source_shared?', autofill=True, default=False)
+option: Str('source_user?')
+option: Str('source_vault?')
 option: Str('username?', cli_name='user')
 option: Str('version?', exclude='webui')
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
diff --git a/VERSION b/VERSION
index 
941b5356378582d4d620896acdf5c284a4f66b8d..19302efe7a36441b1ffef0d8db609f7be9a26c83
 100644
--- a/VERSION
+++ b/VERSION
@@ -90,5 +90,5 @@ IPA_DATA_VERSION=2010061412
 #  #
 
 IPA_API_VERSION_MAJOR=2
-IPA_API_VERSION_MINOR=152
-# Last change: edewata - Added support for changing vault encryption.
+IPA_API_VERSION_MINOR=153
+# Last change: edewata - Added mechanism to copy vault secrets.
diff --git a/ipalib/plugins/vault.py b/ipalib/plugins/vault.py
index 
54dbf94b7de53a218df38550daa5e21450d2b9bd..47610fee56069a85296445d4eda3c640f835ebcf
 100644
--- a/ipalib/plugins/vault.py
+++ b/ipalib/plugins/vault.py
@@ -254,6 +254,74 @@ vault_options = (
 ),
 )
 
+source_vault_options = (
+Str(
+'source_vault?',
+doc=_('Name of the source service vault'),
+),
+Str(
+'source_service?',
+doc=_('Service name of the source service vault'),
+),
+Flag(
+'source_shared?',
+doc=_('Source

Re: [Freeipa-devel] [PATCH] 371 Added support for changing vault encryption.

2015-08-20 Thread Endi Sukma Dewata

On 8/19/2015 4:58 AM, Martin Basti wrote:

On 08/13/2015 07:11 PM, Endi Sukma Dewata wrote:

On 8/13/2015 8:06 AM, Martin Basti wrote:

The vault-mod command has been modified to support changing vault
encryption attributes (i.e. type, password, public/private keys)
in addition to normal attributes (i.e. description). Changing the
encryption requires retrieving the stored secret with the old
attributes and rearchieving it with the new attributes.

https://fedorahosted.org/freeipa/ticket/5176


Hello, does this patch require any additional patches?

I have current master branch and I cannot apply it.

git am
freeipa-edewata-0371-Added-support-for-changing-vault-encryption.patch -3

Applying: Added support for changing vault encryption.
error: invalid object 100644 3b62822366a62c90f843a6293589c28383e782ef
for 'ipalib/plugins/vault.py'
fatal: git-write-tree: error building trees
Repository lacks necessary blobs to fall back on 3-way merge.


Martin^2


New patch attached. It requires patch #0369-3.


I cannot apply patch (and 369-3 was pushed)

git am
freeipa-edewata-0371-1-Added-support-for-changing-vault-encryption.patch -3
Applying: Added support for changing vault encryption.
error: invalid object 100644 5d367b376ef41427ed983f3eafe120ed477018d2
for 'ipalib/plugins/vault.py'
fatal: git-write-tree: error building trees
Repository lacks necessary blobs to fall back on 3-way merge.
Cannot fall back to three-way merge.


Rebased.

--
Endi S. Dewata
From 78d31f8201ab63ecfe93a3f126654a57e9c14d3f Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Fri, 31 Jul 2015 07:53:15 +0200
Subject: [PATCH] Added support for changing vault encryption.

The vault-mod command has been modified to support changing vault
encryption attributes (i.e. type, password, public/private keys)
in addition to normal attributes (i.e. description). Changing the
encryption requires retrieving the stored secret with the old
attributes and rearchiving it with the new attributes.

https://fedorahosted.org/freeipa/ticket/5176
---
 API.txt   |  27 +++-
 VERSION   |   4 +-
 ipalib/plugins/vault.py   | 231 +--
 ipatests/test_xmlrpc/test_vault_plugin.py | 249 ++
 4 files changed, 497 insertions(+), 14 deletions(-)

diff --git a/API.txt b/API.txt
index 
f23d9a40c647a3c4d209419631794cd36e8e5e2f..749aa41d5cab60e4f2acf7486135ad066db7a8a6
 100644
--- a/API.txt
+++ b/API.txt
@@ -5466,11 +5466,12 @@ output: Output('completed', type 'int', None)
 output: Output('failed', type 'dict', None)
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 command: vault_archive
-args: 1,10,3
+args: 1,11,3
 arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
 option: Bytes('data?')
 option: Str('in?')
+option: Flag('override_password?', autofill=True, default=False)
 option: Str('password?', cli_name='password')
 option: Str('password_file?', cli_name='password_file')
 option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
@@ -5530,6 +5531,30 @@ output: ListOfEntries('result', (type 'list', type 
'tuple'), Gettext('A list
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: Output('truncated', type 'bool', None)
 command: vault_mod
+args: 1,18,3
+arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Flag('change_password?', autofill=True, default=False)
+option: Str('description?', cli_name='desc')
+option: Bytes('ipavaultpublickey?', cli_name='public_key')
+option: Bytes('ipavaultsalt?', cli_name='salt')
+option: Str('ipavaulttype?', cli_name='type')
+option: Str('new_password?', cli_name='new_password')
+option: Str('new_password_file?', cli_name='new_password_file')
+option: Str('old_password?', cli_name='old_password')
+option: Str('old_password_file?', cli_name='old_password_file')
+option: Bytes('private_key?', cli_name='private_key')
+option: Str('private_key_file?', cli_name='private_key_file')
+option: Str('public_key_file?', cli_name='public_key_file')
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Str('service?')
+option: Flag('shared?', autofill=True, default=False)
+option: Str('username?', cli_name='user')
+option: Str('version?', exclude='webui')
+output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
+output: Output('summary', (type 'unicode', type 'NoneType'), None)
+output: PrimaryKey('value

Re: [Freeipa-devel] [PATCH] 374 Fixed vault container ownership.

2015-08-13 Thread Endi Sukma Dewata

On 8/13/2015 9:18 AM, Martin Basti wrote:

The vault-add command has been fixed such that if the user/service
private vault container does not exist yet it will be created and
owned by the user/service instead of the vault creator.

https://fedorahosted.org/freeipa/ticket/5194


I cannot apply this patch, are there any additional required patches?

I have current ipa master branch

git am freeipa-edewata-0374-Fixed-vault-container-ownership.patch -3
Applying: Fixed vault container ownership.
error: invalid object 100644 427b1ea1588af2fb09a99181b8773abdf8099b8d
for 'ipalib/plugins/vault.py'
fatal: git-write-tree: error building trees
Repository lacks necessary blobs to fall back on 3-way merge.
Cannot fall back to three-way merge.


Rebased. This patch doesn't have any dependency.

--
Endi S. Dewata
From da35d235bc8196062b208095aa904e3b7a1905e2 Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Mon, 10 Aug 2015 20:57:58 +0200
Subject: [PATCH] Fixed vault container ownership.

The vault-add command has been fixed such that if the user/service
private vault container does not exist yet it will be created and
owned by the user/service instead of the vault creator.

https://fedorahosted.org/freeipa/ticket/5194
---
 ipalib/plugins/vault.py | 25 +++--
 1 file changed, 23 insertions(+), 2 deletions(-)

diff --git a/ipalib/plugins/vault.py b/ipalib/plugins/vault.py
index 
1150d5f3b8cd0001f24756548d7f280494161d19..9a2995ca04cb99b0be46076541cea2638bf3ca56
 100644
--- a/ipalib/plugins/vault.py
+++ b/ipalib/plugins/vault.py
@@ -712,12 +712,33 @@ class vault_add_internal(LDAPCreate):
 else:
 owner_dn = self.api.Object.user.get_dn(name)
 
+parent_dn = DN(*dn[1:])
+
+container_dn = DN(self.api.Object.vault.container_dn,
+  self.api.env.basedn)
+
+services_dn = DN(('cn', 'services'), container_dn)
+users_dn = DN(('cn', 'users'), container_dn)
+
+if dn.endswith(services_dn):
+# service container should be owned by the service
+service = parent_dn[0]['cn']
+parent_owner_dn = self.api.Object.service.get_dn(service)
+
+elif dn.endswith(users_dn):
+# user container should be owned by the user
+user = parent_dn[0]['cn']
+parent_owner_dn = self.api.Object.user.get_dn(user)
+
+else:
+parent_owner_dn = owner_dn
+
 try:
-parent_dn = DN(*dn[1:])
-self.obj.create_container(parent_dn, owner_dn)
+self.obj.create_container(parent_dn, parent_owner_dn)
 except errors.DuplicateEntry as e:
 pass
 
+# vault should be owned by the creator
 entry_attrs['owner'] = owner_dn
 
 return dn
-- 
2.1.0

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH] 371 Added support for changing vault encryption.

2015-08-13 Thread Endi Sukma Dewata

On 8/13/2015 8:06 AM, Martin Basti wrote:

The vault-mod command has been modified to support changing vault
encryption attributes (i.e. type, password, public/private keys)
in addition to normal attributes (i.e. description). Changing the
encryption requires retrieving the stored secret with the old
attributes and rearchieving it with the new attributes.

https://fedorahosted.org/freeipa/ticket/5176


Hello, does this patch require any additional patches?

I have current master branch and I cannot apply it.

git am
freeipa-edewata-0371-Added-support-for-changing-vault-encryption.patch -3
Applying: Added support for changing vault encryption.
error: invalid object 100644 3b62822366a62c90f843a6293589c28383e782ef
for 'ipalib/plugins/vault.py'
fatal: git-write-tree: error building trees
Repository lacks necessary blobs to fall back on 3-way merge.


Martin^2


New patch attached. It requires patch #0369-3.

--
Endi S. Dewata
From b06e859e51a177369c27c52bff8a70263aed59c0 Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Fri, 31 Jul 2015 07:53:15 +0200
Subject: [PATCH] Added support for changing vault encryption.

The vault-mod command has been modified to support changing vault
encryption attributes (i.e. type, password, public/private keys)
in addition to normal attributes (i.e. description). Changing the
encryption requires retrieving the stored secret with the old
attributes and rearchiving it with the new attributes.

https://fedorahosted.org/freeipa/ticket/5176
---
 API.txt   |  27 +++-
 VERSION   |   4 +-
 ipalib/plugins/vault.py   | 232 ++--
 ipatests/test_xmlrpc/test_vault_plugin.py | 246 ++
 4 files changed, 495 insertions(+), 14 deletions(-)

diff --git a/API.txt b/API.txt
index 
9dbf86aedf2a1b62dabab21fb30bbceb2f0f237b..26f05cf9e1e27ec4f714bb34174e17972961bda2
 100644
--- a/API.txt
+++ b/API.txt
@@ -5466,11 +5466,12 @@ output: Output('completed', type 'int', None)
 output: Output('failed', type 'dict', None)
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 command: vault_archive
-args: 1,10,3
+args: 1,11,3
 arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
 option: Bytes('data?')
 option: Str('in?')
+option: Flag('override_password?', autofill=True, default=False)
 option: Str('password?', cli_name='password')
 option: Str('password_file?', cli_name='password_file')
 option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
@@ -5528,6 +5529,30 @@ output: ListOfEntries('result', (type 'list', type 
'tuple'), Gettext('A list
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: Output('truncated', type 'bool', None)
 command: vault_mod
+args: 1,18,3
+arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Flag('change_password?', autofill=True, default=False)
+option: Str('description?', cli_name='desc')
+option: Bytes('ipavaultpublickey?', cli_name='public_key')
+option: Bytes('ipavaultsalt?', cli_name='salt')
+option: Str('ipavaulttype?', cli_name='type')
+option: Str('new_password?', cli_name='new_password')
+option: Str('new_password_file?', cli_name='new_password_file')
+option: Str('old_password?', cli_name='old_password')
+option: Str('old_password_file?', cli_name='old_password_file')
+option: Bytes('private_key?', cli_name='private_key')
+option: Str('private_key_file?', cli_name='private_key_file')
+option: Str('public_key_file?', cli_name='public_key_file')
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Str('service?')
+option: Flag('shared?', autofill=True, default=False)
+option: Str('username?', cli_name='user')
+option: Str('version?', exclude='webui')
+output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
+output: Output('summary', (type 'unicode', type 'NoneType'), None)
+output: PrimaryKey('value', None, None)
+command: vault_mod_internal
 args: 1,15,3
 arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
 option: Str('addattr*', cli_name='addattr', exclude='webui')
diff --git a/VERSION b/VERSION
index 
c42bea06522dae55e1a89ff94ae394594086b467..feb9f4db92c7c7b95e9e5d5907b1f97e96b26886
 100644
--- a/VERSION
+++ b/VERSION
@@ -90,5 +90,5 @@ IPA_DATA_VERSION=2010061412
 #  

Re: [Freeipa-devel] [PATCH] 369 Added CLI param and ACL for vault service operations.

2015-08-13 Thread Endi Sukma Dewata

On 8/13/2015 6:00 AM, Petr Vobornik wrote:

On 08/11/2015 08:42 AM, Jan Cholasta wrote:

On 10.8.2015 21:12, Endi Sukma Dewata wrote:

On 8/4/2015 10:32 AM, Endi Sukma Dewata wrote:

Martin, I do not think going on with business as usual is the right
thing to do here. We know this is going to bite.
I suggest Endy adds a *new* API if making it backwards compatible is
not
possible. The era of bumping whole API version must stop, the sooner
the
better.


My point is that we do not know yet how to do this kind of changes
long term.
So what I did not want to end up are 2 copypasted Vault plugins
maintained
forever, differing in just that.

If you know how to do this without copypasting, I will be fine with
that.


We probably can do it like this:
* the old plugin continues to provide Vault 1.0 functionality
* the new plugin will be a proxy to the old plugin except for the parts
that have changed in Vault 1.1.

Or the other way around:
* the new plugin will provide Vault 1.1 functionality
* the old plugin will be a proxy to the new plugin except for the parts
that needs to be maintained for Vault 1.0.

The first option is probably safer.

In any case, IPA 4.2.1 will only provide a single client for Vault 1.1,
but two services for Vault 1.0 and 1.1.


A new patch #369-1 is attached. It has been rebased on top of #372 and
#373 that fix the conflicting parameter while maintaining backward
compatibility.


I have modified the first version of the patch to maintain backward
compatibility and not require your patches #372 and #373. Should be much
easier to review. See attachment.


Jan approach seems better to me for 4.2.1. Endi, do you agree with the
changes? Could we proceed with the review?


Yes, please see the attached patch. I had to update it to remove buggy 
code and revised the docs. I also had to rebase my other patches to make 
sure they work with this patch.


--
Endi S. Dewata
From c43df23159e5dafd47d5309b3b0f75de4870640b Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Tue, 11 Aug 2015 08:19:59 +0200
Subject: [PATCH] Added CLI param and ACL for vault service operations.

The CLIs to manage vault owners and members have been modified
to accept services with a new parameter. Due to name conflict,
the existing 'service' parameter has been renamed to
'servicename'.

A new ACL has been added to allow a service to create its own
service container.

https://fedorahosted.org/freeipa/ticket/5172
---
 API.txt|  12 ++-
 VERSION|   4 +-
 install/share/vault.update |   1 +
 ipalib/plugins/vault.py| 179 +
 4 files changed, 95 insertions(+), 101 deletions(-)

diff --git a/API.txt b/API.txt
index 
04f2f894f7667239d94a2a7278d0cc80704afeb5..9dbf86aedf2a1b62dabab21fb30bbceb2f0f237b
 100644
--- a/API.txt
+++ b/API.txt
@@ -5434,13 +5434,14 @@ output: Entry('result', type 'dict', Gettext('A 
dictionary representing an LDA
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
 command: vault_add_member
-args: 1,9,3
+args: 1,10,3
 arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
 option: Str('group*', alwaysask=True, cli_name='groups', csv=True)
 option: Flag('no_members', autofill=True, default=False, exclude='webui')
 option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
 option: Str('service?')
+option: Str('services', alwaysask=True, cli_name='services', csv=True, 
multivalue=True, required=False)
 option: Flag('shared?', autofill=True, default=False)
 option: Str('user*', alwaysask=True, cli_name='users', csv=True)
 option: Str('username?', cli_name='user')
@@ -5449,13 +5450,14 @@ output: Output('completed', type 'int', None)
 output: Output('failed', type 'dict', None)
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 command: vault_add_owner
-args: 1,9,3
+args: 1,10,3
 arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
 option: Str('group*', alwaysask=True, cli_name='groups', csv=True)
 option: Flag('no_members', autofill=True, default=False, exclude='webui')
 option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
 option: Str('service?')
+option: Str('services', alwaysask=True, cli_name='services', csv=True, 
multivalue=True, required=False)
 option: Flag('shared?', autofill=True, default=False)
 option: Str('user*', alwaysask=True, cli_name='users', csv=True)
 option: Str('username?', cli_name='user')
@@ -5547,13 +5549,14 @@ output: Entry('result

[Freeipa-devel] [PATCH] 373 Fixed conflicting vault 'service' option.

2015-08-10 Thread Endi Sukma Dewata

A new vault API has been added to rename the 'service' option to
'servicename' to avoid conflicts with 'service' member in a future
patch. The old API is retained for backward compatibility, but the
implementation has been changed to invoke the new API.

https://fedorahosted.org/freeipa/ticket/5193

--
Endi S. Dewata
From 157d38f6954d5453eb494a542f027cc30eb4ea0d Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Thu, 6 Aug 2015 21:46:27 +0200
Subject: [PATCH] Fixed conflicting vault 'service' option.

A new vault API has been added to rename the 'service' option to
'servicename' to avoid conflicts with 'service' member in a future
patch. The old API is retained for backward compatibility, but the
implementation has been changed to invoke the new API.

https://fedorahosted.org/freeipa/ticket/5193
---
 API.txt| 230 -
 VERSION|   4 +-
 ipalib/plugins/vault.py| 595 +++-
 ipatests/test_xmlrpc/test_vault2_plugin.py | 719 +
 4 files changed, 1435 insertions(+), 113 deletions(-)
 create mode 100644 ipatests/test_xmlrpc/test_vault2_plugin.py

diff --git a/API.txt b/API.txt
index 
04f2f894f7667239d94a2a7278d0cc80704afeb5..9a777bd029d88f6882a9db341822544c6d1e7b5a
 100644
--- a/API.txt
+++ b/API.txt
@@ -5396,6 +5396,232 @@ option: Str('version?', exclude='webui')
 output: Output('result', type 'bool', None)
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
+command: vault2_add
+args: 1,14,3
+arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
+option: Str('addattr*', cli_name='addattr', exclude='webui')
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Str('description?', cli_name='desc')
+option: Bytes('ipavaultpublickey?', cli_name='public_key')
+option: Str('ipavaulttype?', cli_name='type')
+option: Str('password?', cli_name='password')
+option: Str('password_file?', cli_name='password_file')
+option: Str('public_key_file?', cli_name='public_key_file')
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Str('servicename?', cli_name='service')
+option: Str('setattr*', cli_name='setattr', exclude='webui')
+option: Flag('shared?', autofill=True, default=False)
+option: Str('username?', cli_name='user')
+option: Str('version?', exclude='webui')
+output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
+output: Output('summary', (type 'unicode', type 'NoneType'), None)
+output: PrimaryKey('value', None, None)
+command: vault2_add_internal
+args: 1,13,3
+arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, required=True)
+option: Str('addattr*', cli_name='addattr', exclude='webui')
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Str('description', attribute=True, cli_name='desc', multivalue=False, 
required=False)
+option: Bytes('ipavaultpublickey', attribute=True, cli_name='public_key', 
multivalue=False, required=False)
+option: Bytes('ipavaultsalt', attribute=True, cli_name='salt', 
multivalue=False, required=False)
+option: Str('ipavaulttype', attribute=True, autofill=True, cli_name='type', 
default=u'standard', multivalue=False, required=False)
+option: Flag('no_members', autofill=True, default=False, exclude='webui')
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Str('servicename?', cli_name='service')
+option: Str('setattr*', cli_name='setattr', exclude='webui')
+option: Flag('shared?', autofill=True, default=False)
+option: Str('username?', cli_name='user')
+option: Str('version?', exclude='webui')
+output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
+output: Output('summary', (type 'unicode', type 'NoneType'), None)
+output: PrimaryKey('value', None, None)
+command: vault2_add_member
+args: 1,9,3
+arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Str('group*', alwaysask=True, cli_name='groups', csv=True)
+option: Flag('no_members', autofill=True, default=False, exclude='webui')
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Str('servicename?', cli_name='service')
+option: Flag('shared?', autofill=True, default=False)
+option: Str('user*', alwaysask=True, cli_name='users', csv=True)
+option: Str('username?', cli_name='user')
+option: Str('version?', 

[Freeipa-devel] [PATCH] 372 Added attribute to specify command's CLI name.

2015-08-10 Thread Endi Sukma Dewata

A new attribute has been added to the HasParams class to allow a
command to specify a different CLI name if necessary. By default
the command's CLI name is the same as the class name.

https://fedorahosted.org/freeipa/ticket/5189

--
Endi S. Dewata
From 6808e44b97148db8700e47d1d436dd0a30a0c9f9 Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Fri, 7 Aug 2015 05:00:14 +0200
Subject: [PATCH] Added attribute to specify command's CLI name.

A new attribute has been added to the HasParams class to allow a
command to specify a different CLI name if necessary. By default
the command's CLI name is the same as the class name.

https://fedorahosted.org/freeipa/ticket/5189
---
 ipalib/backend.py  | 15 --
 ipalib/cli.py  | 59 ++
 ipalib/frontend.py | 19 ++
 3 files changed, 65 insertions(+), 28 deletions(-)

diff --git a/ipalib/backend.py b/ipalib/backend.py
index 
d510bc73396cd8b219836d28083e3165f226d466..48a4445c34a861d65e80f1fdaf711cbd0f643ae2
 100644
--- a/ipalib/backend.py
+++ b/ipalib/backend.py
@@ -121,12 +121,23 @@ class Executioner(Backend):
 def destroy_context(self):
 destroy_context()
 
+def find_command(self, cli_name):
+# TODO: replace loop with a dict to map CLI names to commands
+for name in self.Command:
+cmd = self.Command[name]
+if cmd.NO_CLI:
+continue
+if cmd.cli_name() == cli_name:
+return cmd
+return None
+
 def execute(self, _name, *args, **options):
 error = None
 try:
-if _name not in self.Command:
+cmd = self.find_command(_name)
+if not cmd:
 raise CommandError(name=_name)
-result = self.Command[_name](*args, **options)
+result = cmd(*args, **options)
 except PublicError, e:
 error = e
 except StandardError, e:
diff --git a/ipalib/cli.py b/ipalib/cli.py
index 
4104e6482e4e713d701c6c1a4313ab6ecc899057..0fa00d3a6e32fc6095ca3b76270355cf9cb3dd2d
 100644
--- a/ipalib/cli.py
+++ b/ipalib/cli.py
@@ -771,32 +771,40 @@ class help(frontend.Local):
 if outfile is None:
 outfile = sys.stdout
 writer = self._writer(outfile)
-name = from_cli(key)
-mod_name = '%s.%s' % (self._PLUGIN_BASE_MODULE, name)
+
 if key is None:
 self.api.parser.print_help(outfile)
 return
-if name == topics:
+
+if key == topics:
 self.print_topics(outfile)
 return
-if name in self._topics:
-self.print_commands(name, outfile)
-elif name in self.Command:
-cmd = self.Command[name]
-if cmd.NO_CLI:
-raise HelpError(topic=name)
-self.Backend.cli.build_parser(cmd).print_help(outfile)
-elif mod_name in sys.modules:
-self.print_commands(name, outfile)
-elif name == commands:
+
+if key in self._topics:
+self.print_commands(key, outfile)
+return
+
+if key == commands:
 mcl = max(len(s) for s in (self.Command))
 for cname in self.Command:
 cmd = self.Command[cname]
 if cmd.NO_CLI:
 continue
-writer('%s  %s' % (to_cli(cmd.name).ljust(mcl), cmd.summary))
-else:
-raise HelpError(topic=name)
+writer('%s  %s' % (cmd.cli_name().ljust(mcl), cmd.summary))
+return
+
+cmd = self.find_command(key)
+if cmd:
+self.Backend.cli.build_parser(cmd).print_help(outfile)
+return
+
+name = from_cli(key)
+mod_name = '%s.%s' % (self._PLUGIN_BASE_MODULE, name)
+if mod_name in sys.modules:
+self.print_commands(name, outfile)
+return
+
+raise HelpError(topic=key)
 
 def _writer(self, outfile):
 def writer(string=''):
@@ -848,7 +856,7 @@ class help(frontend.Local):
 writer(_('Topic commands:'))
 for c in commands:
 writer(
-'  %s  %s' % (to_cli(c.name).ljust(mcl), c.summary))
+'  %s  %s' % (c.cli_name().ljust(mcl), c.summary))
 writer()
 writer(_('To get command help, use:'))
 writer(_('  ipa command --help'))
@@ -866,10 +874,10 @@ class show_mappings(frontend.Command):
 has_output = tuple()
 
 def run(self, command_name, **options):
-command_name = from_cli(command_name)
-if command_name not in self.Command:
+cmd = self.find_command(command_name)
+if not cmd:
 raise CommandError(name=command_name)
-params = self.Command[command_name].options
+params = cmd.options
 out = [('Parameter','LDAP attribute'),
 

Re: [Freeipa-devel] [PATCH] 369 Added CLI param and ACL for vault service operations.

2015-08-10 Thread Endi Sukma Dewata

On 8/4/2015 10:32 AM, Endi Sukma Dewata wrote:

Martin, I do not think going on with business as usual is the right
thing to do here. We know this is going to bite.
I suggest Endy adds a *new* API if making it backwards compatible is not
possible. The era of bumping whole API version must stop, the sooner the
better.


My point is that we do not know yet how to do this kind of changes
long term.
So what I did not want to end up are 2 copypasted Vault plugins
maintained
forever, differing in just that.

If you know how to do this without copypasting, I will be fine with that.


We probably can do it like this:
* the old plugin continues to provide Vault 1.0 functionality
* the new plugin will be a proxy to the old plugin except for the parts
that have changed in Vault 1.1.

Or the other way around:
* the new plugin will provide Vault 1.1 functionality
* the old plugin will be a proxy to the new plugin except for the parts
that needs to be maintained for Vault 1.0.

The first option is probably safer.

In any case, IPA 4.2.1 will only provide a single client for Vault 1.1,
but two services for Vault 1.0 and 1.1.


A new patch #369-1 is attached. It has been rebased on top of #372 and 
#373 that fix the conflicting parameter while maintaining backward 
compatibility.


--
Endi S. Dewata
From 7f461c8fe5d567e9ddad3684a60037cdd90e833c Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Thu, 30 Jul 2015 23:20:34 +0200
Subject: [PATCH] Added CLI param and ACL for vault service operations.

The CLIs to manage vault owners and members have been modified
to accept services in addition to users and groups. A new ACL
has been added to allow a service to create its own service
container.

https://fedorahosted.org/freeipa/ticket/5172
---
 API.txt| 12 
 VERSION|  4 ++--
 install/share/vault.update |  1 +
 ipalib/plugins/vault.py| 21 +++--
 4 files changed, 26 insertions(+), 12 deletions(-)

diff --git a/API.txt b/API.txt
index 
9a777bd029d88f6882a9db341822544c6d1e7b5a..81527bf60bb440ddfdacb25d63e211b154182487
 100644
--- a/API.txt
+++ b/API.txt
@@ -5436,12 +5436,13 @@ output: Entry('result', type 'dict', Gettext('A 
dictionary representing an LDA
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
 command: vault2_add_member
-args: 1,9,3
+args: 1,10,3
 arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
 option: Str('group*', alwaysask=True, cli_name='groups', csv=True)
 option: Flag('no_members', autofill=True, default=False, exclude='webui')
 option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Str('service*', alwaysask=True, cli_name='services', csv=True)
 option: Str('servicename?', cli_name='service')
 option: Flag('shared?', autofill=True, default=False)
 option: Str('user*', alwaysask=True, cli_name='users', csv=True)
@@ -5451,12 +5452,13 @@ output: Output('completed', type 'int', None)
 output: Output('failed', type 'dict', None)
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 command: vault2_add_owner
-args: 1,9,3
+args: 1,10,3
 arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
 option: Str('group*', alwaysask=True, cli_name='groups', csv=True)
 option: Flag('no_members', autofill=True, default=False, exclude='webui')
 option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Str('service*', alwaysask=True, cli_name='services', csv=True)
 option: Str('servicename?', cli_name='service')
 option: Flag('shared?', autofill=True, default=False)
 option: Str('user*', alwaysask=True, cli_name='users', csv=True)
@@ -5549,12 +5551,13 @@ output: Entry('result', type 'dict', Gettext('A 
dictionary representing an LDA
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
 command: vault2_remove_member
-args: 1,9,3
+args: 1,10,3
 arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
 option: Str('group*', alwaysask=True, cli_name='groups', csv=True)
 option: Flag('no_members', autofill=True, default=False, exclude='webui')
 option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Str('service*', alwaysask=True, cli_name='services', csv=True)
 option: Str('servicename?', cli_name

[Freeipa-devel] Installation issue on F23

2015-08-05 Thread Endi Sukma Dewata

Hi,

Just FYI, the recent IPA installation issue on F23 has been fixed:
https://bugzilla.redhat.com/show_bug.cgi?id=1250724

by installing a new TomcatJSS package:
https://admin.fedoraproject.org/updates/tomcatjss-7.1.3-1.fc23

The PKI dependency on TomcatJSS will be updated in the following ticket:
https://fedorahosted.org/pki/ticket/1542

Once this is completed, please update the IPA dependency on PKI. Thanks.

--
Endi S. Dewata

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] [PATCH] 369 Added CLI param and ACL for vault service operations.

2015-08-04 Thread Endi Sukma Dewata

On 8/4/2015 8:51 AM, Martin Kosek wrote:

Please also note that my next patch that adds the ability to change vault type,
password, and keys will also require a client upgrade because the functionality
is mainly implemented on the client side. In this case API URL versioning will
be necessary.


Adding new commands and/or attributes is a common thing in FreeIPA. We just do
the work, bump the minor API version and that's it. We planned having better
version support in FreeIPA 4.4, we will see how it goes.


Martin, I do not think going on with business as usual is the right
thing to do here. We know this is going to bite.
I suggest Endy adds a *new* API if making it backwards compatible is not
possible. The era of bumping whole API version must stop, the sooner the
better.


My point is that we do not know yet how to do this kind of changes long term.
So what I did not want to end up are 2 copypasted Vault plugins maintained
forever, differing in just that.

If you know how to do this without copypasting, I will be fine with that.


We probably can do it like this:
* the old plugin continues to provide Vault 1.0 functionality
* the new plugin will be a proxy to the old plugin except for the parts 
that have changed in Vault 1.1.


Or the other way around:
* the new plugin will provide Vault 1.1 functionality
* the old plugin will be a proxy to the new plugin except for the parts 
that needs to be maintained for Vault 1.0.


The first option is probably safer.

In any case, IPA 4.2.1 will only provide a single client for Vault 1.1, 
but two services for Vault 1.0 and 1.1.


--
Endi S. Dewata

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] [PATCH] 369 Added CLI param and ACL for vault service operations.

2015-08-03 Thread Endi Sukma Dewata

On 8/3/2015 2:31 AM, Martin Kosek wrote:

On 07/31/2015 05:07 PM, Endi Sukma Dewata wrote:

The CLIs to manage vault owners and members have been modified
to accept services with a new parameter. Due to name conflict,
the existing 'service' parameter has been renamed to
'servicename'.

A new ACL has been added to allow a service to create its own
service container.

https://fedorahosted.org/freeipa/ticket/5172


I did not do a full review, I just saw some of the changes that made me worry -
like renaming API command attribute. Wouldn't that make the older clients fail?

See for example permission-* commands, we faced similar situation there and had
to rename command attributes potentially used by old clients in the new format.


Yes, it will break older clients. The problem is I cannot keep the 
legacy 'service' parameter for backward compatibility:


Str(
'service?',
cli_name='deprecated_service',
doc=_('DEPRECATED: Service name of the service vault'),
),
Str(
'servicename?',
cli_name='service',
doc=_('Service name of the service vault'),
),

because it will conflict with a new 'service' parameter:

[Mon Aug 03 17:19:00.197040 2015] [wsgi:error] [pid 9409] ipa: ERROR: 
Failed to start IPA: cannot override NameSpace.service value 
Str('service?', cli_name='deprecated_service', doc=Gettext('DEPRECATED: 
Service name of the service vault', domain='ipa', localedir=None)) with 
Str('service*', alwaysask=True, cli_name='services', csv=True, 
doc=u'services to add', label=u'member service')


that was added automatically when I added the 'service' attribute member:

attribute_members = {
'owner': ['user', 'group', 'service'],
'member': ['user', 'group', 'service'],
}

If there's a way to use a different parameter name for the 'service' 
attribute member to avoid conflict with the legacy 'service' parameter 
please let me know.


The other option is to force the client to upgrade to the same version.

Please let me know. Thanks.

--
Endi S. Dewata

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


[Freeipa-devel] [PATCH] 371 Added support for changing vault encryption.

2015-08-03 Thread Endi Sukma Dewata

The vault-mod command has been modified to support changing vault
encryption attributes (i.e. type, password, public/private keys)
in addition to normal attributes (i.e. description). Changing the
encryption requires retrieving the stored secret with the old
attributes and rearchieving it with the new attributes.

https://fedorahosted.org/freeipa/ticket/5176

--
Endi S. Dewata
From e80928fa8e8a099576fcdbff08fd90a634600825 Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Fri, 31 Jul 2015 07:53:15 +0200
Subject: [PATCH] Added support for changing vault encryption.

The vault-mod command has been modified to support changing vault
encryption attributes (i.e. type, password, public/private keys)
in addition to normal attributes (i.e. description). Changing the
encryption requires retrieving the stored secret with the old
attributes and rearchieving it with the new attributes.

https://fedorahosted.org/freeipa/ticket/5176
---
 API.txt |  27 +-
 VERSION |   4 +-
 ipalib/plugins/vault.py | 239 
 3 files changed, 248 insertions(+), 22 deletions(-)

diff --git a/API.txt b/API.txt
index 
00b47b8709c81217d7f2a69e719093f4a04f1734..bd5c48998056ab94729cb1b475bf444707a883cb
 100644
--- a/API.txt
+++ b/API.txt
@@ -5466,11 +5466,12 @@ output: Output('completed', type 'int', None)
 output: Output('failed', type 'dict', None)
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 command: vault_archive
-args: 1,10,3
+args: 1,11,3
 arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
 option: Bytes('data?')
 option: Str('in?')
+option: Flag('override_password?', autofill=True, default=False)
 option: Str('password?', cli_name='password')
 option: Str('password_file?', cli_name='password_file')
 option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
@@ -5528,6 +5529,30 @@ output: ListOfEntries('result', (type 'list', type 
'tuple'), Gettext('A list
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: Output('truncated', type 'bool', None)
 command: vault_mod
+args: 1,18,3
+arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Flag('change_password?', autofill=True, default=False)
+option: Str('description?', cli_name='desc')
+option: Bytes('ipavaultsalt?', cli_name='salt')
+option: Str('ipavaulttype?', cli_name='type')
+option: Str('new_password?', cli_name='new_password')
+option: Str('new_password_file?', cli_name='new_password_file')
+option: Str('old_password?', cli_name='old_password')
+option: Str('old_password_file?', cli_name='old_password_file')
+option: Bytes('private_key?', cli_name='private_key')
+option: Str('private_key_file?', cli_name='private_key_file')
+option: Bytes('public_key?', cli_name='public_key')
+option: Str('public_key_file?', cli_name='public_key_file')
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Str('servicename?', cli_name='service')
+option: Flag('shared?', autofill=True, default=False)
+option: Str('username?', cli_name='user')
+option: Str('version?', exclude='webui')
+output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
+output: Output('summary', (type 'unicode', type 'NoneType'), None)
+output: PrimaryKey('value', None, None)
+command: vault_mod_internal
 args: 1,15,3
 arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
 option: Str('addattr*', cli_name='addattr', exclude='webui')
diff --git a/VERSION b/VERSION
index 
c42bea06522dae55e1a89ff94ae394594086b467..feb9f4db92c7c7b95e9e5d5907b1f97e96b26886
 100644
--- a/VERSION
+++ b/VERSION
@@ -90,5 +90,5 @@ IPA_DATA_VERSION=2010061412
 #  #
 
 IPA_API_VERSION_MAJOR=2
-IPA_API_VERSION_MINOR=149
-# Last change: edewata - Added CLI param and ACL for vault service operations
+IPA_API_VERSION_MINOR=150
+# Last change: edewata - Added support for changing vault encryption.
diff --git a/ipalib/plugins/vault.py b/ipalib/plugins/vault.py
index 
3b62822366a62c90f843a6293589c28383e782ef..5bee6aaae8ddd306d4ee0c273143c9d0ffc913d5
 100644
--- a/ipalib/plugins/vault.py
+++ b/ipalib/plugins/vault.py
@@ -133,19 +133,36 @@ EXAMPLES:
ipa vault-show name --shared
 ) + _(
  Show a user vault:
-   ipa vault-show name --user username
+ 

Re: [Freeipa-devel] [PATCH] 369 Added CLI param and ACL for vault service operations.

2015-08-03 Thread Endi Sukma Dewata

On 8/3/2015 2:47 PM, Martin Kosek wrote:

On 08/03/2015 05:36 PM, Endi Sukma Dewata wrote:

On 8/3/2015 2:31 AM, Martin Kosek wrote:

On 07/31/2015 05:07 PM, Endi Sukma Dewata wrote:

The CLIs to manage vault owners and members have been modified
to accept services with a new parameter. Due to name conflict,
the existing 'service' parameter has been renamed to
'servicename'.

A new ACL has been added to allow a service to create its own
service container.

https://fedorahosted.org/freeipa/ticket/5172


I did not do a full review, I just saw some of the changes that made
me worry -
like renaming API command attribute. Wouldn't that make the older
clients fail?

See for example permission-* commands, we faced similar situation
there and had
to rename command attributes potentially used by old clients in the
new format.


Yes, it will break older clients. The problem is I cannot keep the legacy
'service' parameter for backward compatibility:

 Str(
 'service?',
 cli_name='deprecated_service',
 doc=_('DEPRECATED: Service name of the service vault'),
 ),
 Str(
 'servicename?',
 cli_name='service',
 doc=_('Service name of the service vault'),
 ),

because it will conflict with a new 'service' parameter:

[Mon Aug 03 17:19:00.197040 2015] [wsgi:error] [pid 9409] ipa: ERROR:
Failed to
start IPA: cannot override NameSpace.service value Str('service?',
cli_name='deprecated_service', doc=Gettext('DEPRECATED: Service name
of the
service vault', domain='ipa', localedir=None)) with Str('service*',
alwaysask=True, cli_name='services', csv=True, doc=u'services to add',
label=u'member service')

that was added automatically when I added the 'service' attribute member:

 attribute_members = {
 'owner': ['user', 'group', 'service'],
 'member': ['user', 'group', 'service'],
 }

If there's a way to use a different parameter name for the 'service'
attribute
member to avoid conflict with the legacy 'service' parameter please
let me know.

The other option is to force the client to upgrade to the same version.

Please let me know. Thanks.



Honza, do we have any other options than to break API between 4.2.0 and
4.2.1?


Another option is to keep 2 vault plugins. The old plugin (1.0) will 
handle old IPA 4.2.0 clients. The new plugin (1.1) will handle the new 
IPA 4.2.1 clients. For this to work the plugins need to have different 
API URLs so they won't conflict/confuse the clients.


Please also note that my next patch that adds the ability to change 
vault type, password, and keys will also require a client upgrade 
because the functionality is mainly implemented on the client side. In 
this case API URL versioning will be necessary.


--
Endi S. Dewata

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


[Freeipa-devel] [PATCH] 370 Fixed missing KRA agent cert on replica.

2015-07-31 Thread Endi Sukma Dewata

The code that exports the KRA agent certificate has been moved
such that it will be executed both on master and replica.

https://fedorahosted.org/freeipa/ticket/5174

--
Endi S. Dewata
From 04abaf7354d5c140d2bb88c7a58e54f0691fcbe4 Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Sat, 1 Aug 2015 02:46:26 +0200
Subject: [PATCH] Fixed missing KRA agent cert on replica.

The code that exports the KRA agent certificate has been moved
such that it will be executed both on master and replica.

https://fedorahosted.org/freeipa/ticket/5174
---
 ipaserver/install/krainstance.py | 17 +
 1 file changed, 9 insertions(+), 8 deletions(-)

diff --git a/ipaserver/install/krainstance.py b/ipaserver/install/krainstance.py
index 
50ab424b0e59becfea9e7af4b8d43a32ccbdc823..fa50c3dec897d63b9d3522d196054163f7b3369a
 100644
--- a/ipaserver/install/krainstance.py
+++ b/ipaserver/install/krainstance.py
@@ -256,6 +256,15 @@ class KRAInstance(DogtagInstance):
 os.remove(cfg_file)
 
 shutil.move(paths.KRA_BACKUP_KEYS_P12, paths.KRACERT_P12)
+
+# export ipaCert with private key for client authentication
+args = [/usr/bin/pki,
+-d, paths.HTTPD_ALIAS_DIR,
+-C, paths.ALIAS_PWDFILE_TXT,
+client-cert-show, ipaCert,
+--client-cert, paths.KRA_AGENT_PEM]
+ipautil.run(args)
+
 self.log.debug(completed creating KRA instance)
 
 def __add_ra_user_to_agent_group(self):
@@ -330,14 +339,6 @@ class KRAInstance(DogtagInstance):
 finally:
 os.remove(filename)
 
-# export ipaCert with private key for client authentication
-args = [/usr/bin/pki,
--d, paths.HTTPD_ALIAS_DIR,
--C, paths.ALIAS_PWDFILE_TXT,
-client-cert-show, ipaCert,
---client-cert, paths.KRA_AGENT_PEM]
-ipautil.run(args)
-
 def __add_vault_container(self):
 sub_dict = {
 'SUFFIX': self.suffix,
-- 
2.4.3

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH] Password vault

2015-07-07 Thread Endi Sukma Dewata
- Original Message -
 On 07/07/2015 10:51 AM, Jan Cholasta wrote:
  Dne 3.7.2015 v 15:44 Endi Sukma Dewata napsal(a):
  Here is the rebased patch for vault access control.
 
  
  LGTM, except:
  
  @@ -356,6 +386,13 @@ class vault(LDAPObject):
   {
   'objectclass': ['nsContainer'],
   'cn': rdn['cn'],
  +'aci':
  +'(targetfilter=(objectClass=ipaVault))' +
  +'(version 3.0; ' +
  +'acl User can manage private vaults; ' +
  +'allow(read, search, compare, add, delete) ' +
  +'userdn=ldap:///%s;;)'
  +% owner_dn
   })
  
   # if entry can be added, return
  
  I don't think dynamically creating ACIs with hardcoded userdn is something
  we
  want to do. This should be handled by a single ACI in cn=vaults.
 
 +1. Single ACI like
 
 +default: aci: (targetfilter=(objectClass=ipaVault))(version 3.0; acl
 Vault
 owners can manage the vault; allow(read, search, compare, write)
 userattr=owner#USERDN;)
 
 you already have there is more preferred.

New patch attached. For this to work the container itself needs an 'owner' 
attribute, so I changed the nsContainer into ipaVaultContainer.

--
Endi S. Dewata
From 087f36b888e068ee732af6e0a2c24b1d50849ccd Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Fri, 17 Oct 2014 12:05:34 -0400
Subject: [PATCH] Added vault access control.

New LDAP ACIs have been added to allow vault owners to manage the
vaults and to allow members to access the vaults. New CLIs have
been added to manage the owner and member list. The LDAP schema
has been updated as well.

https://fedorahosted.org/freeipa/ticket/3872
---
 API.txt   |  92 +++
 VERSION   |   4 +-
 install/share/60basev3.ldif   |   3 +-
 install/share/vault.update|  15 +++-
 ipalib/plugins/vault.py   | 118 --
 ipatests/test_xmlrpc/test_vault_plugin.py |  27 +--
 6 files changed, 226 insertions(+), 33 deletions(-)

diff --git a/API.txt b/API.txt
index 99fa528733200fc3d797a9847b1d6df2188b92d5..98eaee8ce8b2804a6d34e42c3eff26ddb3851963 100644
--- a/API.txt
+++ b/API.txt
@@ -5422,27 +5422,58 @@ option: Flag('raw', autofill=True, cli_name='raw', default=False, exclude='webui
 option: Str('service?')
 option: Str('setattr*', cli_name='setattr', exclude='webui')
 option: Flag('shared?', autofill=True, default=False)
-option: Str('user?')
+option: Str('username?', cli_name='user')
 option: Str('version?', exclude='webui')
 output: Entry('result', type 'dict', Gettext('A dictionary representing an LDAP entry', domain='ipa', localedir=None))
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
 command: vault_add_internal
-args: 1,10,3
+args: 1,11,3
 arg: Str('cn', attribute=True, cli_name='name', maxlength=255, multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, required=True)
 option: Flag('all', autofill=True, cli_name='all', default=False, exclude='webui')
 option: Str('description', attribute=True, cli_name='desc', multivalue=False, required=False)
 option: Bytes('ipavaultpublickey', attribute=True, cli_name='public_key', multivalue=False, required=False)
 option: Bytes('ipavaultsalt', attribute=True, cli_name='salt', multivalue=False, required=False)
 option: Str('ipavaulttype', attribute=True, autofill=True, cli_name='type', default=u'standard', multivalue=False, required=False)
+option: Flag('no_members', autofill=True, default=False, exclude='webui')
 option: Flag('raw', autofill=True, cli_name='raw', default=False, exclude='webui')
 option: Str('service?')
 option: Flag('shared?', autofill=True, default=False)
-option: Str('user?')
+option: Str('username?', cli_name='user')
 option: Str('version?', exclude='webui')
 output: Entry('result', type 'dict', Gettext('A dictionary representing an LDAP entry', domain='ipa', localedir=None))
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
+command: vault_add_member
+args: 1,9,3
+arg: Str('cn', attribute=True, cli_name='name', maxlength=255, multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, exclude='webui')
+option: Str('group*', alwaysask=True, cli_name='groups', csv=True)
+option: Flag('no_members', autofill=True, default=False, exclude='webui')
+option: Flag('raw', autofill=True, cli_name='raw', default=False, exclude='webui')
+option: Str('service?')
+option: Flag('shared?', autofill=True, default=False)
+option: Str('user*', alwaysask=True, cli_name='users', csv=True)
+option: Str('username?', cli_name='user')
+option: Str

Re: [Freeipa-devel] [PATCH] Password vault

2015-07-03 Thread Endi Sukma Dewata

On 7/1/2015 1:53 AM, Jan Cholasta wrote:

I think it would be better to use a new attribute type which
inherits
from ipaPublicKey (ipaVaultPublicKey?) rather than ipaPublicKey
directly
for assymetric vault public keys, so that assymetric public key and
escrow public key are on the same level and you can still use
ipaPublicKey to refer to either one:

 ipaPublicKey
 ipaVaultPublicKey
 ipaEscrowPublicKey


OK. To be consistent the parameters need to be renamed too:
--vault-public-key and --vault-public-key-file.


It doesn't need to, there is no requirement for CLI names to always
match attribute names. (Also I don't insist on the name
ipaVaultPublicKey, feel free to change it if you want.)


It's unchanged for now. In a previous discussion it was advised to
reuse
the existing attribute type whenever possible.


Well, in this discussion, it is not. Escrow public key should also reuse
ipaPublicKey, but it can't if you use it for vault public key. By using
ipaPublicKey subtypes you can distinguish between the two uses and still
use ipaPublicKey to refer to either of them.


So what's changed? This is what you said when I posted the same patch
six months ago:


In this patch I'm adding ipaVaultSalt and ipaVaultPublicKey attribute
types to store salt and public key for vault. Are there existing
attribute types that I can use instead? I see there's an ipaPublicKey,
should I use that and maybe add ipaSalt/ipaEncSalt? Thanks.


yes, please re-use existing attributes where possible.

Honza


What changed is that I now know there is also escrow public key, which I
didn't know six months ago.


Here's patch #368 to be applied on top of patch #357-5, but see comments 
below.



Could you show me how to use ipaPublicKey to refer to ipaVaultPublicKey
and ipaEscrowPublicKey? Under what situation would that be useful?


For example for ipaPublicKey searches - if ipaVaultPublicKey and
ipaEscrowPublicKey both inherit from ipaPublicKey, then an ipaPublicKey
search will look in both ipaVaultPublicKey and ipaEscrowPublicKey. This
is not something we actually need right now, but once the schema is
done, it can't be fixed and I don't think we should prevent this,
especially since we can get it for free. BTW even the core LDAP schema
does this, see for example how the cn attribute inherits from the more
general name attribute: https://tools.ietf.org/html/rfc4519#section-2.3.


I don't think that's how LDAP works. The RFC doesn't say that either. 
The cn does inherit from name, but if you search for name it won't 
match/return cn. See queries below:


$ ldapsearch -LLL -x -b dc=example,dc=com (cn=Accounting Managers)
dn: cn=Accounting Managers,ou=Groups,dc=example,dc=com
objectClass: top
objectClass: groupOfUniqueNames
cn: Accounting Managers
ou: groups
description: People who can manage accounting entries
uniqueMember: cn=Directory Manager

$ ldapsearch -LLL -x -b dc=example,dc=com (cn=Accounting Managers) \
  name
dn: cn=Accounting Managers,ou=Groups,dc=example,dc=com
(no cn attribute)

$ ldapsearch -LLL -x -b dc=example,dc=com (name=Accounting Managers)
(no result)

Assuming this is what you meant, which doesn't seem to be working, is 
there still a valid reason to add a new ipaVaultPublicKey instead of 
using the existing ipaPublicKey?



* CLI options will be identical to client and server API options (i.e.
no CLI-only, client-only, or server-only options)


Actually, you can create CLI-only options (add include='cli' to the
param's kwargs).


I need to look at this more closely. If I understand correctly in
user_del there are two 'preserve' options, the Bool preserve is for
client and server API, and the Flag preserve is for CLI. Wouldn't it be
better if they are stored in separate lists (or maybe separate classes)?
And it looks like you still need to delete the CLI options explicitly
anyway.


Well, it would be better if there was no Flag class at all and flags
were handled by CLI exclusively, because parameter classes should
reflect the data type (bool) and not the presentation (flag).


That indicates there should be a separation between client API and the 
CLI too because, as you see in user_del, they can be different.



Does the API.txt actually show the CLI options, the client API options,
or the server API options? I only see the Flag preserve, not the Bool
preserve.


It shows CLI options, see how the API object is initialized in makeapi.


Does that mean we're only doing the versioning on the CLI, and not the 
client API or server API? Suppose there are changes in client or server 
API that do not appear in API.txt but will affect the XML RPC, it might 
cause a compatibility problem. I think it just shows how convoluted the 
CLI, client API, and server API are in this framework.



* a plugin will only access one type of data (i.e. LDAP plugin can only
access LDAP data)


This is not assumed anywhere in the framework, you can access whatever
you want, but you can't expect baseldap to do 

Re: [Freeipa-devel] [PATCH] Password vault

2015-07-03 Thread Endi Sukma Dewata

Here is the rebased patch for vault access control.

--
Endi S. Dewata
From 6bec99d51552a6415c45d655f95627e341fae44b Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Fri, 17 Oct 2014 12:05:34 -0400
Subject: [PATCH] Added vault access control.

New LDAP ACIs have been added to allow vault owners to manage the
vaults and to allow members to access the vaults. New CLIs have
been added to manage the owner and member list. The LDAP schema
has been updated as well.

https://fedorahosted.org/freeipa/ticket/3872
---
 API.txt   |  92 ++
 VERSION   |   4 +-
 install/share/60basev3.ldif   |   2 +-
 install/share/vault.update|   5 ++
 ipalib/plugins/vault.py   | 122 --
 ipatests/test_xmlrpc/test_vault_plugin.py |  27 +--
 6 files changed, 224 insertions(+), 28 deletions(-)

diff --git a/API.txt b/API.txt
index 
d0ae1b72c2ae445a4e2cc168da5fd53f9a4de56d..c182098fe1017d46f9f7980c7b6891a1031f1068
 100644
--- a/API.txt
+++ b/API.txt
@@ -5341,27 +5341,58 @@ option: Flag('raw', autofill=True, cli_name='raw', 
default=False, exclude='webui
 option: Str('service?')
 option: Str('setattr*', cli_name='setattr', exclude='webui')
 option: Flag('shared?', autofill=True, default=False)
-option: Str('user?')
+option: Str('username?', cli_name='user')
 option: Str('version?', exclude='webui')
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
 command: vault_add_internal
-args: 1,10,3
+args: 1,11,3
 arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, required=True)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
 option: Str('description', attribute=True, cli_name='desc', multivalue=False, 
required=False)
 option: Bytes('ipavaultpublickey', attribute=True, cli_name='public_key', 
multivalue=False, required=False)
 option: Bytes('ipavaultsalt', attribute=True, cli_name='salt', 
multivalue=False, required=False)
 option: Str('ipavaulttype', attribute=True, autofill=True, cli_name='type', 
default=u'standard', multivalue=False, required=False)
+option: Flag('no_members', autofill=True, default=False, exclude='webui')
 option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
 option: Str('service?')
 option: Flag('shared?', autofill=True, default=False)
-option: Str('user?')
+option: Str('username?', cli_name='user')
 option: Str('version?', exclude='webui')
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
+command: vault_add_member
+args: 1,9,3
+arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Str('group*', alwaysask=True, cli_name='groups', csv=True)
+option: Flag('no_members', autofill=True, default=False, exclude='webui')
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Str('service?')
+option: Flag('shared?', autofill=True, default=False)
+option: Str('user*', alwaysask=True, cli_name='users', csv=True)
+option: Str('username?', cli_name='user')
+option: Str('version?', exclude='webui')
+output: Output('completed', type 'int', None)
+output: Output('failed', type 'dict', None)
+output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
+command: vault_add_owner
+args: 1,9,3
+arg: Str('cn', attribute=True, cli_name='name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Str('group*', alwaysask=True, cli_name='groups', csv=True)
+option: Flag('no_members', autofill=True, default=False, exclude='webui')
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Str('service?')
+option: Flag('shared?', autofill=True, default=False)
+option: Str('user*', alwaysask=True, cli_name='users', csv=True)
+option: Str('username?', cli_name='user')
+option: Str('version?', exclude='webui')
+output: Output('completed', type 'int', None)
+output: Output('failed', type 'dict', None)
+output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 command: vault_archive
 args: 1,10,3
 arg: Str('cn', attribute=True, cli_name='name', 

Re: [Freeipa-devel] [PATCH] Password vault

2015-06-25 Thread Endi Sukma Dewata

On 6/25/2015 12:35 AM, Jan Cholasta wrote:

I think it would be better to use a new attribute type which inherits
from ipaPublicKey (ipaVaultPublicKey?) rather than ipaPublicKey
directly
for assymetric vault public keys, so that assymetric public key and
escrow public key are on the same level and you can still use
ipaPublicKey to refer to either one:

 ipaPublicKey
 ipaVaultPublicKey
 ipaEscrowPublicKey


OK. To be consistent the parameters need to be renamed too:
--vault-public-key and --vault-public-key-file.


It doesn't need to, there is no requirement for CLI names to always
match attribute names. (Also I don't insist on the name
ipaVaultPublicKey, feel free to change it if you want.)


It's unchanged for now. In a previous discussion it was advised to reuse
the existing attribute type whenever possible.


Well, in this discussion, it is not. Escrow public key should also reuse
ipaPublicKey, but it can't if you use it for vault public key. By using
ipaPublicKey subtypes you can distinguish between the two uses and still
use ipaPublicKey to refer to either of them.


So what's changed? This is what you said when I posted the same patch 
six months ago:



In this patch I'm adding ipaVaultSalt and ipaVaultPublicKey attribute
types to store salt and public key for vault. Are there existing
attribute types that I can use instead? I see there's an ipaPublicKey,
should I use that and maybe add ipaSalt/ipaEncSalt? Thanks.


yes, please re-use existing attributes where possible.

Honza


Could you show me how to use ipaPublicKey to refer to ipaVaultPublicKey 
and ipaEscrowPublicKey? Under what situation would that be useful?



  a) When the non-split vault_{archive,retrieve} was called from a
server API with client-only options, it crashed. This is the broken API
I was talking about.


This is because in the current framework any API called on the server
side will be a server API, so you are not supposed to call it with
client options in the first place. Because of that limitation, the only
way to use client options is to use a separate API on the client side to
call the original API on the server side. The point is, client options
belong to client API, and server options belong to server API. In
vault_add the public key file name belongs to client API because it's
used to load a file on the client side. You should not add public key
file name option to the server API just because it can safely be ignored.


I don't disagree, but file name options do not belong to the general
client API either, as they are strictly CLI-specific.


To my understanding the current framework doesn't have a separate CLI 
class, so you don't have a choice but to put CLI-specific options in the 
client API class too. However, you do have a choice not to combine 
client API class and server API class because otherwise that will put 
CLI-specific options in the server API class too.



2. Since the vault_archive_internal inherits from Update, it accepts
all non
primary-key attributes automatically. This is incorrect since we
don't want to
update these parameters during archival. Can this behavior be
overridden?


Inherit from PKQuery instead (don't forget to add has_output =
output.standard_entry).


Previously you didn't want to use LDAPQuery because of semantics
reasons. Is PKQuery fine semantically?


It's not. Currently there is a set of commands which operate on the LDAP
part of vault and another set of commands which operate on the KRA part
of vault and we don't want the commands in one set to see attributes
related to the other part of vault. If you insist on keeping both parts
in a single object, you have to resort to hackery like using PKQuery,
hence my suggestion to split the data part off to a separate object to
avoid this.


This because the framework was based on simplistic assumptions which
create unnecessary restrictions, for example:
* client API is just a proxy to server API (i.e. client and server
cannot do different things)


They can do different things the same way vault_archive/vault_retrieve
does that, the commands just can't be called the same (which is not
necessarily a bad thing).


Of course different APIs can do different things, like vault_add calling 
vault_archive, or vault_archive calling vault_archive_internal. The 
point is right now the client portion of an API (i.e. the forward() 
method) cannot do anything other than forwarding the request to the 
server, so the API has to be split into different APIs:


 * vault_archive
 * vault_archive_internal

It would be nice to have formal separation between client and server 
APIs so it's clear they are different but still related without 
resorting to ugly names:


 * client.vault_archive
 * server.vault_archive


* CLI options will be identical to client and server API options (i.e.
no CLI-only, client-only, or server-only options)


Actually, you can create CLI-only options (add include='cli' to the
param's kwargs).


I need to 

Re: [Freeipa-devel] [PATCH] Password vault

2015-06-22 Thread Endi Sukma Dewata

Please take a look at the new patch.

On 6/17/2015 1:32 AM, Jan Cholasta wrote:

I think it would be better to use a new attribute type which inherits
from ipaPublicKey (ipaVaultPublicKey?) rather than ipaPublicKey directly
for assymetric vault public keys, so that assymetric public key and
escrow public key are on the same level and you can still use
ipaPublicKey to refer to either one:

 ipaPublicKey
 ipaVaultPublicKey
 ipaEscrowPublicKey


OK. To be consistent the parameters need to be renamed too:
--vault-public-key and --vault-public-key-file.


It doesn't need to, there is no requirement for CLI names to always
match attribute names. (Also I don't insist on the name
ipaVaultPublicKey, feel free to change it if you want.)


It's unchanged for now. In a previous discussion it was advised to reuse 
the existing attribute type whenever possible.



1. The vault_add was split into a client-side vault_add and
server-side
vault_add_internal since the parameters are different (i.e. public
key file and
future escrow-related params). Since vault_add inherits from Local all
non-primary-key attributes have to be added explicitly.


The split is not really necessary, since the only difference is the
public_key_file option, which exists only because of the lack of proper
file support in the framework. This is a different situation from
vault_{archive,retrieve}, which has two different sets of options on
client and server side. Escrow adds only ipaescrowpublickey and
escrow_public_key_file, right? If yes, we can safely keep the command in
a single piece.


We know the vault-add will have at least two client-only parameters:
vault_public_key_file and escrow_public_key_file. Keeping these
parameters on the server API would be wrong and confusing. If the API is
called on the server side with vault_public_key_file the operation will
fail. In the previous discussion you considered this as broken API:


Server API is used not only by the server itself, but also by installers
for example. Anyway the point is that there *can't* be a broken API like
this, you should at least raise an error if the command is called from
server API, although actually separating it into client and server parts
would be preferable.


You are comparing apples and oranges:


Non-identical items are different by definition. Even between 2 apples 
there are differences, but it doesn't mean the distinction is important. 
The latest patch shows that the vault_add needs to be split, not just 
because of the options, but because of what they do differently on the 
client and server.



  a) When the non-split vault_{archive,retrieve} was called from a
server API with client-only options, it crashed. This is the broken API
I was talking about.


This is because in the current framework any API called on the server 
side will be a server API, so you are not supposed to call it with 
client options in the first place. Because of that limitation, the only 
way to use client options is to use a separate API on the client side to 
call the original API on the server side. The point is, client options 
belong to client API, and server options belong to server API. In 
vault_add the public key file name belongs to client API because it's 
used to load a file on the client side. You should not add public key 
file name option to the server API just because it can safely be ignored.



  b) The non-split vault_{archive,retrieve} had server-only options,
which were also accepted on client, but setting them had no effect.


Similarly, in a combined vault_add the public key file name option will 
be accepted by the server, but it will be ignored. If something calls 
vault_add on the server side and provides a file name, the operation 
will crash too because the command expects the public key data to be 
provided via another option. Splitting the vault_add into client and 
server components avoids the potential problems.



  c) The CLI options to read param values from files should be generated
by the framework without having to specify dummy params. Once this is
implemented, the dummy params will go away. However, this will still
leave some client-only options in vault_{archive,retrieve}.


I'm not sure how the options will look like when that's implemented, but 
regardless, the vault_add will still have client-only password option.



None of the above applies to vault_add - it does not have any
server-only options and the only client-only options it has are the
dummy options for file input, which are ignored on the server.


Let's not get fixated with just the options. The vault_add will now 
archive a blank initial data as it was originally designed. The data can 
be used later to verify the vault password in subsequent archival 
operations. The vault_archive must be called by vault_add's client 
component since it takes a password and the password cannot be sent to 
the server.



Also, originally the vault was designed like this: when 

Re: [Freeipa-devel] Need to figure out how to make a schema change

2015-06-18 Thread Endi Sukma Dewata

On 6/18/2015 8:19 PM, Fraser Tweedale wrote:

In order for IPA to use some new functionality in Profile Management and
Sub CAs, we need to add some additional schema to the Dogtag LDAP
instance.

Fraser has written a Dogtag upgrade script to do this upgrade, but this
script expects the DM password to be in password.conf.  Some discussion
on this script can be found here ..
  https://www.redhat.com/archives/pki-devel/2015-June/msg00054.html

In general, I think that while Dogtag will provide a database upgrade
framework and/or upgrade LDIF scripts, we will not - in general - know
how to connect to the DB with a user that has credentials to make schema
changes.

Fortunately, these types of changes are rare.  Note that in all the
years Dogtag has been part of IPA, this is the first time this situation
has arisen.

The question now though is - how can we co-ordinate with IPA to make
this change?  This question may have both a short term (for this
particular change) and long term answer.


What about using LDAPI and autobind functionality?  If the upgrade
script is run locally  as root, then it can autobind to cn=Directory
Manager without requiring a password.


I like this idea, but I'm not sure how to accurately locate the
socket, because the name depends on the domain, e.g.
`/var/run/slapd-EXAMPLE-COM.socket'.


I think the socket name would have to be provided by IPA via PKI 
deployment configuration.


I'm just wondering how LDAPI with autobind would work with nuxwdog. 
Supposedly when nuxwdog is enabled the server can only be started by 
providing the NSS and LDAP database passwords. Does LDAPI with autobind 
make it less secure since the LDAP password is no longer required?


Also, LDAPI wouldn't work if the DS is on a different machine in general 
PKI deployment.


I created this page about PKI database upgrade:
http://pki.fedoraproject.org/wiki/Database_Upgrade


Since the new schema is for now only used by and supported for IPA,
I think the immediate way forward is to provide the new schema LDIF
in the Dogtag package (as the current patch does), and have FreeIPA
use it to update the DS.  I will have patch for IPA and updated
patch for Dogtag shortly.

We will then work out what is the way forward for Dogtag to reliably
manage its schema updates in the variety of authentication
scenarios.

Thanks,
Fraser


--
Endi S. Dewata

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] [PATCH] Password vault

2015-06-15 Thread Endi Sukma Dewata

On 6/15/2015 2:22 AM, Jan Cholasta wrote:

I think it would be better to use a new attribute type which inherits
from ipaPublicKey (ipaVaultPublicKey?) rather than ipaPublicKey directly
for assymetric vault public keys, so that assymetric public key and
escrow public key are on the same level and you can still use
ipaPublicKey to refer to either one:

 ipaPublicKey
 ipaVaultPublicKey
 ipaEscrowPublicKey

 ( 2.16.840.1.113730.3.8.18.2.? NAME 'ipaVaultPublicKey' DESC
'Assymetric vault public key as DER-encoded SubjectPublicKeyInfo (RFC
5280)' SUP ipaPublicKey EQUALITY octetStringMatch SYNTAX
1.3.6.1.4.1.1466.115.121.1.40 X-ORIGIN 'IPA v4.2' )
 ( 2.16.840.1.113730.3.8.18.2.3 NAME 'ipaEscrowPublicKey' DESC 'IPA
escrow public key as DER-encoded SubjectPublicKeyInfo (RFC 5280)' SUP
ipaPublicKey EQUALITY octetStringMatch SYNTAX
1.3.6.1.4.1.1466.115.121.1.40 X-ORIGIN 'IPA v4.2' )


OK. To be consistent the parameters need to be renamed too: 
--vault-public-key and --vault-public-key-file.



1. The vault_add was split into a client-side vault_add and server-side
vault_add_internal since the parameters are different (i.e. public
key file and
future escrow-related params). Since vault_add inherits from Local all
non-primary-key attributes have to be added explicitly.


The split is not really necessary, since the only difference is the
public_key_file option, which exists only because of the lack of proper
file support in the framework. This is a different situation from
vault_{archive,retrieve}, which has two different sets of options on
client and server side. Escrow adds only ipaescrowpublickey and
escrow_public_key_file, right? If yes, we can safely keep the command in
a single piece.


We know the vault-add will have at least two client-only parameters: 
vault_public_key_file and escrow_public_key_file. Keeping these 
parameters on the server API would be wrong and confusing. If the API is 
called on the server side with vault_public_key_file the operation will 
fail. In the previous discussion you considered this as broken API:



Server API is used not only by the server itself, but also by installers
for example. Anyway the point is that there *can't* be a broken API like
this, you should at least raise an error if the command is called from
server API, although actually separating it into client and server parts
would be preferable.


Also, originally the vault was designed like this: when you create a 
symmetric vault you're supposed to specify the password as well, similar 
to adding a public key when creating an asymmetric vault. When you 
archive, you're supposed to enter the same password for verification, 
not a new password. So it would look like this:


$ ipa vault-add test --type symmetric
New password: 
Verify password: 

$ ipa vault-archive test --in secret1.txt
Password:  (same password)

$ ipa vault-archive test --in secret2.txt
Password:  (same password)

In the original design the vault-add would also archive a blank data, 
which later could be used to verify the password during vault-archive by 
decrypting the existing data first. There's also a plan to add a 
mechanism to change the password after the ACL patch.


In the current design the vault-add doesn't archive anything, so during 
vault-archive it cannot verify the password because there is nothing to 
decrypt. In other words you can specify different passwords on each 
archival, regardless of previous archivals:


$ ipa vault-add test --type symmetric

$ ipa vault-archive test --in secret1.txt
New password: 
Verify password: 

$ ipa vault-archive test --in secret2.txt
New password: 
Verify password: 

So basically here are the options:

1. Specify the crypto parameters once during vault creation, then 
reuse/verify the parameters on each archival  retrieval. You can change 
the parameters only with a special command.


2. Don't specify the crypto parameters during vault creation, but 
specify new parameters on each archival. For retrieval you'd have to 
use/verify the parameters specified in the last archival.


I think the first one makes more sense and is easier to use. That also 
means the vault-add will have additional client-only parameters such as 
--password and --password-file.



2. Since the vault_archive_internal inherits from Update, it accepts
all non
primary-key attributes automatically. This is incorrect since we
don't want to
update these parameters during archival. Can this behavior be
overridden?


Inherit from PKQuery instead (don't forget to add has_output =
output.standard_entry).


Previously you didn't want to use LDAPQuery because of semantics 
reasons. Is PKQuery fine semantically? Why not use LDAPQuery since vault 
is an LDAPObject? And to be consistent should vault_retrieve_internal 
inherit from the same class?



BTW the correct solution would be to have a separate object and commands
for vault data (e.g. 

Re: [Freeipa-devel] [PATCH] Password vault

2015-06-05 Thread Endi Sukma Dewata

On 6/5/2015 7:13 AM, Jan Cholasta wrote:

If KRA is not installed, vault-archive and vault-retrieve fail with
internal error.


Added a code to check KRA installation in all vault commands. If you
know a way not to load the vault plugin if the KRA is not installed
please let me know, that's probably even better. Not sure how that will
work on the client side though.


I see this has been already resolved in the other thread.


The other thread was talking about removing the pki-base dependency on 
the client side, but the vault plugin is still loaded on both client and 
server regardless of KRA installation. Ideally the vault plugin should 
not even be loaded so you cannot even execute the commands.



The commands still behave differently based on whether they were called
from API which was initialized with in_server set to True or False.


That is unfortunately a restriction imposed by the framework. In order
to guarantee the security, the vault is designed to have separate client
and server code. The client code encrypts the secret, the server code
forwards the encrypted secret to KRA. To archive a secret into a vault
properly, you are supposed to call the client code. If you're calling
the server code directly, you are responsible to do your own encryption
(i.e. generating session key, nonce, and vault data).


I understand why the code has to be separated, what I don't understand
is why it is in fact *not* separated and crammed into a single command,
making weird and undefined behavior possible.


If another plugin wants to use vault, it should implement a client code
which calls the vault client code to perform the archival from the
client side.

What is the use case for calling the vault API from the server side
anyway? Wouldn't that defeat the purpose of having a vault? If a secret
exists on the server side in an unencrypted form doesn't it mean the
secret may already have been compromised?


Server API is used not only by the server itself, but also by installers
for example. Anyway the point is that there *can't* be a broken API like
this, you should at least raise an error if the command is called from
server API, although actually separating it into client and server parts
would be preferable.


There is no point in exposing the session_key, nonce and vault_data
options in CLI when their value is always overwritten in forward().


I agree there is no need to expose them in CLI, but in this framework
the API also defines the CLI. If there's a way to keep them in the
server API but not expose them in the CLI please let me know. Or, if
there's a way to define completely separate server API (without a
matching client CLI) and client CLI (without a matching server API) that
will work too.


As I suggested above, you can split the commands into separate client
and server commands. The client command should inherit from
frontend.Local so that it is always executed locally and the server
command should have a NO_CLI = True attribute so that it is not
available in the CLI.


I see the vault_archive and vault_retrieve now inherit from PKQuery, and 
there is a hack to execute the forward() even on the server side. A few 
things below:


1. Why didn't you use frontend.Local as you initially suggested? If 
there's a problem with frontend.Local please attach the ticket number in 
the code.


2. The forward() can be merged into run(). There is no need to keep the 
code in forward(). It would make more sense to have a run() method that 
runs both on client and server, rather than a forward() that is supposed 
to run on the client only but now forced to run on server too, 
semantically speaking.



Attached is a patch including the requested changes.

I have also changed vault_config to vaultconfig_show, for consistency
with {,dns}config_show (it also makes the transport certificate
retrieval code in vault_{archive,retrieve} simpler).


3. The parameter description for nonce should be just 'Nonce' instead of 
'Nonce encrypted'.


4. There's a PEP8 error.

5. The VERSION needs to be updated.

Assuming the above issues are addressed, ACK.


I have noticed that triple-length DES is used for the session key.
Wouldn't AES be better?

 # generate session key
 mechanism = nss.CKM_DES3_CBC_PAD


That's the default used by the KRA's client library, and that's what the 
KRA has been tested with. We probably can change it to AES later. It 
shouldn't be blocking this patch.



BTW, ipa-kra-install is broken with pki-core-10.2.4-1, but it works with
pki-core-10.2.1-3.


There's a bug in IPA: https://bugzilla.redhat.com/show_bug.cgi?id=1228671

--
Endi S. Dewata

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] [PATCH] Password vault

2015-06-03 Thread Endi Sukma Dewata

On 6/3/2015 8:52 AM, Alexander Bokovoy wrote:

Having to use the same plugins for client and server is a framework
limitation/poor design. Having to use conditional imports to work
around the limitation is a bad programming practice. The fact that
trust plugin has to implement a similar workaround is not a
justification, it just shows that the problem is not vault-specific.



There is another thing. Even when splitting client/server sides, we'll
need to check on the server side that certain functionality is
available. In trust case we have ID Views (a separate plugin) which does
use information about trusts to resolve users from AD to their
normalized references (SIDs) and few other places would be depending on
functionality only provided when Samba packages are installed.

To continue your approach, we would need to split also server-side parts
of plugins into separate callable units that would only be provided and
called when appropriate rpm subpackages are installed. This is unneeded
complication in place where we can simply handle dependencies in run
time and make sure the packaging deps are managed separately.


So there are two issues:
1. Conditional imports due to combined client and server plugin.
2. Conditional imports due to feature availability.

Issue #1 is generic and I think we pretty much agree that this is 
supposed to be fixed somehow.


Issue #2 is plugin-specific. I think there are different ways to solve 
this, some might be better than others. The solution that you pick will 
only affect that particular plugin and should not be generalized to 
other plugins or to justify #1.


In my opinion a code should have a fixed dependency, but some features 
can be enabled/disabled based on the configuration. Enabling a feature 
shouldn't be based on package installation because people might install 
a package for different reasons. So the code may look something like this:


  import module

  if feature is enabled:
  do something with the module

It shouldn't be like this:

  if package is installed:
  import module

  if package is installed:
  do something with the module

Of course this assumes that the package is lightweight enough to be 
installed regardless whether it will be used. I don't know if it's 
applicable to your case, but again, there are different ways to address 
issue #2.


--
Endi S. Dewata

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] [PATCH] Password vault

2015-06-03 Thread Endi Sukma Dewata

On 6/3/2015 1:41 AM, Martin Kosek wrote:

On 06/02/2015 11:22 PM, Alexander Bokovoy wrote:

On Tue, 02 Jun 2015, Endi Sukma Dewata wrote:

I think ideally the
client and server code should be in separate files (so they can be deployed
separately too), but the framework doesn't seem to allow that.



This exactly the case we have to use here and we are using that in
trusts case as well -- some code has to run on server only and shouldn't
cause to install Samba related packages on the client. This is because
IPA client is actually using the same IPA plugins that server uses, to
have access to the API calls metadata and client-side callbacks are
defined in the same place where server-side callbacks are. It is IPA
framework design, so we have to use what we have.


This is planned to be changed BTW, when we start with the Thin Client concept
and have different code/plugins for FreeIPA server side and client side.


Is there a ticket for this?


In other words, it is not necessarily an evil under conditions we are
dealing with.


Having to use the same plugins for client and server is a framework 
limitation/poor design. Having to use conditional imports to work around 
the limitation is a bad programming practice. The fact that trust plugin 
has to implement a similar workaround is not a justification, it just 
shows that the problem is not vault-specific.


--
Endi S. Dewata

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] [PATCH] Password vault

2015-06-03 Thread Endi Sukma Dewata

On 6/2/2015 1:34 PM, Simo Sorce wrote:

On Tue, 2015-06-02 at 12:04 +0200, Jan Cholasta wrote:

Dne 2.6.2015 v 02:02 Endi Sukma Dewata napsal(a):

On 5/28/2015 12:46 AM, Jan Cholasta wrote:

On a related note, since KRA is optional, can we move the vaults
container to cn=kra,cn=vaults? This is the convetion used by the other
optional components (DNS and recently CA).


I mean cn=vaults,cn=kra of course.


If you are talking about the o=kra,PKI suffix, I'm not sure whether
the IPA framework will work with it.

If you are talking about adding a new cn=kra,IPA suffix entry on top
of cn=vaults, what is the purpose of this entry? Is the entry going to
be created/deleted automatically when the KRA is installed/removed? Is
it going to be used for something else other than vaults?


I'm talking about cn=kra,IPA suffix. It should be created only when
KRA is installed, although I think this can be done later after the
release, moving vaults to cn=kra should be good enough for now. It's
going to be used for everything KRA-specific.



There are a lot of questions that need to be answered before we can make
this change.


This is about sticking to a convention, which everyone should do, and
everyone except KRA already does.

I'm sorry I didn't realize this earlier, but the change must be done now.


We probably should revisit this issue after the core vault
functionality is added.



We can't revisit it later because after release we are stuck with
whatever is there forever.

See attachment for a patch which implements the change.



Shouldn't we s/kra/vault/ ?
After all the feature is called Vault, not KRA.

Simo.



Here are the options:
1. the original code uses cn=vaults,IPA suffix.
2. Honza proposed cn=vaults,cn=kra,IPA suffix, ACKed by Martin.

Are you proposing a third option cn=vaults,cn=vault,IPA suffix or 
did you mean the first option?


I think the first option would make more sense since we're not 
introducing KRA to the end user, but I'll let the IPA team decide. My 
next patch will be based on whatever pushed at the time.


--
Endi S. Dewata

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] [PATCH] Password vault

2015-06-02 Thread Endi Sukma Dewata

Please take a look at the new patch.

On 6/2/2015 10:05 AM, Martin Kosek wrote:

4) In the vault-archive forward method, you use pki module. However,
this module will be only available on FreeIPA PKI-powered servers and
not on FreeIPA clients - so this will not work unless freeipa-client
gets a dependency on pki-base - which is definitely not something we
want...


In my opinion it should be fine to require pki-base on the client because it
contains just the client library, unless you have other concerns? Any
objections to having pki-nss and pki-cryptography dependencies on the client?

Even if we can change the client code not to depend on pki module, since in
this framework the client and server code are written in the same plugin, the
import pki still cannot be removed since it's still needed by the server
code, and I don't think conditional import is a good programming practice.


I have major concerns here. Look at the different between installing
freeipa-client and freeipa-client + pki-base on my F21:

~~
$ sudo yum install freeipa-client
...
Install  1 Package (+4 Dependent packages)

Total download size: 2.6 M
Installed size: 14 M
~~
$ sudo yum install freeipa-client pki-base
...
Install  2 Packages (+288 Dependent packages)

Total download size: 160 M
Installed size: 235 M
~~

This is obviously a no-go for client. The conditional import is smaller concern
that big dependency growth on the client. We do them in trust plugin for
example and it works fine (though I agree it is not ideal programming practice).

IMO, we should limit new freeipa-client dependencies only to
python-cryptography (or also python-nss in the worst case, in case
python-cryptography is not enough) - there should be no pki dependencies at
all, these should be only on the server side.


OK. I opened a ticket to split the pki-base into separate Python and 
Java packages:

https://fedorahosted.org/pki/ticket/1399

For now in this patch I added conditional imports for pki.account and 
pki.key which are needed to access KRA on the server side. I removed 
dependency on pki.crypto on the client side and replaced it with direct 
python-nss code.


On 6/2/2015 1:40 PM, Simo Sorce wrote:
 I have coded in python (jwcrypto)
 support for some key wrapping not yet available in python-cryptography
 and can lend you the code as needed.

Thanks. I'll get back to you when it's time to add support for 
python-cryptography in KRA:

https://fedorahosted.org/pki/ticket/1400

On 6/2/2015 10:16 AM, Alexander Bokovoy wrote:

Yes, please use conditional import here, it is perfectly valid use case
for the client side.


On 6/2/2015 1:40 PM, Simo Sorce wrote:
 conditional import is just fine

The conditional imports that I've seen usually are used for importing 
different versions of the same module, which I think is acceptable 
because the dependency always exists. In the vault case we're 
selectively importing a module depending on where the code runs. I think 
that is bad because it adds complexity and it's easy to make mistakes. 
Any code that depends on that module would have to be (a) guarded:


  if pki_is_loaded:
  ... call pki ...

or (b) used in a method that's only called if the module is loaded:

  def do_something(self): # runs only on server
  ... call pki ...

The (a) is similar to #ifdef's which should be avoidable using OOD, and 
in (b) we may inadvertently call a wrong method indirectly. I think 
ideally the client and server code should be in separate files (so they 
can be deployed separately too), but the framework doesn't seem to allow 
that.


Regardless, the conditional imports are in.

--
Endi S. Dewata
From 0e9d3868423c21dc47d125f4b3c23e8261c4655f Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Tue, 21 Oct 2014 10:57:08 -0400
Subject: [PATCH] Added vault-archive and vault-retrieve commands.

New commands have been added to archive and retrieve
data into and from a vault, also to retrieve the
transport certificate.

https://fedorahosted.org/freeipa/ticket/3872
---
 API.txt   |  28 ++
 VERSION   |   4 +-
 ipalib/plugins/vault.py   | 517 +-
 ipatests/test_xmlrpc/test_vault_plugin.py |  71 +++-
 4 files changed, 616 insertions(+), 4 deletions(-)

diff --git a/API.txt b/API.txt
index 
da69f32de5c12c0d85a7d61d9027385aa3c0ee05..3741e6f16689e43838c2d31a44872d1ea47589c7
 100644
--- a/API.txt
+++ b/API.txt
@@ -4768,6 +4768,24 @@ option: Str('version?', exclude='webui')
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
+command: vault_archive
+args: 1,9,1
+arg: Str('cn', cli_name='name', 

Re: [Freeipa-devel] [PATCH] Password vault

2015-06-02 Thread Endi Sukma Dewata

On 6/2/2015 1:10 AM, Martin Kosek wrote:

Hi Endi,

Quickly skimming through your patches raised couple questions on my side:

1) Will it be possible to also store plain text password via Vault? It
talks about taking in the binary data or the text file, but will it also
work with plain user secrets (passwords)? I am talking about use like this:

# ipa vault-archive name --user mkosek --data Secret123


For security the plain text password should be stored in a file first:

  # vi password.txt
  # ipa vault-archive name --user mkosek --in password.txt

It's also possible to specify the password as base-64 encoded data:

  # echo -n Secret123 | base64
  # ipa vault-archive name --user mkosek --data U2VjcmV0MTIz

But it's not recommended since the data will be stored in the command 
history and someone could see and decode it. I think passing a plain 
text password as command line argument would be even worse. The --data 
parameter is mainly used for unit testing.


Later we might be able to add an option to read from standard input:

  # cat password.txt | ipa vault-archive name --user mkosek --std-in


2) Didn't we discuss a dependency of IPA/Vault on python-cryptography in
the past? I rather see use of python-nss for cryptography...


Yes. I might have mentioned that it would be in the 2nd (current) vault 
patch. Actually it will be in the 3rd patch when we add the symmetric 
and asymmetric vaults. The symmetric and asymmetric encryption will be 
implemented using python-cryptography. You can also see this in an old 
patch (#358) but it's obsolete now.


The standard vault in the current patch uses python-nss for transport 
encryption because when the KRA interface was written 
python-cryptography wasn't available on Fedora, it didn't support 
certificates, and I'm not sure if it supports key wrapping.


The symmetric and asymmetric vaults add an additional layer of 
encryption on top of the standard transport encryption, so it will 
depend on both python-nss and python-cryptography.


In the future if the KRA can support python-cryptography without 
python-nss we may be able to drop the python-nss dependency from vaults.



3) You do a lot of actions in the forward() method (as planned in
https://www.freeipa.org/page/V4/Password_Vault#Archival). But how do you
envision that this is consumed by the Web UI? It does not have access to
the forward() method. Would it need to also include some crypto library?


If Web UI wants to access vault (not sure if everybody agrees with 
that), it would have to perform an encryption on the browser side. In 
that case we will need to use either WebCrypto or a browser-specific 
extension to implement something similar to vault_archive.forward(), 
assuming the required cryptographic functionalities are available. In 
the future PKI might be able to provide a JavaScript interface for KRA.



4) In the vault-archive forward method, you use pki module. However,
this module will be only available on FreeIPA PKI-powered servers and
not on FreeIPA clients - so this will not work unless freeipa-client
gets a dependency on pki-base - which is definitely not something we
want...


In my opinion it should be fine to require pki-base on the client 
because it contains just the client library, unless you have other 
concerns? Any objections to having pki-nss and pki-cryptography 
dependencies on the client?


Even if we can change the client code not to depend on pki module, 
since in this framework the client and server code are written in the 
same plugin, the import pki still cannot be removed since it's still 
needed by the server code, and I don't think conditional import is a 
good programming practice.



Thanks,
Martin


--
Endi S. Dewata

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] [PATCH] Password vault

2015-06-01 Thread Endi Sukma Dewata

Please take a look at the updated patch.

On 5/27/2015 12:39 AM, Jan Cholasta wrote:

21) vault_archive is not a retrieve operation, it should be based on
LDAPUpdate instead of LDAPRetrieve. Or Command actually, since it
does
not do anything with LDAP. The same applies to vault_retrieve.


The vault_archive does not actually modify the LDAP entry because it
stores the data in KRA. It is actually an LDAPRetrieve operation
because
it needs to get the vault info before it can perform the archival
operation. Same thing with vault_retrieve.


It is not a LDAPRetrieve operation, because it has different
semantics.
Please use Command as base class and either use ldap2 for direct
LDAP or
call vault_show instead of hacking around LDAPRetrieve.


It's been changed to inherit from LDAPQuery instead.


NACK, it's not a LDAPQuery operation, because it has different
semantics. There is more to a command than executing code, so you should
use a correct base class.


Changed to inherit from Command as requested. Now these commands no
longer have a direct access to the vault object (self.obj) although they
are accessing vault objects like other vault commands. Also now the
vault name argument has to be added explicitly on each command.


You can inherit from crud.Retrieve and crud.Update to get self.obj and
the argument back.


I tried this:

  class vault_retrieve(Command, crud.Retrieve):

and it gave me an error:

  TypeError: Error when calling the metaclass bases
  Cannot create a consistent method resolution
  order (MRO) for bases Retrieve, Command

I'm sticking with the original code since it works fine although not 
ideal. I'm not a Python expert, so if you know how to fix this properly 
please feel free to post a patch on top of this.



If KRA is not installed, vault-archive and vault-retrieve fail with
internal error.


Added a code to check KRA installation in all vault commands. If you 
know a way not to load the vault plugin if the KRA is not installed 
please let me know, that's probably even better. Not sure how that will 
work on the client side though.



The commands still behave differently based on whether they were called
from API which was initialized with in_server set to True or False.


That is unfortunately a restriction imposed by the framework. In order 
to guarantee the security, the vault is designed to have separate client 
and server code. The client code encrypts the secret, the server code 
forwards the encrypted secret to KRA. To archive a secret into a vault 
properly, you are supposed to call the client code. If you're calling 
the server code directly, you are responsible to do your own encryption 
(i.e. generating session key, nonce, and vault data).


If another plugin wants to use vault, it should implement a client code 
which calls the vault client code to perform the archival from the 
client side.


What is the use case for calling the vault API from the server side 
anyway? Wouldn't that defeat the purpose of having a vault? If a secret 
exists on the server side in an unencrypted form doesn't it mean the 
secret may already have been compromised?



There is no point in exposing the session_key, nonce and vault_data
options in CLI when their value is always overwritten in forward().


I agree there is no need to expose them in CLI, but in this framework 
the API also defines the CLI. If there's a way to keep them in the 
server API but not expose them in the CLI please let me know. Or, if 
there's a way to define completely separate server API (without a 
matching client CLI) and client CLI (without a matching server API) that 
will work too.



Will this always succeed?

+# deactivate vault record in KRA
+response = kra_client.keys.list_keys(
+client_key_id, pki.key.KeyClient.KEY_STATUS_ACTIVE)


Yes. If there's no active keys it will return an empty collection.


+for key_info in response.key_infos:
+kra_client.keys.modify_key_status(
+key_info.get_key_id(),
+pki.key.KeyClient.KEY_STATUS_INACTIVE)


This loop will do nothing given an empty collection.


If not, we might get into an inconsistent state, where the vault is
deleted in LDAP but still active in KRA. (I'm not sure if this is
actually a problem or not.)


That can only happen if the server crashes after deleting the vault but 
before deactivating the key. Regardless, it will not be a problem 
because the key is identified by vault ID/path so it will not conflict 
with other vaults, and it will get overwritten if the same vault is 
recreated again.


--
Endi S. Dewata
From d1123f07745fea856ced305a814d933cd793dbf2 Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Tue, 21 Oct 2014 10:57:08 -0400
Subject: [PATCH] Added vault-archive and vault-retrieve commands.

New commands have been added to archive and retrieve
data into and from a vault, also to retrieve the
transport certificate.


Re: [Freeipa-devel] [PATCH] Password vault

2015-06-01 Thread Endi Sukma Dewata

On 5/28/2015 12:46 AM, Jan Cholasta wrote:

On a related note, since KRA is optional, can we move the vaults
container to cn=kra,cn=vaults? This is the convetion used by the other
optional components (DNS and recently CA).


I mean cn=vaults,cn=kra of course.


If you are talking about the o=kra,PKI suffix, I'm not sure whether 
the IPA framework will work with it.


If you are talking about adding a new cn=kra,IPA suffix entry on top 
of cn=vaults, what is the purpose of this entry? Is the entry going to 
be created/deleted automatically when the KRA is installed/removed? Is 
it going to be used for something else other than vaults?


There are a lot of questions that need to be answered before we can make 
this change. We probably should revisit this issue after the core vault 
functionality is added.


--
Endi S. Dewata

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] [PATCH] Password vault

2015-05-26 Thread Endi Sukma Dewata
Please take a look at the attached patch to add vault-archive/retrieve 
commands.


On 4/20/2015 1:12 AM, Jan Cholasta wrote:

16) You do way too much stuff in vault_add.forward(). Only code that
must be done on the client needs to be there, i.e. handling of the
data, text and in options.

The vault_archive call must be in vault_add.execute(), otherwise a) we
will be making 2 RPC calls from the client and b) it won't be
called at
all when api.env.in_server is True.


This is done by design. The vault_add.forward() generates the salt and
the keys. The vault_archive.forward() will encrypt the data. These
operations have to be done on the client side to secure the
transport of
the data from the client through the server and finally to KRA. This
mechanism prevents the server from looking at the unencrypted data.


OK, but that does not justify that it's broken in server-side API. It
can and should be done so that it works the same way on both client and
server.

I think the best solution would be to split the command into two
commands, server-side vault_archive_raw to archive already encrypted
data, and client-side vault_archive to encrypt data and archive them
with vault_archive_raw in its .execute(). Same thing for vault_retrieve.


Actually I think it's better to just merge the add and archive, reducing
the number of RPC calls. The vault_archive now will support two types of
operations:

(a) Archive data into a new vault (it will create the vault just before
archiving the data):

   $ ipa vault-archive testvault --create --in data ...

(b) Archive data into an existing vault:

   $ ipa vault-archive testvault --in data ...

The vault_add is now just a wrapper for the vault_archive(a).


If that's just an implementation detail, OK.

If it's possible to modify existing vault objects using vault-add or
create new objects using vault-archive, then NACK.


The vault-archive is now completely separate from vault-add. You can no 
longer archive data while creating a vault:


  $ ipa vault-add test
  $ ipa vault-archive test --in secret.bin


BTW, I also think it would be better if there were 2 separate sets of
commands for binary and textual data
(vault_{archive,retrieve}_{data,text}) rather than trying to handle
everything in vault_{archive,retrieve}.


I don't think we want to provide a separate command of every possible
data type  operation combination. Users would get confused. The archive
 retrieve commands should be able to handle all current  future data
types with options.


A command with two sets of mutually exclusive options is really two
commands in disguise, which is a good sign it should be divided into two
actual commands.

Who are you to say users would get confused? I say they would be more
confused by a command with a plethora of mutually exclusive options.

What other possible data types are there?


The patch has been simplified to support only binary data. The data can 
be specified using either of these options:


  $ ipa vault-archive test --data base-64 encoded data
  $ ipa vault-archive test --in input file

I don't think we want to provide two separate commands for these options 
although they are mutually exclusive, do we?



The add  archive combination was added for convenience, not for
optimization. This way you would be able to archive data into a new
vault using a single command. Without this, you'd have to execute two
separate commands: add  archive, which will result in 2 RPC calls
anyway.


I think I would prefer if it was separate, as that would be consistent
with other plugins (e.g. for objects with members, we don't allow adding
members directly in -add, you have to use -add-member after -add).


The vault data is similar to group description, not group members. When
creating a group we can supply the description. If not specified it will
be blank. Archiving vault data is similar to updating the group
description.


It's similar to group members because there are separate commands to
manipulate them.


Just because there are separate commands doesn't mean vault data 
(single-valued) is similar to group members (multi-valued). It uses 
separate commands because of the encryption steps involved while 
archiving/retrieving data.



You have to choose one of:

   a) vault data is settable using vault-add and vault-mod and gettable
using vault-mod, vault-show and vault-find

   b) vault data is settable using vault-archive and gettable using
vault-retrieve

Anything in between is not permitted.


As mentioned above, the add and archive commands are now separate.


21) vault_archive is not a retrieve operation, it should be based on
LDAPUpdate instead of LDAPRetrieve. Or Command actually, since it does
not do anything with LDAP. The same applies to vault_retrieve.


The vault_archive does not actually modify the LDAP entry because it
stores the data in KRA. It is actually an LDAPRetrieve operation
because
it needs to get the vault info before it can perform the archival
operation. 

Re: [Freeipa-devel] [PATCH] Password vault

2015-05-19 Thread Endi Sukma Dewata

Before I send another patch I have some questions below.

On 5/19/2015 3:27 AM, Jan Cholasta wrote:

I changed the 'host vaults' to become 'service vaults'. The interface
will look like this:

$ ipa vault-find --service HTTP/server.example.com
$ ipa vault-add test --service HTTP/server.example.com

I also added user vaults:

$ ipa vault-find --user testuser
$ ipa vault-add test --user testuser

Private vaults is a special case of user vaults where username=you.

Host vaults can be added later once we define the use case.


OK.


I suppose you meant you're OK with not adding host vaults now?


4. The following code in get_dn() is incorrect:

   principal = getattr(context, 'principal')
   (name, realm) = split_principal(principal)
   name = name.split('/')
   if len(name) == 1:
   container_dn = DN(('cn', 'users'), container_dn)
   else:
   container_dn = DN(('cn', 'services'), container_dn)
   container_dn = DN(('cn', name[-1]), container_dn)

A service does not have a private container like users (cn=username,
cn=users, cn=vaults). The entry cn=name, cn=host, cn=services,
cn=vaults is a service vault, not a container. The service vault is
used
by the admin to provide a secret for a service.

I'm not sure what the behavior should be if a service is executing a
vault command that uses a private container such as:

   $ ipa vault-add test

Maybe it should just generate an error.


Users, hosts and services are all user-like objects, is there a reason
not to support private vaults for all of them?


As mentioned above, it's not required in the design doc, but we can add
it if there's a clear use case. I agree that at least for now we can
change the service vault into a service container to store multiple
service's private vaults.


I don't really care about having a clear use case, I would prefer if the
design was elegant enough to handle *all* the cases without any extra
effort.


The only way to know if the design will be future proof is if we have at 
least some idea how it will be used. Without that there is no guarantee.


Host principals have this form: host/hostname@realm, so with the 
current code they will be considered a service and will have a service 
container.


Do you want to add a new cn=hosts container just for hosts? Unless we 
have a specific reason (i.e. use case) I don't see a need to add 
specific code for hosts now, or at least until we get the core vault 
functionality working.



5. In create_container() why do you need to reconstruct the
container_dn
on each invocation even though the value is fixed?

   container_dn = DN(self.container_dn, self.api.env.basedn)


Because self.api may not necessarily be the same as ipalib.api.


Under what scenario would that be a problem?


When someone uses the plugin with a different API object than ipalib.api.


The original code seems to

be working fine with ipalib.api.


The current best practice is to use self.api and *all* new plugin code
should do that.



If it is a problem, why do we still use ipalib.api to initialize
container_dn vault class attribute?

   container_dn = api.env.container_vault

Then in get_dn() we basically construct the container_dn variable with
values from both self.api and ipalib.api:

   container_dn = DN(self.container_dn, self.api.env.basedn)

When is the self.api actually initialized? Can we initialize the
container_dn (or base_dn as in the original code) attribute immediately
after that?


Not yet, but this will be fixed in the future. (Also, container_dn is
part of the LDAPObject API, unlike base_dn used in the original code.)


Is there a ticket for this?


This change is not included. The code will now obtain the values from
apilib.api.env at init time and store it in class attributes so it can
be reused.

 container_dn = api.env.container_vault
 base_dn = DN(container_dn, api.env.basedn)


Sorry, but no. Please just follow the best practice instead of trying to
invent something new. This is not the right time and place to discuss
this. We should be discussing the vault, not framework idiosyncracies.


OK.


6. The loop in create_container() is incorrect. Suppose we're
creating a
container cn=A, cn=B, suffix and the parent container cn=B, suffix
doesn't exist yet. The first add_entry() invocation will fail as
expected, but instead of adding the parent entry the whole method will
fail.


Right, fixed.


It's still not working. The new code is trying to add cn=vaults first,
and it stops because it already exists, but the intermediary entries are
still not added. Also the DN displayed in the message misleading:

$ ipa vault-add test
ipa: ERROR: container entry (cn=vaults) not found

$ ipa vault-add test --host server.example.com
ipa: ERROR: container entry (cn=vaults) not found

The unit tests are failing because of this.


I forgot to remove the break statement in the for loop.



This change is not included. The original code and the tests work just
fine.


I would prefer if it was done 

Re: [Freeipa-devel] [PATCH] Password vault

2015-05-18 Thread Endi Sukma Dewata
Please take a look at the attached new patch which includes some of your 
changes you proposed.


On 5/14/2015 7:17 PM, Endi Sukma Dewata wrote:

On 5/14/2015 1:42 PM, Jan Cholasta wrote:

Question: Services in IPA are identified by Kerberos principal. Why are
service vaults identified by hostname alone?


The service vaults are actually identified by the hostname and service
name assuming the principal is in this format: name/host@realm.
The vault is stored in cn=name, cn=host, cn=services, cn=vaults,
suffix. When accessing a vault service you are supposed to specify the
name and the host (except for vault-find which will return all services
in the same host):

$ ipa vault-find --host host
$ ipa vault-add name --host host

Are there other service principal formats that we need to support? Do we
need to support services of different realms?


Well, users are identified by username (uid attribute), hosts by
hostname (fqdn attribute) and services by principal name
(krbprincipalname attribute). Each of them has separate container and is
also uniquely identified by principal name. I guess it would make sense
to follow that for vaults as well, in which case service vaults should
be called host vaults (because they are created in a host-specific
container, not a service-specific one) and maybe real service vaults
should be added.


There's no plan for host vaults in the design doc, but it's not
impossible to add it in the future. What is the use case?

I suppose we can change the service vault into a container, and possibly
add generic user vaults too (in addition to private user vaults), the
interface will look like this:

$ ipa vault-add PrivateVault
$ ipa vault-add ServiceVault --service name/hostname
$ ipa vault-add SharedVault --shared
$ ipa vault-add UsersVault --user username
$ ipa vault-add HostVault --host hostname

Basically we'll need a new parameter for each new container. For the
initial implementation we only need the first 2 or 3.


I changed the 'host vaults' to become 'service vaults'. The interface 
will look like this:


$ ipa vault-find --service HTTP/server.example.com
$ ipa vault-add test --service HTTP/server.example.com

I also added user vaults:

$ ipa vault-find --user testuser
$ ipa vault-add test --user testuser

Private vaults is a special case of user vaults where username=you.

Host vaults can be added later once we define the use case.


1. The following code in get_dn() is causing problems for vault-find.

   dn = super(vault, self).get_dn(*keys, **options)
   rdn = dn[0]
   container_dn = DN(*dn[1:])

The super.get_dn() will return cn=vaults, suffix, so the rdn will
become cn=vaults and container_dn will become suffix. When combined
with the subcontainer (private/shared/service) it will produce an
invalid DN.


Right, fixed.

I should have tested the patch before posting it, my bad, sorry.


OK.


This change has been included in this patch.


2. Not sure if it'related to #1, the vault-find is failing.

$ ipa vault-find
ipa: ERROR: an internal error has occurred

The error_log shows TypeError: 'NoneType' object is not iterable


Fixed. It was caused by missing return value in vault_find.exc_callback.


We can actually ignore all NotFound errors since now all containers are
either created automatically or already created by default.

   if call_func.__name__ == 'find_entries':
   if isinstance(exc, errors.NotFound):
   shared = options.get('shared')

   # if private or service container has not been created, ignore
   if not shared:
   raise errors.EmptyResult(reason=str(exc))

The original code was only ignoring NotFound errors on user vaults
because previously only the user containers was supposed to be created
automatically, and there wasn't a plan to provide service container.


This change has been included and it will ignore all NotFound errors.


3. The --shared option in vault-find is now requiring an argument even
though it's a Flag.

$ ipa vault-find --shared
Usage: ipa [global-options] vault-find [CRITERIA] [options]

ipa: error: --shared option requires an argument


Fixed.


OK.


Not sure which code you changed, but the new patch doesn't exhibit this 
problem.



4. The following code in get_dn() is incorrect:

   principal = getattr(context, 'principal')
   (name, realm) = split_principal(principal)
   name = name.split('/')
   if len(name) == 1:
   container_dn = DN(('cn', 'users'), container_dn)
   else:
   container_dn = DN(('cn', 'services'), container_dn)
   container_dn = DN(('cn', name[-1]), container_dn)

A service does not have a private container like users (cn=username,
cn=users, cn=vaults). The entry cn=name, cn=host, cn=services,
cn=vaults is a service vault, not a container. The service vault is used
by the admin to provide a secret for a service.

I'm not sure what the behavior should be if a service is executing a
vault command that uses a private container such as:

   $ ipa vault-add test

Maybe

Re: [Freeipa-devel] [PATCH] Password vault

2015-05-14 Thread Endi Sukma Dewata

On 5/14/2015 1:42 PM, Jan Cholasta wrote:

Question: Services in IPA are identified by Kerberos principal. Why are
service vaults identified by hostname alone?


The service vaults are actually identified by the hostname and service
name assuming the principal is in this format: name/host@realm.
The vault is stored in cn=name, cn=host, cn=services, cn=vaults,
suffix. When accessing a vault service you are supposed to specify the
name and the host (except for vault-find which will return all services
in the same host):

$ ipa vault-find --host host
$ ipa vault-add name --host host

Are there other service principal formats that we need to support? Do we
need to support services of different realms?


Well, users are identified by username (uid attribute), hosts by
hostname (fqdn attribute) and services by principal name
(krbprincipalname attribute). Each of them has separate container and is
also uniquely identified by principal name. I guess it would make sense
to follow that for vaults as well, in which case service vaults should
be called host vaults (because they are created in a host-specific
container, not a service-specific one) and maybe real service vaults
should be added.


There's no plan for host vaults in the design doc, but it's not 
impossible to add it in the future. What is the use case?


I suppose we can change the service vault into a container, and possibly 
add generic user vaults too (in addition to private user vaults), the 
interface will look like this:


$ ipa vault-add PrivateVault
$ ipa vault-add ServiceVault --service name/hostname
$ ipa vault-add SharedVault --shared
$ ipa vault-add UsersVault --user username
$ ipa vault-add HostVault --host hostname

Basically we'll need a new parameter for each new container. For the 
initial implementation we only need the first 2 or 3.



Do we
need to support services of different realms?


That depends. Do we want to support vaults for users and services from
AD trusts?


It wasn't mentioned in the design doc either, but probably in the future 
we can support external vaults too:


$ ipa vault-add ExternalVault --principal principal

  cn=vaults
  + cn=external
+ cn=name/hostname@realm
  + cn=vault 1
  + cn=vault 2
+ cn=user@realm
  + cn=vault 3
  + cn=vault 4


1. The following code in get_dn() is causing problems for vault-find.

   dn = super(vault, self).get_dn(*keys, **options)
   rdn = dn[0]
   container_dn = DN(*dn[1:])

The super.get_dn() will return cn=vaults, suffix, so the rdn will
become cn=vaults and container_dn will become suffix. When combined
with the subcontainer (private/shared/service) it will produce an
invalid DN.


Right, fixed.

I should have tested the patch before posting it, my bad, sorry.


OK.


2. Not sure if it'related to #1, the vault-find is failing.

$ ipa vault-find
ipa: ERROR: an internal error has occurred

The error_log shows TypeError: 'NoneType' object is not iterable


Fixed. It was caused by missing return value in vault_find.exc_callback.


We can actually ignore all NotFound errors since now all containers are 
either created automatically or already created by default.


  if call_func.__name__ == 'find_entries':
  if isinstance(exc, errors.NotFound):
  shared = options.get('shared')

  # if private or service container has not been created, ignore
  if not shared:
  raise errors.EmptyResult(reason=str(exc))

The original code was only ignoring NotFound errors on user vaults 
because previously only the user containers was supposed to be created 
automatically, and there wasn't a plan to provide service container.



3. The --shared option in vault-find is now requiring an argument even
though it's a Flag.

$ ipa vault-find --shared
Usage: ipa [global-options] vault-find [CRITERIA] [options]

ipa: error: --shared option requires an argument


Fixed.


OK.


4. The following code in get_dn() is incorrect:

   principal = getattr(context, 'principal')
   (name, realm) = split_principal(principal)
   name = name.split('/')
   if len(name) == 1:
   container_dn = DN(('cn', 'users'), container_dn)
   else:
   container_dn = DN(('cn', 'services'), container_dn)
   container_dn = DN(('cn', name[-1]), container_dn)

A service does not have a private container like users (cn=username,
cn=users, cn=vaults). The entry cn=name, cn=host, cn=services,
cn=vaults is a service vault, not a container. The service vault is used
by the admin to provide a secret for a service.

I'm not sure what the behavior should be if a service is executing a
vault command that uses a private container such as:

   $ ipa vault-add test

Maybe it should just generate an error.


Users, hosts and services are all user-like objects, is there a reason
not to support private vaults for all of them?


As mentioned above, it's not required in the design doc, but we can add 
it if there's a clear use case. I agree that at least for now we can 
change the service 

Re: [Freeipa-devel] [PATCH] Password vault

2015-05-13 Thread Endi Sukma Dewata

On 5/13/2015 4:09 AM, Jan Cholasta wrote:

Dne 12.5.2015 v 12:52 Endi Sukma Dewata napsal(a):

Please take a look at the attached patch (#353-9). It obsoletes all
previous patches. See comments below.

On 4/20/2015 1:12 AM, Jan Cholasta wrote:

I'm planning to merge the vault and vault container object and use the
vault type attribute to distinguish between the two. See more
discussion
about that below.


OK.


The vault container plugin has been removed instead of merged (see
explanation below). Internally the vaults are still stored in built-in
containers in the DS, but there won't be an interface to manage them.
The following containers are available for use: private, shared, and
services, but they are flat, not hierarchical.


To speed up the review, I have amended your patch with code and coding
style fixes (attached), please review my changes.

Question: Services in IPA are identified by Kerberos principal. Why are
service vaults identified by hostname alone?


The service vaults are actually identified by the hostname and service 
name assuming the principal is in this format: name/host@realm. 
The vault is stored in cn=name, cn=host, cn=services, cn=vaults, 
suffix. When accessing a vault service you are supposed to specify the 
name and the host (except for vault-find which will return all services 
in the same host):


$ ipa vault-find --host host
$ ipa vault-add name --host host

Are there other service principal formats that we need to support? Do we 
need to support services of different realms?


Some comments about your changes:

1. The following code in get_dn() is causing problems for vault-find.

  dn = super(vault, self).get_dn(*keys, **options)
  rdn = dn[0]
  container_dn = DN(*dn[1:])

The super.get_dn() will return cn=vaults, suffix, so the rdn will 
become cn=vaults and container_dn will become suffix. When combined 
with the subcontainer (private/shared/service) it will produce an 
invalid DN.


2. Not sure if it'related to #1, the vault-find is failing.

$ ipa vault-find
ipa: ERROR: an internal error has occurred

The error_log shows TypeError: 'NoneType' object is not iterable

3. The --shared option in vault-find is now requiring an argument even 
though it's a Flag.


$ ipa vault-find --shared
Usage: ipa [global-options] vault-find [CRITERIA] [options]

ipa: error: --shared option requires an argument

4. The following code in get_dn() is incorrect:

  principal = getattr(context, 'principal')
  (name, realm) = split_principal(principal)
  name = name.split('/')
  if len(name) == 1:
  container_dn = DN(('cn', 'users'), container_dn)
  else:
  container_dn = DN(('cn', 'services'), container_dn)
  container_dn = DN(('cn', name[-1]), container_dn)

A service does not have a private container like users (cn=username, 
cn=users, cn=vaults). The entry cn=name, cn=host, cn=services, 
cn=vaults is a service vault, not a container. The service vault is used 
by the admin to provide a secret for a service.


I'm not sure what the behavior should be if a service is executing a 
vault command that uses a private container such as:


  $ ipa vault-add test

Maybe it should just generate an error.

5. In create_container() why do you need to reconstruct the container_dn 
on each invocation even though the value is fixed?


  container_dn = DN(self.container_dn, self.api.env.basedn)

6. The loop in create_container() is incorrect. Suppose we're creating a 
container cn=A, cn=B, suffix and the parent container cn=B, suffix 
doesn't exist yet. The first add_entry() invocation will fail as 
expected, but instead of adding the parent entry the whole method will fail.


  while dn.endswith(container_dn):
  entry = self.backend.make_entry(
  dn,
  objectclass=['nsContainer'],
  cn=[dn[0]['cn']],
  )

  try:
  self.backend.add_entry(entry)
  except errors.DuplicateEntry:
  break

  dn = DN(*dn[1:])

7. The host and shared parameters are no longer available in vault-show 
and vault-del commands. The unit tests are failing because of that. Why 
do these commands not automatically inherit all parameters from the class?


--
Endi S. Dewata

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] [PATCH] Password vault

2015-05-12 Thread Endi Sukma Dewata
Please take a look at the attached patch (#353-9). It obsoletes all 
previous patches. See comments below.


On 4/20/2015 1:12 AM, Jan Cholasta wrote:

I'm planning to merge the vault and vault container object and use the
vault type attribute to distinguish between the two. See more discussion
about that below.


OK.


The vault container plugin has been removed instead of merged (see 
explanation below). Internally the vaults are still stored in built-in 
containers in the DS, but there won't be an interface to manage them. 
The following containers are available for use: private, shared, and 
services, but they are flat, not hierarchical.



3) The container_vault config option should be renamed to
container_vaultcontainer, as it is used in the vaultcontainer plugin,
not the vault plugin.


It was named container_vault because it defines the DN for of the
subtree that contains all vault-related entries. I moved the base_dn
variable from vaultcontainer object to the vault object for clarity.


That does not make much sense to me. Vault objects are contained in
their respective vaultcontainer objects, not directly in cn=vaults.


The cn=vaults itself is actually a vault container (i.e.
ipaVaultContainer). Theoretically you could store a vault in any
container including cn=vaults, but we just don't want people to use it
that way.

I think this is consistent with other plugins. For example, the
container_user points to cn=users, which is an nsContainer. There is no
concept of 'user container' other than the cn=users itself. But even if
there is one, the container_user will still be stored in the user class.


In fact it is not consistent with other plugins. All entries managed by
the user plugin are stored *directly* under cn=users. Entries managed by
the vault plugin are not stored directly under cn=vaults, but rather
anywhere in the cn=vaults subtree and their DN is derived from the DN of
the parent vault container. For such objects, we don't set
plugin.container_dn and don't have container_plugin constant, but
rather define them as child objects of their container objects.


When the vault  vaultcontainer is merged, this will no longer be an
issue.


OK.


The vaults are still stored in the cn=vaults subtree, but now the 
containers will use nsContainer instead of ipaVaultContainer. The 
container_vault variable still defines the root container DN, i.e. 
cn=vaults.



4) The vault object should be child of the vaultcontainer object.

Not only is this correct from the object model perspective, but it
would
also make all the container_id hacks go away.


It's a bit difficult because it will affect how the container  vault
ID's are represented on the CLI.


Yes, but the API should be done right (without hacks) first. You can
tune the CLI after that if you want.


I think the current framework is rather limiting. It's kind of hard to
build an interface that looks exactly what you want then add the
implementation later to match the interface because many things are
interrelated. In this particular case the object hierarchy on the server
side would affect how the vault ID will be represented on the client
side.


It indeed is limiting and that's a good thing. We don't want people to
be able to create any crazy interfaces they can imagine, inconsistent
with everything else in IPA.


So the vault container plugin was removed because the current framework 
cannot support the hierarchical structure described in the vault design 
without overriding the default parameter handling (which was referred to 
as 'hacks', although it was actually suggested by the previous 
reviewer). Adding the missing functionality will require modifications 
to the base framework classes. Such changes should only be done after 
thoroughly evaluating the impact on the existing plugins, probably in a 
future release.



In the design the container ID would be a single value like this:

   $ ipa vault-add /services/server.example.com/HTTP

And if the vault ID is relative (without initial slash), it will be
appended to the user's private container (i.e. /users/username/):

   $ ipa vault-add PrivateVault

The implementation is not complete yet. Currently it accepts this
format:

   $ ipa vault-add vault name [--container container ID]

and I'm still planning to add this:

   $ ipa vault-add vault ID


This is actually now done in the latest patch. Internally the ID is
still split into name  parent ID.


If the vault must be a child of vaultcontainer, and the vaultcontainer
must be a child of a vaultcontainer, does it mean the vault ID would
have to be split into separate arguments like this?

   $ ipa vaultcontainer-add services server.example.com HTTP

If that's the case we'd lose the ability to specify a relative vault
ID.


Yes, that's the case.

But I don't think relative IDs should be a problem, we can do this:

 $ ipa vaultcontainer-add a b c  # absolute
 $ ipa vaultcontainer-add . c# relative


I think a . will be 

Re: [Freeipa-devel] [PATCH] Password vault

2015-03-16 Thread Endi Sukma Dewata

On 3/13/2015 2:27 AM, Endi Sukma Dewata wrote:

On 3/11/2015 9:12 PM, Endi Sukma Dewata wrote:

Thanks for the review. New patch attached to be applied on top of all
previous patches. Please see comments below.


New patch #362-1 attached replacing #362. It fixed some issues in
handle_not_found().


New patch #363 attached. It adds supports for vault  vaultcontainer ID 
parameter.


--
Endi S. Dewata
From e1d2a3a62e6d16c1c9b19f4cb19b900427ea5e1f Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Thu, 12 Mar 2015 09:21:02 -0400
Subject: [PATCH] Vault ID improvements.

The vault plugin has been modified to accept a single vault ID
in addition to separate name and parent ID attributes. The vault
container has also been modified in the same way. New test cases
have been added to verify this functionality.

https://fedorahosted.org/freeipa/ticket/3872
---
 ipalib/plugins/vault.py| 143 +--
 ipalib/plugins/vaultcontainer.py   | 137 +--
 ipalib/plugins/vaultsecret.py  |  10 +-
 ipatests/test_xmlrpc/test_vault_plugin.py  | 151 +
 ipatests/test_xmlrpc/test_vaultcontainer_plugin.py |  90 ++--
 ipatests/test_xmlrpc/test_vaultsecret_plugin.py| 115 
 6 files changed, 565 insertions(+), 81 deletions(-)

diff --git a/ipalib/plugins/vault.py b/ipalib/plugins/vault.py
index 
d47067758186601365e5924f5d13c7ab51ba66e5..38693d0710e000695cae21fb4db5dfb4c85b5c74
 100644
--- a/ipalib/plugins/vault.py
+++ b/ipalib/plugins/vault.py
@@ -61,7 +61,7 @@ EXAMPLES:
ipa vault-find
 ) + _(
  List shared vaults:
-   ipa vault-find --container-id /shared
+   ipa vault-find /shared
 ) + _(
  Add a standard vault:
ipa vault-add MyVault
@@ -171,8 +171,8 @@ class vault(LDAPObject):
 cli_name='vault_name',
 label=_('Vault name'),
 primary_key=True,
-pattern='^[a-zA-Z0-9_.-]+$',
-pattern_errmsg='may only include letters, numbers, _, ., and -',
+pattern='^[a-zA-Z0-9_.-/]+$',
+pattern_errmsg='may only include letters, numbers, _, ., -, and /',
 maxlength=255,
 ),
 Str('vault_id?',
@@ -217,7 +217,7 @@ class vault(LDAPObject):
 
 # get vault ID from parameters
 name = keys[-1]
-container_id = 
self.api.Object.vaultcontainer.normalize_id(options.get('container_id'))
+container_id = 
self.api.Object.vaultcontainer.absolute_id(options.get('container_id'))
 vault_id = container_id + name
 
 dn = self.base_dn
@@ -250,6 +250,81 @@ class vault(LDAPObject):
 
 return id
 
+def split_id(self, id):
+
+Splits a vault ID into (vault name, container ID) tuple.
+
+
+if not id:
+return (None, None)
+
+# split ID into container ID and vault name
+parts = id.rsplit(u'/', 1)
+
+if len(parts) == 2:
+vault_name = parts[1]
+container_id = u'%s/' % parts[0]
+
+else:
+vault_name = parts[0]
+container_id = None
+
+if not vault_name:
+vault_name = None
+
+return (vault_name, container_id)
+
+def merge_id(self, vault_name, container_id):
+
+Merges a vault name and a container ID into a vault ID.
+
+
+if not vault_name:
+id = container_id
+
+elif vault_name.startswith('/') or not container_id:
+id = vault_name
+
+else:
+id = container_id + vault_name
+
+return id
+
+def normalize_params(self, *args, **options):
+
+Normalizes the vault ID in the parameters.
+
+
+vault_id = self.parse_params(*args, **options)
+(vault_name, container_id) = self.split_id(vault_id)
+return self.update_params(vault_name, container_id, *args, **options)
+
+def parse_params(self, *args, **options):
+
+Extracts the vault name and container ID in the parameters.
+
+
+# get vault name and container ID from parameters
+vault_name = args[0]
+if type(vault_name) is tuple:
+vault_name = vault_name[0]
+container_id = 
self.api.Object.vaultcontainer.normalize_id(options.get('container_id'))
+
+return self.merge_id(vault_name, container_id)
+
+def update_params(self, new_vault_name, new_container_id, *args, 
**options):
+
+Stores vault name and container ID back into the parameters.
+
+
+args_list = list(args)
+args_list[0] = new_vault_name
+args = tuple(args_list)
+
+options['container_id'] = new_container_id
+
+return (args, options)
+
 def get_kra_id(self, id):
 
 Generates a client key ID to store/retrieve data in KRA.
@@ -363,10 +438,14 @@ class vault_add(LDAPCreate):
 
 msg_summary = _('Added

Re: [Freeipa-devel] [PATCH] Password vault

2015-03-12 Thread Endi Sukma Dewata
Thanks for the review. New patch attached to be applied on top of all 
previous patches. Please see comments below.


On 3/6/2015 3:53 PM, Jan Cholasta wrote:

Patch 353:

1) Please follow PEP8 in new code.

The pep8 tool reports these errors in existing files:

./ipalib/constants.py:98:80: E501 line too long (84  79 characters)
./ipalib/plugins/baseldap.py:1527:80: E501 line too long (81  79
characters)
./ipalib/plugins/user.py:915:80: E501 line too long (80  79 characters)

as well as many errors in the files this patch adds.


For some reason pylint keeps crashing during build so I cannot run it 
for all files. I'm fixing the errors that I can see. If you see other 
errors please let me know while I'm still trying to figure out the problem.


Is there an existing ticket for fixing PEP8 errors? Let's use that for 
fixing the errors in the existing code.



2) Pylint reports the following error:

ipatests/test_xmlrpc/test_vault_plugin.py:153:
[E0102(function-redefined), test_vault] class already defined line 27)


Fixed.


3) The container_vault config option should be renamed to
container_vaultcontainer, as it is used in the vaultcontainer plugin,
not the vault plugin.


It was named container_vault because it defines the DN for of the 
subtree that contains all vault-related entries. I moved the base_dn 
variable from vaultcontainer object to the vault object for clarity.



4) The vault object should be child of the vaultcontainer object.

Not only is this correct from the object model perspective, but it would
also make all the container_id hacks go away.


It's a bit difficult because it will affect how the container  vault 
ID's are represented on the CLI.


In the design the container ID would be a single value like this:

  $ ipa vault-add /services/server.example.com/HTTP

And if the vault ID is relative (without initial slash), it will be 
appended to the user's private container (i.e. /users/username/):


  $ ipa vault-add PrivateVault

The implementation is not complete yet. Currently it accepts this format:

  $ ipa vault-add vault name [--container container ID]

and I'm still planning to add this:

  $ ipa vault-add vault ID

If the vault must be a child of vaultcontainer, and the vaultcontainer 
must be a child of a vaultcontainer, does it mean the vault ID would 
have to be split into separate arguments like this?


  $ ipa vaultcontainer-add services server.example.com HTTP

If that's the case we'd lose the ability to specify a relative vault ID.


5) When specifying param flags, use set literals.

This is especially wrong, because it's not a tuple, but a string in
parentheses:

+flags=('virtual_attribute'),


Fixed.


6) The `container` param of vault should actually be an option in
vault_* commands.

Also it should be renamed to `container_id`, for consistency with
vaultcontainer.


Fixed. It was actually made to be consistent with the 'parent' attribute 
in the vaultcontainer class. Now the 'parent' has been renamed to 
'parent_id' as well.



7) The `vault_id` param of vault should have no_option in flags, since
it is output-only.


Fixed.


8) Don't translate docstrings where not necessary:

+def get_dn(self, *keys, **options):
+__doc__ = _(
+Generates vault DN from vault ID.
+)

Only plugin modules and classes should have translated docstrings.


Fixed.


9) This looks wrong in vault.get_dn() and vaultcontainer.get_dn():

+name = None
+if keys:
+name = keys[0]

Primary key of the object should always be set, so the if statement
should not be there.


Fixed.


Also, primary key of any given object is always last in keys, so use
keys[-1] instead of keys[0].


Fixed.


10) Use self.api instead of api to access the API in plugins.


Fixed.


11) No clever optimizations like this please:

+# vault DN cannot be the container base DN
+if len(dn) == len(api.Object.vaultcontainer.base_dn):
+raise ValueError('Invalid vault DN: %s' % dn)

Compare the DNs by value instead.


Actually the DN values have already been compared in the code right 
above it:


# make sure the DN is a vault DN
if not dn.endswith(self.api.Object.vaultcontainer.base_dn):
raise ValueError('Invalid vault DN: %s' % dn)

This code confirms that the incoming vault DN is within the vault 
subtree. After that, the DN length comparison above is just to make sure 
the incoming vault DN is not the root of the vault subtree itself. It 
doesn't need to compare the values again.



12) vault.split_id() is not used anywhere.


Removed.


13) Bytes is not base64-encoded data:

+Bytes('data?',
+cli_name='data',
+doc=_('Base-64 encoded binary data to archive'),
+),

It is base64-encoded in the CLI, but on the API level it is not. The doc
should say just Binary data to archive.


Fixed.


14) Use File instead of Str for input files:

+Str('in?',
+cli_name='in',
+ 

Re: [Freeipa-devel] [PATCH] Password vault

2015-03-12 Thread Endi Sukma Dewata

On 3/11/2015 9:12 PM, Endi Sukma Dewata wrote:

Thanks for the review. New patch attached to be applied on top of all
previous patches. Please see comments below.


New patch #362-1 attached replacing #362. It fixed some issues in 
handle_not_found().


--
Endi S. Dewata
From 6741138b647427cd3448ff72369dfc6fa21354aa Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Mon, 9 Mar 2015 12:09:20 -0400
Subject: [PATCH] Vault improvements.

The vault plugins have been modified to clean up the code, to fix
some issues, and to improve error handling. The LDAPCreate and
LDAPSearch classes have been refactored to allow subclasses to
provide custom error handling. The test scripts have been updated
accordingly.

https://fedorahosted.org/freeipa/ticket/3872
---
 API.txt|  50 ++--
 ipalib/plugins/baseldap.py |  35 +--
 ipalib/plugins/user.py |   6 +-
 ipalib/plugins/vault.py| 273 ++---
 ipalib/plugins/vaultcontainer.py   | 226 +
 ipalib/plugins/vaultsecret.py  | 108 
 ipatests/test_xmlrpc/test_vault_plugin.py  |   2 +-
 ipatests/test_xmlrpc/test_vaultcontainer_plugin.py |  24 +-
 ipatests/test_xmlrpc/test_vaultsecret_plugin.py|   2 +-
 9 files changed, 371 insertions(+), 355 deletions(-)

diff --git a/API.txt b/API.txt
index 
ffbffa78cde372d5c7027b758be58bf07caebbc6..3a741755ab3e15e0175599a16a090b04d46d6be8
 100644
--- a/API.txt
+++ b/API.txt
@@ -4518,7 +4518,7 @@ args: 1,20,3
 arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, required=True)
 option: Str('addattr*', cli_name='addattr', exclude='webui')
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
-option: Str('container', attribute=False, cli_name='container', 
multivalue=False, pattern='^[a-zA-Z0-9_.-/]+$', required=False)
+option: Str('container_id?', cli_name='container_id')
 option: Bytes('data?', cli_name='data')
 option: Str('description', attribute=True, cli_name='desc', multivalue=False, 
required=False)
 option: Str('escrow_public_key_file?', cli_name='escrow_public_key_file')
@@ -4543,7 +4543,7 @@ command: vault_add_member
 args: 1,7,3
 arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
-option: Str('container?', cli_name='container')
+option: Str('container_id?', cli_name='container_id')
 option: Str('group*', alwaysask=True, cli_name='groups', csv=True)
 option: Flag('no_members', autofill=True, default=False, exclude='webui')
 option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
@@ -4556,7 +4556,7 @@ command: vault_add_owner
 args: 1,7,3
 arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
-option: Str('container?', cli_name='container')
+option: Str('container_id?', cli_name='container_id')
 option: Str('group*', alwaysask=True, cli_name='groups', csv=True)
 option: Flag('no_members', autofill=True, default=False, exclude='webui')
 option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
@@ -4569,7 +4569,7 @@ command: vault_archive
 args: 1,15,3
 arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
-option: Str('container?', cli_name='container')
+option: Str('container_id?', cli_name='container_id')
 option: Bytes('data?', cli_name='data')
 option: Bytes('encryption_key?', cli_name='encryption_key')
 option: Str('in?', cli_name='in')
@@ -4589,7 +4589,7 @@ output: PrimaryKey('value', None, None)
 command: vault_del
 args: 1,3,3
 arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=True, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
-option: Str('container?', cli_name='container')
+option: Str('container_id?', cli_name='container_id')
 option: Flag('continue', autofill=True, cli_name='continue', default=False)
 option: Str('version?', exclude='webui')
 output: Output('result', type 'dict', None)
@@ -4600,7 +4600,7 @@ args: 1,15,4
 arg: Str('criteria?', noextrawhitespace=False)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
 option: Str('cn', attribute=True, autofill=False, cli_name='vault_name', 
maxlength=255, multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True

Re: [Freeipa-devel] [PATCH] 353 Added initial vault implementation.

2015-01-26 Thread Endi Sukma Dewata
Sorry for the long delay. Attached is an updated patch addressing most 
of the concerns. I think the rest can be addressed in subsequent patches.


On 11/5/2014 4:06 AM, Petr Viktorin wrote:

ipapython/dn.py: This change is not needed. If you have a sequence of
RNDs you can do `DN(*seq)`.


This is actually needed to construct a parent DN from an existing DN:

   parent_dn = DN(dn[1:])


Right, for that you can do
 parent_dn = DN(*dn[1:])
(note the asterisk)


Fixed.


All existing plugins are actually using LDAPObject directly, not
baseldap.LDAPObject. Is there a concern doing that? I fixed the code to
do the same thing.


Your choice.
If there are many objects imported from a module I prefer importing the
module itself, but individual names work too.


For now I'll keep it consistent with the other plugins. There aren't 
that many objects imported, and it's easier to copy codes between plugins.



The pattern and pattern_errmsg for the 'cn' param don't match. Which one
is right? Shouldn't a similar check be there for parent?


This is actually copied verbatim from user's uid. Could you give some
sample values that show the problem?


These messages are misleading:

$ ipa vaultcontainer-add '$$'
ipa: ERROR: invalid 'container_name': may only include letters, numbers,
_, -, . and $
$ ipa vaultcontainer-add -- '-myvault-'
ipa: ERROR: invalid 'container_name': may only include letters, numbers,
_, -, . and $

(Note that in webUI you wouldn't need the awkward quoting and `--`)


I think ideally there should be a similar check for parent, but I
haven't figured out the proper pattern yet. I think we can do this as a
later enhancement. Or maybe we should remove the validation for cn for
now.


Well, what are the PKI limitations on ids?
Are there any best practices for the id names?


I don't think KRA itself puts any restrictions on the ID, but see the 
problem with get_active_key_info() below.



I think we want to limit the names to avoid nasty surprises later;
'[a-zA-Z0-9_.-]+' seems like a good set.
For parent you also need the slash, otherwise use the same pattern.

I think this is quite an important decision; weird names could lead to
security issues.


OK, I changed the container name to use that pattern and added slash in 
the parent's pattern.



vaultcontainer.get_dn: Why put '/' at the end of container_id, if the
empty string is ignored later anyway?


I'm still considering some options for container ID:
a) use a slash at the end for all containers (e.g. /users/)
b) don't use a trailing slash, except for root container (/)

Maybe a better way is to use an array of names internally and the slash
is only used during input/output. The problem is when calling another
command the array has to be converted into string and parsed back into
array, so it may even introduce more redundancies. I don't see a way to
pass an object parameter to a command. Any suggestion?


That would indeed be better. Hopefully you can avoid re-parsing and
re-validating by not calling nested Commands, see below.


In vaultcontainer.get_id, what's the purpose of the len() check? Did you
want dn.endswith() instead?


Yes, but my assumption is the DN will always be within the namespace
because it's only used internally, and using len() will be faster than
endswith(). The error case will never happen, but I was just making sure
there won't be an infinite loop. Maybe instead of calling the method
recursively I should just use a loop? That will avoid repetitive
validations.


Well, if you end up with representing the id as a list the problem will
largely go away, but: use an internal error (e.g. ValueError) for
internal sanity checking. The public errors are for user errors, they
are presented nicely which makes them much harder to debug. And
Invalid container DN with no indication of what the DN is isn't
helpful to the user either.


I revised the code to use endswith() and a loop instead of recursion, 
but I'm keeping the string ID for now. I changed the exception to 
ValueError and included the DN value. We can improve this later.



I sugggest writing doctests for the id manipulation methods (get_id,
get_private_id, ...) – it's not always obvious what exactly they do.


Do you mean sample code in docs? Is it still necessary if we have some
test code?


Not strictly necessary, but it better explains what the method does.


I'm not sure how to write doctests for plugin objects/commands. I don't 
see any example in other plugins. Do I need to write the API 
initialization code? What if the output depends on the authenticated 
user? I suppose we can add this later. I already added some unit tests.



vaultcontainer_add: You should use ldap backend directly. Calling
Commands is costly, most of the call is spent doing validation of what
here is already validated. You should add a function to recursively
create vault container using just the ldap client, and call that here
and in vault_add.


Hmm.. I'm not sure if we should write a 

Re: [Freeipa-devel] [PATCH] 356 Added command to retrieve vault transport certificate.

2015-01-26 Thread Endi Sukma Dewata

On 11/4/2014 12:29 AM, Endi Sukma Dewata wrote:

On 10/28/2014 6:26 PM, Endi Sukma Dewata wrote:

On 10/23/2014 6:18 AM, Jan Cholasta wrote:

Hi,

Dne 22.10.2014 v 22:06 Endi Sukma Dewata napsal(a):

A new command has been added to retrieve the vault transport
certificate and optionally save it into a file. The vault archive
and retrieve command has been modified to retrieve the transport
certificate and store it locally for subsequent usage. This way
it's no longer necessary to manually import the transport
certificate into the client's NSS database.


As part of the CA certificate renewal feature in 4.1, I have added a
LDAP certificate store to IPA, see
http://www.freeipa.org/page/V4/CA_certificate_renewal. Currently it
supports only CA certificates, but can be extended to support end entity
certificates rather easily. If you use it for the vault transport
certificate, it can be added to the client NSS database automatically on
install.

Honza



I'm attaching a new patch that's identical to the previous one with
ticket URL updated. I'm thinking we should check this patch in first
because it's already done, and then investigate the use of CA cert
management utility as a separate enhancement since the it seems to need
to be generalized before it can be used to manage KRA transport cert.
I'll also need to investigate the KRA transport cert replacement process
to make sure it can be accommodated via IPA's cert management utility.


Revised the patch to always download the transport certificate (no local
caching). Further optimization can be done later.


Rebased on top of #355-3, no code changes.

--
Endi S. Dewata
From 56e1211f42faa1e4c36184efdcddc3af23310cda Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Wed, 22 Oct 2014 10:02:25 -0400
Subject: [PATCH] Added command to retrieve vault transport certificate.

A new command has been added to retrieve the vault transport
certificate and optionally save it into a file. The vault archive
and retrieve command has been modified to retrieve the transport
certificate and store it locally for subsequent usage. This way
it's no longer necessary to manually import the transport
certificate into the client's NSS database.

https://fedorahosted.org/freeipa/ticket/3872
---
 API.txt |  5 
 VERSION |  4 +--
 ipalib/plugins/vault.py | 80 +++--
 3 files changed, 84 insertions(+), 5 deletions(-)

diff --git a/API.txt b/API.txt
index 
f6fd2686a49dfabc053a772818904ca6c14f3b53..f64fd1570efff9a2eb2e277e16ceef2ebb1a9ae3
 100644
--- a/API.txt
+++ b/API.txt
@@ -4676,6 +4676,11 @@ option: Str('version?', exclude='webui')
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
+command: vault_transport_cert
+args: 0,2,1
+option: Str('out?', cli_name='out')
+option: Str('version?', exclude='webui')
+output: Output('result', None, None)
 command: vaultcontainer_add
 args: 1,9,3
 arg: Str('cn', attribute=True, cli_name='container_name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, required=True)
diff --git a/VERSION b/VERSION
index 
b0250045145d17e40df3828d0c50be1db8867625..c75e5376c1041a701908ea7e23c8b4e570eb21b6
 100644
--- a/VERSION
+++ b/VERSION
@@ -90,5 +90,5 @@ IPA_DATA_VERSION=2010061412
 #  #
 
 IPA_API_VERSION_MAJOR=2
-IPA_API_VERSION_MINOR=114
-# Last change: edewata - added vault access control
+IPA_API_VERSION_MINOR=115
+# Last change: edewata - added vault transport certificate
diff --git a/ipalib/plugins/vault.py b/ipalib/plugins/vault.py
index 
7ec36836a741bc0669ab1be59192532136e8e126..58f0ae494c0d0c2c3a66f708cbe7c1545bdeec84
 100644
--- a/ipalib/plugins/vault.py
+++ b/ipalib/plugins/vault.py
@@ -25,6 +25,8 @@ import shutil
 import string
 import tempfile
 
+import nss.nss as nss
+
 import pki
 import pki.account
 import pki.crypto
@@ -118,7 +120,6 @@ EXAMPLES:
 )
 
 register = Registry()
-transport_cert_nickname = 'KRA Transport Certificate'
 
 @register()
 class vaultcontainer(LDAPObject):
@@ -845,6 +846,63 @@ class vault_show(LDAPRetrieve):
 
 
 @register()
+class vault_transport_cert(Command):
+__doc__ = _('Retrieve vault transport certificate.')
+
+
+# list of attributes we want exported to JSON
+json_friendly_attributes = (
+'takes_args',
+)
+
+takes_options = (
+Str('out?',
+cli_name='out',
+doc=_('Output file to store the transport certificate'),
+),
+)
+
+has_output_params = (
+Str('certificate',
+label=_('Certificate'),
+),
+)
+
+def __json__(self):
+json_dict = dict(
+(a, getattr(self

Re: [Freeipa-devel] [PATCH] 355 Added vault access control.

2015-01-26 Thread Endi Sukma Dewata

On 11/4/2014 12:28 AM, Endi Sukma Dewata wrote:

On 10/28/2014 5:35 PM, Endi Sukma Dewata wrote:

On 10/22/2014 3:04 PM, Endi Sukma Dewata wrote:

New LDAP ACIs have been added to allow users to create their own
private vault container, to allow owners to manage vaults and
containers, and to allow members to use the vaults. New CLIs have
been added to manage the owner and member list. For archive and
retrieve operations the access control has to be enforced by the
plugins because the operations only affects KRA. The LDAP schema
has been updated as well.

Ticket #3872

This patch depends on #353-2.


New patch attached to fix the ticket URL. It depends on #353-3.


New patch attached for some cleanups.


Rebased on top of #353-5 to include access control tests.

--
Endi S. Dewata
From b4591d364a9e38de66536de596347dd08301173e Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Fri, 17 Oct 2014 12:05:34 -0400
Subject: [PATCH] Added vault access control.

New LDAP ACIs have been added to allow users to create their own
private vault container, to allow owners to manage vaults and
containers, and to allow members to use the vaults. New CLIs have
been added to manage the owner and member list. For archive and
retrieve operations the access control has to be enforced by the
plugins because the operations only affects KRA. The LDAP schema
has been updated as well.

https://fedorahosted.org/freeipa/ticket/3872
---
 API.txt   | 134 +++--
 VERSION   |   4 +-
 install/share/60basev3.ldif   |   4 +-
 install/updates/40-vault.update   |   7 +
 ipalib/plugins/vault.py   | 238 +-
 ipatests/test_xmlrpc/test_vault_plugin.py |  17 +++
 6 files changed, 389 insertions(+), 15 deletions(-)

diff --git a/API.txt b/API.txt
index 
559f4b97fad334f037cb61bbb787f7dfcbd6e23c..f6fd2686a49dfabc053a772818904ca6c14f3b53
 100644
--- a/API.txt
+++ b/API.txt
@@ -4518,12 +4518,13 @@ output: Output('result', type 'bool', None)
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
 command: vault_add
-args: 1,10,3
+args: 1,11,3
 arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, required=True)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
 option: Bytes('data?', cli_name='data')
 option: Str('description', attribute=True, cli_name='desc', multivalue=False, 
required=False)
 option: Str('in?', cli_name='in')
+option: Flag('no_members', autofill=True, default=False, exclude='webui')
 option: Str('parent', attribute=False, cli_name='parent', multivalue=False, 
pattern='^[a-zA-Z0-9_.-/]+$', required=False)
 option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
 option: Flag('rights', autofill=True, default=False)
@@ -4533,13 +4534,40 @@ option: Str('version?', exclude='webui')
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
+command: vault_add_member
+args: 1,7,3
+arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Str('group*', alwaysask=True, cli_name='groups', csv=True)
+option: Flag('no_members', autofill=True, default=False, exclude='webui')
+option: Str('parent?', cli_name='parent')
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Str('user*', alwaysask=True, cli_name='users', csv=True)
+option: Str('version?', exclude='webui')
+output: Output('completed', type 'int', None)
+output: Output('failed', type 'dict', None)
+output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
+command: vault_add_owner
+args: 1,7,3
+arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, pattern='^[a-zA-Z0-9_.-]+$', primary_key=True, query=True, 
required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Str('group*', alwaysask=True, cli_name='groups', csv=True)
+option: Flag('no_members', autofill=True, default=False, exclude='webui')
+option: Str('parent?', cli_name='parent')
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Str('user*', alwaysask=True, cli_name='users', csv=True)
+option: Str('version?', exclude='webui')
+output: Output('completed', type 'int', None)
+output: Output('failed', type 'dict', None)
+output: Entry('result', type 'dict', Gettext('A dictionary representing

Re: [Freeipa-devel] FreeIPA integration with external DNS services

2014-12-10 Thread Endi Sukma Dewata

On 12/10/2014 9:59 PM, Petr Spacek wrote:

Alternatively we can use Vault for TSIG key storage and use Vault's capability
to share keys among many users. In that case we don't have problem with key
distribution nor authorization.


I am not convinced we should grow Vault dependency for this functionality, it
requires the whole PKI machinery to be available. Many deployments do not use
it though.


Another possibility is to say that replica-deletion can be done only by root
user or that updates to external DNS are supported only when running
ipa-replica-manage as root.


Why does root help? So that TSIG keys will be available on all IPA servers,
regardless whether they have DNS service running or not?

The point is that we need to store TSIG keys somewhere to make them
available to ipa* scripts on all replica but at the same not to human-users.

BTW there don't need to be any 'DNS service' if only external DNS integration
is in use.


It would be better to have the TSIG keys available using standard FreeIPA RBAC
roles, i.e. DS ACIs so that privileged users can also use the TSIG. Or am I
missing anything?


Ideologically - no, it sounds fine.
Technically - the problem is in implementation. We need a mechanism for secure
key distribution *among users*, i.e. Vault-like functionality. I would rather
not re-implement Vault just for external DNS.

I think that external DNS could depend on Vault (assuming that external DNS
support will be purely optional).

DNSSEC key distribution is very different because it distributes keys to
*servers* and there is no ACI mechanism on top of that - all DNS servers need
to know all the keys anyway.


So IIUC, the goal would be to put TSIG keys in the Vault and then make them
available for all IPA servers?

I am now not sure if the Vault as proposed can easily select a group of
principals to allow reading the Vault or if it is only confined for the
owner/user principal. We would need to ask Endi (CCed).


I assumed that very first sentence of
http://www.freeipa.org/page/V4/Password_Vault_Implementation#Vault
Vault is a secure place to store data or a collection of secrets. A vault may
be privately owned by a user, or shared among users, groups, or services.
is correct :-)

Endi, we would like to have one secret which is shared among services  users
at the same time. Is it possible to do that with Vault?


Yes, the access to a particular vault is limited to the vault members  
owners which can be a set of users, groups, and/or services. See 
http://www.freeipa.org/page/V4/Password_Vault_Implementation#Access_Control


A vault member can list, archive, and retrieve the secrets in a standard 
vault. With symmetric vault the member will need a vault password in 
order to archive and retrieve secrets. With asymmetric vault the member 
can only archive the secrets but not retrieve them since it only has the 
public key and not the private key.


A vault owner can list, archive, and retrieve the secrets like vault 
members, but it has the private key so it can retrieve secrets from 
asymmetric vault. The owner can also manage the list of members and 
owners of the vault, and change the vault password/keys.


There are commands to manage the members/owners. See 
http://www.freeipa.org/page/V4/Password_Vault_Implementation#Adding_vault_member


$ ipa vault-add-member MyVault --groups group1,group2
$ ipa vault-add-owner MyVault --users user1,user2
(we may need a separate option for services)

Below are the vault LDAP ACIs. See 
http://www.freeipa.org/page/V4/Password_Vault_Implementation#Access_Control_Attributes


aci: (targetfilter=(objectClass=ipaVault))
  (targetattr=*)
  (version 3.0; acl Vault members can access the vault;
   allow(read, search, compare) userattr=member#USERDN;)
aci: (targetfilter=(objectClass=ipaVault))
  (targetattr=*)
  (version 3.0; acl Indirect vault members can access the vault;
   allow(read, search, compare) userattr=member#GROUPDN;)

aci: (targetfilter=(objectClass=ipaVault))
  (targetattr=*)
  (version 3.0; acl Vault owners can modify the vault;
   allow(read, search, compare, write) userattr=owner#USERDN;)
aci: (targetfilter=(objectClass=ipaVault))
  (targetattr=*)
  (version 3.0; acl Indirect vault owners can modify the vault;
   allow(read, search, compare, write) userattr=owner#GROUPDN;)

Note that the secrets are stored separately in KRA, not in the IPA LDAP 
tree, so they are not directly controlled by the above ACIs. The vault 
service will first verify that you have access to the vault based on the 
above ACIs then it will let you archive/retrieve secrets in KRA.


--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 353 Added initial vault implementation.

2014-11-04 Thread Endi Sukma Dewata
Thanks for the review. I have some questions below. I'll post a new 
patch after the issues are addressed.


On 11/4/2014 11:36 AM, Petr Viktorin wrote:

The new schema can go to 60basev3.ldif, no need for a new file.


Fixed. Also removed nsContainer as suggested by Simo.


ipalib/plugins/user.py: Make the try: block as small as possible; maybe
something like this:

  try:
  vaultcontainer_id = ...
  except errors.NotFound:
  pass
  else:
 (vaultcontainer_name, vaultcontainer_parent_id) = ...
 self.api.Command.vaultcontainer_del(...)


Actually the command that generates the NotFound exception is the last 
command. The first two commands are preparing the parameters. I moved 
them before the try-block.



ipapython/dn.py: This change is not needed. If you have a sequence of
RNDs you can do `DN(*seq)`.


This is actually needed to construct a parent DN from an existing DN:

  parent_dn = DN(dn[1:])

Note that the parameter is an array of RDNs, not a sequence of RDNs. I 
don't see an existing mechanism to get a parent DN from a DN. Without 
this change it would generate an error:


ValueError: tuple or list must be 2-valued, not 
[ipapython.dn.RDN('cn=admin'), ipapython.dn.RDN('cn=users'), 
ipapython.dn.RDN('cn=vaults'), ipapython.dn.RDN('dc=idm'), 
ipapython.dn.RDN('dc=lab'), ipapython.dn.RDN('dc=bos'), 
ipapython.dn.RDN('dc=redhat'), ipapython.dn.RDN('dc=com')]



ipalib/plugins/vault.py:
Do not use star imports in new code; remove the line
  from ipalib.plugins.baseldap import *
and use e.g. `baseldap.LDAPObject` instead of just `LDAPObject`.
Hopefully the only star-imported used are the base classes?


I found 20 star imports in the existing plugins :/

All existing plugins are actually using LDAPObject directly, not 
baseldap.LDAPObject. Is there a concern doing that? I fixed the code to 
do the same thing.



To make translators' jobs easier, split the help text in __doc__ into
strings that can be translated individually.
Replace every blank line by:
) + _(


Fixed.


The pattern and pattern_errmsg for the 'cn' param don't match. Which one
is right? Shouldn't a similar check be there for parent?


This is actually copied verbatim from user's uid. Could you give some 
sample values that show the problem?


I think ideally there should be a similar check for parent, but I 
haven't figured out the proper pattern yet. I think we can do this as a 
later enhancement. Or maybe we should remove the validation for cn for now.



vaultcontainer.get_dn: Why put '/' at the end of container_id, if the
empty string is ignored later anyway?


I'm still considering some options for container ID:
a) use a slash at the end for all containers (e.g. /users/)
b) don't use a trailing slash, except for root container (/)

I think option (a) is more consistent, container ID always ends with 
slash, although in cases like this it can be redundant. This is how the 
code is currently written. We can make an exception for 
vaultcontainer.get_dn() though.


Option (b) will require more careful coding in many places (e.g. 
concatenations) because it needs to handle a special case for root 
container.


Maybe a better way is to use an array of names internally and the slash 
is only used during input/output. The problem is when calling another 
command the array has to be converted into string and parsed back into 
array, so it may even introduce more redundancies. I don't see a way to 
pass an object parameter to a command. Any suggestion?



Nitpick: In vaultcontainer.get_dn, prefer the if statement over the
if-else expressions; it's a bit longer but more readable.


Fixed.


In vaultcontainer.get_id, what's the purpose of the len() check? Did you
want dn.endswith() instead?


Yes, but my assumption is the DN will always be within the namespace 
because it's only used internally, and using len() will be faster than 
endswith(). The error case will never happen, but I was just making sure 
there won't be an infinite loop. Maybe instead of calling the method 
recursively I should just use a loop? That will avoid repetitive 
validations.



I sugggest writing doctests for the id manipulation methods (get_id,
get_private_id, ...) – it's not always obvious what exactly they do.


Do you mean sample code in docs? Is it still necessary if we have some 
test code?



According to the design doc, container IDs shouldn't end in '/'. If you
did that I think the manipulation would be simpler, and it could be
shared with the vault equivalents.


Yeah, as mentioned above I'm still considering several options. The 
design may need to be adjusted.



vaultcontainer_add: You should use ldap backend directly. Calling
Commands is costly, most of the call is spent doing validation of what
here is already validated. You should add a function to recursively
create vault container using just the ldap client, and call that here
and in vault_add.


Hmm.. I'm not sure if we should write a code that will 

Re: [Freeipa-devel] [PATCH] 355 Added vault access control.

2014-11-03 Thread Endi Sukma Dewata

On 10/28/2014 5:35 PM, Endi Sukma Dewata wrote:

On 10/22/2014 3:04 PM, Endi Sukma Dewata wrote:

New LDAP ACIs have been added to allow users to create their own
private vault container, to allow owners to manage vaults and
containers, and to allow members to use the vaults. New CLIs have
been added to manage the owner and member list. For archive and
retrieve operations the access control has to be enforced by the
plugins because the operations only affects KRA. The LDAP schema
has been updated as well.

Ticket #3872

This patch depends on #353-2.


New patch attached to fix the ticket URL. It depends on #353-3.


New patch attached for some cleanups.

--
Endi S. Dewata
From a9714d34d180ce26c3d484fe313c991fa101ccfb Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Fri, 17 Oct 2014 12:05:34 -0400
Subject: [PATCH] Added vault access control.

New LDAP ACIs have been added to allow users to create their own
private vault container, to allow owners to manage vaults and
containers, and to allow members to use the vaults. New CLIs have
been added to manage the owner and member list. For archive and
retrieve operations the access control has to be enforced by the
plugins because the operations only affects KRA. The LDAP schema
has been updated as well.

https://fedorahosted.org/freeipa/ticket/3872
---
 API.txt | 134 +++---
 VERSION |   4 +-
 install/share/60basev4.ldif |   4 +-
 install/updates/40-vault.update |   7 ++
 ipalib/plugins/vault.py | 177 +++-
 5 files changed, 310 insertions(+), 16 deletions(-)

diff --git a/API.txt b/API.txt
index 
b73da0af55a3c514de73ae4e1b2a4d13c01c903d..ee33af74eb2870fcdf1ab7e6781797b348243a8f
 100644
--- a/API.txt
+++ b/API.txt
@@ -4476,12 +4476,13 @@ output: Output('result', type 'bool', None)
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
 command: vault_add
-args: 1,10,3
+args: 1,11,3
 arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, required=True)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
 option: Bytes('data?', cli_name='data')
 option: Str('description', attribute=True, cli_name='desc', multivalue=False, 
required=False)
 option: Str('in?', cli_name='in')
+option: Flag('no_members', autofill=True, default=False, exclude='webui')
 option: Str('parent', attribute=False, cli_name='parent', multivalue=False, 
required=False)
 option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
 option: Flag('rights', autofill=True, default=False)
@@ -4491,13 +4492,40 @@ option: Str('version?', exclude='webui')
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
+command: vault_add_member
+args: 1,7,3
+arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, query=True, required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Str('group*', alwaysask=True, cli_name='groups', csv=True)
+option: Flag('no_members', autofill=True, default=False, exclude='webui')
+option: Str('parent?', cli_name='parent')
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Str('user*', alwaysask=True, cli_name='users', csv=True)
+option: Str('version?', exclude='webui')
+output: Output('completed', type 'int', None)
+output: Output('failed', type 'dict', None)
+output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
+command: vault_add_owner
+args: 1,7,3
+arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, query=True, required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Str('group*', alwaysask=True, cli_name='groups', csv=True)
+option: Flag('no_members', autofill=True, default=False, exclude='webui')
+option: Str('parent?', cli_name='parent')
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Str('user*', alwaysask=True, cli_name='users', csv=True)
+option: Str('version?', exclude='webui')
+output: Output('completed', type 'int', None)
+output: Output('failed', type 'dict', None)
+output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 command: vault_archive
-args: 1,11,3
+args: 1,12,3
 arg: Str('cn

Re: [Freeipa-devel] [PATCH] 356 Added command to retrieve vault transport certificate.

2014-11-03 Thread Endi Sukma Dewata

On 10/28/2014 6:26 PM, Endi Sukma Dewata wrote:

On 10/23/2014 6:18 AM, Jan Cholasta wrote:

Hi,

Dne 22.10.2014 v 22:06 Endi Sukma Dewata napsal(a):

A new command has been added to retrieve the vault transport
certificate and optionally save it into a file. The vault archive
and retrieve command has been modified to retrieve the transport
certificate and store it locally for subsequent usage. This way
it's no longer necessary to manually import the transport
certificate into the client's NSS database.


As part of the CA certificate renewal feature in 4.1, I have added a
LDAP certificate store to IPA, see
http://www.freeipa.org/page/V4/CA_certificate_renewal. Currently it
supports only CA certificates, but can be extended to support end entity
certificates rather easily. If you use it for the vault transport
certificate, it can be added to the client NSS database automatically on
install.

Honza



I'm attaching a new patch that's identical to the previous one with
ticket URL updated. I'm thinking we should check this patch in first
because it's already done, and then investigate the use of CA cert
management utility as a separate enhancement since the it seems to need
to be generalized before it can be used to manage KRA transport cert.
I'll also need to investigate the KRA transport cert replacement process
to make sure it can be accommodated via IPA's cert management utility.


Revised the patch to always download the transport certificate (no local 
caching). Further optimization can be done later.


--
Endi S. Dewata
From ef9af13e577343d3ff2564e81b9b6dc895a03b09 Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Wed, 22 Oct 2014 10:02:25 -0400
Subject: [PATCH] Added command to retrieve vault transport certificate.

A new command has been added to retrieve the vault transport
certificate and optionally save it into a file. The vault archive
and retrieve command has been modified to retrieve the transport
certificate and store it locally for subsequent usage. This way
it's no longer necessary to manually import the transport
certificate into the client's NSS database.

https://fedorahosted.org/freeipa/ticket/3872
---
 API.txt |  5 
 VERSION |  4 +--
 ipalib/plugins/vault.py | 80 +++--
 3 files changed, 84 insertions(+), 5 deletions(-)

diff --git a/API.txt b/API.txt
index 
ee33af74eb2870fcdf1ab7e6781797b348243a8f..7668e8ceebb1a2b6e6ebcd6d70c9209f5a874627
 100644
--- a/API.txt
+++ b/API.txt
@@ -4633,6 +4633,11 @@ option: Str('version?', exclude='webui')
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
+command: vault_transport_cert
+args: 0,2,1
+option: Str('out?', cli_name='out')
+option: Str('version?', exclude='webui')
+output: Output('result', None, None)
 command: vaultcontainer_add
 args: 1,9,3
 arg: Str('cn', attribute=True, cli_name='container_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, required=True)
diff --git a/VERSION b/VERSION
index 
c471ed80af6a2c26be7fc89281ae60fac6c68577..d0ada131b700e93faa8c4946b811db36d76341a9
 100644
--- a/VERSION
+++ b/VERSION
@@ -90,5 +90,5 @@ IPA_DATA_VERSION=2010061412
 #  #
 
 IPA_API_VERSION_MAJOR=2
-IPA_API_VERSION_MINOR=110
-# Last change: edewata - added vault access control
+IPA_API_VERSION_MINOR=111
+# Last change: edewata - added vault transport certificate
diff --git a/ipalib/plugins/vault.py b/ipalib/plugins/vault.py
index 
20948b267e2c585bd59e801d02f9e0b752399609..00a7a3471d79eee741a1986224c3ad534ec5ff3d
 100644
--- a/ipalib/plugins/vault.py
+++ b/ipalib/plugins/vault.py
@@ -24,6 +24,8 @@ import shutil
 import string
 import tempfile
 
+import nss.nss as nss
+
 import pki
 import pki.account
 import pki.crypto
@@ -109,7 +111,6 @@ EXAMPLES:
 )
 
 register = Registry()
-transport_cert_nickname = KRA Transport Certificate
 
 @register()
 class vaultcontainer(LDAPObject):
@@ -693,6 +694,63 @@ class vault_show(LDAPRetrieve):
 
 
 @register()
+class vault_transport_cert(Command):
+__doc__ = _('Retrieve vault transport certificate.')
+
+
+# list of attributes we want exported to JSON
+json_friendly_attributes = (
+'takes_args',
+)
+
+takes_options = (
+Str('out?',
+cli_name='out',
+doc=_('Output file to store the transport certificate'),
+),
+)
+
+has_output_params = (
+Str('certificate',
+label=_('Certificate'),
+),
+)
+
+def __json__(self):
+json_dict = dict(
+(a, getattr(self, a)) for a in self.json_friendly_attributes
+)
+json_dict

[Freeipa-devel] [PATCH] 357 Added symmetric and asymmetric vaults.

2014-11-03 Thread Endi Sukma Dewata

The IPA vault has been modified to support symmetric and asymmetric
vaults to allow client to pre-encrypt the data. Due to the status
of the crypto library the actual encryption will be added separately
later.

New LDAP attribute types have been added to store vault type, salt
and public key.

https://fedorahosted.org/freeipa/ticket/3872

--
Endi S. Dewata
From 062d86c4bf4f58eadb863cbcd01bd39ef30691d8 Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Fri, 24 Oct 2014 19:53:16 -0400
Subject: [PATCH] Added symmetric and asymmetric vaults.

The IPA vault has been modified to support symmetric and asymmetric
vaults to allow client to pre-encrypt the data. Due to the status
of the crypto library the actual encryption will be added separately
later.

New LDAP attribute types have been added to store vault type, salt
and public key.

https://fedorahosted.org/freeipa/ticket/3872
---
 API.txt |  27 +++-
 VERSION |   4 +-
 install/share/60basev4.ldif |   7 +-
 ipalib/plugins/vault.py | 332 +++-
 4 files changed, 358 insertions(+), 12 deletions(-)

diff --git a/API.txt b/API.txt
index 
7668e8ceebb1a2b6e6ebcd6d70c9209f5a874627..7c4a87dcab4a523977cc63341801120816088db1
 100644
--- a/API.txt
+++ b/API.txt
@@ -4476,14 +4476,20 @@ output: Output('result', type 'bool', None)
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
 command: vault_add
-args: 1,11,3
+args: 1,17,3
 arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, required=True)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
 option: Bytes('data?', cli_name='data')
 option: Str('description', attribute=True, cli_name='desc', multivalue=False, 
required=False)
 option: Str('in?', cli_name='in')
+option: Bytes('ipavaultpublickey', attribute=True, cli_name='public_key', 
multivalue=False, required=False)
+option: Bytes('ipavaultsalt', attribute=True, cli_name='salt', 
multivalue=False, required=False)
+option: Str('ipavaulttype', attribute=True, autofill=True, cli_name='type', 
default=u'standard', multivalue=False, required=False)
 option: Flag('no_members', autofill=True, default=False, exclude='webui')
 option: Str('parent', attribute=False, cli_name='parent', multivalue=False, 
required=False)
+option: Str('password?', cli_name='password')
+option: Str('password_file?', cli_name='password_file')
+option: Str('public_key_file?', cli_name='public_key_file')
 option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
 option: Flag('rights', autofill=True, default=False)
 option: Str('text?', cli_name='text')
@@ -4519,7 +4525,7 @@ output: Output('completed', type 'int', None)
 output: Output('failed', type 'dict', None)
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 command: vault_archive
-args: 1,12,3
+args: 1,14,3
 arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, query=True, required=True)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
 option: Bytes('data?', cli_name='data')
@@ -4528,6 +4534,8 @@ option: Str('in?', cli_name='in')
 option: Flag('no_members', autofill=True, default=False, exclude='webui')
 option: Bytes('nonce?', cli_name='nonce')
 option: Str('parent?', cli_name='parent')
+option: Str('password?', cli_name='password')
+option: Str('password_file?', cli_name='password_file')
 option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
 option: Flag('rights', autofill=True, default=False)
 option: Str('text?', cli_name='text')
@@ -4546,11 +4554,14 @@ output: Output('result', type 'dict', None)
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: ListOfPrimaryKeys('value', None, None)
 command: vault_find
-args: 1,11,4
+args: 1,14,4
 arg: Str('criteria?', noextrawhitespace=False)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
 option: Str('cn', attribute=True, autofill=False, cli_name='vault_name', 
maxlength=255, multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, query=True, required=False)
 option: Str('description', attribute=True, autofill=False, cli_name='desc', 
multivalue=False, query=True, required=False)
+option: Bytes('ipavaultpublickey', attribute=True, autofill=False, 
cli_name='public_key', multivalue=False, query=True, required=False)
+option: Bytes('ipavaultsalt', attribute=True, autofill=False, cli_name='salt', 
multivalue=False, query=True, required=False)
+option: Str('ipavaulttype', attribute=True, autofill=False, 

Re: [Freeipa-devel] [PATCH] 352 Fixed KRA backend.

2014-10-29 Thread Endi Sukma Dewata

New patch attached.

On 10/29/2014 7:58 AM, Petr Viktorin wrote:

Dependency is bumped to 10.2.1-0.1 which is available from my
COPR repo:

   dnf copr enable edewata/pki


OK. We should get that to an IPA COPR before merging this.


How do we do that? Here is the SRPM:
https://edewata.fedorapeople.org/pki/copr/pki-core-10.2.1-0.1.fc20.src.rpm


ipaserver.plugins.dogtag.kra.get_client:
Should every caller check if this returns None?
If not, raise an exception instead.
If yes, at least mention it in a docstring.


Fixed. It's now raising a generic exception.

Is there an existing exception that is more appropriate for backend
issues like this?


I'd go for RuntimeError.


Fixed.


Don't use translatable strings (the _ function) if you're not using
ipalib.PublicError subclasses.


Fixed.


How can I do some basic smoke check on this? Is there something I still
need to to besides ipa-kra-istall? Any other patches?
I tried:

from ipalib import api
from pki.key import KeyClient
api.bootstrap(context='server')
api.finalize()
keyclient = api.Backend.kra.get_client()
keyclient.keys.archive_key('test3', KeyClient.PASS_PHRASE_TYPE, 'tkey')

which gives me:

Traceback (most recent call last):
   File stdin, line 1, in module
   File /usr/lib/python2.7/site-packages/pki/__init__.py, line 295, in
handler
 return fn_call(inst, *args, **kwargs)
   File /usr/lib/python2.7/site-packages/pki/key.py, line 687, in
archive_key
 nonce_iv = self.crypto.generate_nonce_iv()
   File /usr/lib/python2.7/site-packages/pki/crypto.py, line 176, in
generate_nonce_iv
 iv_data = nss.generate_random(iv_length)
nss.error.NSPRError: (SEC_ERROR_NO_TOKEN) The security card or token
does not exist, needs to be initialized, or has been removed.


The simplest test is probably this:

from ipalib import api

api.bootstrap(context='server')
api.finalize()

kra_client = api.Backend.kra.get_client()
transport_cert = kra_client.system_certs.get_transport_cert()

print Serial number: %s % transport_cert.serial_number
print Issuer DN: %s % transport_cert.issuer_dn
print Subject DN: %s % transport_cert.subject_dn

print transport_cert.encoded

If you want to test the key archival it would require installing a 
transport certificate and add some authentication operations. A better 
way to do that is to install patch #354-1, #353-3, #355-1, and #356-1 
and test the vault-archive command. It will install the transport cert 
automatically and perform the required authentication.


--
Endi S. Dewata
From 7b68ff1c93554975abd75e6d672ee18a7a0bcf04 Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Wed, 1 Oct 2014 14:59:46 -0400
Subject: [PATCH] Fixed KRA backend.

The KRA backend has been simplified since most of the tasks have
been moved somewhere else. The transport certificate will be
installed on the client, and it is not needed by KRA backend. The
KRA agent's PEM certificate is now generated during installation
due to permission issue. The kra_host() for now is removed since
the current ldap_enable() cannot register the KRA service, so it
is using the kra_host environment variable.

The KRA installer has been modified to use Dogtag's CLI to create
KRA agent and setup the client authentication.

The proxy settings have been updated to include KRA's URLs.

Some constants have been renamed for clarity. The DOGTAG_AGENT_P12
has been renamed to DOGTAG_ADMIN_P12 since file actually contains
the Dogtag admin's certificate and private key and it can be used
to access both CA and KRA. The DOGTAG_AGENT_PEM has been renamed
to KRA_AGENT_PEM since it can only be used for KRA.

The Dogtag dependency has been updated to 10.2.1-0.1.

https://fedorahosted.org/freeipa/ticket/4503
---
 freeipa.spec.in  |   4 +-
 install/conf/ipa-pki-proxy.conf  |   2 +-
 ipaplatform/base/paths.py|   4 +-
 ipaserver/install/cainstance.py  |   4 +-
 ipaserver/install/ipa_backup.py  |   3 +-
 ipaserver/install/krainstance.py |  83 +++---
 ipaserver/plugins/dogtag.py  | 122 ++-
 7 files changed, 101 insertions(+), 121 deletions(-)

diff --git a/freeipa.spec.in b/freeipa.spec.in
index 
8fcb535e229db4f7a8eaaee3c99b18446eef7f1e..dc04be48b2bb52ff05f9fab371c4b333a15d24ca
 100644
--- a/freeipa.spec.in
+++ b/freeipa.spec.in
@@ -130,8 +130,8 @@ Requires(post): systemd-units
 Requires: selinux-policy = %{selinux_policy_version}
 Requires(post): selinux-policy-base
 Requires: slapi-nis = 0.54-1
-Requires: pki-ca = 10.2.0-3
-Requires: pki-kra = 10.2.0
+Requires: pki-ca = 10.2.1-0.1
+Requires: pki-kra = 10.2.1-0.1
 %if 0%{?rhel}
 Requires: subscription-manager
 %endif
diff --git a/install/conf/ipa-pki-proxy.conf b/install/conf/ipa-pki-proxy.conf
index 
2370b4d7a7467a7e47c0d223915e018c9a009e83..5d21156848f3b5ddf14c42d92a26a30a9f94af36
 100644
--- a/install/conf/ipa-pki-proxy.conf
+++ b/install/conf/ipa-pki-proxy.conf
@@ -19,7 +19,7 @@ ProxyRequests Off
 /LocationMatch
 
 # matches for agent 

Re: [Freeipa-devel] [PATCH] 352 Fixed KRA backend.

2014-10-28 Thread Endi Sukma Dewata

Thanks for the review. New patch attached.

On 10/23/2014 3:59 AM, Petr Viktorin wrote:

In IPA we usually include the full ticket URL, not just the number.


Fixed.


The build fails with a lint message:
* Module ipaserver.plugins.dogtag
ipaserver/plugins/dogtag.py:1903: [E1123(unexpected-keyword-arg),
kra.get_client] Unexpected keyword argument 'password_file' in
constructor call)
ipaserver/plugins/dogtag.py:1903: [E1120(no-value-for-parameter),
kra.get_client] No value for argument 'certdb_password' in constructor
call)

I have pki-base-10.2.0-3.fc21.noarch, where NSSCryptoProvider indeed
takes password and not password_file. If a newer version is required you
should put it in the spec.


Fixed. Dependency is bumped to 10.2.1-0.1 which is available from my 
COPR repo:


  dnf copr enable edewata/pki


ipaserver.install.certs.CertDB.install_pem_from_p12:
If p12_passwd is missing and pwd_fname is None, this will crash.
Please document how the method should be called. And assert that exactly
one of p12_passwd and pwd_fname is given.


I reverted this change because the KRA backend actually no longer uses 
install_pem_from_p12(). The KRA backend is now using the CLI from the 
new Dogtag which generates the proper PEM format for client 
authentication, so I'll leave install_pem_from_p12() unmodified because 
it's still used by KrbInstance.



ipaserver.plugins.dogtag.kra.get_client:
Should every caller check if this returns None?
If not, raise an exception instead.
If yes, at least mention it in a docstring.


Fixed. It's now raising a generic exception.

Is there an existing exception that is more appropriate for backend 
issues like this?



Typo in commit message: modified to use Dogtag's CLI *go* create


Fixed.

--
Endi S. Dewata
From 6f1f289f32dd68c85c09a41422e5b2e0c204ee4c Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Wed, 1 Oct 2014 14:59:46 -0400
Subject: [PATCH] Fixed KRA backend.

The KRA backend has been simplified since most of the tasks have
been moved somewhere else. The transport certificate will be
installed on the client, and it is not needed by KRA backend. The
KRA agent's PEM certificate is now generated during installation
due to permission issue. The kra_host() for now is removed since
the current ldap_enable() cannot register the KRA service, so it
is using the kra_host environment variable.

The KRA installer has been modified to use Dogtag's CLI to create
KRA agent and setup the client authentication.

The proxy settings have been updated to include KRA's URLs.

Some constants have been renamed for clarity. The DOGTAG_AGENT_P12
has been renamed to DOGTAG_ADMIN_P12 since file actually contains
the Dogtag admin's certificate and private key and it can be used
to access both CA and KRA. The DOGTAG_AGENT_PEM has been renamed
to KRA_AGENT_PEM since it can only be used for KRA.

The Dogtag dependency has been updated to 10.2.1-0.1.

https://fedorahosted.org/freeipa/ticket/4503
---
 freeipa.spec.in  |   4 +-
 install/conf/ipa-pki-proxy.conf  |   2 +-
 ipaplatform/base/paths.py|   4 +-
 ipaserver/install/cainstance.py  |   4 +-
 ipaserver/install/ipa_backup.py  |   3 +-
 ipaserver/install/krainstance.py |  83 +++---
 ipaserver/plugins/dogtag.py  | 122 ++-
 7 files changed, 101 insertions(+), 121 deletions(-)

diff --git a/freeipa.spec.in b/freeipa.spec.in
index 
8fcb535e229db4f7a8eaaee3c99b18446eef7f1e..dc04be48b2bb52ff05f9fab371c4b333a15d24ca
 100644
--- a/freeipa.spec.in
+++ b/freeipa.spec.in
@@ -130,8 +130,8 @@ Requires(post): systemd-units
 Requires: selinux-policy = %{selinux_policy_version}
 Requires(post): selinux-policy-base
 Requires: slapi-nis = 0.54-1
-Requires: pki-ca = 10.2.0-3
-Requires: pki-kra = 10.2.0
+Requires: pki-ca = 10.2.1-0.1
+Requires: pki-kra = 10.2.1-0.1
 %if 0%{?rhel}
 Requires: subscription-manager
 %endif
diff --git a/install/conf/ipa-pki-proxy.conf b/install/conf/ipa-pki-proxy.conf
index 
2370b4d7a7467a7e47c0d223915e018c9a009e83..5d21156848f3b5ddf14c42d92a26a30a9f94af36
 100644
--- a/install/conf/ipa-pki-proxy.conf
+++ b/install/conf/ipa-pki-proxy.conf
@@ -19,7 +19,7 @@ ProxyRequests Off
 /LocationMatch
 
 # matches for agent port and eeca port
-LocationMatch 
^/ca/agent/ca/displayBySerial|^/ca/agent/ca/doRevoke|^/ca/agent/ca/doUnrevoke|^/ca/agent/ca/updateDomainXML|^/ca/eeca/ca/profileSubmitSSLClient|^/kra/agent/kra/connector|^/kra/rest/agent/keyrequests|^/kra/rest/agent/keys|^/ca/rest/admin/kraconnector/remove
+LocationMatch 
^/ca/agent/ca/displayBySerial|^/ca/agent/ca/doRevoke|^/ca/agent/ca/doUnrevoke|^/ca/agent/ca/updateDomainXML|^/ca/eeca/ca/profileSubmitSSLClient|^/kra/agent/kra/connector|^/kra/rest/account|^/kra/rest/agent/keyrequests|^/kra/rest/agent/keys|^/ca/rest/admin/kraconnector/remove
 NSSOptions +StdEnvVars +ExportCertData +StrictRequire +OptRenegotiate
 NSSVerifyClient require
 ProxyPassMatch 

Re: [Freeipa-devel] [PATCH] 354 Modififed NSSConnection not to shutdown existing database.

2014-10-28 Thread Endi Sukma Dewata

On 10/22/2014 9:15 AM, Endi Sukma Dewata wrote:

The NSSConnection class has been modified not to shutdown the
existing NSS database if the database is already opened to
establish an SSL connection, or is already opened by another
code that uses an NSS database without establishing an SSL
connection such as vault CLIs.

Ticket #4638


New patch attached. It's identical except for the ticket URL in the 
commit log.


--
Endi S. Dewata
From 34bd77959687673db9fbf71c443b6ffe5ed4ca71 Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Tue, 16 Sep 2014 20:11:35 -0400
Subject: [PATCH] Modififed NSSConnection not to shutdown existing database.

The NSSConnection class has been modified not to shutdown the
existing NSS database if the database is already opened to
establish an SSL connection, or is already opened by another
code that uses an NSS database without establishing an SSL
connection such as vault CLIs.

https://fedorahosted.org/freeipa/ticket/4638
---
 ipalib/rpc.py   | 34 +++---
 ipapython/nsslib.py | 35 +++
 2 files changed, 42 insertions(+), 27 deletions(-)

diff --git a/ipalib/rpc.py b/ipalib/rpc.py
index 
5934f0c26e4b7c0a44adbab978c1f9b319d72e9f..001b7f1ca06edadfc7aad635d9d564e517008a63
 100644
--- a/ipalib/rpc.py
+++ b/ipalib/rpc.py
@@ -63,6 +63,7 @@ from ipaplatform.paths import paths
 from ipapython.cookie import Cookie
 from ipapython.dnsutil import DNSName
 from ipalib.text import _
+import ipapython.nsslib
 from ipapython.nsslib import NSSHTTPS, NSSConnection
 from ipalib.krb_utils import KRB5KDC_ERR_S_PRINCIPAL_UNKNOWN, 
KRB5KRB_AP_ERR_TKT_EXPIRED, \
  KRB5_FCC_PERM, KRB5_FCC_NOFILE, KRB5_CC_FORMAT, 
KRB5_REALM_CANT_RESOLVE
@@ -450,14 +451,10 @@ class LanguageAwareTransport(MultiProtocolTransport):
 class SSLTransport(LanguageAwareTransport):
 Handles an HTTPS transaction to an XML-RPC server.
 
-def __nss_initialized(self, dbdir):
+def get_connection_dbdir(self):
 
-If there is another connections open it may have already
-initialized NSS. This is likely to lead to an NSS shutdown
-failure.  One way to mitigate this is to tell NSS to not
-initialize if it has already been done in another open connection.
-
-Returns True if another connection is using the same db.
+If there is a connections open it may have already initialized
+NSS database. Return the database location used by the connection.
 
 for value in context.__dict__.values():
 if not isinstance(value, Connection):
@@ -466,25 +463,32 @@ class SSLTransport(LanguageAwareTransport):
 getattr(value.conn, '_ServerProxy__transport', None),
 SSLTransport):
 continue
-if hasattr(value.conn._ServerProxy__transport, 'dbdir') and \
-  value.conn._ServerProxy__transport.dbdir == dbdir:
-return True
-return False
+if hasattr(value.conn._ServerProxy__transport, 'dbdir'):
+return value.conn._ServerProxy__transport.dbdir
+return None
 
 def make_connection(self, host):
 host, self._extra_headers, x509 = self.get_host_info(host)
 # Python 2.7 changed the internal class used in xmlrpclib from
 # HTTP to HTTPConnection. We need to use the proper subclass
 
-# If we an existing connection exists using the same NSS database
-# there is no need to re-initialize. Pass thsi into the NSS
-# connection creator.
 if sys.version_info = (2, 7):
 if self._connection and host == self._connection[0]:
 return self._connection[1]
 
 dbdir = getattr(context, 'nss_dir', paths.IPA_NSSDB_DIR)
-no_init = self.__nss_initialized(dbdir)
+connection_dbdir = self.get_connection_dbdir()
+
+if connection_dbdir:
+# If an existing connection is already using the same NSS
+# database there is no need to re-initialize.
+no_init = dbdir == connection_dbdir
+
+else:
+# If the NSS database is already being used there is no
+# need to re-initialize.
+no_init = dbdir == ipapython.nsslib.current_dbdir
+
 if sys.version_info  (2, 7):
 conn = NSSHTTPS(host, 443, dbdir=dbdir, no_init=no_init)
 else:
diff --git a/ipapython/nsslib.py b/ipapython/nsslib.py
index 
93b0c56fcff4fc69841a6823aae8f694c1f76ff0..1452a2a5844a5fb017d4408aadf56f7fcfc7fa25
 100644
--- a/ipapython/nsslib.py
+++ b/ipapython/nsslib.py
@@ -31,6 +31,9 @@ import nss.ssl as ssl
 import nss.error as error
 from ipaplatform.paths import paths
 
+# NSS database currently open
+current_dbdir = None
+
 def auth_certificate_callback(sock, check_sig, is_server, certdb):
 cert_is_valid = False
 
@@ -184,19 +187,27 @@ class NSSConnection(httplib.HTTPConnection

Re: [Freeipa-devel] [PATCH] 353 Added initial vault implementation.

2014-10-28 Thread Endi Sukma Dewata

On 10/22/2014 3:04 PM, Endi Sukma Dewata wrote:

On 10/16/2014 4:12 PM, Endi Sukma Dewata wrote:

On 10/15/2014 10:59 PM, Endi Sukma Dewata wrote:

The NSSConnection class has to be modified not to shutdown existing
database because some of the vault clients (e.g. vault-archive and
vault-retrieve) also use a database to encrypt/decrypt the secret.


The problem is described in more detail in this ticket:
https://fedorahosted.org/freeipa/ticket/4638

The changes to the NSSConnection in the first patch caused the
installation to fail. Attached is a new patch that uses the solution
proposed by jdennis.


New patch attached. It's now using the correct OID's for the schema. It
also has been rebased on top of #352-1 and #354.


New patch attached to fix the ticket URL. It depends on #352-2 and #354-1.

--
Endi S. Dewata
From cd3daa901f7139801ea61ae1e2716810da131bcc Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Tue, 21 Oct 2014 10:57:08 -0400
Subject: [PATCH] Added initial vault implementation.

This patch provides the initial vault implementation which allows
the admin to create a vault, archive a secret, and retrieve the
secret using a standard vault. It also included the initial LDAP
schema.

It currently has limitations including:
 - The vault only supports the standard vault type.
 - The vault can only be used by the admin user.
 - The transport certificate has to be installed manually.

These limitations, other vault features, schema and ACL changes will
be addressed in subsequent patches.

https://fedorahosted.org/freeipa/ticket/3872
---
 API.txt| 160 
 VERSION|   4 +-
 install/share/60basev4.ldif|   3 +
 install/share/Makefile.am  |   1 +
 install/share/copy-schema-to-ca.py |   1 +
 install/updates/40-vault.update|  27 ++
 install/updates/Makefile.am|   1 +
 ipa-client/man/default.conf.5  |   1 +
 ipalib/constants.py|   1 +
 ipalib/plugins/user.py |   9 +
 ipalib/plugins/vault.py| 724 +
 ipaserver/install/dsinstance.py|   1 +
 12 files changed, 931 insertions(+), 2 deletions(-)
 create mode 100644 install/share/60basev4.ldif
 create mode 100644 install/updates/40-vault.update
 create mode 100644 ipalib/plugins/vault.py

diff --git a/API.txt b/API.txt
index 
491d7a76fd1d2d50208d314d1600839ce295..cfa6558fcf678e5915a90407da517f9a591a41bf
 100644
--- a/API.txt
+++ b/API.txt
@@ -4475,6 +4475,166 @@ option: Str('version?', exclude='webui')
 output: Output('result', type 'bool', None)
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
+command: vault_add
+args: 1,8,3
+arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Str('description', attribute=True, cli_name='desc', multivalue=False, 
required=False)
+option: Str('in?', cli_name='in')
+option: Str('parent', attribute=False, cli_name='parent', multivalue=False, 
required=False)
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Flag('rights', autofill=True, default=False)
+option: Bytes('secret', attribute=True, cli_name='secret', multivalue=False, 
required=False)
+option: Str('version?', exclude='webui')
+output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
+output: Output('summary', (type 'unicode', type 'NoneType'), None)
+output: PrimaryKey('value', None, None)
+command: vault_archive
+args: 1,10,3
+arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, query=True, required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Bytes('encrypted_data?', cli_name='encrypted_data')
+option: Str('in?', cli_name='in')
+option: Bytes('nonce?', cli_name='nonce')
+option: Str('parent?', cli_name='parent')
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Flag('rights', autofill=True, default=False)
+option: Bytes('secret?', cli_name='secret')
+option: Str('version?', exclude='webui')
+option: Bytes('wrapped_session_key?', cli_name='wrapped_session_key')
+output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
+output: Output('summary', (type 'unicode', type 'NoneType'), None)
+output: PrimaryKey('value', None, None)
+command: vault_del
+args: 1,3,3
+arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=True, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_

Re: [Freeipa-devel] [PATCH] 355 Added vault access control.

2014-10-28 Thread Endi Sukma Dewata

On 10/22/2014 3:04 PM, Endi Sukma Dewata wrote:

New LDAP ACIs have been added to allow users to create their own
private vault container, to allow owners to manage vaults and
containers, and to allow members to use the vaults. New CLIs have
been added to manage the owner and member list. For archive and
retrieve operations the access control has to be enforced by the
plugins because the operations only affects KRA. The LDAP schema
has been updated as well.

Ticket #3872

This patch depends on #353-2.


New patch attached to fix the ticket URL. It depends on #353-3.

--
Endi S. Dewata
From 6f464581e4e30e6105522ff25047764ec97e5a53 Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Fri, 17 Oct 2014 12:05:34 -0400
Subject: [PATCH] Added vault access control.

New LDAP ACIs have been added to allow users to create their own
private vault container, to allow owners to manage vaults and
containers, and to allow members to use the vaults. New CLIs have
been added to manage the owner and member list. For archive and
retrieve operations the access control has to be enforced by the
plugins because the operations only affects KRA. The LDAP schema
has been updated as well.

https://fedorahosted.org/freeipa/ticket/3872
---
 API.txt | 134 +--
 VERSION |   4 +-
 install/share/60basev4.ldif |   4 +-
 install/updates/40-vault.update |   7 ++
 ipalib/plugins/vault.py | 233 +++-
 5 files changed, 366 insertions(+), 16 deletions(-)

diff --git a/API.txt b/API.txt
index 
cfa6558fcf678e5915a90407da517f9a591a41bf..a46592ec9e82e618154bf09393c83d4b854315c5
 100644
--- a/API.txt
+++ b/API.txt
@@ -4476,11 +4476,12 @@ output: Output('result', type 'bool', None)
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
 command: vault_add
-args: 1,8,3
+args: 1,9,3
 arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, required=True)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
 option: Str('description', attribute=True, cli_name='desc', multivalue=False, 
required=False)
 option: Str('in?', cli_name='in')
+option: Flag('no_members', autofill=True, default=False, exclude='webui')
 option: Str('parent', attribute=False, cli_name='parent', multivalue=False, 
required=False)
 option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
 option: Flag('rights', autofill=True, default=False)
@@ -4489,12 +4490,39 @@ option: Str('version?', exclude='webui')
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
+command: vault_add_member
+args: 1,7,3
+arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, query=True, required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Str('group*', alwaysask=True, cli_name='groups', csv=True)
+option: Flag('no_members', autofill=True, default=False, exclude='webui')
+option: Str('parent?', cli_name='parent')
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Str('user*', alwaysask=True, cli_name='users', csv=True)
+option: Str('version?', exclude='webui')
+output: Output('completed', type 'int', None)
+output: Output('failed', type 'dict', None)
+output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
+command: vault_add_owner
+args: 1,7,3
+arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, query=True, required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Str('group*', alwaysask=True, cli_name='groups', csv=True)
+option: Flag('no_members', autofill=True, default=False, exclude='webui')
+option: Str('parent?', cli_name='parent')
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Str('user*', alwaysask=True, cli_name='users', csv=True)
+option: Str('version?', exclude='webui')
+output: Output('completed', type 'int', None)
+output: Output('failed', type 'dict', None)
+output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 command: vault_archive
-args: 1,10,3
+args: 1,11,3
 arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_

Re: [Freeipa-devel] [PATCH] 356 Added command to retrieve vault transport certificate.

2014-10-28 Thread Endi Sukma Dewata

On 10/23/2014 6:18 AM, Jan Cholasta wrote:

Hi,

Dne 22.10.2014 v 22:06 Endi Sukma Dewata napsal(a):

A new command has been added to retrieve the vault transport
certificate and optionally save it into a file. The vault archive
and retrieve command has been modified to retrieve the transport
certificate and store it locally for subsequent usage. This way
it's no longer necessary to manually import the transport
certificate into the client's NSS database.


As part of the CA certificate renewal feature in 4.1, I have added a
LDAP certificate store to IPA, see
http://www.freeipa.org/page/V4/CA_certificate_renewal. Currently it
supports only CA certificates, but can be extended to support end entity
certificates rather easily. If you use it for the vault transport
certificate, it can be added to the client NSS database automatically on
install.

Honza



I'm attaching a new patch that's identical to the previous one with 
ticket URL updated. I'm thinking we should check this patch in first 
because it's already done, and then investigate the use of CA cert 
management utility as a separate enhancement since the it seems to need 
to be generalized before it can be used to manage KRA transport cert. 
I'll also need to investigate the KRA transport cert replacement process 
to make sure it can be accommodated via IPA's cert management utility.


--
Endi S. Dewata
From 1bffa29d35fee0ac06cb1bc943f9de8beee58d05 Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Wed, 22 Oct 2014 10:02:25 -0400
Subject: [PATCH] Added command to retrieve vault transport certificate.

A new command has been added to retrieve the vault transport
certificate and optionally save it into a file. The vault archive
and retrieve command has been modified to retrieve the transport
certificate and store it locally for subsequent usage. This way
it's no longer necessary to manually import the transport
certificate into the client's NSS database.

https://fedorahosted.org/freeipa/ticket/3872
---
 API.txt |  5 +++
 VERSION |  4 +--
 ipalib/plugins/vault.py | 85 +++--
 3 files changed, 89 insertions(+), 5 deletions(-)

diff --git a/API.txt b/API.txt
index 
a46592ec9e82e618154bf09393c83d4b854315c5..95b86ce84f5bc9f1d879e561e07b0348d719c90e
 100644
--- a/API.txt
+++ b/API.txt
@@ -4629,6 +4629,11 @@ option: Str('version?', exclude='webui')
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
+command: vault_transport_cert
+args: 0,2,1
+option: Str('out?', cli_name='out')
+option: Str('version?', exclude='webui')
+output: Output('result', None, None)
 command: vaultcontainer_add
 args: 1,8,3
 arg: Str('cn', attribute=True, cli_name='container_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, required=True)
diff --git a/VERSION b/VERSION
index 
c471ed80af6a2c26be7fc89281ae60fac6c68577..d0ada131b700e93faa8c4946b811db36d76341a9
 100644
--- a/VERSION
+++ b/VERSION
@@ -90,5 +90,5 @@ IPA_DATA_VERSION=2010061412
 #  #
 
 IPA_API_VERSION_MAJOR=2
-IPA_API_VERSION_MINOR=110
-# Last change: edewata - added vault access control
+IPA_API_VERSION_MINOR=111
+# Last change: edewata - added vault transport certificate
diff --git a/ipalib/plugins/vault.py b/ipalib/plugins/vault.py
index 
95f96859235af1c477c8f5738a27571d64aabe3a..871c3e3a25c688a64ba0ecfde5ccbd50b47fbe01
 100644
--- a/ipalib/plugins/vault.py
+++ b/ipalib/plugins/vault.py
@@ -24,6 +24,8 @@ import shutil
 import string
 import tempfile
 
+import nss.nss as nss
+
 import pki
 import pki.account
 import pki.crypto
@@ -109,7 +111,7 @@ EXAMPLES:
 )
 
 register = Registry()
-transport_cert_nickname = KRA Transport Certificate
+transport_cert_filename = vault-transport.pem
 
 @register()
 class vaultcontainer(LDAPObject):
@@ -628,6 +630,63 @@ class vault_show(LDAPRetrieve):
 
 
 @register()
+class vault_transport_cert(Command):
+__doc__ = _('Retrieve vault transport certificate.')
+
+
+# list of attributes we want exported to JSON
+json_friendly_attributes = (
+'takes_args',
+)
+
+takes_options = (
+Str('out?',
+cli_name='out',
+doc=_('Output file to store the transport certificate'),
+),
+)
+
+has_output_params = (
+Str('certificate',
+label=_('Certificate'),
+),
+)
+
+def __json__(self):
+json_dict = dict(
+(a, getattr(self, a)) for a in self.json_friendly_attributes
+)
+json_dict['takes_options'] = list(self.get_json_options())
+return json_dict
+
+def execute(self, *args, **options

[Freeipa-devel] [PATCH] 354 Modififed NSSConnection not to shutdown existing database.

2014-10-22 Thread Endi Sukma Dewata

The NSSConnection class has been modified not to shutdown the
existing NSS database if the database is already opened to
establish an SSL connection, or is already opened by another
code that uses an NSS database without establishing an SSL
connection such as vault CLIs.

Ticket #4638

--
Endi S. Dewata
From d7b05ef151a59e6828b537ac2077d05b74b25903 Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Tue, 16 Sep 2014 20:11:35 -0400
Subject: [PATCH] Modififed NSSConnection not to shutdown existing database.

The NSSConnection class has been modified not to shutdown the
existing NSS database if the database is already opened to
establish an SSL connection, or is already opened by another
code that uses an NSS database without establishing an SSL
connection such as vault CLIs.

Ticket #4638
---
 ipalib/rpc.py   | 34 +++---
 ipapython/nsslib.py | 35 +++
 2 files changed, 42 insertions(+), 27 deletions(-)

diff --git a/ipalib/rpc.py b/ipalib/rpc.py
index 
5934f0c26e4b7c0a44adbab978c1f9b319d72e9f..001b7f1ca06edadfc7aad635d9d564e517008a63
 100644
--- a/ipalib/rpc.py
+++ b/ipalib/rpc.py
@@ -63,6 +63,7 @@ from ipaplatform.paths import paths
 from ipapython.cookie import Cookie
 from ipapython.dnsutil import DNSName
 from ipalib.text import _
+import ipapython.nsslib
 from ipapython.nsslib import NSSHTTPS, NSSConnection
 from ipalib.krb_utils import KRB5KDC_ERR_S_PRINCIPAL_UNKNOWN, 
KRB5KRB_AP_ERR_TKT_EXPIRED, \
  KRB5_FCC_PERM, KRB5_FCC_NOFILE, KRB5_CC_FORMAT, 
KRB5_REALM_CANT_RESOLVE
@@ -450,14 +451,10 @@ class LanguageAwareTransport(MultiProtocolTransport):
 class SSLTransport(LanguageAwareTransport):
 Handles an HTTPS transaction to an XML-RPC server.
 
-def __nss_initialized(self, dbdir):
+def get_connection_dbdir(self):
 
-If there is another connections open it may have already
-initialized NSS. This is likely to lead to an NSS shutdown
-failure.  One way to mitigate this is to tell NSS to not
-initialize if it has already been done in another open connection.
-
-Returns True if another connection is using the same db.
+If there is a connections open it may have already initialized
+NSS database. Return the database location used by the connection.
 
 for value in context.__dict__.values():
 if not isinstance(value, Connection):
@@ -466,25 +463,32 @@ class SSLTransport(LanguageAwareTransport):
 getattr(value.conn, '_ServerProxy__transport', None),
 SSLTransport):
 continue
-if hasattr(value.conn._ServerProxy__transport, 'dbdir') and \
-  value.conn._ServerProxy__transport.dbdir == dbdir:
-return True
-return False
+if hasattr(value.conn._ServerProxy__transport, 'dbdir'):
+return value.conn._ServerProxy__transport.dbdir
+return None
 
 def make_connection(self, host):
 host, self._extra_headers, x509 = self.get_host_info(host)
 # Python 2.7 changed the internal class used in xmlrpclib from
 # HTTP to HTTPConnection. We need to use the proper subclass
 
-# If we an existing connection exists using the same NSS database
-# there is no need to re-initialize. Pass thsi into the NSS
-# connection creator.
 if sys.version_info = (2, 7):
 if self._connection and host == self._connection[0]:
 return self._connection[1]
 
 dbdir = getattr(context, 'nss_dir', paths.IPA_NSSDB_DIR)
-no_init = self.__nss_initialized(dbdir)
+connection_dbdir = self.get_connection_dbdir()
+
+if connection_dbdir:
+# If an existing connection is already using the same NSS
+# database there is no need to re-initialize.
+no_init = dbdir == connection_dbdir
+
+else:
+# If the NSS database is already being used there is no
+# need to re-initialize.
+no_init = dbdir == ipapython.nsslib.current_dbdir
+
 if sys.version_info  (2, 7):
 conn = NSSHTTPS(host, 443, dbdir=dbdir, no_init=no_init)
 else:
diff --git a/ipapython/nsslib.py b/ipapython/nsslib.py
index 
93b0c56fcff4fc69841a6823aae8f694c1f76ff0..1452a2a5844a5fb017d4408aadf56f7fcfc7fa25
 100644
--- a/ipapython/nsslib.py
+++ b/ipapython/nsslib.py
@@ -31,6 +31,9 @@ import nss.ssl as ssl
 import nss.error as error
 from ipaplatform.paths import paths
 
+# NSS database currently open
+current_dbdir = None
+
 def auth_certificate_callback(sock, check_sig, is_server, certdb):
 cert_is_valid = False
 
@@ -184,19 +187,27 @@ class NSSConnection(httplib.HTTPConnection, 
NSSAddressFamilyFallback):
 httplib.HTTPConnection.__init__(self, host, port, strict)
 NSSAddressFamilyFallback.__init__(self, family)
 
-if 

Re: [Freeipa-devel] [PATCH] 352 Fixed KRA backend.

2014-10-22 Thread Endi Sukma Dewata

On 10/15/2014 10:59 PM, Endi Sukma Dewata wrote:

The KRA backend has been simplified since most of the tasks have
been moved somewhere else. The transport certificate will be
installed on the client, and it is not needed by KRA backend. The
KRA agent's PEM certificate is now generated during installation
due to permission issue. The kra_host() for now is removed since
the current ldap_enable() cannot register the KRA service, so it
is using the kra_host environment variable.

The KRA installer has been modified to use Dogtag's CLI go create
KRA agent and setup the client authentication.

The proxy settings have been updated to include KRA's URLs.

The certs.install_pem_from_p12() has been updated to generate the
proper client certificate using the -clcerts option and also take
a password file.

Some constants have been renamed for clarity. The DOGTAG_AGENT_P12
has been renamed to DOGTAG_ADMIN_P12 since file actually contains
the Dogtag admin's certificate and private key and it can be used
to access both CA and KRA. The DOGTAG_AGENT_PEM has been renamed
to KRA_AGENT_PEM since it can only be used for KRA.

Ticket #3872


New patch attached. It's identical to the previous one except I changed 
the ticket number to #4503.


--
Endi S. Dewata
From a3e5c4f872fade7e88f954452a64c310ba4ae380 Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Wed, 1 Oct 2014 14:59:46 -0400
Subject: [PATCH] Fixed KRA backend.

The KRA backend has been simplified since most of the tasks have
been moved somewhere else. The transport certificate will be
installed on the client, and it is not needed by KRA backend. The
KRA agent's PEM certificate is now generated during installation
due to permission issue. The kra_host() for now is removed since
the current ldap_enable() cannot register the KRA service, so it
is using the kra_host environment variable.

The KRA installer has been modified to use Dogtag's CLI go create
KRA agent and setup the client authentication.

The proxy settings have been updated to include KRA's URLs.

The certs.install_pem_from_p12() has been updated to generate the
proper client certificate using the -clcerts option and also take
a password file.

Some constants have been renamed for clarity. The DOGTAG_AGENT_P12
has been renamed to DOGTAG_ADMIN_P12 since file actually contains
the Dogtag admin's certificate and private key and it can be used
to access both CA and KRA. The DOGTAG_AGENT_PEM has been renamed
to KRA_AGENT_PEM since it can only be used for KRA.

Ticket #4503
---
 install/conf/ipa-pki-proxy.conf  |   2 +-
 ipaplatform/base/paths.py|   4 +-
 ipaserver/install/cainstance.py  |   4 +-
 ipaserver/install/certs.py   |  10 ++--
 ipaserver/install/ipa_backup.py  |   3 +-
 ipaserver/install/krainstance.py |  83 ---
 ipaserver/plugins/dogtag.py  | 120 +--
 7 files changed, 100 insertions(+), 126 deletions(-)

diff --git a/install/conf/ipa-pki-proxy.conf b/install/conf/ipa-pki-proxy.conf
index 
2370b4d7a7467a7e47c0d223915e018c9a009e83..5d21156848f3b5ddf14c42d92a26a30a9f94af36
 100644
--- a/install/conf/ipa-pki-proxy.conf
+++ b/install/conf/ipa-pki-proxy.conf
@@ -19,7 +19,7 @@ ProxyRequests Off
 /LocationMatch
 
 # matches for agent port and eeca port
-LocationMatch 
^/ca/agent/ca/displayBySerial|^/ca/agent/ca/doRevoke|^/ca/agent/ca/doUnrevoke|^/ca/agent/ca/updateDomainXML|^/ca/eeca/ca/profileSubmitSSLClient|^/kra/agent/kra/connector|^/kra/rest/agent/keyrequests|^/kra/rest/agent/keys|^/ca/rest/admin/kraconnector/remove
+LocationMatch 
^/ca/agent/ca/displayBySerial|^/ca/agent/ca/doRevoke|^/ca/agent/ca/doUnrevoke|^/ca/agent/ca/updateDomainXML|^/ca/eeca/ca/profileSubmitSSLClient|^/kra/agent/kra/connector|^/kra/rest/account|^/kra/rest/agent/keyrequests|^/kra/rest/agent/keys|^/ca/rest/admin/kraconnector/remove
 NSSOptions +StdEnvVars +ExportCertData +StrictRequire +OptRenegotiate
 NSSVerifyClient require
 ProxyPassMatch ajp://localhost:$DOGTAG_PORT
diff --git a/ipaplatform/base/paths.py b/ipaplatform/base/paths.py
index 
bbe6eed76ccb3c5f325fd368694ac6a2afbb72f0..01505594a7af926c860f867b817bd397c54efff5
 100644
--- a/ipaplatform/base/paths.py
+++ b/ipaplatform/base/paths.py
@@ -138,8 +138,8 @@ class BasePathNamespace(object):
 HOME_DIR = /home
 ROOT_IPA_CACHE = /root/.ipa_cache
 ROOT_PKI = /root/.pki
-DOGTAG_AGENT_P12 = /root/ca-agent.p12
-DOGTAG_AGENT_PEM = /etc/httpd/alias/agent.pem
+DOGTAG_ADMIN_P12 = /root/ca-agent.p12
+KRA_AGENT_PEM = /etc/httpd/alias/kra-agent.pem
 CACERT_P12 = /root/cacert.p12
 ROOT_IPA_CSR = /root/ipa.csr
 ROOT_TMP_CA_P12 = /root/tmp-ca.p12
diff --git a/ipaserver/install/cainstance.py b/ipaserver/install/cainstance.py
index 
1ae39639ac9702651851e6c3964faa69788db31e..fe95201517a577b9f6dba7642afe09b4eef2328d
 100644
--- a/ipaserver/install/cainstance.py
+++ b/ipaserver/install/cainstance.py
@@ -514,7 +514,7 @@ class CAInstance(DogtagInstance

[Freeipa-devel] [PATCH] 355 Added vault access control.

2014-10-22 Thread Endi Sukma Dewata

New LDAP ACIs have been added to allow users to create their own
private vault container, to allow owners to manage vaults and
containers, and to allow members to use the vaults. New CLIs have
been added to manage the owner and member list. For archive and
retrieve operations the access control has to be enforced by the
plugins because the operations only affects KRA. The LDAP schema
has been updated as well.

Ticket #3872

This patch depends on #353-2.

--
Endi S. Dewata
From ccbfa01f40e2ac4c978e5ef0f1fbe167f96793a2 Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Fri, 17 Oct 2014 12:05:34 -0400
Subject: [PATCH] Added vault access control.

New LDAP ACIs have been added to allow users to create their own
private vault container, to allow owners to manage vaults and
containers, and to allow members to use the vaults. New CLIs have
been added to manage the owner and member list. For archive and
retrieve operations the access control has to be enforced by the
plugins because the operations only affects KRA. The LDAP schema
has been updated as well.

Ticket #3872
---
 API.txt | 134 +--
 VERSION |   4 +-
 install/share/60basev4.ldif |   4 +-
 install/updates/40-vault.update |   7 ++
 ipalib/plugins/vault.py | 233 +++-
 5 files changed, 366 insertions(+), 16 deletions(-)

diff --git a/API.txt b/API.txt
index 
cfa6558fcf678e5915a90407da517f9a591a41bf..a46592ec9e82e618154bf09393c83d4b854315c5
 100644
--- a/API.txt
+++ b/API.txt
@@ -4476,11 +4476,12 @@ output: Output('result', type 'bool', None)
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
 command: vault_add
-args: 1,8,3
+args: 1,9,3
 arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, required=True)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
 option: Str('description', attribute=True, cli_name='desc', multivalue=False, 
required=False)
 option: Str('in?', cli_name='in')
+option: Flag('no_members', autofill=True, default=False, exclude='webui')
 option: Str('parent', attribute=False, cli_name='parent', multivalue=False, 
required=False)
 option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
 option: Flag('rights', autofill=True, default=False)
@@ -4489,12 +4490,39 @@ option: Str('version?', exclude='webui')
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
+command: vault_add_member
+args: 1,7,3
+arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, query=True, required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Str('group*', alwaysask=True, cli_name='groups', csv=True)
+option: Flag('no_members', autofill=True, default=False, exclude='webui')
+option: Str('parent?', cli_name='parent')
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Str('user*', alwaysask=True, cli_name='users', csv=True)
+option: Str('version?', exclude='webui')
+output: Output('completed', type 'int', None)
+output: Output('failed', type 'dict', None)
+output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
+command: vault_add_owner
+args: 1,7,3
+arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, query=True, required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Str('group*', alwaysask=True, cli_name='groups', csv=True)
+option: Flag('no_members', autofill=True, default=False, exclude='webui')
+option: Str('parent?', cli_name='parent')
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Str('user*', alwaysask=True, cli_name='users', csv=True)
+option: Str('version?', exclude='webui')
+output: Output('completed', type 'int', None)
+output: Output('failed', type 'dict', None)
+output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 command: vault_archive
-args: 1,10,3
+args: 1,11,3
 arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, query=True, required=True)
 option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
 option: 

Re: [Freeipa-devel] [PATCH] 353 Added initial vault implementation.

2014-10-22 Thread Endi Sukma Dewata

On 10/16/2014 4:12 PM, Endi Sukma Dewata wrote:

On 10/15/2014 10:59 PM, Endi Sukma Dewata wrote:

The NSSConnection class has to be modified not to shutdown existing
database because some of the vault clients (e.g. vault-archive and
vault-retrieve) also use a database to encrypt/decrypt the secret.


The problem is described in more detail in this ticket:
https://fedorahosted.org/freeipa/ticket/4638

The changes to the NSSConnection in the first patch caused the
installation to fail. Attached is a new patch that uses the solution
proposed by jdennis.


New patch attached. It's now using the correct OID's for the schema. It 
also has been rebased on top of #352-1 and #354.


--
Endi S. Dewata
From 2284f5684149e9fdfb7cde13865fe28e265ff5a3 Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Tue, 21 Oct 2014 10:57:08 -0400
Subject: [PATCH] Added initial vault implementation.

This patch provides the initial vault implementation which allows
the admin to create a vault, archive a secret, and retrieve the
secret using a standard vault. It also included the initial LDAP
schema.

It currently has limitations including:
 - The vault only supports the standard vault type.
 - The vault can only be used by the admin user.
 - The transport certificate has to be installed manually.

These limitations, other vault features, schema and ACL changes will
be addressed in subsequent patches.

Ticket #3872
---
 API.txt| 160 
 VERSION|   4 +-
 install/share/60basev4.ldif|   3 +
 install/share/Makefile.am  |   1 +
 install/share/copy-schema-to-ca.py |   1 +
 install/updates/40-vault.update|  27 ++
 install/updates/Makefile.am|   1 +
 ipa-client/man/default.conf.5  |   1 +
 ipalib/constants.py|   1 +
 ipalib/plugins/user.py |   9 +
 ipalib/plugins/vault.py| 724 +
 ipaserver/install/dsinstance.py|   1 +
 12 files changed, 931 insertions(+), 2 deletions(-)
 create mode 100644 install/share/60basev4.ldif
 create mode 100644 install/updates/40-vault.update
 create mode 100644 ipalib/plugins/vault.py

diff --git a/API.txt b/API.txt
index 
491d7a76fd1d2d50208d314d1600839ce295..cfa6558fcf678e5915a90407da517f9a591a41bf
 100644
--- a/API.txt
+++ b/API.txt
@@ -4475,6 +4475,166 @@ option: Str('version?', exclude='webui')
 output: Output('result', type 'bool', None)
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
+command: vault_add
+args: 1,8,3
+arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Str('description', attribute=True, cli_name='desc', multivalue=False, 
required=False)
+option: Str('in?', cli_name='in')
+option: Str('parent', attribute=False, cli_name='parent', multivalue=False, 
required=False)
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Flag('rights', autofill=True, default=False)
+option: Bytes('secret', attribute=True, cli_name='secret', multivalue=False, 
required=False)
+option: Str('version?', exclude='webui')
+output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
+output: Output('summary', (type 'unicode', type 'NoneType'), None)
+output: PrimaryKey('value', None, None)
+command: vault_archive
+args: 1,10,3
+arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, query=True, required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Bytes('encrypted_data?', cli_name='encrypted_data')
+option: Str('in?', cli_name='in')
+option: Bytes('nonce?', cli_name='nonce')
+option: Str('parent?', cli_name='parent')
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Flag('rights', autofill=True, default=False)
+option: Bytes('secret?', cli_name='secret')
+option: Str('version?', exclude='webui')
+option: Bytes('wrapped_session_key?', cli_name='wrapped_session_key')
+output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
+output: Output('summary', (type 'unicode', type 'NoneType'), None)
+output: PrimaryKey('value', None, None)
+command: vault_del
+args: 1,3,3
+arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=True, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, query=True, required=True)
+option: Flag('continue', autofill=True, cli_name='continue', default=False)
+option: Str('parent?', cli_name

[Freeipa-devel] [PATCH] 356 Added command to retrieve vault transport certificate.

2014-10-22 Thread Endi Sukma Dewata

A new command has been added to retrieve the vault transport
certificate and optionally save it into a file. The vault archive
and retrieve command has been modified to retrieve the transport
certificate and store it locally for subsequent usage. This way
it's no longer necessary to manually import the transport
certificate into the client's NSS database.

Ticket #3872

This patch depends on #355.

--
Endi S. Dewata
From abeda85904f7247f1f0d679a71a7094bb2cefe0c Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Wed, 22 Oct 2014 10:02:25 -0400
Subject: [PATCH] Added command to retrieve vault transport certificate.

A new command has been added to retrieve the vault transport
certificate and optionally save it into a file. The vault archive
and retrieve command has been modified to retrieve the transport
certificate and store it locally for subsequent usage. This way
it's no longer necessary to manually import the transport
certificate into the client's NSS database.

Ticket #3872
---
 API.txt |  5 +++
 VERSION |  4 +--
 ipalib/plugins/vault.py | 85 +++--
 3 files changed, 89 insertions(+), 5 deletions(-)

diff --git a/API.txt b/API.txt
index 
a46592ec9e82e618154bf09393c83d4b854315c5..95b86ce84f5bc9f1d879e561e07b0348d719c90e
 100644
--- a/API.txt
+++ b/API.txt
@@ -4629,6 +4629,11 @@ option: Str('version?', exclude='webui')
 output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
+command: vault_transport_cert
+args: 0,2,1
+option: Str('out?', cli_name='out')
+option: Str('version?', exclude='webui')
+output: Output('result', None, None)
 command: vaultcontainer_add
 args: 1,8,3
 arg: Str('cn', attribute=True, cli_name='container_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, required=True)
diff --git a/VERSION b/VERSION
index 
c471ed80af6a2c26be7fc89281ae60fac6c68577..d0ada131b700e93faa8c4946b811db36d76341a9
 100644
--- a/VERSION
+++ b/VERSION
@@ -90,5 +90,5 @@ IPA_DATA_VERSION=2010061412
 #  #
 
 IPA_API_VERSION_MAJOR=2
-IPA_API_VERSION_MINOR=110
-# Last change: edewata - added vault access control
+IPA_API_VERSION_MINOR=111
+# Last change: edewata - added vault transport certificate
diff --git a/ipalib/plugins/vault.py b/ipalib/plugins/vault.py
index 
95f96859235af1c477c8f5738a27571d64aabe3a..871c3e3a25c688a64ba0ecfde5ccbd50b47fbe01
 100644
--- a/ipalib/plugins/vault.py
+++ b/ipalib/plugins/vault.py
@@ -24,6 +24,8 @@ import shutil
 import string
 import tempfile
 
+import nss.nss as nss
+
 import pki
 import pki.account
 import pki.crypto
@@ -109,7 +111,7 @@ EXAMPLES:
 )
 
 register = Registry()
-transport_cert_nickname = KRA Transport Certificate
+transport_cert_filename = vault-transport.pem
 
 @register()
 class vaultcontainer(LDAPObject):
@@ -628,6 +630,63 @@ class vault_show(LDAPRetrieve):
 
 
 @register()
+class vault_transport_cert(Command):
+__doc__ = _('Retrieve vault transport certificate.')
+
+
+# list of attributes we want exported to JSON
+json_friendly_attributes = (
+'takes_args',
+)
+
+takes_options = (
+Str('out?',
+cli_name='out',
+doc=_('Output file to store the transport certificate'),
+),
+)
+
+has_output_params = (
+Str('certificate',
+label=_('Certificate'),
+),
+)
+
+def __json__(self):
+json_dict = dict(
+(a, getattr(self, a)) for a in self.json_friendly_attributes
+)
+json_dict['takes_options'] = list(self.get_json_options())
+return json_dict
+
+def execute(self, *args, **options):
+
+kra_client = api.Backend.kra.get_client()
+transport_cert = kra_client.system_certs.get_transport_cert()
+return {
+'result': {
+'certificate': transport_cert.encoded
+}
+}
+
+def forward(self, *args, **options):
+
+file = options.get('out')
+
+# don't send these parameters to server
+if 'out' in options:
+del options['out']
+
+response = super(vault_transport_cert, self).forward(*args, **options)
+
+if file:
+with open(file, 'w') as f:
+f.write(response['result']['certificate'])
+
+return response
+
+
+@register()
 class vault_archive(LDAPRetrieve):
 __doc__ = _('Archive a secret into a vault.')
 
@@ -743,7 +802,17 @@ class vault_archive(LDAPRetrieve):
 
 nonce = crypto.generate_nonce_iv()
 session_key = crypto.generate_session_key()
-nss_transport_cert = crypto.get_cert(transport_cert_nickname)
+
+   

Re: [Freeipa-devel] [PATCH] 779 webui: do not show closed dialog

2014-10-20 Thread Endi Sukma Dewata

On 10/20/2014 12:22 PM, Petr Vobornik wrote:

Fixes issues when dialog is not removed from `IPA.opened_dialogs`
registry when dialog.close() is called while the dialog is not shown,
i.e., while other dialog is shown. Without it, the dialog is could be
incorrectly displayed.

New dialog's property `opened` handles whether dialog is intended to be
opened.

How to test:

Add new host with IP address outside of managed reverse zones to get
error 4304.


Didn't have a chance to test it, but the code looks fine. ACK.
Perhaps the 'opened' property can be renamed to 'enabled' or something 
like that to avoid confusions with 'is_shown'.


--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 771 webui: do not offer ipa users to Default Trust View

2014-10-19 Thread Endi Sukma Dewata

On 10/19/2014 8:22 AM, Petr Vobornik wrote:

On 17.10.2014 22:51, Endi Sukma Dewata wrote:

On 10/10/2014 6:45 AM, Petr Vobornik wrote:

https://fedorahosted.org/freeipa/ticket/4616


This patch does not apply. Does it depend on another patch?



rebased version attached. Should be applicable on master, ipa-4-1 and
patchset 767-770.


ACK.

--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 767-770 webui: hide applied to hosts tab for Default Trust View

2014-10-19 Thread Endi Sukma Dewata

On 10/17/2014 4:55 PM, Petr Vobornik wrote:

On 17.10.2014 22:51, Endi Sukma Dewata wrote:

On 10/10/2014 6:44 AM, Petr Vobornik wrote:

Web UI part of:

https://fedorahosted.org/freeipa/ticket/4615

Patch 767 is a little refactoring needed for $pre_op(as plain object)
work as intended even with instantiated objects + fixes a bug where
Evented objects were not considered a framework object.

Patch 768 switches tabs so we can hide it later

Patch 769 hides the tab

PAtch 770 is not really needed(would like to hear options whether to
include it). It's in effect only if user somehow manages to open
'Applies to hosts' facet for 'Default trust view'. Maybe redirection
would be better - if we need to act.


For some reason I don't see the Default Trust View in the
database/CLI/UI with a brand new server installation. Alexander said he
will investigate on Monday.

The patches seem to be fine, I don't have any objections, feel free to
push. The missing Default Trust View is most likely unrelated to UI.


It should be added when you run ipa-adtrust-install.


OK, that fixed it. Some comments:

1. Shouldn't the Default Trust View entry be added during the initial 
installation? Although it's unlikely to conflict with user-defined 
entries, it's kind of strange to add a 'built-in' entry after the 
initial installation.


2. The description field in the Settings page for Default Trust View 
should be read-only since the entry cannot be modified.


3. The Delete action in the Settings page for Default Trust View should 
not exist since the entry cannot be deleted. Probably the Actions 
drop-down list can be disabled.


4. I think this was discussed before, but I'm just not sure what the 
plan is. The current facet tab titles seem to be redundant since we 
already have facet group headers that say ID view overrides/applies 
to. Are we going to change User/Group ID overrides into 
Users/Groups and Applied to hosts into Hosts?


No major issue. ACK.

--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 764 webui: management of keytab permissions

2014-10-17 Thread Endi Sukma Dewata

On 10/17/2014 8:36 AM, Petr Vobornik wrote:

Server part has been pushed. Version which matches its API attached.


ACK.

--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 765 webui: allow --force in dnszone-mod and dnsrecord-add

2014-10-17 Thread Endi Sukma Dewata

On 10/3/2014 11:48 AM, Petr Vobornik wrote:

Allow to use --force when changing authoritative nameserver address in
DNS zone.

Same for dnsrecord-add for NS record.

https://fedorahosted.org/freeipa/ticket/4573


ACK, just some minor issues:

1. The 'Check DNS' button might be interpreted as 'check the DNS without 
updating the DNS zone' but in fact it also updates the DNS zone. I'd 
suggest using a regular 'Update' button, but add a 'Force' checkbox.


2. A 'Force' checkbox would be more consistent with other dialog boxes 
than 'Skip DNS check' checkbox. Also, there might be other checks being 
done, not just DNS check.


3. When the authoritative nameserver is updated I get this message:

  semantic of '--name-server' option was changed: the option is used
  only for setting up the SOA MNAME attribute.

  To edit NS record(s) in zone apex, use command 'dnsrecord-mod [zone]
  @ --ns-rec=nameserver'.

I think the message shouldn't be displayed because it's irrelevant to 
the UI, a bit too long to read, and not enough time to read it before it 
disappears.


4. Server issue. I don't see a --force option in the dnsrecord-add help 
message.



Note: dnsrecord-mod doesn't support --force option to skip dns
resolution when changing NS record. Question is whether it should be
added, or if 'edit' button should be hidden in web ui to force
dnsrecord-add + dnsrecord-del combo instead of dnsrecord-mod, or if it's
good as is.
Note2: -add + -del combo has better UX than -del + -add if there is only
one value in the dns record.


I think if we're keeping the dnsrecord-mod CLI it should have a --force 
option too regardless whether the UI uses -mod or -add/del. Once that's 
added, the Edit dialog should provide a checkbox for that too.


The Edit dialog in DNS records is useful to edit records that consist of 
multiple fields (e.g. MX, SRV), so I think we should keep that, but 
internally it could be implemented either using -add/del or -mod, 
whichever works better.


--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 767-770 webui: hide applied to hosts tab for Default Trust View

2014-10-17 Thread Endi Sukma Dewata

On 10/10/2014 6:44 AM, Petr Vobornik wrote:

Web UI part of:

https://fedorahosted.org/freeipa/ticket/4615

Patch 767 is a little refactoring needed for $pre_op(as plain object)
work as intended even with instantiated objects + fixes a bug where
Evented objects were not considered a framework object.

Patch 768 switches tabs so we can hide it later

Patch 769 hides the tab

PAtch 770 is not really needed(would like to hear options whether to
include it). It's in effect only if user somehow manages to open
'Applies to hosts' facet for 'Default trust view'. Maybe redirection
would be better - if we need to act.


For some reason I don't see the Default Trust View in the 
database/CLI/UI with a brand new server installation. Alexander said he 
will investigate on Monday.


The patches seem to be fine, I don't have any objections, feel free to 
push. The missing Default Trust View is most likely unrelated to UI.


--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 771 webui: do not offer ipa users to Default Trust View

2014-10-17 Thread Endi Sukma Dewata

On 10/10/2014 6:45 AM, Petr Vobornik wrote:

https://fedorahosted.org/freeipa/ticket/4616


This patch does not apply. Does it depend on another patch?

--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 353 Added initial vault implementation.

2014-10-16 Thread Endi Sukma Dewata

On 10/15/2014 10:59 PM, Endi Sukma Dewata wrote:

The NSSConnection class has to be modified not to shutdown existing
database because some of the vault clients (e.g. vault-archive and
vault-retrieve) also use a database to encrypt/decrypt the secret.


The problem is described in more detail in this ticket:
https://fedorahosted.org/freeipa/ticket/4638

The changes to the NSSConnection in the first patch caused the 
installation to fail. Attached is a new patch that uses the solution 
proposed by jdennis.


--
Endi S. Dewata
From 65929b64da8d273a8b991845e47e69c1b3182f79 Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Tue, 16 Sep 2014 20:11:35 -0400
Subject: [PATCH] Added initial vault implementation.

This patch provides the initial vault implementation which allows
the admin to create a vault, archive a secret, and retrieve the
secret using a standard vault.

It currently has limitations including:
 - The vault only supports the standard vault type.
 - The vault can only be used by the admin user.
 - The transport certificate has to be installed manually.

These limitations, other vault features, schema and ACL changes will
be addressed in subsequent patches.

The NSSConnection class has been modified not to shutdown existing
database because some of the vault clients (e.g. vault-archive and
vault-retrieve) also use a database to encrypt/decrypt the secret.

Ticket #4638, #3872
---
 API.txt| 160 
 VERSION|   4 +-
 install/share/60basev4.ldif|   3 +
 install/share/Makefile.am  |   1 +
 install/share/copy-schema-to-ca.py |   1 +
 install/updates/40-vault.update|  27 ++
 install/updates/Makefile.am|   1 +
 ipa-client/man/default.conf.5  |   1 +
 ipalib/constants.py|   1 +
 ipalib/plugins/user.py |   9 +
 ipalib/plugins/vault.py| 726 +
 ipalib/rpc.py  |  34 +-
 ipapython/nsslib.py|  31 +-
 ipaserver/install/dsinstance.py|   1 +
 14 files changed, 973 insertions(+), 27 deletions(-)
 create mode 100644 install/share/60basev4.ldif
 create mode 100644 install/updates/40-vault.update
 create mode 100644 ipalib/plugins/vault.py

diff --git a/API.txt b/API.txt
index 
1af78509732b13eec07208114cea00e56c1059b4..1eec3527e36bc250acddbf0e2fe7a6baa30abd74
 100644
--- a/API.txt
+++ b/API.txt
@@ -4373,6 +4373,166 @@ option: Str('version?', exclude='webui')
 output: Output('result', type 'bool', None)
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
+command: vault_add
+args: 1,8,3
+arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Str('description', attribute=True, cli_name='desc', multivalue=False, 
required=False)
+option: Str('in?', cli_name='in')
+option: Str('parent', attribute=False, cli_name='parent', multivalue=False, 
required=False)
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Flag('rights', autofill=True, default=False)
+option: Bytes('secret', attribute=True, cli_name='secret', multivalue=False, 
required=False)
+option: Str('version?', exclude='webui')
+output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
+output: Output('summary', (type 'unicode', type 'NoneType'), None)
+output: PrimaryKey('value', None, None)
+command: vault_archive
+args: 1,10,3
+arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, query=True, required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Bytes('encrypted_data?', cli_name='encrypted_data')
+option: Str('in?', cli_name='in')
+option: Bytes('nonce?', cli_name='nonce')
+option: Str('parent?', cli_name='parent')
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Flag('rights', autofill=True, default=False)
+option: Bytes('secret?', cli_name='secret')
+option: Str('version?', exclude='webui')
+option: Bytes('wrapped_session_key?', cli_name='wrapped_session_key')
+output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
+output: Output('summary', (type 'unicode', type 'NoneType'), None)
+output: PrimaryKey('value', None, None)
+command: vault_del
+args: 1,3,3
+arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=True, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, query=True, required=True)
+option: Flag('continue

[Freeipa-devel] [PATCH] 353 Added initial vault implementation.

2014-10-15 Thread Endi Sukma Dewata

This patch provides the initial vault implementation which allows
the admin to create a vault, archive a secret, and retrieve the
secret using a standard vault.

It currently has limitations including:
 - The vault only supports the standard vault type.
 - The vault can only be used by the admin user.
 - The transport certificate has to be installed manually.

These limitations, other vault features, schema and ACL changes will
be addressed in subsequent patches.

The NSSConnection class has to be modified not to shutdown existing
database because some of the vault clients (e.g. vault-archive and
vault-retrieve) also use a database to encrypt/decrypt the secret.

Ticket #3872

--
Endi S. Dewata
From 1ad4307323c9e76ed51e5cdbd736e8834864f6fc Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Tue, 16 Sep 2014 20:11:35 -0400
Subject: [PATCH] Added initial vault implementation.

This patch provides the initial vault implementation which allows
the admin to create a vault, archive a secret, and retrieve the
secret using a standard vault.

It currently has limitations including:
 - The vault only supports the standard vault type.
 - The vault can only be used by the admin user.
 - The transport certificate has to be installed manually.

These limitations, other vault features, schema and ACL changes will
be addressed in subsequent patches.

The NSSConnection class has to be modified not to shutdown existing
database because some of the vault clients (e.g. vault-archive and
vault-retrieve) also use a database to encrypt/decrypt the secret.

Ticket #3872
---
 API.txt| 160 
 VERSION|   4 +-
 install/share/60basev4.ldif|   3 +
 install/share/Makefile.am  |   1 +
 install/share/copy-schema-to-ca.py |   1 +
 install/updates/40-vault.update|  27 ++
 install/updates/Makefile.am|   1 +
 ipa-client/man/default.conf.5  |   1 +
 ipalib/constants.py|   1 +
 ipalib/plugins/user.py |   9 +
 ipalib/plugins/vault.py| 726 +
 ipapython/nsslib.py|  22 +-
 ipaserver/install/dsinstance.py|   1 +
 13 files changed, 943 insertions(+), 14 deletions(-)
 create mode 100644 install/share/60basev4.ldif
 create mode 100644 install/updates/40-vault.update
 create mode 100644 ipalib/plugins/vault.py

diff --git a/API.txt b/API.txt
index 
1af78509732b13eec07208114cea00e56c1059b4..1eec3527e36bc250acddbf0e2fe7a6baa30abd74
 100644
--- a/API.txt
+++ b/API.txt
@@ -4373,6 +4373,166 @@ option: Str('version?', exclude='webui')
 output: Output('result', type 'bool', None)
 output: Output('summary', (type 'unicode', type 'NoneType'), None)
 output: PrimaryKey('value', None, None)
+command: vault_add
+args: 1,8,3
+arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Str('description', attribute=True, cli_name='desc', multivalue=False, 
required=False)
+option: Str('in?', cli_name='in')
+option: Str('parent', attribute=False, cli_name='parent', multivalue=False, 
required=False)
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Flag('rights', autofill=True, default=False)
+option: Bytes('secret', attribute=True, cli_name='secret', multivalue=False, 
required=False)
+option: Str('version?', exclude='webui')
+output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
+output: Output('summary', (type 'unicode', type 'NoneType'), None)
+output: PrimaryKey('value', None, None)
+command: vault_archive
+args: 1,10,3
+arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=False, 
pattern='^[a-zA-Z0-9_.][a-zA-Z0-9_.-]{0,252}[a-zA-Z0-9_.$-]?$', 
primary_key=True, query=True, required=True)
+option: Flag('all', autofill=True, cli_name='all', default=False, 
exclude='webui')
+option: Bytes('encrypted_data?', cli_name='encrypted_data')
+option: Str('in?', cli_name='in')
+option: Bytes('nonce?', cli_name='nonce')
+option: Str('parent?', cli_name='parent')
+option: Flag('raw', autofill=True, cli_name='raw', default=False, 
exclude='webui')
+option: Flag('rights', autofill=True, default=False)
+option: Bytes('secret?', cli_name='secret')
+option: Str('version?', exclude='webui')
+option: Bytes('wrapped_session_key?', cli_name='wrapped_session_key')
+output: Entry('result', type 'dict', Gettext('A dictionary representing an 
LDAP entry', domain='ipa', localedir=None))
+output: Output('summary', (type 'unicode', type 'NoneType'), None)
+output: PrimaryKey('value', None, None)
+command: vault_del
+args: 1,3,3
+arg: Str('cn', attribute=True, cli_name='vault_name', maxlength=255, 
multivalue=True, 

[Freeipa-devel] [PATCH] 352 Fixed KRA backend.

2014-10-15 Thread Endi Sukma Dewata

The KRA backend has been simplified since most of the tasks have
been moved somewhere else. The transport certificate will be
installed on the client, and it is not needed by KRA backend. The
KRA agent's PEM certificate is now generated during installation
due to permission issue. The kra_host() for now is removed since
the current ldap_enable() cannot register the KRA service, so it
is using the kra_host environment variable.

The KRA installer has been modified to use Dogtag's CLI go create
KRA agent and setup the client authentication.

The proxy settings have been updated to include KRA's URLs.

The certs.install_pem_from_p12() has been updated to generate the
proper client certificate using the -clcerts option and also take
a password file.

Some constants have been renamed for clarity. The DOGTAG_AGENT_P12
has been renamed to DOGTAG_ADMIN_P12 since file actually contains
the Dogtag admin's certificate and private key and it can be used
to access both CA and KRA. The DOGTAG_AGENT_PEM has been renamed
to KRA_AGENT_PEM since it can only be used for KRA.

Ticket #3872

--
Endi S. Dewata
From 018d849db0f08285aeeb26e3fc886c8c161b3f33 Mon Sep 17 00:00:00 2001
From: Endi S. Dewata edew...@redhat.com
Date: Wed, 1 Oct 2014 14:59:46 -0400
Subject: [PATCH] Fixed KRA backend.

The KRA backend has been simplified since most of the tasks have
been moved somewhere else. The transport certificate will be
installed on the client, and it is not needed by KRA backend. The
KRA agent's PEM certificate is now generated during installation
due to permission issue. The kra_host() for now is removed since
the current ldap_enable() cannot register the KRA service, so it
is using the kra_host environment variable.

The KRA installer has been modified to use Dogtag's CLI go create
KRA agent and setup the client authentication.

The proxy settings have been updated to include KRA's URLs.

The certs.install_pem_from_p12() has been updated to generate the
proper client certificate using the -clcerts option and also take
a password file.

Some constants have been renamed for clarity. The DOGTAG_AGENT_P12
has been renamed to DOGTAG_ADMIN_P12 since file actually contains
the Dogtag admin's certificate and private key and it can be used
to access both CA and KRA. The DOGTAG_AGENT_PEM has been renamed
to KRA_AGENT_PEM since it can only be used for KRA.

Ticket #3872
---
 install/conf/ipa-pki-proxy.conf  |   2 +-
 ipaplatform/base/paths.py|   4 +-
 ipaserver/install/cainstance.py  |   4 +-
 ipaserver/install/certs.py   |  10 ++--
 ipaserver/install/ipa_backup.py  |   3 +-
 ipaserver/install/krainstance.py |  83 ---
 ipaserver/plugins/dogtag.py  | 120 +--
 7 files changed, 100 insertions(+), 126 deletions(-)

diff --git a/install/conf/ipa-pki-proxy.conf b/install/conf/ipa-pki-proxy.conf
index 
2370b4d7a7467a7e47c0d223915e018c9a009e83..5d21156848f3b5ddf14c42d92a26a30a9f94af36
 100644
--- a/install/conf/ipa-pki-proxy.conf
+++ b/install/conf/ipa-pki-proxy.conf
@@ -19,7 +19,7 @@ ProxyRequests Off
 /LocationMatch
 
 # matches for agent port and eeca port
-LocationMatch 
^/ca/agent/ca/displayBySerial|^/ca/agent/ca/doRevoke|^/ca/agent/ca/doUnrevoke|^/ca/agent/ca/updateDomainXML|^/ca/eeca/ca/profileSubmitSSLClient|^/kra/agent/kra/connector|^/kra/rest/agent/keyrequests|^/kra/rest/agent/keys|^/ca/rest/admin/kraconnector/remove
+LocationMatch 
^/ca/agent/ca/displayBySerial|^/ca/agent/ca/doRevoke|^/ca/agent/ca/doUnrevoke|^/ca/agent/ca/updateDomainXML|^/ca/eeca/ca/profileSubmitSSLClient|^/kra/agent/kra/connector|^/kra/rest/account|^/kra/rest/agent/keyrequests|^/kra/rest/agent/keys|^/ca/rest/admin/kraconnector/remove
 NSSOptions +StdEnvVars +ExportCertData +StrictRequire +OptRenegotiate
 NSSVerifyClient require
 ProxyPassMatch ajp://localhost:$DOGTAG_PORT
diff --git a/ipaplatform/base/paths.py b/ipaplatform/base/paths.py
index 
0ba6b46c562a3d3f97e3631fd9dc0f27536bc6bc..3652a3213d2d116b34d7c4239964c511f572ed6b
 100644
--- a/ipaplatform/base/paths.py
+++ b/ipaplatform/base/paths.py
@@ -127,8 +127,8 @@ class BasePathNamespace(object):
 HOME_DIR = /home
 ROOT_IPA_CACHE = /root/.ipa_cache
 ROOT_PKI = /root/.pki
-DOGTAG_AGENT_P12 = /root/ca-agent.p12
-DOGTAG_AGENT_PEM = /etc/httpd/alias/agent.pem
+DOGTAG_ADMIN_P12 = /root/ca-agent.p12
+KRA_AGENT_PEM = /etc/httpd/alias/kra-agent.pem
 CACERT_P12 = /root/cacert.p12
 ROOT_IPA_CSR = /root/ipa.csr
 ROOT_TMP_CA_P12 = /root/tmp-ca.p12
diff --git a/ipaserver/install/cainstance.py b/ipaserver/install/cainstance.py
index 
2c912206a37accfdf217c955755b82ed0d2056af..aa6de498726e779e62cd96ec13d9e52cd1262714
 100644
--- a/ipaserver/install/cainstance.py
+++ b/ipaserver/install/cainstance.py
@@ -514,7 +514,7 @@ class CAInstance(DogtagInstance):
 config.set(CA, pki_admin_nickname, ipa-ca-agent)
 config.set(CA, pki_admin_subject_dn,
 str(DN(('cn', 'ipa-ca-agent'), 

Re: [Freeipa-devel] [PATCH] 757 webui: do not offer ipa-ad-winsync and ipa-ipa-trust range types

2014-09-29 Thread Endi Sukma Dewata

On 9/29/2014 2:40 AM, Petr Vobornik wrote:

On 24.9.2014 12:17, Petr Vobornik wrote:

webui: do not offer ipa-ad-winsync and ipa-ipa-trust range types

They are not supported by API.


Forgot to attach patch...


ACK.

--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 749-754 webui: new ID views section

2014-09-28 Thread Endi Sukma Dewata

On 9/26/2014 10:25 AM, Petr Vobornik wrote:

On 25.9.2014 19:07, Petr Vobornik wrote:

All issues will be done separately as already stated in other
sub-thread. I've removed issues which are discussed in the other
sub-thread.


2. The tab titles in the ID view details page are quite long, and the
User ID overrides and Group ID overrides labels aren't quite
appropriate because the ID view can override other attributes too. How
about using facet groups like in User Groups? For example:
- ID view applies to:
  - Hosts
- ID view overrides:
  - Users
  - Groups
- Settings


Attached patch 758


ACK.


7. Related to #6, there probably should be a tab in the Host details
page showing which ID views apply to it.

There is only a single view and yes, it would be good to add a property
there, linking it to the ID view tab, if possible.


Will add simple readonly field (link to view). It will be improved later
(based on ipa-4-1 priorities)


Attached patch 760


ACK.


9. This probably requires server support. In the Apply to hosts
association dialog, if a host is already added it will still appear in
the dialog box. As a comparison, a User that has been added into a
User Group will not appear in the association dialog anymore.

Could be trivially filtered out on Web UI side.


Will be implemented.


Attached patch 759 - quite ugly hack but does the job


ACK.

--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 749-754 webui: new ID views section

2014-09-25 Thread Endi Sukma Dewata

On 9/25/2014 2:25 AM, Alexander Bokovoy wrote:

On Wed, 24 Sep 2014, Endi Sukma Dewata wrote:

4. If I understand correctly the description field for the User ID
Overrides and Group ID Overrides should be optional too because it's
also used to optionally override the description attribute of the
original entry.



No, this is description of the override itself. We don't want to
override original description field, if any, we want to provide a way to
document why this override was done.


In that case the 'description' probably should have been a MUST.

objectClasses: (2.16.840.1.113730.3.8.12.30 NAME 'ipaOverrideAnchor' SUP 
top STRUCTURAL MUST ( ipaAnchorUUID ) MAY ( description ) X-ORIGIN 'IPA 
v4' )


BTW, the LDAP schema in the wiki page is outdated:
http://www.freeipa.org/page/V4/Migrating_existing_environments_to_Trust


6. Can multiple ID views be applied to the same host? Does the order
matter? If so, how would the UI manage the order?



No. Single ID view per host. The scheme is actually a bit more complex:
- IPA users: data from main tree is overridden with a data from a
   host-specific ID view
- AD users: data from AD is overridden by a data from a default trust
   view which is then overridden by a data from a host-specific ID view


OK, right now if I apply an ID view to a host that already uses another 
ID view, the host will be removed silently from the other ID view. 
Should the operation fail/provide a warning if the host already uses 
another ID view?



7. Related to #6, there probably should be a tab in the Host details
page showing which ID views apply to it.



There is only a single view and yes, it would be good to add a property
there, linking it to the ID view tab, if possible.


I think that field should be editable as well so you can select the ID 
view from Host details page.



9. This probably requires server support. In the Apply to hosts
association dialog, if a host is already added it will still appear in
the dialog box. As a comparison, a User that has been added into a
User Group will not appear in the association dialog anymore.



Could be trivially filtered out on Web UI side.


It may not be possible if the list of hosts is paged. The UI will not 
get the full list of hosts, so it won't be able to filter out hosts that 
are already added but not currently displayed. I'm not sure how 
important is this, but we did this for some other pages.


Since #4 is not a UI issue, patch #754 is ACKed. Other issues can be 
addressed later.


--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 755 webui-ci: case-insensitive record check

2014-09-24 Thread Endi Sukma Dewata

On 9/24/2014 8:26 AM, Petr Vobornik wrote:

On 24.9.2014 04:43, Endi Sukma Dewata wrote:

On 9/22/2014 9:49 AM, Petr Vobornik wrote:

[PATCH] webui-ci: case-insensitive record check

Indirect association are no longer lower cased, which caused a issue
in CI.


Is the use of |= operator intentional? I don't see the has variable
defined anywhere else in this method.

   has |= self.has_record(pkey.lower(), parent, table_name)

If this has been tested to work, then ACK.



it's defined on the previous line:

   has = self.has_record(pkey, parent, table_name)
   has |= self.has_record(pkey.lower(), parent, table_name)


Ah, I see. I thought the old line was being replaced.
ACK.

--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 749-754 webui: new ID views section

2014-09-24 Thread Endi Sukma Dewata

On 9/19/2014 7:29 AM, Petr Vobornik wrote:

Hello,

attached patches implements Web UI part of ID Views. Backend is
currently on review as well - thread [PATCHES 247-259] ID views -
management part.

https://fedorahosted.org/freeipa/ticket/4535

I expect that backed can change and that the UI might influence it as
well. Therefore no UI integration tests nor static data files are
included in this patchset. They will follow when backend is stable.

First 5 patches extends UI framework to support the main patch (#754).


== [PATCH] 749 webui: improve breadcrumb navigation ==

Fixes issue when:
- user navigates to a nested facet
- refreshes browser
- uses breadcrumb navigation to go to parent entity page which requires
a pkey. E.g. from automount keys to maps.

The old code relies on the facet, that user visited the parent facet
before and therefore the facet has pkey stored. It fails after the
browser reload.

Allows to specify a containing_facet. It allows breadcrumb navigation to
return to a different facet than the 'default'.


ACK.


== [PATCH] 750 webui: treat value as pkey in link widget ==

Current default mechanism of a link widget assumes that pkeys of a
current facet are pkeys for the link. It works for the only usage - in
password policy. It's rather inflexible since it can't be used if the
keys are in other attribute. This behavior is also bad in nested
entities - creates a link to itself which is pointless.

This patch changes the default behavior to assume that the supplied
value are the pkeys and that the last pkey is the value to display.

It also keeps the old method of overriding `other_pkeys` method so if
the last and only pkey is the actual value to display then the method
can tranform it into the pkeys which keeps compatibility with descendant
widgets (`host_dnsrecord_entity_link_widget`,
`dnsrecord_host_link_widget`).


ACK.


== [PATCH] 751 webui: do not show internal facet name to user ==


ACK.


== [PATCH] 752 webui: allow to skip link widget link validation ==


ACK.


== [PATCH] 753 webui: add simple link column suppor ==

Usual link columns are link with primary key of current entity.

This patch allows to create a link to arbitrary non-nested entity.


ACK.


== [PATCH] 754 webui: new ID views section ==


I'll test this separately. Does your patch #754-1 work with Tomas' 
latest patches (#247-2 - 270)?



--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 749-754 webui: new ID views section

2014-09-24 Thread Endi Sukma Dewata

On 9/24/2014 9:43 AM, Petr Vobornik wrote:

On 24.9.2014 16:30, Endi Sukma Dewata wrote:

On 9/19/2014 7:29 AM, Petr Vobornik wrote:

Hello,

attached patches implements Web UI part of ID Views. Backend is
currently on review as well - thread [PATCHES 247-259] ID views -
management part.

https://fedorahosted.org/freeipa/ticket/4535

I expect that backed can change and that the UI might influence it as
well. Therefore no UI integration tests nor static data files are
included in this patchset. They will follow when backend is stable.



snip




== [PATCH] 754 webui: new ID views section ==


I'll test this separately. Does your patch #754-1 work with Tomas'
latest patches (#247-2 - 270)?



It was tested with tomas' git branch which matched
http://www.redhat.com/archives/freeipa-devel/2014-September/msg00336.html



OK, some comments/questions:

1. For consistency, the ID view should be capitalized into ID View 
in the navigation tab, page title, and dialog title. See ID Ranges as 
an example.


2. The tab titles in the ID view details page are quite long, and the 
User ID overrides and Group ID overrides labels aren't quite 
appropriate because the ID view can override other attributes too. How 
about using facet groups like in User Groups? For example:

 - ID view applies to:
   - Hosts
 - ID view overrides:
   - Users
   - Groups
 - Settings

3. Since the tab already says Applied to hosts, the current button 
labels is kind of redundant. How about renaming and reorder the buttons 
like this?

 - Refresh
 - Remove
 - Add
 - Add hosts in host group
 - Remove hosts in host group

4. If I understand correctly the description field for the User ID 
Overrides and Group ID Overrides should be optional too because it's 
also used to optionally override the description attribute of the 
original entry.


5. Not sure if this is a problem. The search field in User/Group ID 
Overrides can be used to find the overriding attributes, but not the 
User/Group to override.


6. Can multiple ID views be applied to the same host? Does the order 
matter? If so, how would the UI manage the order?


7. Related to #6, there probably should be a tab in the Host details 
page showing which ID views apply to it.


8. If we implement #7, are the Un-apply from hosts/host groups buttons 
in the ID views search page still necessary? Or can it be moved into 
that page (i.e. unapplying one host at a time)?


9. This probably requires server support. In the Apply to hosts 
association dialog, if a host is already added it will still appear in 
the dialog box. As a comparison, a User that has been added into a User 
Group will not appear in the association dialog anymore.


10. The use of association dialog for Apply to host groups and 
Un-apply from host groups is a bit unusual because it's used to select 
host groups, and once selected the host groups are not added to/removed 
from the main list because the main list contains hosts, not host 
groups. Would it be very common to select multiple host groups at once 
to apply ID view? If not, it might make more sense to just use a plain 
dialog to select one host group at a time. The dialog could probably 
show information about the host groups (i.e. host members) before 
applying/unapplying the ID view.


I think the only major issue is #4. Once it's addressed, ACK.
The others are minor issues, questions, and suggestions.

--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 755 webui-ci: case-insensitive record check

2014-09-23 Thread Endi Sukma Dewata

On 9/22/2014 9:49 AM, Petr Vobornik wrote:

[PATCH] webui-ci: case-insensitive record check

Indirect association are no longer lower cased, which caused a issue in CI.


Is the use of |= operator intentional? I don't see the has variable 
defined anywhere else in this method.


  has |= self.has_record(pkey.lower(), parent, table_name)

If this has been tested to work, then ACK.

--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 756 webui: fix regression in association facet preop

2014-09-23 Thread Endi Sukma Dewata

On 9/22/2014 9:50 AM, Petr Vobornik wrote:

Association facet specs use 'add_method' instead of 'add_command'

origin: https://fedorahosted.org/freeipa/ticket/4507


ACK.

--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 720-729 OTP usability improvements

2014-09-10 Thread Endi Sukma Dewata

On 9/9/2014 10:48 AM, Petr Vobornik wrote:

[PATCH] 727 webui: hide empty fields and sections



Will the counter field strictly have a
value with HOTP only and clock offset  interval fields strictly have
value with TOTP only? Do these fields contain the configured values or
the effective values? I was just thinking maybe in the future some of
these fields might be configured empty and they will use the default
values instead. If that's not a problem then ACK.


Respective fields are present only in corresponding object classes -
there won't be an HTOP token with 'clock offset'. If they are present,
they have a default value. - No false hiding - Shouldn't be a problem.


ACK.


What do you think about setting `hide_empty_widgets` global setting to
True?


On a read-only page I think it's OK to hide empty fields. But on edit
page I don't think we should do that by default.


Maybe we should first clarify what is a read-only page. All details
pages are writable if user has the rights or read-only if he doesn't.
But mostly it depends on individual fields/attributes.


Read-only page is a page that does not provide editable interface for 
all fields displayed in the page. For example, the fields in the 
certificate details page are never editable, so suppose it has empty 
optional fields (e.g. cert extensions?) we probably can hide those 
fields without confusing users.


Other details pages have read-only and edit modes, so it probably makes 
more sense to keep the structure identical in both modes so optional 
fields like 'Job Title' will always be visible. However, it's also 
possible to hide the empty fields in the read-only mode if necessary to 
clean up the display.



Does empty mean unset, or blank like empty string/array? Does
unset always mean non-editable and invisible and blank always mean
editable and visible but it's just empty? If this definition isn't
strictly followed there could be an editable field that accidentally
gets hidden because it's empty.


My intent was to show everything what user can edit. And hide fields:
1. for which he doesn't have read rights
2. have no (undefined) or empty('') value
3. are explicitly hidden by other logic - not related to this patch

When thinking about #1. Maybe we should base it on a presence or rather
not presence of a specific objectclass, rather than on a value. That way
it would be less confusing for newcomers. The logic is: if fields is
bound to an object class and the class is not present (that also means
we don't have attribute level rights) - hide it.


That would be hiding/displaying based on structural reason instead of 
user rights. How about an attribute that exists but the user doesn't 
have the read permission? I think we need to check both structural and 
user rights.



#2 is questionable. IMHO it would require user tests. Also hiding on ''
value might not be always desirable. Nevertheless I like the behavior
but it may be caused by the fact that I already know what to expect.


Before we can make this behavior the default/global, how do we make sure 
that optional fields such as 'Job Title' will always visible in the edit 
page even if it's empty? I think to avoid unexpected behavior hiding 
based on empty value should only be done in read-only page as I 
mentioned above.



Generally if a field is supposed to be hidden in an edit page because of
other condition (e.g. token type), not because of the value, the code
should also use that condition instead of relying on the value being
unset. In this particular case, instead of:

   if (field !== undefined) {
   // display field
   }

we probably should do:

   var type = ...

   if (type == 'hotp') {
   // display hotp fields

   } else { // totp
   // display totp fields
   }


I agree - use case #3.


This is actually hiding/displaying based on structural reason too. In 
this case the structure is reflected in the (virtual) 'type' field.



4. If no Owner is specified when adding a token, it will default to
admin. Is this the intended behavior?


Sadly yes.


Maybe the adder dialog should show admin (or the current user?) as the
default owner instead of empty?



Note: all patches except for 727 are ACKed.


It's all ACKed now. Everything else can be handled separately.

--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 746-747 append domain into network.negotiate-auth.trusted-uris

2014-09-10 Thread Endi Sukma Dewata

On 9/10/2014 9:59 AM, Petr Vobornik wrote:

On 4.9.2014 21:26, Endi Sukma Dewata wrote:

On 8/29/2014 11:00 AM, Petr Vobornik wrote:

[PATCH] 746 webui: append network.negotiate-auth.trusted-uris

https://fedorahosted.org/freeipa/ticket/4478


Some comments/questions:

1. If I'm reading this correctly, if the preference is currently empty,
the method will just return without setting the new value.


Fixed. Yes, even if there was 'continue', it would be wrong.



2. If the new value already exists, the method will just return.
Shouldn't it continue with the rest of the loop instead of return?
This applies to #1 as well.


Yes, fixed.



3. Using indexOf() to find a URI in a string can produce false matches.
For example, aa.com will match baa.com. Ideally the existing value
should be parsed into a collection of URI's, then the new URI should be
matched using a proper URI matching algorithm.


Fixed with function which matches splitted and striped values using
strict equality.


ACK. A possible improvement is to normalize the current value to remove 
excessive white spaces.


--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 744 webui: switch associators if default doesn't work

2014-09-04 Thread Endi Sukma Dewata

On 8/22/2014 11:29 AM, Petr Vobornik wrote:

Ticket: https://fedorahosted.org/freeipa/ticket/4507

Support for delegating RBAC roles to service principals added new
attribute members. [1][2] Most of Web UI was automatically extended but
the defaults chose wrong associator for service's memberof_role facet

traditionally it would be solved by

{
 $type: 'association',
 name: 'memberof_role',
 associator: IPA.serial_associator
 }

This patch tries to make the auto-magic functionality little bit less
stupid to eliminate a need for ^^ patches. It's far from perfect -
doesn't support things like:

{
 $type: 'association',
 name: 'memberof_sudorule',
 associator: IPA.serial_associator,
 add_method: 'add_user',
 remove_method: 'remove_user'
 }

[1]
https://git.fedorahosted.org/cgit/freeipa.git/commit/?id=8fabd6dde152fc394bd4f093d93c8a46e5b2851b

[2] https://fedorahosted.org/freeipa/ticket/3164


ACK.

--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 745 webui: notify psw change success only once

2014-09-04 Thread Endi Sukma Dewata

On 8/29/2014 3:40 AM, Petr Vobornik wrote:

Password change initiated from header menu notified success twice.
First one in `dialogs.password.dialog` and second one in a success
callback. The second notification was removed.

Caused by:
https://fedorahosted.org/freeipa/changeset/870db2f677dff01750aeec104c90fce3ca0e54be/


ACK.

--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 746-747 append domain into network.negotiate-auth.trusted-uris

2014-09-04 Thread Endi Sukma Dewata

On 8/29/2014 11:00 AM, Petr Vobornik wrote:

[PATCH] 746 webui: append network.negotiate-auth.trusted-uris

https://fedorahosted.org/freeipa/ticket/4478


Some comments/questions:

1. If I'm reading this correctly, if the preference is currently empty, 
the method will just return without setting the new value.


2. If the new value already exists, the method will just return. 
Shouldn't it continue with the rest of the loop instead of return? 
This applies to #1 as well.


3. Using indexOf() to find a URI in a string can produce false matches. 
For example, aa.com will match baa.com. Ideally the existing value 
should be parsed into a collection of URI's, then the new URI should be 
matched using a proper URI matching algorithm.



[PATCH] 747 install: create ff krb extension on every install, replica
install and upgrade

We don't want to copy the extension from master to replica because the
replica may use newer version of FreeIPA and therefore the extension
code might be obsolete. Same reason for upgrades.

https://fedorahosted.org/freeipa/ticket/4478


ACK.

--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 720-729 OTP usability improvements

2014-09-03 Thread Endi Sukma Dewata

On 8/22/2014 3:31 AM, Petr Vobornik wrote:

On 12.8.2014 17:59, Endi Sukma Dewata wrote:

On 8/5/2014 6:31 AM, Petr Vobornik wrote:

ticket: https://fedorahosted.org/freeipa/ticket/4402


snip (ACK of 720, 721) but patch 720 was replaced by a new version


ACK.


[PATCH] 724 webui: display fields based on otp token type

- in adder dialog


is it an ACK?


Sorry, missed this one. ACK.


[PATCH] 727 webui: hide empty fields and sections

Hide widgets without a value. Must be explicitly turned on. In widget by
`hidden_if_empty` flag. Or globally by `hide_empty_widgets` flag. Global
hiding can be individually turned off by `ignore_empty_hiding` flag.


In item #2 of ticket #4402 I was originally thinking the widget
visibility would be determined by the token type.


Originally I wrote it that way but with this patch it became redundant.


Any plan to add the
token type field in the future?


maybe, I don't have any strong feelings about it. Will users benefit
from adding it? If yes, it should be also added to CLI.


Can the users tell which type of token they have based on the existing 
fields? There is a model field which is populated with the type (e.g. 
totp or hotp). But since the value can be changed to anything it's not a 
reliable way to determine the type. I don't think it's very 
user-friendly to ask the user to see whether the type-specific fields 
are shown in order to determine the type.


I can't say there's a big benefit by adding the field, but maybe some 
admins might find it useful, and it can be used to sort/filter out 
search results.



Atm. token type is derived from object classes. It exists only in add
operation as a virtual attribute. We can check a presence of
ipatokenhotp or ipatokentotp object classes to simulate the field.


Yes, if people can add an attribute usually they will expect to be able 
to see what they added.



Will the counter field strictly have a
value with HOTP only and clock offset  interval fields strictly have
value with TOTP only? Do these fields contain the configured values or
the effective values? I was just thinking maybe in the future some of
these fields might be configured empty and they will use the default
values instead. If that's not a problem then ACK.


Respective fields are present only in corresponding object classes -
there won't be an HTOP token with 'clock offset'. If they are present,
they have a default value. - No false hiding - Shouldn't be a problem.

Btw: Other of my other original intents was to hide it based on ACI
rights. The issues is that the rights are not present without
corresponding OC. Hiding in such case is dangerous - explicitly disabled
in patch 728.

What do you think about setting `hide_empty_widgets` global setting to
True?


On a read-only page I think it's OK to hide empty fields. But on edit 
page I don't think we should do that by default.


Does empty mean unset, or blank like empty string/array? Does 
unset always mean non-editable and invisible and blank always mean 
editable and visible but it's just empty? If this definition isn't 
strictly followed there could be an editable field that accidentally 
gets hidden because it's empty.


Generally if a field is supposed to be hidden in an edit page because of 
other condition (e.g. token type), not because of the value, the code 
should also use that condition instead of relying on the value being 
unset. In this particular case, instead of:


  if (field !== undefined) {
  // display field
  }

we probably should do:

  var type = ...

  if (type == 'hotp') {
  // display hotp fields

  } else { // totp
  // display totp fields
  }

For now the type can be determined by the availability of certain 
type-specific fields, but in the future we may add the type itself.



3. The Clock offset field doesn't have a unit.


Fixed in patch 720-1, patch 729 was rebased


ACK.


4. If no Owner is specified when adding a token, it will default to
admin. Is this the intended behavior?


Sadly yes.


Maybe the adder dialog should show admin (or the current user?) as the 
default owner instead of empty?


--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 748 webui: extract complex pkey on Add and Edit

2014-09-03 Thread Endi Sukma Dewata

On 9/2/2014 10:15 AM, Petr Vobornik wrote:

DNS zone 'Add and Edit' failed because of new DNS name encoding.

This patch makes sure that keys are extracted properly.

https://fedorahosted.org/freeipa/ticket/4520


ACK.

--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 742 webui: adjust behavior of bounce url

2014-09-03 Thread Endi Sukma Dewata

On 8/21/2014 11:06 AM, Petr Vobornik wrote:

based  on:
http://www.redhat.com/archives/freeipa-devel/2014-August/msg00073.html

- bounce url param was renamed from 'redirect' to 'url'
- support for 'delay' param added

Behavior:

- Continue to next page link is shown if 'url' is present
- page is no longer automatically redirected if 'url' is present
- automatic redirect is controlled by 'delay' param - it specifies
   number of seconds until redirection
- info message 'You will be redirected in Xs' is show to notify
   the user that something will happen. It's useful even if delay
   is 0 or negative because redirection might be slow.
- counter is decremented every second
- delay is ignored if parsed as NaN

https://fedorahosted.org/freeipa/ticket/4440


ACK.

Just one thing, when the delay=0 and the direction happens quickly, the 
users might see the confirmation and the redirection messages displayed 
briefly on the screen but they cannot read it because it's too quick, 
which might leave them wondering what it was.


I think delay=0 is a special case where we want a seamless integration 
with 3rd party application. If the password reset is completed 
successfully, it should just display the next page in the 3rd party 
application. Users shouldn't see a 'redirection' message. To them it's 
all one application.


--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 743 webui: do not show login error when switching back from otp sync screen

2014-09-03 Thread Endi Sukma Dewata

On 8/22/2014 6:51 AM, Petr Vobornik wrote:

Errors should reflect only a result of last operation.

https://fedorahosted.org/freeipa/ticket/4470

Fixes issue found by Endi:


Try logging in with an incorrect password/OTP. After you get a login
error click Sync OTP Token. Once the sync is completed it will go
back to the login page with a Token was synchronized message that
disappears in a few seconds, but the old login error still appears
which is confusing. Error messages in the UI should only reflect the
last executed operation.


ACK.

--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 736-740 webui: various minor fixes

2014-08-22 Thread Endi Sukma Dewata

On 8/21/2014 7:18 AM, Simo Sorce wrote:

On Thu, 2014-08-21 at 14:11 +0200, Petr Vobornik wrote:

On 13.8.2014 17:20, Endi Sukma Dewata wrote:

2. Can the UI parse the new key and display it the same way as other
keys that are already saved? That will make it more seamless. 

Would be nice, but is it worth the effort? We would have to
reimplement ipapython.ssh into JavaScript + pull in crypto.js or other
lib for sha1and sha256 functions since Web Cryptography API is still
only a draft [1].

I do not do this lightly, but you have my veto to do any crypto in
javascript unless you convince me first it make sense.

Simo.


Just to clarify, the point is to display some kind of information about 
the public keys the user just entered in the UI (that have not been 
saved) so that:
a) If user enters multiple keys at once, the user can distinguish which 
keys have been added rather than displaying a generic New: key set for 
all new keys.

b) The UI can detect if the key already exists on the server.
If this can be done without any cryptographic operations that would be 
great, but I'm not sure about the details.


For (a) I was wondering if the UI can decode the base-64 encoded public 
key entered by the user and display some user-friendly information about 
the key itself, maybe just the hexdump of the first few bytes of the 
key, or maybe the key type too if possible, or at least just the first 
few chars of the undecoded key.


For (b) the UI would have to use some crypto functions to generate the 
key fingerprints as generated by the server. But even if we do that, we 
are not doing any data encryption or dealing with secret information.


Alternatively, instead of displaying the fingerprints of the saved keys, 
the UI can display the first few bytes/chars of hexdump/encoded key to 
match (a) such that they can be compared without cryptography.


BTW, WebCrypto implementation has been making its way into Firefox:
https://docs.google.com/spreadsheet/ccc?key=0AiAcidBZRLxndE9LWEs2R1oxZ0xidUVoU3FQbFFobkEusp=sharing

--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 736-740 webui: various minor fixes

2014-08-22 Thread Endi Sukma Dewata

On 8/22/2014 12:18 PM, Petr Vobornik wrote:

On 22.8.2014 17:51, Simo Sorce wrote:

On Fri, 2014-08-22 at 09:52 -0500, Endi Sukma Dewata wrote:

On 8/21/2014 7:18 AM, Simo Sorce wrote:

On Thu, 2014-08-21 at 14:11 +0200, Petr Vobornik wrote:

On 13.8.2014 17:20, Endi Sukma Dewata wrote:

2. Can the UI parse the new key and display it the same way as other
keys that are already saved? That will make it more seamless.

Would be nice, but is it worth the effort? We would have to
reimplement ipapython.ssh into JavaScript + pull in crypto.js or 
other

lib for sha1and sha256 functions since Web Cryptography API is still
only a draft [1].

I do not do this lightly, but you have my veto to do any crypto in
javascript unless you convince me first it make sense.

Simo.


Just to clarify, the point is to display some kind of information about
the public keys the user just entered in the UI (that have not been
saved) so that:
a) If user enters multiple keys at once, the user can distinguish which
keys have been added rather than displaying a generic New: key set 
for

all new keys.
b) The UI can detect if the key already exists on the server.
If this can be done without any cryptographic operations that would be
great, but I'm not sure about the details.

For (a) I was wondering if the UI can decode the base-64 encoded public
key entered by the user and display some user-friendly information 
about

the key itself, maybe just the hexdump of the first few bytes of the
key, or maybe the key type too if possible, or at least just the first
few chars of the undecoded key.

For (b) the UI would have to use some crypto functions to generate the
key fingerprints as generated by the server. But even if we do that, we
are not doing any data encryption or dealing with secret information.


I do not think you need crypto functions for (a) and it is unclear to me
what (b) is, if the key already exist on the IPA server you can find it
out with a simple memcmp, why should you need a fingeprint ?


in both (a) and (b) the key does not exist on the server - user just 
added the key(s) but he had not yet clicked on 'update' button to 
upload them.




Right, all the UI has is the base-64 encoded key entered by the user. So 
without crypto, all the UI can do is probably decoding the key and, if 
possible, parsing the key structure to obtain some useful information. 
Can we obtain the key type this way, or does it require crypto?


The existing keys on the other hand are displayed in the UI with their 
fingerprints and key type, for example:

  A8:50:ED:5A:8A:78:81:8D:B9:34:CC:99:D4:6A:E1:32 (ssh-rsa)
The UI does not show the base-64 encoded key unless the user clicks the 
Show/Set key button.


To check if a new key already exists on the server, the UI can compare 
either the base-64 encoded keys or the fingerprints (if available).


Will displaying of the fingerprints prior saving help(improve UX) the 
user? The issue is that user can't distinguish multiple added keys. 
The first few chars of the undecoded key variant of (a) will solve 
the issue in the most efficient way - user will see what he just 
added. No crypto needed.


Displaying the fingerprints of the new keys is not necessary to solve 
(a), but the display will not be seamless without it. For example, it 
could become like this:


  A8:50:ED:5A:8A:78:81:8D:B9:34:CC:99:D4:6A:E1:32 (ssh-rsa)
  B3NzaC1yc2EBIwAAAQEAueevoxw+... (new key)

On the other hand, since existing keys are displayed with their 
fingerprints, it's not possible to open a public key file and compare it 
to the list before adding it to the UI.


Are the fingerprints actually only used internally by the server? Do the 
users have any use of the fingerprints? If the fingerprints are internal 
only, the UI shouldn't need to display it. I'd say the UI should just 
display the key itself, not the whole key but just long enough to be 
distinguishable, either as a hexdump or base-64 encoded.





BTW, WebCrypto implementation has been making its way into Firefox:
https://docs.google.com/spreadsheet/ccc?key=0AiAcidBZRLxndE9LWEs2R1oxZ0xidUVoU3FQbFFobkEusp=sharing 



Yes and I find it a particularly bad/dangerous thing, I tend to agree
with this: http://tonyarcieri.com/whats-wrong-with-webcrypto see also
the links it points to which explains why crypto in javascript (ie in
the user-hostile sandbox running in the browser) is mostly a bad idea,
and can give a false sense of security and a slippery slope in actually
unprotecting data.

Simo.



If for some reason we decided to generate fingerprints in the UI, in my 
opinion using WebCrypto will not pose any security problem or mislead 
users into a false sense of security because the operation is purely 
informational, it does not do any encryption/decryption/validation. Any 
fingerprint generated on the UI side will be discarded anyway, the 
server will still generate its own fingerprints.


This belongs to a separate discussion, but the article

Re: [Freeipa-devel] [PATCH] 733-735 webui: Better description for User authentication types

2014-08-13 Thread Endi Sukma Dewata

On 8/5/2014 6:38 AM, Petr Vobornik wrote:

[PATCH] 733 webui: rename tooltip to title

- use title for input's elements 'title' attribute
- tooltip for Bootstrap's tooltip component

https://fedorahosted.org/freeipa/ticket/4471


ACK.


[PATCH] 734 webui: tooltip support

Allow to set 'tooltip' attribute in spec. It displays info icon
with Bootstrap's tooltip near field's label.

https://fedorahosted.org/freeipa/ticket/4471


ACK.


[PATCH] 735 webui: better authentication types description

Tooltips were added to User authentication types and Default user
objectclasses to describe their relationship and a meaning of
not-setting a value.

https://fedorahosted.org/freeipa/ticket/4471


Just one thing, in the patch comment you probably meant Default user 
authentication types. ACK.


--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 736-740 webui: various minor fixes

2014-08-13 Thread Endi Sukma Dewata

On 8/5/2014 6:43 AM, Petr Vobornik wrote:

[PATCH] 736 webui: convert widget.less indentation to spaces


ACK.


[PATCH] 737 webui: improve rule table css

- category radio line has line-height large enough to contain
  undo button - content doesn't move several pixels on change
- remove vertical padding from btns in table headers to maintain
  about the same height
- remove invisible border from link buttons to have the same height
  for disabled and enabled button


ACK.


[PATCH] 738 webui: sshkey widget - usability fixes

- save one click by opening edit dialog right after adding new row
- add margin between fingerprint and show/edit button
- fix honoring of writable/read-only flags upon row creation


ACK. Possible improvements:

1. How about removing the row if the user cancels the addition or enters 
blank value? That way the rows will always have values, so we don't need 
the New: key set/not set labels anymore.


2. Can the UI parse the new key and display it the same way as other 
keys that are already saved? That will make it more seamless.


3. If we do #2, the Show/Set key button probably can be changed to 
Edit or Modify.



[PATCH] 739 webui: disable batch action buttons by default

action buttons associated with batch actions were enabled by default, 
but they were disabled right after facet creation and a load of data. 
It caused a visual flicker.


UX is enhanced by making them disabled by default.


ACK.


[PATCH] 740 webui: fix group type padding


ACK.

--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 741 webui: add link to OTP token app

2014-08-13 Thread Endi Sukma Dewata


On 8/5/2014 10:11 AM, Petr Vobornik wrote:

- display info message which points user to FreeOTP project page
- the link or the text can be easily changed by a plugin if needed

https://fedorahosted.org/freeipa/ticket/4469

Notes:
- the design can be a subject of discussion.
- the FreeOTP project page [1] is not very end-user friendly but 
serves the purpose
- it's not fully configurable, as decided at today's meeting, but a 
very simple plugin can hide or modify the message


[1] https://fedorahosted.org/freeotp/


ACK.

--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


Re: [Freeipa-devel] [PATCH] 715 webui: add bounce url to reset_password.html

2014-08-13 Thread Endi Sukma Dewata

On 7/29/2014 5:53 AM, Petr Vobornik wrote:

Just one thing, there is no pause between clicking the Reset button

and the redirection, so the Password reset was successful.
confirmation message might only appear very briefly. A possible
alternative is to show a confirmation page/message, but the user will
have to click to continue to the next page.


I don't believe there is a universal solution. I would say that it
depends on personal preferences and a use case. I.e., if it's part 
of  a
login procedure I would prefer immediate redirection back to login 
page.

If it's invoked from a user action - just to change the password, some
delay might be good.

We might add a URL param(s) to configure the delay/link.


How about 2 URL params?
1. the link to the next page
2. an option whether to
a) redirect to the link immediately
b) show a confirmation page with the link

Just my preference, but I don't really like a 'delay' on a web page.
It's either too short or too long, and we can't put important info
during the delay because there's no guarantee people will see it.



Yes, how about this:

1. redirection=url_to_the_page
2. in=x, where x is number of seconds or a string

- redirect immediately if only `redirection` is supplied or `in=0`
- show Continue to anext page/a link if `in` is present
- show count-down timer if `in` is  1 with You will be redirected in 
`x`s text.
- don't redirect if `in` is negative or NaN (your scenario), e.g.: 
`in=no` up to user

- ignore `in` if `redirection` is not present

Then we will support all described scenarios and the decision will be 
on web-site admin. Feel free to propose better name for `in`.


How about 'url/link' or 'next_url' for the first parameter and 'delay' 
for the second one? I think in this case 'delay' would describe its 
function better than 'in'. If delay=0 that means we're redirecting 
immediately. If delay parameter is not specified that means we're 
waiting for the user to click the link.


--
Endi S. Dewata

___
Freeipa-devel mailing list
Freeipa-devel@redhat.com
https://www.redhat.com/mailman/listinfo/freeipa-devel


  1   2   3   4   5   6   7   8   9   10   >