Re: [Freeipa-devel] OOO 2015-03-31-2015-04-01

2015-03-31 Thread Tomas Babej

Sorry about the noise.

On 03/31/2015 07:23 AM, Tomas Babej wrote:

Hours already accumulated this month.

Tomas



--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] Use sessions for mod_auth_gssapi ?

2015-03-31 Thread Martin Kosek
On 03/31/2015 08:04 AM, Jan Cholasta wrote:
 Dne 30.3.2015 v 22:09 Adam Young napsal(a):
 On 03/30/2015 11:52 AM, Simo Sorce wrote:
 Since we now merged in a change from mod_auth_kerb to mod_auth_gssapi I
 was wondering if we want to press further and emable by default the use
 of native mod_auth_gssapi sessions ?

 The old mod_auth_kerb didn't have this feature so, in order to have
 decent performace we introduced split paths where some are always
 incurring the full negotiate penalty and other are and instead rely on a
 session cookie.

 mod_auth_gssapi can be configured to use a session cookie directly which
 avoids the negotiate auth performance hit. Integration would require
 that the FreeIPA code learns how to delete the cookie when someone hits
 a logout button, but it would be otherwise transparent.

 It would be especially useful for 3rd party clients that want to use the
 json/xmlrpc enpoints, as all they have to do is just support sending
 back cookies and they do not have to learn how to contact multiple
 endopints to get credentials and then switch to the session only based
 ones.

 Thoughts ?

 Simo.

 I always wanted this.  It would be awesome, very valuable.
 
 Yes please.

We should have a ticket with all the details then...

 

 REcall that when we looked into it we were on Apache 1.3, and seesion
 support, mod_seesion, was not avaialble.  Fairly certain the landscape
 has changed since then.

 

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] Use sessions for mod_auth_gssapi ?

2015-03-31 Thread Jan Cholasta

Dne 30.3.2015 v 22:09 Adam Young napsal(a):

On 03/30/2015 11:52 AM, Simo Sorce wrote:

Since we now merged in a change from mod_auth_kerb to mod_auth_gssapi I
was wondering if we want to press further and emable by default the use
of native mod_auth_gssapi sessions ?

The old mod_auth_kerb didn't have this feature so, in order to have
decent performace we introduced split paths where some are always
incurring the full negotiate penalty and other are and instead rely on a
session cookie.

mod_auth_gssapi can be configured to use a session cookie directly which
avoids the negotiate auth performance hit. Integration would require
that the FreeIPA code learns how to delete the cookie when someone hits
a logout button, but it would be otherwise transparent.

It would be especially useful for 3rd party clients that want to use the
json/xmlrpc enpoints, as all they have to do is just support sending
back cookies and they do not have to learn how to contact multiple
endopints to get credentials and then switch to the session only based
ones.

Thoughts ?

Simo.


I always wanted this.  It would be awesome, very valuable.


Yes please.



REcall that when we looked into it we were on Apache 1.3, and seesion
support, mod_seesion, was not avaialble.  Fairly certain the landscape
has changed since then.



--
Jan Cholasta

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


[Freeipa-devel] [PATCH 0025] proper client host setup/teardown in forced client reenrollment integration test suite

2015-03-31 Thread Martin Babinsky
During the investigation of https://fedorahosted.org/freeipa/ticket/4614 
I discovered a bug (?) in forced client reenrollment integration test.


During test scenario, master and replica are setup correctly at the 
beginning of the test, but the client is never setup resulting in a 
couple of tracebacks.


After some investigation I realized that the setUp/tearDown methods are 
actually never called because they are supposed to be inherited from 
unittest.TestCase. However, IntegrationTest no longer inherits from this 
class, hence the bug.


I have tried to fix this by adding a fixture which runs client 
fixup/teardown and doing some other small modifications. Tests now work 
as expected, but I need a review from QE guys or someone well-versed in 
pytest framework.


TL;DR: I think I have fixed a bug in integration test but I need someone 
to review the fix because I may not know what I'm doing.


--
Martin^3 Babinsky
From 9cb41407f2db6a353969977016c326fe76169bc9 Mon Sep 17 00:00:00 2001
From: Martin Babinsky mbabi...@redhat.com
Date: Tue, 31 Mar 2015 09:33:53 +0200
Subject: [PATCH] proper client host setup/teardown in forced client
 reenrollment integration test suite

Replace setUp()/tearDown() methods with a pytest.fixture for proper client
setup/teardown during test_forced_client_reenrollment

---
 .../test_forced_client_reenrollment.py | 43 --
 1 file changed, 24 insertions(+), 19 deletions(-)

diff --git a/ipatests/test_integration/test_forced_client_reenrollment.py b/ipatests/test_integration/test_forced_client_reenrollment.py
index ed9800d49f77f6f292f59c22509a9f568041200b..e1edff9b7f2d535a341d1e8ca55917012943818e 100644
--- a/ipatests/test_integration/test_forced_client_reenrollment.py
+++ b/ipatests/test_integration/test_forced_client_reenrollment.py
@@ -19,6 +19,7 @@
 import os
 import subprocess
 from ipaplatform.paths import paths
+import pytest
 
 from ipatests.test_integration.base import IntegrationTest
 from ipatests.test_integration import tasks
@@ -44,15 +45,7 @@ class TestForcedClientReenrollment(IntegrationTest):
 'krb5.keytab'
 )
 
-def setUp(self):
-tasks.prepare_host(self.clients[0])
-tasks.install_client(self.master, self.clients[0])
-
-def tearDown(self):
-tasks.uninstall_client(self.clients[0])
-self.delete_client_host_entry()
-
-def test_reenroll_with_force_join(self):
+def test_reenroll_with_force_join(self, client):
 
 Client re-enrollment using admin credentials (--force-join)
 
@@ -63,7 +56,7 @@ class TestForcedClientReenrollment(IntegrationTest):
 sshfp_record_post = self.get_sshfp_record()
 assert sshfp_record_pre == sshfp_record_post
 
-def test_reenroll_with_keytab(self):
+def test_reenroll_with_keytab(self, client):
 
 Client re-enrollment using keytab
 
@@ -76,7 +69,7 @@ class TestForcedClientReenrollment(IntegrationTest):
 sshfp_record_post = self.get_sshfp_record()
 assert sshfp_record_pre == sshfp_record_post
 
-def test_reenroll_with_both_force_join_and_keytab(self):
+def test_reenroll_with_both_force_join_and_keytab(self, client):
 
 Client re-enrollment using both --force-join and --keytab options
 
@@ -89,7 +82,7 @@ class TestForcedClientReenrollment(IntegrationTest):
 sshfp_record_post = self.get_sshfp_record()
 assert sshfp_record_pre == sshfp_record_post
 
-def test_reenroll_to_replica(self):
+def test_reenroll_to_replica(self, client):
 
 Client re-enrollment using keytab, to a replica
 
@@ -102,7 +95,7 @@ class TestForcedClientReenrollment(IntegrationTest):
 sshfp_record_post = self.get_sshfp_record()
 assert sshfp_record_pre == sshfp_record_post
 
-def test_try_to_reenroll_with_disabled_host(self):
+def test_try_to_reenroll_with_disabled_host(self, client):
 
 Client re-enrollment using keytab, with disabled host
 
@@ -113,7 +106,7 @@ class TestForcedClientReenrollment(IntegrationTest):
 self.restore_keytab()
 self.reenroll_client(keytab=self.BACKUP_KEYTAB, expect_fail=True)
 
-def test_try_to_reenroll_with_uninstalled_host(self):
+def test_try_to_reenroll_with_uninstalled_host(self, client):
 
 Client re-enrollment using keytab, with uninstalled host
 
@@ -124,7 +117,7 @@ class TestForcedClientReenrollment(IntegrationTest):
 self.restore_keytab()
 self.reenroll_client(keytab=self.BACKUP_KEYTAB, expect_fail=True)
 
-def test_try_to_reenroll_with_deleted_host(self):
+def test_try_to_reenroll_with_deleted_host(self, client):
 
 Client re-enrollment using keytab, with deleted host
 
@@ -135,7 +128,7 @@ class TestForcedClientReenrollment(IntegrationTest):
 self.restore_keytab()
 self.reenroll_client(keytab=self.BACKUP_KEYTAB, 

[Freeipa-devel] [PATCH] 809 speed up convert_attribute_members

2015-03-31 Thread Petr Vobornik

A workaround to avoid usage of slow LDAPEntry._sync_attr #4946.

I originally wanted to avoid DN processing as well but we can't do that 
because of DNs which are encoded - e.g. contains '+' or ','. Therefore 
patch 811 - faster DN implementation is very useful. Also patch 809 is 
useful to avoid high load of 389.


https://fedorahosted.org/freeipa/ticket/4965
--
Petr Vobornik
From 7d55a7e2bd88cb9c754cb8cc8a05fd947fd5438a Mon Sep 17 00:00:00 2001
From: Petr Vobornik pvobo...@redhat.com
Date: Wed, 18 Mar 2015 18:48:54 +0100
Subject: [PATCH] speed up convert_attribute_members

A workaround to avoid usage of slow LDAPEntry._sync_attr #4946

https://fedorahosted.org/freeipa/ticket/4965
---
 ipalib/plugins/baseldap.py | 36 
 1 file changed, 28 insertions(+), 8 deletions(-)

diff --git a/ipalib/plugins/baseldap.py b/ipalib/plugins/baseldap.py
index 4b1c701924d57919538e0c428ea181c2e898505e..42509d63de6078b01cc794a005a528d27a168f26 100644
--- a/ipalib/plugins/baseldap.py
+++ b/ipalib/plugins/baseldap.py
@@ -631,16 +631,36 @@ class LDAPObject(Object):
 def convert_attribute_members(self, entry_attrs, *keys, **options):
 if options.get('raw', False):
 return
+
+container_dns = {}
+new_attrs = {}
+
+def get_container_dn(ldap_obj_name):
+dn = container_dns.get(ldap_obj_name, None)
+if not dn:
+ldap_obj = self.api.Object[ldap_obj_name]
+dn = DN(ldap_obj.container_dn, api.env.basedn)
+container_dns[ldap_obj_name] = dn
+return dn
+
+def get_new_attr(attr, ldap_obj_name):
+name = u'%s_%s' % (attr, ldap_obj_name)
+new_attr = new_attrs.get(name, None)
+if not new_attr:
+new_attr = entry_attrs.setdefault(name, [])
+new_attrs[name] = new_attr
+return new_attr
+
 for attr in self.attribute_members:
-for member in entry_attrs.setdefault(attr, []):
+for member in entry_attrs.raw.setdefault(attr, []):
 for ldap_obj_name in self.attribute_members[attr]:
-ldap_obj = self.api.Object[ldap_obj_name]
-container_dn = DN(ldap_obj.container_dn, api.env.basedn)
-if member.endswith(container_dn):
-new_attr = '%s_%s' % (attr, ldap_obj.name)
-entry_attrs.setdefault(new_attr, []).append(
-ldap_obj.get_primary_key_from_dn(member)
-)
+container_dn = get_container_dn(ldap_obj_name)
+memberdn = DN(member)
+if memberdn.endswith(container_dn):
+ldap_obj = self.api.Object[ldap_obj_name]
+pkey = ldap_obj.get_primary_key_from_dn(memberdn)
+get_new_attr(attr, ldap_obj_name).append(pkey)
+break
 del entry_attrs[attr]
 
 def get_password_attributes(self, ldap, dn, entry_attrs):
-- 
2.1.0

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

[Freeipa-devel] [PATCH] 810 speed up indirect member processing

2015-03-31 Thread Petr Vobornik
the old implementation tried to get all entries which are member of 
group. That means also user. User can't have any members therefore this 
costly processing was unnecessary.


New implementation reduces the search only to entries which can have 
entries.


Also page size was removed to avoid paging by small pages(default size: 
100) which is very slow for many members.


https://fedorahosted.org/freeipa/ticket/4947

Useful to test with #809
--
Petr Vobornik
From 2d07532771a4a542cb220f87cdc7c1ce115014f6 Mon Sep 17 00:00:00 2001
From: Petr Vobornik pvobo...@redhat.com
Date: Tue, 31 Mar 2015 10:59:37 +0200
Subject: [PATCH] speed up indirect member processing

the old implementation tried to get all entries which are member of group.
That means also user. User can't have any members therefore this costly
processing was unnecessary.

New implementation reduces the search only to entries which can have entries.

Also page size was removed to avoid paging by small pages(default size: 100)
which is very slow for many members.

https://fedorahosted.org/freeipa/ticket/4947
---
 ipalib/plugins/baseldap.py   | 91 
 ipalib/plugins/group.py  |  1 +
 ipalib/plugins/host.py   |  2 +-
 ipalib/plugins/hostgroup.py  |  1 +
 ipalib/plugins/netgroup.py   |  1 +
 ipalib/plugins/permission.py |  1 +
 ipalib/plugins/role.py   |  8 ++--
 ipaserver/plugins/ldap2.py   | 90 ---
 8 files changed, 101 insertions(+), 94 deletions(-)

diff --git a/ipalib/plugins/baseldap.py b/ipalib/plugins/baseldap.py
index 42509d63de6078b01cc794a005a528d27a168f26..56f871ac309b562130f52abce11816d811f9c89c 100644
--- a/ipalib/plugins/baseldap.py
+++ b/ipalib/plugins/baseldap.py
@@ -538,6 +538,7 @@ class LDAPObject(Object):
 rdn_attribute = ''
 uuid_attribute = ''
 attribute_members = {}
+memberindirect_config = ('*', None)
 rdn_is_primary_key = False # Do we need RDN change to do a rename?
 password_attributes = []
 # Can bind as this entry (has userPassword or krbPrincipalKey)
@@ -663,6 +664,79 @@ class LDAPObject(Object):
 break
 del entry_attrs[attr]
 
+def get_memberindirect(self, group_entry):
+
+Get indirect members
+
+Other objects with memberindirect attribute should define
+`memberindirect_config`. This object class filter is used to limit the
+number of members returned to only the members which can contain other
+members.
+
+
+(member_oc, basedn) = self.memberindirect_config
+
+if not member_oc:
+self.log.error(
+('Missing `memberindirect_object_class` IPA object property.\n'
+ 'Skipping indirect member processing.'))
+return
+
+if not basedn:
+basedn = self.api.env.basedn
+
+filter = self.backend.make_filter(
+{'memberof': group_entry.dn, 'objectclass': member_oc},
+None, self.backend.MATCH_ALL
+)
+
+try:
+result, truncated = self.backend.find_entries(
+base_dn=basedn,
+filter=filter,
+attrs_list=['member'],
+size_limit=-1, # paged search will get everything anyway
+paged_search=True)
+if truncated:
+raise errors.LimitsExceeded()
+except errors.NotFound:
+result = []
+
+indirect = set()
+for entry in result:
+indirect.update(entry.raw.get('member', []))
+indirect.difference_update(group_entry.raw.get('member', []))
+
+if indirect:
+group_entry.raw['memberindirect'] = list(indirect)
+
+def get_memberofindirect(self, entry):
+
+dn = entry.dn
+filter = self.backend.make_filter(
+{'member': dn, 'memberuser': dn, 'memberhost': dn})
+try:
+result, truncated = self.backend.find_entries(
+base_dn=self.api.env.basedn,
+filter=filter,
+attrs_list=[''])
+if truncated:
+raise errors.LimitsExceeded()
+except errors.NotFound:
+result = []
+
+direct = set()
+indirect = set(entry.raw.get('memberof', []))
+for group_entry in result:
+dn = str(group_entry.dn)
+if dn in indirect:
+indirect.remove(dn)
+direct.add(dn)
+
+entry.raw['memberof'] = list(direct)
+if indirect:
+entry.raw['memberofindirect'] = list(indirect)
+
 def get_password_attributes(self, ldap, dn, entry_attrs):
 
 Search on the entry to determine if it has a password or
@@ -1106,6 +1180,12 @@ last, after all sets and adds.),
 )
 break
 
+def get_indirect_members(self, entry_attrs, attrs_list):
+if 'memberindirect' in 

Re: [Freeipa-devel] [PATCH 0025] proper client host setup/teardown in forced client reenrollment integration test suite

2015-03-31 Thread Petr Viktorin

On 03/31/2015 10:42 AM, Martin Babinsky wrote:

During the investigation of https://fedorahosted.org/freeipa/ticket/4614
I discovered a bug (?) in forced client reenrollment integration test.

During test scenario, master and replica are setup correctly at the
beginning of the test, but the client is never setup resulting in a
couple of tracebacks.

After some investigation I realized that the setUp/tearDown methods are
actually never called because they are supposed to be inherited from
unittest.TestCase. However, IntegrationTest no longer inherits from this
class, hence the bug.

I have tried to fix this by adding a fixture which runs client
fixup/teardown and doing some other small modifications. Tests now work
as expected, but I need a review from QE guys or someone well-versed in
pytest framework.


LGTM, from a quick glance.


--
Petr Viktorin

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


[Freeipa-devel] [PATCH] 811 performance: faster DN implementation

2015-03-31 Thread Petr Vobornik
The only different thing is a lack of utf-8 encoded str support(as 
input). I don't know how much important the support is.


maybe it could be attached to ticket 
https://fedorahosted.org/freeipa/ticket/4947

-
DN code was optimized to be faster if DNs are created from string. This 
is the major use case, since most DNs come from LDAP.


With this patch, DN creation is almost 8-10x faster (with 30K-100K DNs).

Second mojor use case - deepcopy in LDAPEntry is about 20x faster - done 
by custom __deepcopy__ function.


The major change is that DN is no longer internally composed  of RDNs 
and AVAs but it rather keeps the data in open ldap format - the same as 
output of str2dn function. Therefore, for immutable DNs, no other 
transformations are required on instantiation.


The format is:

DN: [RDN, RDN,...]
RDN: [AVA, AVA,...]
AVA: ['utf-8 encoded str - attr', 'utf-8 encode str -value', FLAG]
FLAG: int

Further indexing of DN object constructs an RDN which is just an 
encapsulation of the RDN part of open ldap representation. Indexing of 
RDN constructs AVA in the same fashion.


Obtained EditableAVA, EditableRDN from EditableDN shares the respected 
lists of the open ldap repr. so that the change of value or attr is 
reflected in parent object.

--
Petr Vobornik
From 1de87815183b9e9ddef5710eab1e83ba36127986 Mon Sep 17 00:00:00 2001
From: Petr Vobornik pvobo...@redhat.com
Date: Wed, 25 Mar 2015 13:39:43 +0100
Subject: [PATCH] performance: faster DN implementation

DN code was optimized to be faster if DNs are created from string. This is
the major use case, since most DNs come from LDAP.

With this patch, DN creation is almost 8-10x faster (with 30K-100K DNs).

Second mojor use case - deepcopy in LDAPEntry is about 20x faster - done by
custom __deepcopy__ function.

The major change is that DN is no longer internally composed  of RDNs and
AVAs but it rather keeps the data in open ldap format - the same as output
of str2dn function. Therefore, for immutable DNs, no other transformations
are required on instantiation.

The format is:

DN: [RDN, RDN,...]
RDN: [AVA, AVA,...]
AVA: ['utf-8 encoded str - attr', 'utf-8 encode str -value', FLAG]
FLAG: int

Further indexing of DN object constructs an RDN which is just an encapsulation
of the RDN part of open ldap representation. Indexing of RDN constructs AVA in
the same fashion.

Obtained EditableAVA, EditableRDN from EditableDN shares the respected lists
of the open ldap repr. so that the change of value or attr is reflected in
parent object.
---
 ipapython/dn.py| 558 ++---
 ipatests/test_ipapython/test_dn.py |  74 ++---
 2 files changed, 306 insertions(+), 326 deletions(-)

diff --git a/ipapython/dn.py b/ipapython/dn.py
index 834291fbe8696622162efa5193622d74f11f25ca..5d1f15c27abcea11bc3acafa5e4297692b236e98 100644
--- a/ipapython/dn.py
+++ b/ipapython/dn.py
@@ -497,6 +497,89 @@ def _adjust_indices(start, end, length):
 
 return start, end
 
+
+def str2rdn(value):
+try:
+rdns = str2dn(value.encode('utf-8'))
+except DECODING_ERROR:
+raise ValueError(malformed AVA string = \%s\ % value)
+if len(rdns) != 1:
+raise ValueError(multiple RDN's specified by \%s\ % (value))
+return rdns[0]
+
+
+def get_ava(*args, **kwds):
+
+Get AVA from args in open ldap format(raw). Optimized for construction
+from openldap format.
+
+Allowed formats of argument list:
+1) three args - open ldap format (attr and value have to be utf-8 encoded):
+a) ['attr', 'value', 0]
+2) two args:
+a) ['attr', 'value']
+3) one arg:
+a) [('attr', 'value')]
+b) [['attr', 'value']]
+c) [AVA(..)]
+d) ['attr=value']
+
+ava = None
+l = len(args)
+if l == 3:  # raw values - constructed FROM RDN
+if kwds.get('mutable', False):
+ava = args
+else:
+ava = (args[0], args[1], args[2])
+elif l == 2:  # user defined values
+ava = [unicode(args[0]).encode('utf-8'), unicode(args[1]).encode('utf-8'), 0]
+elif l == 1:  # slow mode, tuple, string,
+arg = args[0]
+if isinstance(arg, AVA):
+ava = arg.to_openldap()
+elif isinstance(arg, (tuple, list)):
+if len(arg) != 2:
+raise ValueError(tuple or list must be 2-valued, not \%s\ % (arg))
+ava = [unicode(arg[0]).encode('utf-8'), unicode(arg[1]).encode('utf-8'), 0]
+elif isinstance(arg, basestring):
+rdn = str2rdn(arg)
+if len(rdn)  1:
+raise TypeError(multiple AVA's specified by \%s\ % (arg))
+ava = list(rdn[0])
+else:
+raise TypeError(with 1 argument, argument must be str, unicode, tuple or list, got %s instead %
+arg.__class__.__name__)
+else:
+raise TypeError(invalid number of arguments. 1-3 allowed)
+return ava
+
+
+def 

[Freeipa-devel] [QE] Test categorization into tiers and acceptance testing - tagging proposals

2015-03-31 Thread Martin Koci
Hi all,
I'd like to open discussion on test categorization into tiers and
acceptance testing, respectively test tagging which should help us to
accomplish following goals:

1) Acceptance test - other FreeIPA partner projects (389/DS/PKI) should
be able to have an Acceptance test that would run basic *stable* test
suite that would check if anything significant broke. It should be fast
enough so that the projects can run it in a Jenkins CI after commits.

If we also have tags @dogtag or @sssd, the projects could simply run
just the tests affecting the projects - faster execution.

2) FreeIPA test run optimization. Currently, all FreeIPA tests are
running when new commit is pushed. This takes lot of resources. It would
be nice to at least be able to NOT run Tier 2 tests if Tier1 tests are
failing. Or it would be nice to not run some very expensive tests after
each commit, but maybe once per day/week.

*TIERS*
So after discussions with couple of developers and QE's we have created
and summarized following proposal for sorting current IPA tests into
tiers. 

Currently used tests reside in freeipa/ipatests. From these the only
unit tests (tier 0 candidate) are test_{ipalib,ipapython} with the
exception of test_ipalib/test_rpc.py which requires kerberos.

The rest of the tests either require ipa/lite-server or are an
integration test. The rest of the tests (majority XML RPC, UI
tests, ...) then fall under the definition of Tier 1 test, as they
require at least running IPA instance and admin TGT.

As for the tagging of the test cases, pytest's capabilities can be used
[2]. Though pytest.mark currently does not work with declarative tests
(it marks all of them), when the test is an ordinary function/method the
marking works as expected. The declarative tests could be rewritten in
the future to more pytest specific form, e.g.
test_xmlrpc/test_host_plugin.py

Official guideline for this categorization will be created on the
upstream wiki once we agree on that. 


*ACCEPTANCE TESTING*
As for the acceptance testing Similar to `Test categorization into
tiers` (1) proposal, there is a need to define a subset of freeipa tests
that could be run by other projects or users to find out whether or not
their changes (e.g. new build, feature) works with IPA.

This run could be composed from tier {0,1} execution followed by a
subset of integration tests test cases. The proposed mechanism for this
is the same as in [4], using pytest.mark to select the classes/tests to
run in this context.

What I'd like to ask you here is to share any ideas on the form of the
acceptance run as well as to help me identify the areas (and tests) that
are considered important and should be a part of this test set.

*TAGGING* 
Tagging the actual tests classes with pytest decorator
(http://pytest.org/latest/mark.html). would be better than let
developers manually maintain lists of tests for different projects. The
benefit for pytest mark kept in the code is that whatever we do with the
test class (rename, move, merge), the tag goes with it, not extra list
needs to be maintained.

As for tagging itself, the original idea which Martin Kosek was
proposing was to use just the acceptance tag for marking the base T2
tests that would be part of FreeIPA acceptance tests.

However, it seems there is a value in tagging the tests that exercise
also certain sub-component of FreeIPA - SSSD, Dogtag. As long as we do
not get too wild with the tags, it should be OK. 

So we could agreed on followings tags:
- tier0, tier1, tier2
- acceptance
- sssd
- dogtag

This would lead to e.g.

@pytest.mark.dogtag
@pytest.mark.acceptance
@pytest.mark.tier2
class TestExternalCA(IntegrationTest):
...

or simpler

@dogtag
@acceptance
@tier2
class TestExternalCA(IntegrationTest):

Hope it's not too long and that it makes sense. 
Can I get your thoughts on this, please?
Thank you.

Regards,
/koca

*[1] - https://fedorahosted.org/freeipa/ticket/4922
*[2] - http://pytest.org/latest/mark.html


-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


[Freeipa-devel] [PATCH 0082] Update python-yubico dependency version

2015-03-31 Thread Nathaniel McCallum
This change enables support for all current YubiKey hardware.From 54f74bebe5149d3be4e2772cb0199dda30fb0088 Mon Sep 17 00:00:00 2001
From: Nathaniel McCallum npmccal...@redhat.com
Date: Tue, 31 Mar 2015 10:17:18 -0400
Subject: [PATCH] Update python-yubico dependency version

This change enables support for all current YubiKey hardware.
---
 freeipa.spec.in | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/freeipa.spec.in b/freeipa.spec.in
index 8d58f2568e1de418c25cb1bd34fc7d4736a15e54..608242b5adbc43efbbf0ae30a6d7a933bebc1084 100644
--- a/freeipa.spec.in
+++ b/freeipa.spec.in
@@ -86,7 +86,7 @@ BuildRequires:  libverto-devel
 BuildRequires:  systemd
 BuildRequires:  libunistring-devel
 BuildRequires:  python-lesscpy
-BuildRequires:  python-yubico
+BuildRequires:  python-yubico = 1.2.3
 BuildRequires:  python-backports-ssl_match_hostname
 BuildRequires:  softhsm-devel = 2.0.0b1-3
 BuildRequires:  openssl-devel
@@ -290,7 +290,7 @@ Requires: libipa_hbac-python
 Requires: python-qrcode-core = 5.0.0
 Requires: python-pyasn1
 Requires: python-dateutil
-Requires: python-yubico
+Requires: python-yubico = 1.2.3
 %if (0%{?fedora} = 22)
 Requires: python-sss-murmur
 %endif
-- 
2.3.4

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

[Freeipa-devel] [PATCH] webui: use no_members option in entity select search

2015-03-31 Thread Petr Vobornik

Obtaining member information for entity selects is not needed and it
causes unwanted performance hit, especially with larger groups.

This patch removes it.

https://fedorahosted.org/freeipa/ticket/4948
--
Petr Vobornik
From f4cc39c016749fd73f0cc8bc67aa170ee248507a Mon Sep 17 00:00:00 2001
From: Petr Vobornik pvobo...@redhat.com
Date: Tue, 31 Mar 2015 15:54:37 +0200
Subject: [PATCH] webui: use no_members option in entity select search

Obtaining member information for entity selects is not needed and it
causes unwanted performance hit, especially with larger groups.

This patch removes it.

https://fedorahosted.org/freeipa/ticket/4948
---
 install/ui/src/freeipa/widget.js | 13 ++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/install/ui/src/freeipa/widget.js b/install/ui/src/freeipa/widget.js
index 6767b520ee3d388a36de743122774a462d4a4be5..29d320c07fe156a807ad2bf4415b3058a8a71e98 100644
--- a/install/ui/src/freeipa/widget.js
+++ b/install/ui/src/freeipa/widget.js
@@ -37,6 +37,7 @@ define(['dojo/_base/array',
'./entity',
'./ipa',
'./jquery',
+   './metadata',
'./navigation',
'./phases',
'./reg',
@@ -45,8 +46,9 @@ define(['dojo/_base/array',
'./util',
'exports'
],
-   function(array, lang, construct, Evented, has, keys, on, string, topic, builder, config,
-datetime, entity_mod, IPA, $, navigation, phases, reg, rpc, text, util, exp) {
+   function(array, lang, construct, Evented, has, keys, on, string,
+topic, builder, config, datetime, entity_mod, IPA, $,
+metadata, navigation, phases, reg, rpc, text, util, exp) {
 
 /**
  * Widget module
@@ -4051,12 +4053,17 @@ IPA.entity_select_widget = function(spec) {
 that.filter_options = spec.filter_options || {};
 
 that.create_search_command = function(filter) {
-return rpc.command({
+var cmd  = rpc.command({
 entity: that.other_entity.name,
 method: 'find',
 args: [filter],
 options: that.filter_options
 });
+var no_members = metadata.get('@mc-opt:' + cmd.get_command() + ':no_members');
+if (no_members) {
+cmd.set_option('no_members', true);
+}
+return cmd;
 };
 
 that.search = function(filter, on_success, on_error) {
-- 
2.1.0

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] Use sessions for mod_auth_gssapi ?

2015-03-31 Thread Nathaniel McCallum
On Mon, 2015-03-30 at 11:52 -0400, Simo Sorce wrote:
 Since we now merged in a change from mod_auth_kerb to 
 mod_auth_gssapi I
 was wondering if we want to press further and emable by default the 
 use
 of native mod_auth_gssapi sessions ?
 
 The old mod_auth_kerb didn't have this feature so, in order to have
 decent performace we introduced split paths where some are always
 incurring the full negotiate penalty and other are and instead rely 
 on a
 session cookie.
 
 mod_auth_gssapi can be configured to use a session cookie directly 
 which
 avoids the negotiate auth performance hit. Integration would require
 that the FreeIPA code learns how to delete the cookie when someone 
 hits
 a logout button, but it would be otherwise transparent.
 
 It would be especially useful for 3rd party clients that want to use 
 the
 json/xmlrpc enpoints, as all they have to do is just support sending
 back cookies and they do not have to learn how to contact multiple
 endopints to get credentials and then switch to the session only 
 based
 ones.
 
 Thoughts ?

+1. It is about time. :)

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] [PATCH 0025] proper client host setup/teardown in forced client reenrollment integration test suite

2015-03-31 Thread Martin Babinsky

On 03/31/2015 12:06 PM, Petr Viktorin wrote:

On 03/31/2015 10:42 AM, Martin Babinsky wrote:

During the investigation of https://fedorahosted.org/freeipa/ticket/4614
I discovered a bug (?) in forced client reenrollment integration test.

During test scenario, master and replica are setup correctly at the
beginning of the test, but the client is never setup resulting in a
couple of tracebacks.

After some investigation I realized that the setUp/tearDown methods are
actually never called because they are supposed to be inherited from
unittest.TestCase. However, IntegrationTest no longer inherits from this
class, hence the bug.

I have tried to fix this by adding a fixture which runs client
fixup/teardown and doing some other small modifications. Tests now work
as expected, but I need a review from QE guys or someone well-versed in
pytest framework.


LGTM, from a quick glance.



Thank Petr, anyone else has some opinion on this?

--
Martin^3 Babinsky

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


[Freeipa-devel] [PATCH] 786 webui: unable to select single value in CB by enter key

2015-03-31 Thread Petr Vobornik
This little fellow was hiding in a cupboard (patchset 784-786 was 
abandoned).


Fix: If editable combobox has one value, the value is selected and 
changed by hand, it can't be re-selected by enter key.

--
Petr Vobornik
From 210ee41d957c9efb818b721ef40ffc4d469cf15a Mon Sep 17 00:00:00 2001
From: Petr Vobornik pvobo...@redhat.com
Date: Wed, 12 Nov 2014 15:41:44 +0100
Subject: [PATCH] webui: unable to select single value in CB by enter key

Fix: If editable combobox has one value, the value is selected and changed by hand, it can't be re-selected by enter key.
---
 install/ui/src/freeipa/widget.js | 1 +
 1 file changed, 1 insertion(+)

diff --git a/install/ui/src/freeipa/widget.js b/install/ui/src/freeipa/widget.js
index 1ef1a2bf22b735edcfcca44cfc1e69bc8d36a740..6767b520ee3d388a36de743122774a462d4a4be5 100644
--- a/install/ui/src/freeipa/widget.js
+++ b/install/ui/src/freeipa/widget.js
@@ -3818,6 +3818,7 @@ IPA.combobox_widget = function(spec) {
 that.list_on_keyup = function(e) {
 if (e.which === keys.ENTER || e.which === keys.SPACE) {
 e.stopPropagation();
+that.list_on_change();
 that.close();
 IPA.select_range(that.input, 0, 0);
 return false;
-- 
2.1.0

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Re: [Freeipa-devel] [PATCH] 786 webui: unable to select single value in CB by enter key

2015-03-31 Thread Martin Babinsky

On 03/31/2015 04:19 PM, Petr Vobornik wrote:

This little fellow was hiding in a cupboard (patchset 784-786 was
abandoned).

Fix: If editable combobox has one value, the value is selected and
changed by hand, it can't be re-selected by enter key.



Works as expected, ACK.

--
Martin^3 Babinsky

--
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code


Re: [Freeipa-devel] Use sessions for mod_auth_gssapi ?

2015-03-31 Thread Anthony Messina
On Monday, March 30, 2015 11:52:07 AM Simo Sorce wrote:
 Since we now merged in a change from mod_auth_kerb to mod_auth_gssapi I
 was wondering if we want to press further and emable by default the use
 of native mod_auth_gssapi sessions ?
 
 The old mod_auth_kerb didn't have this feature so, in order to have
 decent performace we introduced split paths where some are always
 incurring the full negotiate penalty and other are and instead rely on a
 session cookie.
 
 mod_auth_gssapi can be configured to use a session cookie directly which
 avoids the negotiate auth performance hit. Integration would require
 that the FreeIPA code learns how to delete the cookie when someone hits
 a logout button, but it would be otherwise transparent.
 
 It would be especially useful for 3rd party clients that want to use the
 json/xmlrpc enpoints, as all they have to do is just support sending
 back cookies and they do not have to learn how to contact multiple
 endopints to get credentials and then switch to the session only based
 ones.
 
 Thoughts ?
 
 Simo.

This is a good thing, Simo.  Yes.  -A

-- 
Anthony - https://messinet.com/ - https://messinet.com/~amessina/gallery
8F89 5E72 8DF0 BCF0 10BE 9967 92DC 35DC B001 4A4E


signature.asc
Description: This is a digitally signed message part.
-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code