Sorry for late reply, I was waiting how the discussion with tracker improvement 
will end, but since there's no progress and I'm leaving soon, I'm attaching new 
patch. I also created mapping between old and new tests [1], to make life of 
reviewer easier. Number in first column denotes line number in original file, 
followed by line number in new tests file and its name. Tests that are not 
mentioned in there extend previous coverage.

Regards,
Peter

[1] http://etherpad.corp.redhat.com/Vk3tRLaPSK


----- Original Message -----
From: "Martin Basti" <mba...@redhat.com>
To: "Peter Lacko" <pla...@redhat.com>
Cc: freeipa-devel@redhat.com
Sent: Monday, June 6, 2016 12:29:29 PM
Subject: Re: [Freeipa-devel] [PATCH] 0002 New User Role Tests



On 02.06.2016 16:16, Peter Lacko wrote:
> Rebased with updated tests.
>
> Peter
>
> ----- Original Message -----
> From: "Martin Basti" <mba...@redhat.com>
> To: "Peter Lacko" <pla...@redhat.com>
> Cc: freeipa-devel@redhat.com
> Sent: Thursday, June 2, 2016 1:50:06 PM
> Subject: Re: [Freeipa-devel] [PATCH] 0002 New User Role Tests
>
> Your patch doesn't apply anymore on master, there were changes in role
> test and patch have to be updated and rebased
>
> Please look at this for new changes in test_role_plugin.py
> https://git.fedorahosted.org/cgit/freeipa.git/commit/?id=5f42b42bd4557a669ab5cfcf1af6596f1a2535f1
>
> Martin^2
>
>
> On 02.06.2016 11:49, Peter Lacko wrote:
>> Sorry for late response, I wasn't working these days.
>> Fixed patch is in attachment.
>>
>> Peter
>>
>>
>> ----- Original Message -----
>> From: "Martin Basti" <mba...@redhat.com>
>> To: "Peter Lacko" <pla...@redhat.com>, freeipa-devel@redhat.com
>> Sent: Monday, May 9, 2016 1:06:08 PM
>> Subject: Re: [Freeipa-devel] [PATCH] 0002 New User Role Tests
>>
>>
>>
>> On 09.05.2016 13:04, Martin Basti wrote:
>>> On 09.05.2016 12:19, Peter Lacko wrote:
>>>> +# pylint: disable=unicode-builtin
>>> I'm not doing complete review, just the line above hit my eyes.
>>>
>>> unicode() is not in Py3 because all strings there are unicode, thus
>>> you cannot use it directly, you need something like
>>>
>>> if six.PY2:
>>>       str = unicode
>>>
>>> and use str() everywhere and remove that #pylint line
>>>
>>> FYI all enabled pylint checks are there for a good reason, be careful
>>> with disabling it (mainly disabling it for a whole module) rather ask
>>> before if you are not sure.
>>>
>>> Martin^2
>>>
>> Nope, sorry, I temporarily forgot how to python
>>
>> instead of pylint disable use this
>>
>> if six.PY3:
>>        unicode =str
>>
>> and keep unicode there. Sorry
>>
>> Martin^2
Hi,

I don't have time to continue with full review, maybe somebody else can 
continue instead of me, but anyway NACK, because using str(unicode()) or 
unicode(str()) is bad in both ways, it should be just unicode() in case 
of strings (with correct mapping unicode=str in Py3). I just  I noticed 
this by reading code, but I didn't do deeper review, so there might be 
more mistakes.

Martin^2
From a5192241e29181e68a480b2b2e802a4267cea4d6 Mon Sep 17 00:00:00 2001
From: Peter Lacko <pla...@redhat.com>
Date: Tue, 31 May 2016 14:34:19 +0200
Subject: [PATCH] New User Role tests.

Commit contains set of User Role tests, implementation of UserTracker
and basic implementation of PrivilegeTracker necessary for Role testing.
Tests replace previous declarative tests for User Role plugin and extends
their coverage.

Commit contains 74 tests split into classes as follows:

    TestNonexistentRole
        Test that no operation can be performed on non-existing role.
    TestDeletedRole
        Test that no operation can be performed on deleted role.
    TestRoleBasic
        Test basic role commands: create, update, delete.
    TestRoleUpdate
        Test role-mod command.
    TestRolePrivileges
        Test correct functionality role-privilege-* command.
    TestRoleMembers
        Test correct functionality of role-member-* command.
    TestRoleFind
        Test correct functionality of role-find command.

Other than previously mentiond functionality has not beed added, removed or modified.
---
 ipatests/test_xmlrpc/test_role_plugin.py         | 1334 +++++++++++-----------
 ipatests/test_xmlrpc/tracker/privilege_plugin.py |   96 ++
 ipatests/test_xmlrpc/tracker/role_plugin.py      |  583 ++++++++++
 3 files changed, 1343 insertions(+), 670 deletions(-)
 create mode 100644 ipatests/test_xmlrpc/tracker/privilege_plugin.py
 create mode 100644 ipatests/test_xmlrpc/tracker/role_plugin.py

diff --git a/ipatests/test_xmlrpc/test_role_plugin.py b/ipatests/test_xmlrpc/test_role_plugin.py
index 6771cfad503e998a2c1433cc4c012210e815eab9..b6163cbf785e76d267f059658c314d80fbd64f51 100644
--- a/ipatests/test_xmlrpc/test_role_plugin.py
+++ b/ipatests/test_xmlrpc/test_role_plugin.py
@@ -2,8 +2,9 @@
 #   Rob Crittenden <rcrit...@redhat.com>
 #   Pavel Zuna <pz...@redhat.com>
 #   John Dennis <jden...@redhat.com>
+#   Peter Lacko <pla...@redhat.com>
 #
-# Copyright (C) 2009  Red Hat
+# Copyright (C) 2009, 2016  Red Hat
 # see file 'COPYING' for use and warranty information
 #
 # This program is free software; you can redistribute it and/or modify
@@ -22,681 +23,674 @@
 Test the `ipaserver/plugins/role.py` module.
 """
 
-from ipalib import api, errors
-from ipatests.test_xmlrpc import objectclasses
-from ipatests.test_xmlrpc.xmlrpc_test import Declarative, fuzzy_uuid
-from ipapython.dn import DN
+
 import pytest
+import six
+from uuid import uuid4
 
-search = u'test-role'
+from ipalib import errors
+from ipatests.test_xmlrpc.tracker.group_plugin import GroupTracker
+from ipatests.test_xmlrpc.tracker.host_plugin import HostTracker
+from ipatests.test_xmlrpc.tracker.hostgroup_plugin import HostGroupTracker
+from ipatests.test_xmlrpc.tracker.privilege_plugin import PrivilegeTracker
+from ipatests.test_xmlrpc.tracker.role_plugin import RoleTracker
+from ipatests.test_xmlrpc.tracker.user_plugin import UserTracker
+from ipatests.test_xmlrpc.xmlrpc_test import (
+    raises_exact,
+    XMLRPC_test,
+)
+from ipatests.util import assert_deepequal
 
-role1 = u'test-role-1'
-role1_dn = DN(('cn',role1),api.env.container_rolegroup,
-              api.env.basedn)
-renamedrole1 = u'test-role'
-invalidrole1 = u' whitespace '
+_DESCRIPTION = u'Description string'
+_NEW_DESC = u'New description'
+_INVALID_NAME = u'  Invalid Name  '
 
-role2 = u'test-role-2'
-role2_dn = DN(('cn',role2),api.env.container_rolegroup,
-              api.env.basedn)
+if six.PY3:
+    unicode = str
 
-group1 = u'testgroup1'
-group1_dn = DN(('cn',group1),api.env.container_group,
-               api.env.basedn)
 
-privilege1 = u'r,w privilege 1'
-privilege1_dn = DN(('cn', privilege1), DN(api.env.container_privilege),
-                   api.env.basedn)
+@pytest.fixture(scope='class')
+def role(request):
+    """Default testing role."""
+    tracker = RoleTracker(name=u'test-role1', description=_DESCRIPTION)
+    return tracker.make_fixture(request)
+
+
+@pytest.fixture(scope='class')
+def role2(request):
+    """Additional testing role."""
+    tracker = RoleTracker(name=u'test-role2', description=_DESCRIPTION)
+    return tracker.make_fixture(request)
+
+
+@pytest.fixture(scope='class')
+def random_role(request):
+    """Testing role with random name."""
+    tracker = RoleTracker(
+        name=unicode(uuid4()), description=_DESCRIPTION
+    )
+    return tracker.make_fixture(request)
+
+
+@pytest.fixture(scope='class')
+def invalid_role(request):
+    """Role with invalid name."""
+    tracker = RoleTracker(name=_INVALID_NAME)
+    return tracker.make_fixture(request)
+
+
+@pytest.fixture(scope='class')
+def user(request):
+    """Default testing user."""
+    tracker = UserTracker(name=u'user', givenname=u'UserGN', sn=u'UserSN')
+    return tracker.make_fixture(request)
+
+
+@pytest.fixture(scope='class')
+def group(request):
+    """Default testing group."""
+    tracker = GroupTracker(name=u'group1', description=u'Group description')
+    return tracker.make_fixture(request)
+
+
+@pytest.fixture(scope='class')
+def host(request):
+    """Default testing host."""
+    tracker = HostTracker(name=u'testhost')
+    return tracker.make_fixture(request)
+
+
+@pytest.fixture(scope='class')
+def hostgroup(request):
+    """Default testing hostgroup."""
+    tracker = HostGroupTracker(name=u'hostgroup')
+    return tracker.make_fixture(request)
+
+
+@pytest.fixture(scope='class')
+def privilege(request):
+    """Default testing privilege."""
+    tracker = PrivilegeTracker(name=u'privilege1')
+    return tracker.make_fixture(request)
+
+
+@pytest.mark.tier1
+class TestNonexistentRole(XMLRPC_test):
+    """Test for correct handling of operations on nonexisting role."""
+    def test_retrieve(self, random_role):
+        """Try to retrieve non-existent role."""
+        random_role.ensure_missing()
+        command = random_role.make_retrieve_command()
+        with raises_exact(errors.NotFound(
+                reason=u'%s: role not found' % random_role.cn)):
+            command()
+
+    def test_update(self, random_role):
+        """Try to update non-existent role."""
+        command = random_role.make_update_command(
+            updates={'description': u'Foo'})
+        with raises_exact(errors.NotFound(
+                reason=u'%s: role not found' % random_role.cn)):
+            command()
+
+    def test_delete(self, random_role):
+        """Try to delete non-existent role."""
+        command = random_role.make_delete_command()
+        with raises_exact(errors.NotFound(
+                reason=u'%s: role not found' % random_role.cn)):
+            command()
+
+    def test_rename(self, random_role):
+        """Try to rename non-existent role."""
+        new_name = uuid4()
+        command = random_role.make_update_command(
+            updates={'setattr': u'cn=%s' % new_name})
+        with raises_exact(errors.NotFound(
+                reason=u'%s: role not found' % random_role.cn)):
+            command()
+
+    def test_find(self, random_role):
+        """Try to find non-existent role."""
+        result = random_role.find(random_role.cn)
+        expected = RoleTracker.find_tracker_roles([random_role])
+        assert_deepequal(expected, result)
+
+    def test_create_invalid(self, invalid_role):
+        """Try to create role with an invalid name."""
+        command = invalid_role.make_create_command()
+        with raises_exact(
+                errors.ValidationError(
+                    name='name',
+                    error=u'Leading and trailing spaces are not allowed')):
+            command()
+
+    def test_add_privilege(self, random_role, privilege):
+        """Try to add a privilege to non-existing role."""
+        privilege.ensure_exists()
+        command = random_role.make_command(
+            'role_add_privilege', random_role.cn, privilege=privilege.cn)
+        with raises_exact(errors.NotFound(
+                reason=u'%s: role not found' % random_role.cn)):
+            command()
+
+    def test_remove_privileges(self, random_role, privilege):
+        """Try to remove privilege from non-existent role."""
+        privilege.ensure_exists()
+        command = random_role.make_command(
+            'role_remove_privilege', random_role.cn, privilege=privilege.cn)
+        with raises_exact(errors.NotFound(
+                reason=u'%s: role not found' % random_role.cn)):
+            command()
+
+    def test_add_member(self, random_role, group):
+        """Try to add a member to non-existing role."""
+        group.ensure_exists()
+        command = random_role.make_command(
+            'role_add_member', random_role.cn, group=group.cn)
+        with raises_exact(errors.NotFound(
+                reason=u'%s: role not found' % random_role.cn)):
+            command()
+
+    def test_remove_member(self, random_role, group):
+        """Try to remove member from non-existing role."""
+        group.ensure_exists()
+        command = random_role.make_command(
+            'role_remove_member', random_role.cn, group=group.cn)
+        with raises_exact(errors.NotFound(
+                reason=u'%s: role not found' % random_role.cn)):
+            command()
+
+
+@pytest.mark.tier1
+class TestDeletedRole(XMLRPC_test):
+    """Test for correct handling of operations on deleted role."""
+
+    def setUpClass(self, role):
+        """Create and delete role."""
+        role.ensure_exists()
+        role.ensure_missing()
+
+    def test_retrieve(self, role):
+        """Try to retrieve deleted role."""
+        command = role.make_retrieve_command()
+        with raises_exact(errors.NotFound(
+                reason=u'%s: role not found' % role.cn)):
+            command()
+
+    def test_update(self, role):
+        """Try to update deleted role."""
+        command = role.make_update_command(updates={'description': u'Foo'})
+        with raises_exact(errors.NotFound(
+                reason=u'%s: role not found' % role.cn)):
+            command()
+
+    def test_delete(self, role):
+        """Try to remove deleted role."""
+        command = role.make_delete_command()
+        with raises_exact(errors.NotFound(
+                reason=u'%s: role not found' % role.cn)):
+            command()
+
+    def test_rename(self, role):
+        """Try to rename deleted role."""
+        new_name = uuid4()
+        command = role.make_update_command(
+            updates={'setattr': u'cn=%s' % new_name})
+        with raises_exact(errors.NotFound(
+                reason=u'%s: role not found' % role.cn)):
+            command()
+
+    def test_find(self, role):
+        """Try to search for deleted role."""
+        result = role.find(role.cn)
+        expected = RoleTracker.find_tracker_roles([role])
+        assert_deepequal(expected, result)
+
+    def test_add_privilege(self, role, privilege):
+        """Try to add a privilege to deleted role."""
+        privilege.ensure_exists()
+        command = role.make_command(
+            'role_add_privilege', role.cn, privilege=privilege.cn)
+        with raises_exact(errors.NotFound(
+                reason=u'%s: role not found' % role.cn)):
+            command()
+
+    def test_remove_privileges(self, role, privilege):
+        """Try to remove privilege from deleted role."""
+        privilege.ensure_exists()
+        command = role.make_command(
+            'role_remove_privilege', role.cn, privilege=privilege.cn)
+        with raises_exact(errors.NotFound(
+                reason=u'%s: role not found' % role.cn)):
+            command()
+
+    def test_add_member(self, role, group):
+        """Try to add a member to deleted role."""
+        group.ensure_exists()
+        command = role.make_command(
+            'role_add_member', role.cn, group=group.cn)
+        with raises_exact(errors.NotFound(
+                reason=u'%s: role not found' % role.cn)):
+            command()
+
+    def test_remove_member(self, role, group):
+        """Try to remove member from deleted role."""
+        group.ensure_exists()
+        command = role.make_command(
+            'role_remove_member', role.cn, group=group.cn)
+        with raises_exact(errors.NotFound(
+                reason=u'%s: role not found' % role.cn)):
+            command()
+
+
+@pytest.mark.tier1
+class TestRoleBasic(XMLRPC_test):
+    """Test basic functionality of role plugin."""
+
+    def test_create_retrieve(self, role):
+        """Create and retrieve role, with checks performed in role_plugin."""
+        role.ensure_exists()
+        result = role.retrieve()
+        role.check_retrieve(result)
+
+    def test_retrieve_all(self, role):
+        """Test role-show command with --all parameter."""
+        result = role.retrieve(all=True)
+        role.check_retrieve(result, all=True)
+
+    def test_create_duplicate(self, role):
+        """Make sure it is not possible to create duplicate role."""
+        command = role.make_create_command()
+        with raises_exact(
+                errors.DuplicateEntry(
+                    message=u'role with name "%s" already exists' % role.cn)):
+            command()
+
+    def test_delete(self, role):
+        """Delete role and make sure we can not retrieve it.
+
+        All checks are performed in tracker.role_plugin.RoleTracker class.
+        """
+        role.delete()
 
 
 @pytest.mark.tier1
-class test_role(Declarative):
-
-    cleanup_commands = [
-        ('role_del', [role1], {}),
-        ('role_del', [role2], {}),
-        ('group_del', [group1], {}),
-        ('privilege_del', [privilege1], {}),
-    ]
-
-    tests = [
-
-        dict(
-            desc='Try to retrieve non-existent %r' % role1,
-            command=('role_show', [role1], {}),
-            expected=errors.NotFound(reason=u'%s: role not found' % role1),
-        ),
-
-
-        dict(
-            desc='Try to update non-existent %r' % role1,
-            command=('role_mod', [role1], dict(description=u'Foo')),
-            expected=errors.NotFound(reason=u'%s: role not found' % role1),
-        ),
-
-
-        dict(
-            desc='Try to delete non-existent %r' % role1,
-            command=('role_del', [role1], {}),
-            expected=errors.NotFound(reason=u'%s: role not found' % role1),
-        ),
-
-
-        dict(
-            desc='Try to rename non-existent %r' % role1,
-            command=('role_mod', [role1], dict(setattr=u'cn=%s' % renamedrole1)),
-            expected=errors.NotFound(reason=u'%s: role not found' % role1),
-        ),
-
-
-        dict(
-            desc='Search for non-existent %r' % role1,
-            command=('role_find', [role1], {}),
-            expected=dict(
-                count=0,
-                truncated=False,
-                summary=u'0 roles matched',
-                result=[],
-            ),
-        ),
-
-
-        dict(
-            desc='Create invalid %r' % invalidrole1,
-            command=('role_add', [invalidrole1],
-                dict(description=u'role desc 1')
-            ),
-            expected=errors.ValidationError(name='name',
-                error=u'Leading and trailing spaces are not allowed'),
-        ),
-
-
-        dict(
-            desc='Create %r' % role1,
-            command=('role_add', [role1],
-                dict(description=u'role desc 1')
-            ),
-            expected=dict(
-                value=role1,
-                summary=u'Added role "%s"' % role1,
-                result=dict(
-                    dn=role1_dn,
-                    cn=[role1],
-                    description=[u'role desc 1'],
-                    objectclass=objectclasses.role,
-                ),
-            ),
-        ),
-
-
-        dict(
-            desc='Retrieve %r' % role1,
-            command=('role_show', [role1], {}),
-            expected=dict(
-                value=role1,
-                summary=None,
-                result=dict(
-                    dn=role1_dn,
-                    cn=[role1],
-                    description=[u'role desc 1'],
-                ),
-            ),
-        ),
-
-
-        dict(
-            desc='Create %r' % group1,
-            command=(
-                'group_add', [group1], dict(description=u'group desc 1',
-                nonposix=True,)
-            ),
-            expected=dict(
-                value=group1,
-                summary=u'Added group "testgroup1"',
-                result=dict(
-                    dn=group1_dn,
-                    cn=[group1],
-                    description=[u'group desc 1'],
-                    objectclass=objectclasses.group,
-                    ipauniqueid=[fuzzy_uuid],
-                ),
-            ),
-        ),
-
-
-        dict(
-            desc='Create %r' % privilege1,
-            command=('privilege_add', [privilege1],
-                dict(description=u'privilege desc. 1')
-            ),
-            expected=dict(
-                value=privilege1,
-                summary=u'Added privilege "%s"' % privilege1,
-                result=dict(
-                    dn=privilege1_dn,
-                    cn=[privilege1],
-                    description=[u'privilege desc. 1'],
-                    objectclass=objectclasses.privilege,
-                ),
-            ),
-        ),
-
-
-        dict(
-            desc='Add privilege %r to role %r' % (privilege1, role1),
-            command=('role_add_privilege', [role1],
-                dict(privilege=privilege1)
-            ),
-            expected=dict(
-                completed=1,
-                failed=dict(
-                    member=dict(
-                        privilege=[],
-                    ),
-                ),
-                result={
-                    'dn': role1_dn,
-                    'cn': [role1],
-                    'description': [u'role desc 1'],
-                    'memberof_privilege': [privilege1],
-                }
-            ),
-        ),
-
-
-        dict(
-            desc='Add zero privileges to role %r' % role1,
-            command=('role_add_privilege', [role1], dict(privilege=None)
-            ),
-            expected=dict(
-                completed=0,
-                failed=dict(
-                    member=dict(
-                        privilege=[],
-                    ),
-                ),
-                result={
-                    'dn': role1_dn,
-                    'cn': [role1],
-                    'description': [u'role desc 1'],
-                    'memberof_privilege': [privilege1],
-                }
-            ),
-        ),
-
-
-        dict(
-            desc='Remove zero privileges from role %r' % role1,
-            command=('role_remove_privilege', [role1], dict(privilege=None)
-            ),
-            expected=dict(
-                completed=0,
-                failed=dict(
-                    member=dict(
-                        privilege=[],
-                    ),
-                ),
-                result={
-                    'dn': role1_dn,
-                    'cn': [role1],
-                    'description': [u'role desc 1'],
-                    'memberof_privilege': [privilege1],
-                }
-            ),
-        ),
-
-
-        dict(
-            desc='Add member %r to %r' % (group1, role1),
-            command=('role_add_member', [role1], dict(group=group1)),
-            expected=dict(
-                completed=1,
-                failed=dict(
-                    member=dict(
-                        user=[],
-                        group=[],
-                        host=[],
-                        hostgroup=[],
-                        service=[],
-                    ),
-                ),
-                result={
-                    'dn': role1_dn,
-                    'cn': [role1],
-                    'description': [u'role desc 1'],
-                    'member_group': [group1],
-                    'memberof_privilege': [privilege1],
-                }
-            ),
-        ),
-
-
-        dict(
-            desc='Retrieve %r to verify member-add' % role1,
-            command=('role_show', [role1], {}),
-            expected=dict(
-                value=role1,
-                summary=None,
-                result={
-                    'dn': role1_dn,
-                    'cn': [role1],
-                    'description': [u'role desc 1'],
-                    'member_group': [group1],
-                    'memberof_privilege': [privilege1],
-                },
-            ),
-        ),
-
-
-        dict(
-            desc='Search for %r with members' % role1,
-            command=('role_find', [role1], {'no_members': False}),
-            expected=dict(
-                count=1,
-                truncated=False,
-                summary=u'1 role matched',
-                result=[
-                    {
-                        'dn': role1_dn,
-                        'cn': [role1],
-                        'description': [u'role desc 1'],
-                        'member_group': [group1],
-                        'memberof_privilege': [privilege1],
-                    },
-                ],
-            ),
-        ),
-
-
-        dict(
-            desc='Search for %r' % role1,
-            command=('role_find', [role1], {}),
-            expected=dict(
-                count=1,
-                truncated=False,
-                summary=u'1 role matched',
-                result=[
-                    {
-                        'dn': role1_dn,
-                        'cn': [role1],
-                        'description': [u'role desc 1'],
-                    },
-                ],
-            ),
-        ),
-
-
-        dict(
-            desc='Search for %r with members' % search,
-            command=('role_find', [search], {'no_members': False}),
-            expected=dict(
-                count=1,
-                truncated=False,
-                summary=u'1 role matched',
-                result=[
-                    {
-                        'dn': role1_dn,
-                        'cn': [role1],
-                        'description': [u'role desc 1'],
-                        'member_group': [group1],
-                        'memberof_privilege': [privilege1],
-                    },
-                ],
-            ),
-        ),
-
-
-        dict(
-            desc='Search for %r' % search,
-            command=('role_find', [search], {}),
-            expected=dict(
-                count=1,
-                truncated=False,
-                summary=u'1 role matched',
-                result=[
-                    {
-                        'dn': role1_dn,
-                        'cn': [role1],
-                        'description': [u'role desc 1'],
-                    },
-                ],
-            ),
-        ),
-
-
-        dict(
-            desc='Create %r' % role2,
-            command=('role_add', [role2],
-                dict(description=u'role desc 2')
-            ),
-            expected=dict(
-                value=role2,
-                summary=u'Added role "%s"' % role2,
-                result=dict(
-                    dn=role2_dn,
-                    cn=[role2],
-                    description=[u'role desc 2'],
-                    objectclass=objectclasses.role,
-                ),
-            ),
-        ),
-
-
-        dict(
-            desc='Search for %r with members' % role1,
-            command=('role_find', [role1], {'no_members': False}),
-            expected=dict(
-                count=1,
-                truncated=False,
-                summary=u'1 role matched',
-                result=[
-                    {
-                        'dn': role1_dn,
-                        'cn': [role1],
-                        'description': [u'role desc 1'],
-                        'member_group': [group1],
-                        'memberof_privilege': [privilege1],
-                    },
-                ],
-            ),
-        ),
-
-
-        dict(
-            desc='Search for %r' % role1,
-            command=('role_find', [role1], {}),
-            expected=dict(
-                count=1,
-                truncated=False,
-                summary=u'1 role matched',
-                result=[
-                    {
-                        'dn': role1_dn,
-                        'cn': [role1],
-                        'description': [u'role desc 1'],
-                    },
-                ],
-            ),
-        ),
-
-
-        dict(
-            desc='Search for %r with members' % search,
-            command=('role_find', [search], {'no_members': False}),
-            expected=dict(
-                count=2,
-                truncated=False,
-                summary=u'2 roles matched',
-                result=[
-                    {
-                        'dn': role1_dn,
-                        'cn': [role1],
-                        'description': [u'role desc 1'],
-                        'member_group': [group1],
-                        'memberof_privilege': [privilege1],
-                    },
-                    {
-                        'dn': role2_dn,
-                        'cn': [role2],
-                        'description': [u'role desc 2'],
-                    },
-                ],
-            ),
-        ),
-
-
-        dict(
-            desc='Search for %r' % search,
-            command=('role_find', [search], {}),
-            expected=dict(
-                count=2,
-                truncated=False,
-                summary=u'2 roles matched',
-                result=[
-                    {
-                        'dn': role1_dn,
-                        'cn': [role1],
-                        'description': [u'role desc 1'],
-                    },
-                    {
-                        'dn': role2_dn,
-                        'cn': [role2],
-                        'description': [u'role desc 2'],
-                    },
-                ],
-            ),
-        ),
-
-
-        dict(
-            desc='Update %r' % role1,
-            command=(
-                'role_mod', [role1], dict(description=u'New desc 1')
-            ),
-            expected=dict(
-                value=role1,
-                summary=u'Modified role "%s"' % role1,
-                result=dict(
-                    cn=[role1],
-                    description=[u'New desc 1'],
-                    member_group=[group1],
-                    memberof_privilege=[privilege1],
-                ),
-            ),
-        ),
-
-
-        dict(
-            desc='Retrieve %r to verify update' % role1,
-            command=('role_show', [role1], {}),
-            expected=dict(
-                value=role1,
-                summary=None,
-                result={
-                    'dn': role1_dn,
-                    'cn': [role1],
-                    'description': [u'New desc 1'],
-                    'member_group': [group1],
-                    'memberof_privilege': [privilege1],
-                },
-            ),
-        ),
-
-
-        dict(
-            desc='Remove member %r from %r' % (group1, role1),
-            command=('role_remove_member', [role1], dict(group=group1)),
-            expected=dict(
-                completed=1,
-                failed=dict(
-                    member=dict(
-                        user=[],
-                        group=[],
-                        host=[],
-                        hostgroup=[],
-                        service=[],
-                    ),
-                ),
-                result={
-                    'dn': role1_dn,
-                    'cn': [role1],
-                    'description': [u'New desc 1'],
-                    'memberof_privilege': [privilege1],
-                },
-            ),
-        ),
-
-
-        dict(
-            desc='Retrieve %r to verify member-del' % role1,
-            command=('role_show', [role1], {}),
-            expected=dict(
-                value=role1,
-                summary=None,
-                result={
-                    'dn': role1_dn,
-                    'cn': [role1],
-                    'description': [u'New desc 1'],
-                    'memberof_privilege': [privilege1],
-                },
-            ),
-        ),
-
-
-        dict(
-            desc='Delete %r' % group1,
-            command=('group_del', [group1], {}),
-            expected=dict(
-                result=dict(failed=[]),
-                value=[group1],
-                summary=u'Deleted group "testgroup1"',
-            )
-        ),
-
-
-        dict(
-            desc='Rename %r' % role1,
-            command=('role_mod', [role1], dict(setattr=u'cn=%s' % renamedrole1)),
-            expected=dict(
-                value=role1,
-                result=dict(
-                    cn=[renamedrole1],
-                    description=[u'New desc 1'],
-                    memberof_privilege=[privilege1],
-                ),
-                summary=u'Modified role "%s"' % role1
-            )
-        ),
-
-
-        dict(
-            desc='Rename %r back' % renamedrole1,
-            command=('role_mod', [renamedrole1], dict(setattr=u'cn=%s' % role1)),
-            expected=dict(
-                value=renamedrole1,
-                result=dict(
-                    cn=[role1],
-                    description=[u'New desc 1'],
-                    memberof_privilege=[privilege1],
-                ),
-                summary=u'Modified role "%s"' % renamedrole1
-            )
-        ),
-
-
-        dict(
-            desc='Remove privilege %r from role %r' % (privilege1, role1),
-            command=('role_remove_privilege', [role1],
-                dict(privilege=privilege1)
-            ),
-            expected=dict(
-                completed=1,
-                failed=dict(
-                    member=dict(
-                        privilege=[],
-                    ),
-                ),
-                result={
-                    'dn': role1_dn,
-                    'cn': [role1],
-                    'description': [u'New desc 1'],
-                }
-            ),
-        ),
-
-
-        dict(
-            desc='Remove privilege %r from role %r again' % (privilege1, role1),
-            command=('role_remove_privilege', [role1],
-                dict(privilege=privilege1)
-            ),
-            expected=dict(
-                completed=0,
-                failed=dict(
-                    member=dict(
-                        privilege=[(u'%s' % privilege1, u'This entry is not a member'),],
-                    ),
-                ),
-                result={
-                    'dn': role1_dn,
-                    'cn': [role1],
-                    'description': [u'New desc 1'],
-                }
-            ),
-        ),
-
-
-
-        dict(
-            desc='Delete %r' % role1,
-            command=('role_del', [role1], {}),
-            expected=dict(
-                result=dict(failed=[]),
-                value=[role1],
-                summary=u'Deleted role "%s"' % role1,
-            )
-        ),
-
-
-        dict(
-            desc='Try to delete non-existent %r' % role1,
-            command=('role_del', [role1], {}),
-            expected=errors.NotFound(reason=u'%s: role not found' % role1),
-        ),
-
-
-        dict(
-            desc='Try to show non-existent %r' % role1,
-            command=('role_show', [role1], {}),
-            expected=errors.NotFound(reason=u'%s: role not found' % role1),
-        ),
-
-
-        dict(
-            desc='Try to update non-existent %r' % role1,
-            command=('role_mod', [role1], dict(description=u'Foo')),
-            expected=errors.NotFound(reason=u'%s: role not found' % role1),
-        ),
-
-
-        dict(
-            desc='Search for %r' % search,
-            command=('role_find', [search], {}),
-            expected=dict(
-                count=1,
-                truncated=False,
-                summary=u'1 role matched',
-                result=[
-                    {
-                        'dn': role2_dn,
-                        'cn': [role2],
-                        'description': [u'role desc 2'],
-                    },
-                ],
-            ),
-        ),
-
-
-        dict(
-            desc='Delete %r' % role2,
-            command=('role_del', [role2], {}),
-            expected=dict(
-                result=dict(failed=[]),
-                value=[role2],
-                summary=u'Deleted role "%s"' % role2,
-            )
-        ),
-
-
-        dict(
-            desc='Search for %r' % search,
-            command=('role_find', [search], {}),
-            expected=dict(
-                count=0,
-                truncated=False,
-                summary=u'0 roles matched',
-                result=[],
-            ),
-        ),
-
-    ]
+class TestRoleUpdate(XMLRPC_test):
+    """Class for testing ipa role-mod command."""
+
+    def test_update_description(self, role):
+        """Test updating role description."""
+        role.ensure_exists()
+        result = role.update({'description': _NEW_DESC})
+        role.update_tracker({'description': _NEW_DESC})
+        role.check_update(result)
+
+    def test_retrieve(self, role):
+        """Check that description was changed."""
+        result = role.retrieve()
+        role.check_retrieve(result)
+
+    def test_rename(self, role):
+        """Rename role to random identifier."""
+        new_name = unicode(uuid4())
+        role.original_name = role.cn
+        result = role.update({'rename': new_name})
+        role.update_tracker({'rename': new_name})
+        role.check_update(result)
+        # we additionaly have to set cn for further testing
+        role.cn = new_name
+
+    def test_rename_again(self, role):
+        """Try to rename role to a same name."""
+        with raises_exact(errors.EmptyModlist(
+                message=u'no modifications to be performed')):
+            role.update({'rename': role.cn})
+
+    def test_rename_to_existing(self, role, random_role):
+        """Try to rename role to already taken name."""
+        random_role.ensure_exists()
+        with raises_exact(errors.DuplicateEntry(
+                message=u'This entry already exists')):
+            role.update({'rename': random_role.cn})
+
+    def test_rename_to_invalid(self, role):
+        """Try to rename role to invalid name."""
+        with raises_exact(errors.ValidationError(
+                message=(u"invalid 'rename': "
+                         u"Leading and trailing spaces are not allowed"))):
+            role.update({'rename': _INVALID_NAME})
+
+    def test_rename_back(self, role):
+        """Rename role back to its original name."""
+        result = role.update({'rename': role.original_name})
+        role.update_tracker({'rename': role.original_name})
+        role.check_update(result)
+        # we additionaly have to set cn for further testing
+        role.cn = role.original_name
+
+    def test_retrieve_again(self, role):
+        """Check that previous tests didn't change role's state."""
+        result = role.retrieve()
+        role.check_retrieve(result)
+
+
+@pytest.mark.tier1
+class TestRolePrivileges(XMLRPC_test):
+    """Test handling of privileges in roles."""
+
+    def test_add_privilege_to_role(self, role, privilege):
+        """Add valid privilege to an existing role."""
+        role.ensure_exists()
+        privilege.ensure_exists()
+        result = role.add_privilege(privilege.cn)
+        role.check_add_privilege(result)
+
+    def test_add_privilege_to_role_again(self, role, privilege):
+        """Try to add privilege to role again."""
+        result = role.add_privilege(privilege.cn)
+        role.check_add_duplicate_privilege(result, privilege.cn)
+
+    def test_add_zero_privilege_to_role(self, role):
+        """Add empty privilege to role and check the result."""
+        result = role.add_privilege(None)
+        role.check_add_zero_privilege(result)
+
+    def test_remove_zero_privilege_from_role(self, role):
+        """Remove empty privilege from role and check result."""
+        result = role.remove_privilege(None)
+        role.check_remove_zero_privilege(result)
+
+    def test_show_updates(self, role, privilege):
+        """Ensure that role-show command returns correct result."""
+        result = role.retrieve()
+        role.check_retrieve(result)
+
+    def test_show_all_updates(self, role, privilege):
+        """Ensure that role-show --all command returns correct result."""
+        result = role.retrieve(all=True)
+        role.check_retrieve(result, all=True)
+
+    def test_remove_privilege_from_role(self, role, privilege):
+        """Remove privilege from role."""
+        result = role.remove_privilege(privilege.cn)
+        role.check_remove_privilege(result)
+
+    def test_remove_privilege_from_role_again(self, role, privilege):
+        """Try to remove already removed privilege from role."""
+        result = role.remove_privilege(privilege.cn)
+        role.check_remove_privilege_false(result, privilege.cn)
+
+    def test_remove_privilege(self, role, privilege):
+        """Test physical removal of privilige from system.
+
+        Add privilege to the role, remove privilige from system and check that
+        it is not present in role-show command. This test assumes that
+        test_add_privilige_to_role holds true.
+        """
+        privilege.ensure_exists()
+        role.add_privilege(privilege.cn)
+        privilege.ensure_missing()
+        role.update_tracker({u'memberof_privilege': privilege.cn}, remove=True)
+        result = role.retrieve()
+        role.check_retrieve(result)
+
+    def test_add_nonexistent_privilege_to_role(self, role):
+        """Try to add non-existent privilege to role."""
+        priv_name = unicode(uuid4())
+        result = role.add_privilege(priv_name)
+        # we have to update tracker manually here since it was modified
+        role.update_tracker({u'memberof_privilege': priv_name}, remove=True)
+        role.check_handle_nonexistent_privilege(result, priv_name)
+
+    def test_remove_nonexistent_privilege_from_role(self, role):
+        """Try to remove non-existent privilege from role."""
+        priv_name = unicode(uuid4())
+        result = role.remove_privilege(priv_name)
+        role.check_handle_nonexistent_privilege(result, priv_name)
+
+
+@pytest.mark.tier1
+class TestRoleMembers(XMLRPC_test):
+    """Test member related operations on roles."""
+
+    def test_add_group_to_role(self, group, role):
+        """Add new group to the role."""
+        group.ensure_exists()
+        role.ensure_exists()
+        result = role.add_member(group.cn, member_type='group')
+        role.check_add_member(result)
+
+    def test_add_user_to_role(self, user, role):
+        """Add new user to the role."""
+        user.ensure_exists()
+        result = role.add_member(user.uid, member_type='user')
+        role.check_add_member(result)
+
+    def test_add_host_to_role(self, host, role):
+        """Add new host to the role."""
+        host.ensure_exists()
+        result = role.add_member(host.fqdn, member_type='host')
+        role.check_add_member(result)
+
+    def test_add_hostgroup_to_role(self, hostgroup, role):
+        """Add new hostgroup to the role."""
+        hostgroup.ensure_exists()
+        result = role.add_member(hostgroup.cn, member_type='hostgroup')
+        role.check_add_member(result)
+
+    def test_add_duplicate_member_to_role(self, group, role):
+        """Try to add duplicate group to the role."""
+        result = role.add_member(group.cn, member_type='group')
+        role.check_add_duplicate_member(result, group.cn, member_type='group')
+
+    def test_remove_user_from_role(self, user, role):
+        """Remove user from role and check results."""
+        result = role.remove_member(user.uid, member_type='user')
+        role.check_remove_member(result)
+
+    def test_show_remove_user(self, role):
+        """Check that user was removed from role."""
+        result = role.retrieve()
+        role.check_retrieve(result)
+
+    def test_remove_user_from_role_again(self, user, role):
+        """Try to remove user from the role again and check results."""
+        result = role.remove_member(user.uid, member_type='user')
+        role.check_remove_member_false(result, user.uid, member_type='user')
+
+    def test_remove_group_from_role(self, group, role):
+        """Remove group from role and check results."""
+        result = role.remove_member(group.cn, member_type='group')
+        role.check_remove_member(result)
+
+    def test_remove_host_from_role(self, host, role):
+        """Remove host from role and check results."""
+        result = role.remove_member(host.fqdn, member_type='host')
+        role.check_remove_member(result)
+
+    def test_remove_hostgroup_from_role(self, hostgroup, role):
+        """Remove hostgroup from role and check results."""
+        result = role.remove_member(hostgroup.cn, member_type='hostgroup')
+        role.check_remove_member(result)
+
+    def test_remove_member(self, group, role):
+        """Test physical removal of role's group member.
+
+        Remove group from the system and check it is not present
+        in role-show command. This test assumes that test_add_group_to_role
+        from this class and tests in test_group_plugin file hold true.
+        """
+        role.ensure_exists()
+        role.add_member(group.cn, member_type='group')
+        group.ensure_missing()
+        # we need to update tracker manually
+        role.update_tracker({u'member_group': group.cn}, remove=True)
+        result = role.retrieve()
+        role.check_retrieve(result)
+
+    def test_add_nonexistent_member(self, role):
+        """Try to add non-existent group to the role."""
+        invalid_group = unicode(uuid4())
+        result = role.add_member(invalid_group, member_type='group')
+        # manually update tracker since we expect group to not exists
+        role.update_tracker({u'member_group': invalid_group}, remove=True)
+        role.check_add_nonexistent_member(
+            result, invalid_group, member_type='group')
+
+    def test_remove_nonexistent_member(self, role):
+        """Try to remove non-existent group from the role."""
+        invalid_group = unicode(uuid4())
+        result = role.remove_member(invalid_group, member_type='group')
+        role.check_remove_member_false(
+            result, invalid_group, member_type='group')
+
+
+@pytest.mark.tier1
+class TestRoleFind(XMLRPC_test):
+    """Test role-find command using tracker method find_tracker_roles."""
+
+    def test_find_basic(self, role):
+        """Test basic find command on single role."""
+        role.ensure_exists()
+        expected = RoleTracker.find_tracker_roles([role], pattern=role.cn)
+        result = role.find(role.cn)
+        assert_deepequal(expected, result)
+
+    def test_find_basic_with_members(self, role):
+        """Test basic find command."""
+        role.ensure_exists()
+        expected = RoleTracker.find_tracker_roles(
+            [role], pattern=role.cn, criteria={'no_members': False})
+        result = role.find(role.cn, no_members=False)
+        assert_deepequal(expected, result)
+
+    def test_find_single(self, role):
+        """Find role searching substring."""
+        expected = RoleTracker.find_tracker_roles(
+            [role], pattern=u'test')
+        result = role.find(u'test')
+        assert_deepequal(expected, result)
+
+    def test_find_single_with_members(self, role):
+        """Find role searching by substring retrieving all member keys."""
+        expected = RoleTracker.find_tracker_roles(
+            [role], pattern=u'test', criteria={'no_members': False})
+        result = role.find(u'test', no_members=False)
+        assert_deepequal(expected, result)
+
+    def test_find_similar(self, role, role2):
+        """Find two roles with common prefix."""
+        role2.ensure_exists()
+        expected = RoleTracker.find_tracker_roles(
+            [role, role2], pattern=u'test')
+        result = role.find(u'test')
+        assert_deepequal(expected, result)
+
+    def test_find_similar_with_members(self, role, role2):
+        """Find two roles with common prefix and all member keys."""
+        expected = RoleTracker.find_tracker_roles(
+            [role, role2], pattern=u'test', criteria={'no_members': False})
+        result = role.find(u'test', no_members=False)
+        assert_deepequal(expected, result)
+
+    def test_find_basic_again(self, role):
+        """Test basic find command."""
+        role.ensure_exists()
+        expected = RoleTracker.find_tracker_roles([role], pattern=role.cn)
+        result = role.find(role.cn)
+        assert_deepequal(expected, result)
+
+    def test_find_basic_with_members_again(self, role):
+        """Test basic find command with members."""
+        role.ensure_exists()
+        expected = RoleTracker.find_tracker_roles(
+            [role], pattern=role.cn, criteria={'no_members': False})
+        result = role.find(role.cn, no_members=False)
+        assert_deepequal(expected, result)
+
+    def test_find_with_attributes(self, role, role2, user, privilege):
+        """Test find role with privilege and member attributes."""
+        privilege.ensure_exists()
+        user.ensure_exists()
+        role.add_privilege(privilege.cn)
+        role2.add_member(user.uid, member_type='user')
+        expected = RoleTracker.find_tracker_roles(
+            [role, role2], pattern=u'test')
+        result = role.find(u'test')
+        assert_deepequal(expected, result)
+
+    def test_find_with_attributes_and_members(
+            self, role, role2, user, privilege):
+        """Test find role with privilege and member attributes."""
+        privilege.ensure_exists()
+        user.ensure_exists()
+        role.add_privilege(privilege.cn)
+        role2.add_member(user.uid, member_type='user')
+        expected = RoleTracker.find_tracker_roles(
+            [role, role2], pattern=u'test', criteria={'no_members': False})
+        result = role.find(u'test', no_members=False)
+        assert_deepequal(expected, result)
+
+    def test_find_pkey_only(self, role, role2):
+        """Find with 'pkey-only' attribute enabled."""
+        expected = RoleTracker.find_tracker_roles(
+            [role, role2], pattern=u'test', criteria={'pkey_only': True})
+        result = role.find(u'test', pkey_only=True)
+        assert_deepequal(expected, result)
+
+    def test_find_all(self, role, role2):
+        """Find with 'all' attribute enabled."""
+        expected = RoleTracker.find_tracker_roles(
+            [role, role2], pattern=u'test', criteria={'all': True})
+        result = role.find(u'test', all=True)
+        assert_deepequal(expected, result)
+
+    def test_find_all_pkey_only_(self, role, role2):
+        """Find with 'pkey_only' and 'all' options enabled."""
+        expected = RoleTracker.find_tracker_roles(
+            [role, role2],
+            pattern=u'test',
+            criteria={'pkey_only': True, 'all': True})
+        result = role.find(u'test', pkey_only=True, all=True)
+        assert_deepequal(expected, result)
+
+    def test_find_after_attributes_removal(self, role, role2, user, privilege):
+        """Remove attributes set in previous test and run find."""
+        role.remove_privilege(privilege.cn)
+        role2.remove_member(user.uid, member_type='user')
+        expected = RoleTracker.find_tracker_roles(
+            [role, role2], pattern=u'test')
+        result = role.find(u'test')
+        assert_deepequal(expected, result)
+
+    def test_find_by_description(self, role):
+        """Find role by valid description and name."""
+        expected = RoleTracker.find_tracker_roles(
+            [role], pattern=u'test-role1', criteria={'desc': _DESCRIPTION})
+        result = role.find(u'test-role1', description=_DESCRIPTION)
+        assert_deepequal(expected, result)
+
+    def test_find_by_description_false(self, role):
+        """Try to search for role with valid name and random description."""
+        invalid_desc = unicode(uuid4())
+        expected = RoleTracker.find_tracker_roles(
+            [role], pattern=u'test-role1', criteria={'desc': invalid_desc})
+        result = role.find(u'test-role1', description=invalid_desc)
+        assert_deepequal(expected, result)
+
+    def test_find_simlar_after_removal(self, role, role2):
+        """Remove first role and search for roles."""
+        role.ensure_missing()
+        expected = RoleTracker.find_tracker_roles(
+            [role2], pattern=u'test')
+        result = role.find(u'test')
+        assert_deepequal(expected, result)
+
+    def test_find_simlar_after_removal2(self, role2):
+        """Remove also second role and search for roles."""
+        role2.ensure_missing()
+        expected = RoleTracker.find_tracker_roles(
+            [], pattern=u'test')
+        result = role2.find(u'test')
+        assert_deepequal(expected, result)
+
diff --git a/ipatests/test_xmlrpc/tracker/privilege_plugin.py b/ipatests/test_xmlrpc/tracker/privilege_plugin.py
new file mode 100644
index 0000000000000000000000000000000000000000..ba2114e0e54c44b1b4524b76caf8d45c1f8e703d
--- /dev/null
+++ b/ipatests/test_xmlrpc/tracker/privilege_plugin.py
@@ -0,0 +1,96 @@
+#
+# Copyright (C) 2016  FreeIPA Contributors see COPYING for license
+#
+
+from ipalib import api
+from ipapython.dn import DN
+from ipatests.test_xmlrpc import objectclasses
+from ipatests.test_xmlrpc.tracker.base import Tracker
+from ipatests.util import assert_deepequal
+
+
+class PrivilegeTracker(Tracker):
+    """Minimal working Tracker class necessary for Role plugin testing."""
+
+    retrieve_keys = {u'cn', u'member'}
+    retrieve_all_keys = {u'dn', u'objectclass'}
+    create_keys = retrieve_keys | retrieve_all_keys
+
+    def __init__(self, name, **kwargs):
+        super(PrivilegeTracker, self).__init__(default_version=None)
+        self.cn = name
+        self.dn = DN(
+            ('cn', self.cn), api.env.container_privilege, api.env.basedn
+        )
+        self.kwargs = kwargs
+
+    def check_create(self, result, extra_keys=()):
+        """Check privilege-add command result."""
+        if u'description' in result['result']:
+            expected = self.filter_attrs(
+                self.create_keys | set(extra_keys) | {u'description'})
+        else:
+            expected = self.filter_attrs(self.create_keys | set(extra_keys))
+        assert_deepequal(dict(
+            value=self.cn,
+            summary=u'Added privilege "%s"' % self.cn,
+            result=self.filter_attrs(expected),
+            ), result)
+
+    def check_retrieve(self, result, all=False, raw=False):
+        """ Check retrieve-show command result. """
+        if all:
+            keys = self.retrieve_all_keys
+        else:
+            keys = self.retrieve_keys
+        # description is only returned if exists in IPA role
+        if u'description' in result['result']:
+            keys = keys | {u'description'}
+        expected = self.filter_attrs(keys)
+        assert_deepequal(dict(
+            value=self.cn,
+            summary=None,
+            result=expected,
+        ), result)
+
+    def check_delete(self, result):
+        """Check correct behaviour of 'privilege-del' command."""
+        assert_deepequal({
+            'value': [self.cn],
+            'summary': u'Deleted privilege "%s"' % self.cn,
+            'result': {
+                'failed': []
+            },
+        }, result)
+
+    def make_create_command(self, force=None):
+        """Make function that creates a privilege using privilege-add."""
+        return self.make_command('privilege_add', self.cn, **self.kwargs)
+
+    def make_update_command(self, updates):
+        """Make a function that updates a privilege using privilege-mod."""
+        return self.make_command('privilege_mod', self.cn, **updates)
+
+    def make_delete_command(self, force=None):
+        """Make function that deletes a privilege using privilege-del."""
+        return self.make_command('privilege_del', self.cn)
+
+    def make_find_command(self, *args, **kwargs):
+        """ Make function that finds privilege using privilege-find """
+        return self.make_command('privilege_find', *args, **kwargs)
+
+    def make_retrieve_command(self, rights=False, all=False, raw=False):
+        """Make function that retrieves a privilege using privilege-show."""
+        return self.make_command(
+            'privilege_show', self.cn, rights=rights, raw=raw
+        )
+
+    def track_create(self):
+        self.attrs = dict(
+            dn=self.dn,
+            cn=[self.cn],
+            objectclass=objectclasses.privilege,
+            description=[u'Description string'],
+            memberof='defaultpergroup'
+        )
+        self.exists = True
diff --git a/ipatests/test_xmlrpc/tracker/role_plugin.py b/ipatests/test_xmlrpc/tracker/role_plugin.py
new file mode 100644
index 0000000000000000000000000000000000000000..87aa983e8a92072cf3217ccb4b1f727d24867533
--- /dev/null
+++ b/ipatests/test_xmlrpc/tracker/role_plugin.py
@@ -0,0 +1,583 @@
+#
+# Copyright (C) 2016  FreeIPA Contributors see COPYING for license
+#
+
+from ipalib import _, api, errors
+from ipapython.dn import DN
+from ipatests.test_xmlrpc import objectclasses
+from ipatests.test_xmlrpc.tracker.base import Tracker
+from ipatests.util import assert_deepequal
+
+member_types = ('user', 'group', 'host', 'hostgroup', 'service')
+
+
+class RoleTracker(Tracker):
+    """ Class for representing Role's tracker state in the system.
+
+    Tracker keeps track of all changes made on role through its methods.
+    If you update state of the system related to tested role, e.g. remove
+    user or permission from the system, tracker's `attrs` attribute should
+    be updated manually for correct behavior.
+    """
+
+    retrieve_keys = {
+        u'dn', u'cn', u'description', u'privilege',
+        u'memberof_privilege', u'member_group', u'member_host',
+        u'member_hostgroup', u'member_service',
+        u'member_user'
+    }
+    retrieve_all_keys = retrieve_keys | {u'objectclass'}
+    find_keys = {u'dn', u'cn', u'description', u'privilege'}
+    find_member_keys = find_keys | {
+        u'memberof_privilege', u'member_group', u'member_host',
+        u'member_hostgroup', u'member_service', u'member_user'
+    }
+    find_all_keys = retrieve_keys | retrieve_all_keys
+    create_keys = retrieve_keys | {u'objectclass'}
+    update_keys = retrieve_keys - {u'dn'}
+    privilege_keys = retrieve_keys
+    member_keys = retrieve_keys
+    primary_keys = {u'cn', u'dn'}
+
+    def __init__(self, name, **kwargs):
+        super(RoleTracker, self).__init__(default_version=None)
+        self.cn = name
+        self.dn = self._create_dn(self.cn)
+        self.kwargs = kwargs
+
+    def _create_dn(self, cn):
+        return DN(
+            ('cn', cn), api.env.container_rolegroup, api.env.basedn
+        )
+
+    def add_privilege(self, privilege):
+        """Add privilege to the role and update tracker approprietely.
+
+        :param privilege: Name of the privilege to add.
+        """
+        # calling update with None would delete attribute
+        if privilege is not None:
+            self.update_tracker({u'memberof_privilege': privilege})
+        result = self.run_command(
+            'role_add_privilege', self.cn, privilege=privilege)
+        return result
+
+    def check_add_privilege(self, result):
+        """Check that privilege was correctly added to role.
+
+        :param result: Result of add_privilege command.
+        """
+        expected = {
+            'completed': 1,
+            'failed': {
+                'member': {
+                    'privilege': [],
+                },
+            },
+            'result': self.filter_attrs(self.privilege_keys)
+        }
+        assert_deepequal(expected, result)
+
+    def check_add_zero_privilege(self, result):
+        """Check that adding empty privilege does not change role's state.
+
+        :param result: Result of add_privilege command.
+        """
+        expected = {
+            'completed': 0,
+            'failed': {
+                'member': {
+                    'privilege': [],
+                },
+            },
+            'result': self.filter_attrs(self.privilege_keys)
+        }
+        assert_deepequal(expected, result)
+
+    def check_add_duplicate_privilege(self, result, privilege):
+        """Check that adding duplicat privilege will fail.
+
+        :param result: Result of add_privilege command.
+        :param privilege: Name of duplicated privilege.
+        """
+        expected = {
+            'failed': {
+                'member': {
+                    'privilege': [
+                        [privilege, u'This entry is already a member']
+                    ]
+                }
+            },
+            'completed': 0,
+            'result': self.filter_attrs(self.privilege_keys)
+        }
+        assert_deepequal(expected, result)
+
+    def check_handle_nonexistent_privilege(self, result, privilege):
+        """Check that non-existent privilege will not be added.
+
+        :param result: Result of add_privilege command.
+        """
+        expected = {
+            'completed': 0,
+            'failed': {
+                'member': {
+                    'privilege': [[str(privilege), u'privilege not found']]
+                }
+            },
+            'result': self.filter_attrs(self.privilege_keys)
+        }
+        assert_deepequal(expected, result)
+
+    def remove_privilege(self, privilege):
+        """Remove privilege from role and update tracker.
+
+        :param privilege: Name of the privilege to remove.
+        """
+        if privilege is not None:
+            self.update_tracker(
+                {u'memberof_privilege': privilege}, remove=True)
+        result = self.run_command(
+            'role_remove_privilege', self.cn, privilege=privilege)
+        return result
+
+    def check_remove_privilege(self, result):
+        """Check correct execution of role-remove-privilege command.
+
+        :param result: Result of remove_privilege command.
+        """
+        expected = {
+            'completed': 1,
+            'failed': {
+                'member': {
+                    'privilege': [],
+                },
+            },
+            'result': self.filter_attrs(self.privilege_keys)
+        }
+        assert_deepequal(expected, result)
+
+    def check_remove_privilege_false(self, result, privilege):
+        """Check that removing privilege not in role will fail.
+
+        :param result: Result of remove_privilege command.
+        :param privilege: Name of the non-existing privilege.
+        """
+        expected = {
+            'completed': 0,
+            'failed': {
+                'member': {
+                    'privilege': [(u'%s' % privilege,
+                                   u'This entry is not a member')],
+                },
+            },
+            'result': self.filter_attrs(self.privilege_keys)
+        }
+        assert_deepequal(expected, result)
+
+    def check_remove_zero_privilege(self, result):
+        """Check that removing an empty privilege does not change role's state.
+
+        :param result: Result of remove_privilege command.
+        """
+        expected = {
+            'completed': 0,
+            'failed': {
+                'member': {
+                    'privilege': [],
+                },
+            },
+            'result': self.filter_attrs(self.privilege_keys)
+        }
+        assert_deepequal(expected, result)
+
+    def add_member(self, member, member_type):
+        """Add member to this role.
+
+        :param member: Name of the member to add.
+        :param member_type: Type of member to add, can be 'user', 'group',
+                       'host', 'hostgroup' or 'service'.
+        """
+        if member_type not in member_types:
+            raise TypeError("Invalid member type: %s " % member_type)
+        # name that will be used for updating tracker attrs
+        result_key = 'member_%s' % member_type
+        self.update_tracker({result_key: member})
+
+        result = self.run_command(
+            'role_add_member', self.cn, **{member_type: member})
+        return result
+
+    def check_add_member(self, result):
+        """Check that add-member command was successfull.
+
+        :param result: Result of add_member command.
+        """
+        expected = {
+            'failed': {
+                'member': {
+                    'user': [],
+                    'group': [],
+                    'host': [],
+                    'hostgroup': [],
+                    'service': [],
+                },
+            },
+            'completed': 1,
+            'result': self.filter_attrs(self.member_keys)
+        }
+        assert_deepequal(expected, result)
+
+    def check_add_duplicate_member(self, result, member, member_type):
+        """Check that member can not be added to role again.
+
+        :param result: Result of add_member command.
+        :param member: Name of duplicated member.
+        :param member_type: Name of non-existent member.
+        """
+        if member_type not in member_types:
+            raise TypeError("Invalid member type: %s " % member_type)
+        members = {m: [] for m in member_types}
+        members[member_type] = [[member, u'This entry is already a member']]
+        expected = {
+            'failed': {
+                'member': members
+            },
+            'completed': 0,
+            'result': self.filter_attrs(self.member_keys)
+        }
+        assert_deepequal(expected, result)
+
+    def check_add_nonexistent_member(self, result, member, member_type):
+        """Check that it is not possible to add nonexistent member to role.
+
+        :param result: Result of add_member command.
+        :param member: Name of nonexistent member.
+        :param member_type: Type of non-existent member.
+        """
+        if member_type not in member_types:
+            raise TypeError("Invalid member type: %s " % member_type)
+        members = {m: [] for m in member_types}
+        members[member_type] = [[member, u'no such entry']]
+        expected = {
+            'failed': {
+                'member': members
+            },
+            'completed': 0,
+            'result': self.filter_attrs(self.member_keys)
+        }
+        assert_deepequal(expected, result)
+
+    def remove_member(self, member, member_type):
+        """Remove specified member from the role.
+
+        :param name: Name of the member to remove.
+        :param member: Type of member to remove.
+        """
+        if member_type not in member_types:
+            raise TypeError("Invalid member type: %s " % member_type)
+        # name that will be used for updating tracker attrs
+        result_key = 'member_%s' % member_type
+        self.update_tracker({result_key: member}, remove=True)
+        result = self.run_command(
+            'role_remove_member', self.cn, **{member_type: member})
+        return result
+
+    def check_remove_member(self, result):
+        """Check that member was succesfully removed from role.
+
+        :param result: Result of remove_member command.
+        """
+        expected = {
+            'failed': {
+                'member':
+                {
+                    'user': [],
+                    'group': [],
+                    'host': [],
+                    'hostgroup': [],
+                    'service': [],
+                },
+            },
+            'completed': 1,
+            'result': self.filter_attrs(self.member_keys)
+        }
+        assert_deepequal(expected, result)
+
+    def check_remove_member_false(self, result, member, member_type):
+        """Check that removing member that is not in the role will fail.
+
+        :param result: Result of remove_member command
+        :param member: Type of member to remove.
+        :param member_type: Type of non-existent member.
+        """
+        if member_type not in member_types:
+            raise TypeError("Invalid member type: %s " % member_type)
+        members = {m: [] for m in member_types}
+        members[member_type] = [[member, u'This entry is not a member']]
+        expected = {
+            'completed': 0,
+            'failed': {
+                'member': members
+            },
+            'result': self.filter_attrs(self.member_keys)
+        }
+        assert_deepequal(expected, result)
+
+    def update_tracker(self, updates, expected_updates=None, remove=False):
+        """Helper function to update tracker's attributes.
+
+        This method is used to:
+            * add new attribute to self.attrs
+            * set new value or add another value to existing attribute
+            * remove whole attribute by setting dict value to None, '' or u''
+            * remove single value from attribute's list by setting remove=True
+
+        If remove==True and updates contain None, '' or u'', no action
+        is performed.
+        If remove==True, expected_updates parameter is ignored, since one
+        can remove it by simply setting key=None.
+
+        :param updates: Dictionary of attributes that will be stored in list.
+        :param expected_updates: Dictionary of other attributes.
+        :param remove: Specifies whether to remove value of attribute
+                       from list. List will be deleted if empty after removal.
+        """
+        self.ensure_exists()
+        if expected_updates is None:
+            expected_updates = {}
+
+        if 'rename' in updates.keys():
+            self.attrs['cn'] = [updates['rename']]
+            self.attrs['dn'] = self._create_dn(updates['rename'])
+            updates.pop('rename')
+
+        if not remove:
+            for key, value in updates.items():
+                if value is None or value is '' or value is u'':
+                    del self.attrs[key]
+                else:
+                    if type(value) is list:
+                        self.attrs[key] = value
+                    else:
+                        self.attrs[key] = [value]
+            for key, value in expected_updates.items():
+                if value is None or value is '' or value is u'':
+                    del self.attrs[key]
+                else:
+                    self.attrs[key] = value
+        else:
+            for key, value in updates.items():
+                # if attribute or value is not in attrs, nothing happens...
+                try:
+                    self.attrs[key].remove(value)
+                    if not self.attrs[key]:
+                        del self.attrs[key]
+                except (KeyError, ValueError):
+                    pass
+
+    def update(self, updates):
+        """Run update command on role.
+
+        :param updates: Dictionary of attributes to update.
+        """
+        return self.make_update_command(updates)()
+
+    def check_update(self, result, extra_keys=()):
+        """Check correct execution of role_mod command.
+
+        :param result: Result of update command.
+        :param extra_keys: Extra keys expected in update result.
+        """
+        expected = {
+            'value': self.cn,
+            'summary': u'Modified role "%s"' % self.cn,
+            'result': self.filter_attrs(self.update_keys | set(extra_keys))
+        }
+        assert_deepequal(expected, result)
+
+    def check_create(self, result, extra_keys=()):
+        """Check correct behavior of role-add command.
+
+        :param result: Result of create command.
+        :param extra_keys: Extra keys expected in create result.
+        """
+        expected = self.filter_attrs(self.create_keys | set(extra_keys))
+        assert_deepequal({
+            'value': self.cn,
+            'summary': u'Added role "%s"' % self.cn,
+            'result': self.filter_attrs(expected),
+        }, result)
+
+    def check_delete(self, result):
+        """Check correct behaviour of 'role-del' command.
+
+        :param result: Result of delete command.
+        """
+        assert_deepequal({
+            'value': [self.cn],
+            'summary': u'Deleted role "%s"' % self.cn,
+            'result': {
+                'failed': []
+            },
+        }, result)
+
+    def retrieve(self, rights=False, all=False, raw=False):
+        """Retrieve all command.
+
+        :param rights: Display effective rights for attributes.
+        :param all: Show detailed description.
+        :param raw: Output in raw format.
+        """
+        result = self.make_retrieve_command(rights=rights, all=all, raw=raw)()
+        return result
+
+    def check_retrieve(self, result, rights=False, all=False, raw=False):
+        """Check correct behavior of role-show command.
+
+        :param result: Result of retrieve command.
+        :param rights: Display effective rights for attributes.
+        :param all: Show detailed description.
+        :param raw: Output in raw format.
+        """
+        if all:
+            keys = self.retrieve_all_keys
+        else:
+            keys = self.retrieve_keys
+        expected = {
+            'value': self.cn,
+            'summary': None,
+            'result': self.filter_attrs(keys),
+        }
+        assert_deepequal(expected, result)
+
+    def find(self, *args, **kwargs):
+        """Perform role-find command on roles.
+
+        :param pattern: Role's name pattern we are searching for.
+        :param criteria: Other find criteria.
+        """
+        return self.make_find_command(*args, **kwargs)()
+
+    @classmethod
+    def find_tracker_roles(cls, roles, pattern='', criteria={}):
+        """Perform find command on given roles and return result.
+
+        Criteria can be dictionary or kwargs maybe
+
+        :param roles: List of RoleTracker objects we want to perform search on.
+        :param patter: Filter by name.
+        :param criteria: Other possible criteria, e.g. description.
+        """
+        matching_roles = []
+        keys = cls.find_keys
+
+        for role in roles:
+            if not role.exists:
+                continue
+            # set initially True, change if some attribute do not match
+            full_match = True
+            if pattern in role.cn:
+                for key, value in criteria.items():
+                    if key == 'name':
+                        if value not in role.attrs['cn']:
+                            full_match = False
+                            break
+                    elif key == 'desc':
+                        if value not in role.attrs['description']:
+                            full_match = False
+                            break
+                    elif key == 'pkey_only':
+                        if value is True:
+                            keys = cls.primary_keys
+                    elif key == 'all':
+                        if value is True:
+                            if 'pkey_only' not in criteria.keys():
+                                keys = cls.find_all_keys
+                            elif criteria['pkey_only'] is False:
+                                keys = cls.find_all_keys
+                    elif key == 'no_members':
+                        if value is False:
+                            if 'pkey_only' not in criteria.keys():
+                                keys = cls.find_member_keys
+                            elif criteria['pkey_only'] is False:
+                                keys = cls.find_member_keys
+                    else:
+                        raise Exception('Invalid criterion')
+            else:
+                full_match = False
+            # role matched, add it to final result
+            if full_match:
+                matching_roles.append(role)
+
+        # form a final response
+        expected = {}
+        results = []
+        expected[u'count'] = len(matching_roles)
+        expected[u'truncated'] = False
+        noun = u'role' if expected['count'] == 1 else u'roles'
+        expected[u'summary'] = _('%(count)s %(noun)s matched') % {
+            'count': str(len(matching_roles)), 'noun': noun}
+        for role in matching_roles:
+            results.append(role.filter_attrs(keys))
+        expected[u'result'] = results
+        return expected
+
+    def make_create_command(self, force=False):
+        """Make function that creates a role using role-add."""
+        return self.make_command('role_add', self.cn, **self.kwargs)
+
+    def make_update_command(self, updates):
+        """Make function that updates role with updates."""
+        return self.make_command('role_mod', self.cn, **updates)
+
+    def make_delete_command(self, *args, **kwargs):
+        """Make function that deletes a role using role-del."""
+        return self.make_command('role_del', self.cn, *args, **kwargs)
+
+    def make_find_command(self, *args, **kwargs):
+        """ Make function that finds role using role-find """
+        return self.make_command('role_find', *args, **kwargs)
+
+    def make_retrieve_command(self, rights=False, all=False, raw=False):
+        """Make function that retrieves a role using role-show."""
+        return self.make_command(
+            'role_show', self.cn, rights=rights, all=all, raw=raw)
+
+    def track_create(self):
+        """Initialize tracker instance."""
+        self.attrs = {
+            'dn': self.dn,
+            'cn': [self.cn],
+            'memberof': 'defaultpergroup',
+            'objectclass': objectclasses.role,
+        }
+        for key in self.kwargs:
+            if not type(self.kwargs[key]) is list:
+                self.attrs[key] = [self.kwargs[key]]
+            else:
+                self.attrs[key] = self.kwargs[key]
+        self.exists = True
+
+    def make_fixture(self, request):
+        """Make a pytest fixture for this tracker
+
+        The fixture ensures the plugin entry does not exist before
+        and after the tests that use it.
+        """
+        try:
+            self.make_delete_command()()
+        except errors.NotFound:
+            pass
+
+        def cleanup():
+            existed = self.exists
+            try:
+                self.make_delete_command()()
+            except errors.NotFound:
+                if existed:
+                    raise
+            self.exists = False
+
+        request.addfinalizer(cleanup)
+        return self
-- 
2.7.4

-- 
Manage your subscription for the Freeipa-devel mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-devel
Contribute to FreeIPA: http://www.freeipa.org/page/Contribute/Code

Reply via email to