Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities
IMO, if that's impossible to provide Users pagination/indexing for LDAP from technical point of view, we'd better revise the current UI of Identity-Users, since most likely production-grade OpenStack installations will have LDAP instead of MySQL storage. So I'm for gathering more operators/deployers feedback on the pagination (Users panel particularly). I'm inclined to revising existing UI not because I think the Keystone community is adamant in doing things the way they're going to, but because LDAP storage backend seems to be unavoidable for large OpenStack deployments and Horizon shouldn't ignore this fact (what's the point in having fancy UI when it's broken for real-world installations?). Of course, 'unavoidable' thing is open to discussion. On Tue, Aug 18, 2015 at 6:30 AM Adam Young ayo...@redhat.com wrote: On 08/17/2015 09:53 PM, David Lyle wrote: I think we've conveniently been led off track here. The original request/subject was regarding pagination of projects in the v3 API. Since this is purely a keystone construct it seems implausible to me that ldap or the IdP of choice would be limiting the ability to return a paginated list of all projects. Or groups or domains or roles for that matter. Yeah, SQL can support it. LDAP assignment can't. But that is not going to have a long life. With Hierarchical projects, we'll probably also have to keep nesting in mind for how we display a project list: do we always show a flat list, or is a tree closer to what users expect? Both are going to work poorly for some deployments and work well for others. There is no reason to punt on pagination across the API for one resource type, which actually would also work with select backends. Give me something that I can exhaustively list in the API I can build from. David On Aug 17, 2015 10:53 AM, Fox, Kevin M kevin@pnnl.gov wrote: 1. yes, but probably only if its a short list. It may be feasible to show it only if there are 5 or less pages, and maybe just load all pages of data and paginate it on the client. If too big, ask the user to refine their search? Or always paginate to 5, and then the 6th page have a page requesting further refinement? 2. Not sure what the difference between searching and filtering is in this context? something like facets? If so, probably the 5 or less thing would work here too. 3. Yes, but again, probably within a smaller set of pages? Thanks, Kevin From: Kruithof, Piet [pieter.c.kruithof...@hp.com] Sent: Sunday, August 16, 2015 9:41 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities I like Michael’s response because it moved the thread towards identifying actual user needs before digging into the technical feasibility. IMHO, it would be helpful to have a few people on the list answer his questions: 1 - Do users want to page through search results? 2 - Do users want to page through filter results? (do they use filter results?) 3 - If they want to page, do they want to be able to go back a page and/or know their current page? I understand that even if we answer “yes” to all three questions that there could be issues around implementation, but at least we’ll know a gap exists. Piet Kruithof Sr UX Architect, HP Helion Cloud PTL, OpenStack UX project For every complex problem, there is a solution that is simple, neat and wrong.” H L Menken __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities
How about a settings.py switch that would simply allow the deployer to entirely disable the list users functionality? Best, -jay On 08/18/2015 12:52 PM, Timur Sufiev wrote: IMO, if that's impossible to provide Users pagination/indexing for LDAP from technical point of view, we'd better revise the current UI of Identity-Users, since most likely production-grade OpenStack installations will have LDAP instead of MySQL storage. So I'm for gathering more operators/deployers feedback on the pagination (Users panel particularly). I'm inclined to revising existing UI not because I think the Keystone community is adamant in doing things the way they're going to, but because LDAP storage backend seems to be unavoidable for large OpenStack deployments and Horizon shouldn't ignore this fact (what's the point in having fancy UI when it's broken for real-world installations?). Of course, 'unavoidable' thing is open to discussion. On Tue, Aug 18, 2015 at 6:30 AM Adam Young ayo...@redhat.com mailto:ayo...@redhat.com wrote: On 08/17/2015 09:53 PM, David Lyle wrote: I think we've conveniently been led off track here. The original request/subject was regarding pagination of projects in the v3 API. Since this is purely a keystone construct it seems implausible to me that ldap or the IdP of choice would be limiting the ability to return a paginated list of all projects. Or groups or domains or roles for that matter. Yeah, SQL can support it. LDAP assignment can't. But that is not going to have a long life. With Hierarchical projects, we'll probably also have to keep nesting in mind for how we display a project list: do we always show a flat list, or is a tree closer to what users expect? Both are going to work poorly for some deployments and work well for others. There is no reason to punt on pagination across the API for one resource type, which actually would also work with select backends. Give me something that I can exhaustively list in the API I can build from. David On Aug 17, 2015 10:53 AM, Fox, Kevin M kevin@pnnl.gov mailto:kevin@pnnl.gov wrote: 1. yes, but probably only if its a short list. It may be feasible to show it only if there are 5 or less pages, and maybe just load all pages of data and paginate it on the client. If too big, ask the user to refine their search? Or always paginate to 5, and then the 6th page have a page requesting further refinement? 2. Not sure what the difference between searching and filtering is in this context? something like facets? If so, probably the 5 or less thing would work here too. 3. Yes, but again, probably within a smaller set of pages? Thanks, Kevin From: Kruithof, Piet [pieter.c.kruithof...@hp.com mailto:pieter.c.kruithof...@hp.com] Sent: Sunday, August 16, 2015 9:41 AM To: openstack-dev@lists.openstack.org mailto:openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities I like Michael’s response because it moved the thread towards identifying actual user needs before digging into the technical feasibility. IMHO, it would be helpful to have a few people on the list answer his questions: 1 - Do users want to page through search results? 2 - Do users want to page through filter results? (do they use filter results?) 3 - If they want to page, do they want to be able to go back a page and/or know their current page? I understand that even if we answer “yes” to all three questions that there could be issues around implementation, but at least we’ll know a gap exists. Piet Kruithof Sr UX Architect, HP Helion Cloud PTL, OpenStack UX project For every complex problem, there is a solution that is simple, neat and wrong.” H L Menken __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities
On 08/15/2015 01:15 PM, Michael Krotscheck wrote: On Fri, Aug 14, 2015 at 2:26 PM Adam Young ayo...@redhat.com mailto:ayo...@redhat.com wrote: On 08/14/2015 12:43 PM, Michael Krotscheck wrote: 1- Do users want to page through search results? Does not matter: in Federation, the User list is not available. Let's back up here for a sec: A user wants to page a list of data. This is something horizon needs, traditionally relying on keystone, and now keystone has broken backwards compatibility for horizon because of one use case, without taking responsibility for it and providing (with code) a good alternative. Furthermore, you and your team are saying You should go use a different service that's better at this, which is basically saying We live in this silo, we don't have to care about other silo's. NO. What we are saying is you are asking for infromation in a away that the technoliogies that OpenStack is pulling together cannot support. You broke backwards compatibility. It's your responsibility to address it. No. The world moved on. Keystone pagination in SQL is trivial. It is also useless. LDAP does not support paging. The majority of the deployments us LDAP for the back end. In as SAML/OpenID deployment there is no way to list users. The other argument I'm hearing here is that keystone is responsible for authentication and authorization, but not user management. I actually agree with this, but nobody's started a user management service and/or its delegation plugins, so now we have a rather large hole in horizon's features, late in a release cycle, and nobody has the resources to address it. What do you propose to do about it? We don't maintain MySQL, either. Use an external tool for user management. There are numerous, and OpenStack can integrate with them via LDAP or SAML. Other technologies coming soon, too. Michael __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities
I think we've conveniently been led off track here. The original request/subject was regarding pagination of projects in the v3 API. Since this is purely a keystone construct it seems implausible to me that ldap or the IdP of choice would be limiting the ability to return a paginated list of all projects. Or groups or domains or roles for that matter. There is no reason to punt on pagination across the API for one resource type, which actually would also work with select backends. Give me something that I can exhaustively list in the API I can build from. David On Aug 17, 2015 10:53 AM, Fox, Kevin M kevin@pnnl.gov wrote: 1. yes, but probably only if its a short list. It may be feasible to show it only if there are 5 or less pages, and maybe just load all pages of data and paginate it on the client. If too big, ask the user to refine their search? Or always paginate to 5, and then the 6th page have a page requesting further refinement? 2. Not sure what the difference between searching and filtering is in this context? something like facets? If so, probably the 5 or less thing would work here too. 3. Yes, but again, probably within a smaller set of pages? Thanks, Kevin From: Kruithof, Piet [pieter.c.kruithof...@hp.com] Sent: Sunday, August 16, 2015 9:41 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities I like Michael’s response because it moved the thread towards identifying actual user needs before digging into the technical feasibility. IMHO, it would be helpful to have a few people on the list answer his questions: 1 - Do users want to page through search results? 2 - Do users want to page through filter results? (do they use filter results?) 3 - If they want to page, do they want to be able to go back a page and/or know their current page? I understand that even if we answer “yes” to all three questions that there could be issues around implementation, but at least we’ll know a gap exists. Piet Kruithof Sr UX Architect, HP Helion Cloud PTL, OpenStack UX project For every complex problem, there is a solution that is simple, neat and wrong.” H L Menken __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities
On 08/17/2015 09:53 PM, David Lyle wrote: I think we've conveniently been led off track here. The original request/subject was regarding pagination of projects in the v3 API. Since this is purely a keystone construct it seems implausible to me that ldap or the IdP of choice would be limiting the ability to return a paginated list of all projects. Or groups or domains or roles for that matter. Yeah, SQL can support it. LDAP assignment can't. But that is not going to have a long life. With Hierarchical projects, we'll probably also have to keep nesting in mind for how we display a project list: do we always show a flat list, or is a tree closer to what users expect? Both are going to work poorly for some deployments and work well for others. There is no reason to punt on pagination across the API for one resource type, which actually would also work with select backends. Give me something that I can exhaustively list in the API I can build from. David On Aug 17, 2015 10:53 AM, Fox, Kevin M kevin@pnnl.gov mailto:kevin@pnnl.gov wrote: 1. yes, but probably only if its a short list. It may be feasible to show it only if there are 5 or less pages, and maybe just load all pages of data and paginate it on the client. If too big, ask the user to refine their search? Or always paginate to 5, and then the 6th page have a page requesting further refinement? 2. Not sure what the difference between searching and filtering is in this context? something like facets? If so, probably the 5 or less thing would work here too. 3. Yes, but again, probably within a smaller set of pages? Thanks, Kevin From: Kruithof, Piet [pieter.c.kruithof...@hp.com mailto:pieter.c.kruithof...@hp.com] Sent: Sunday, August 16, 2015 9:41 AM To: openstack-dev@lists.openstack.org mailto:openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities I like Michael’s response because it moved the thread towards identifying actual user needs before digging into the technical feasibility. IMHO, it would be helpful to have a few people on the list answer his questions: 1 - Do users want to page through search results? 2 - Do users want to page through filter results? (do they use filter results?) 3 - If they want to page, do they want to be able to go back a page and/or know their current page? I understand that even if we answer “yes” to all three questions that there could be issues around implementation, but at least we’ll know a gap exists. Piet Kruithof Sr UX Architect, HP Helion Cloud PTL, OpenStack UX project For every complex problem, there is a solution that is simple, neat and wrong.” H L Menken __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities
Paging support for certain Keystone constructs seems reasonable to me (roles, domains and projects). But as Adam Young just wrote in [0], though possible for our SQL backends, why? Do you really expect deployments to have 100s of roles/domains/projects? IMO groups probably should not have paging, as with users, they can be owned by another identity source as well. A minor note about paging through LDAP; in my experience LDAP paging usually requires an administrator authenticate/bind, which most users likely are not, and still uses it's own server side settings to determine how many results to return. [0] http://openstack.markmail.org/search/?q=keystone#query:keystone%20list%3Aorg.openstack.lists.openstack-dev%20order%3Adate-backward +page:1+mid:kdw5wf5tcc6ad36j+state:results Thanks, Steve Martinelli OpenStack Keystone Core From: David Lyle dkly...@gmail.com To: OpenStack Development Mailing List openstack-dev@lists.openstack.org Date: 2015/08/17 09:54 PM Subject:Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities I think we've conveniently been led off track here. The original request/subject was regarding pagination of projects in the v3 API. Since this is purely a keystone construct it seems implausible to me that ldap or the IdP of choice would be limiting the ability to return a paginated list of all projects. Or groups or domains or roles for that matter. There is no reason to punt on pagination across the API for one resource type, which actually would also work with select backends. Give me something that I can exhaustively list in the API I can build from. David On Aug 17, 2015 10:53 AM, Fox, Kevin M kevin@pnnl.gov wrote: 1. yes, but probably only if its a short list. It may be feasible to show it only if there are 5 or less pages, and maybe just load all pages of data and paginate it on the client. If too big, ask the user to refine their search? Or always paginate to 5, and then the 6th page have a page requesting further refinement? 2. Not sure what the difference between searching and filtering is in this context? something like facets? If so, probably the 5 or less thing would work here too. 3. Yes, but again, probably within a smaller set of pages? Thanks, Kevin From: Kruithof, Piet [pieter.c.kruithof...@hp.com] Sent: Sunday, August 16, 2015 9:41 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities I like Michael’s response because it moved the thread towards identifying actual user needs before digging into the technical feasibility. IMHO, it would be helpful to have a few people on the list answer his questions: 1 - Do users want to page through search results? 2 - Do users want to page through filter results? (do they use filter results?) 3 - If they want to page, do they want to be able to go back a page and/or know their current page? I understand that even if we answer “yes” to all three questions that there could be issues around implementation, but at least we’ll know a gap exists. Piet Kruithof Sr UX Architect, HP Helion Cloud PTL, OpenStack UX project For every complex problem, there is a solution that is simple, neat and wrong.” H L Menken __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities
1. yes, but probably only if its a short list. It may be feasible to show it only if there are 5 or less pages, and maybe just load all pages of data and paginate it on the client. If too big, ask the user to refine their search? Or always paginate to 5, and then the 6th page have a page requesting further refinement? 2. Not sure what the difference between searching and filtering is in this context? something like facets? If so, probably the 5 or less thing would work here too. 3. Yes, but again, probably within a smaller set of pages? Thanks, Kevin From: Kruithof, Piet [pieter.c.kruithof...@hp.com] Sent: Sunday, August 16, 2015 9:41 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities I like Michael’s response because it moved the thread towards identifying actual user needs before digging into the technical feasibility. IMHO, it would be helpful to have a few people on the list answer his questions: 1 - Do users want to page through search results? 2 - Do users want to page through filter results? (do they use filter results?) 3 - If they want to page, do they want to be able to go back a page and/or know their current page? I understand that even if we answer “yes” to all three questions that there could be issues around implementation, but at least we’ll know a gap exists. Piet Kruithof Sr UX Architect, HP Helion Cloud PTL, OpenStack UX project For every complex problem, there is a solution that is simple, neat and wrong.” H L Menken __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities
On Sat, Aug 15, 2015 at 11:02 AM Morgan Fainberg morgan.fainb...@gmail.com wrote: Please do not construe a major api change as backwards incompatible. This pagination was never supported in v3 properly/at all. Sure, let's argue semantics! That's a useful path forward :). As we move towards tools like FreeIPA Or something similar, we will be addressing the gap. Is there a cross-project spec with assigned resources? I'd love to comment on it. I also assert horizon should not be managing users in the same way Keystone should not be managing users. Horizon should show what users have access to openstack (and this *can* be paginated) and allow for searching for a user that is visible to keystone but does not have access to openstack so that they can be grants access. Is there a horizon spec that defines this? I'd love to comment on it. The user management service would be FreeIPA in my previous example (or AD in the environments that deploy it, etc). 1: you do not get to list every user visible to keystone via v3. This is where the problems lie. 2: you can search for users that are visible but do not have an active assignment (this may need some work - and is a reasonable ask). 2a: if the list of users is small the filter/search could return all users 3: you can list users with an active assignment (using the assignment apis) and this *can* be paginated (if it does not already support pagination) 4: (future view) leverage an already existing open source solution for user management such as FreeIPA and remove/deprecate the sql-based data store that is missing basic user management features Hey, all of the above looks like a spec! Neat! You should propose it! Michael Code Trumps Conversation __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities
On Aug 16, 2015, at 08:02, Michael Krotscheck krotsch...@gmail.com wrote: On Sat, Aug 15, 2015 at 11:02 AM Morgan Fainberg morgan.fainb...@gmail.com wrote: Please do not construe a major api change as backwards incompatible. This pagination was never supported in v3 properly/at all. Sure, let's argue semantics! That's a useful path forward :). By this logic every version of every API must be compatible. This is not semantics. As we move towards tools like FreeIPA Or something similar, we will be addressing the gap. Is there a cross-project spec with assigned resources? I'd love to comment on it. I also assert horizon should not be managing users in the same way Keystone should not be managing users. Horizon should show what users have access to openstack (and this *can* be paginated) and allow for searching for a user that is visible to keystone but does not have access to openstack so that they can be grants access. Is there a horizon spec that defines this? I'd love to comment on it. The user management service would be FreeIPA in my previous example (or AD in the environments that deploy it, etc). 1: you do not get to list every user visible to keystone via v3. This is where the problems lie. 2: you can search for users that are visible but do not have an active assignment (this may need some work - and is a reasonable ask). 2a: if the list of users is small the filter/search could return all users 3: you can list users with an active assignment (using the assignment apis) and this *can* be paginated (if it does not already support pagination) 4: (future view) leverage an already existing open source solution for user management such as FreeIPA and remove/deprecate the sql-based data store that is missing basic user management features Hey, all of the above looks like a spec! Neat! You should propose it! Michael Code Trumps Conversation __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities
On Aug 15, 2015, at 10:15, Michael Krotscheck krotsch...@gmail.com wrote: On Fri, Aug 14, 2015 at 2:26 PM Adam Young ayo...@redhat.com wrote: On 08/14/2015 12:43 PM, Michael Krotscheck wrote: 1- Do users want to page through search results? Does not matter: in Federation, the User list is not available. Let's back up here for a sec: A user wants to page a list of data. This is something horizon needs, traditionally relying on keystone, and now keystone has broken backwards compatibility for horizon because of one use case, without taking responsibility for it and providing (with code) a good alternative. Furthermore, you and your team are saying You should go use a different service that's better at this, which is basically saying We live in this silo, we don't have to care about other silo's. You broke backwards compatibility. It's your responsibility to address it. Please do not construe a major api change as backwards incompatible. This pagination was never supported in v3 properly/at all. The v3 api was never claimed to be backwards compatible; this is a major api version. I am offering a different API that gets the information you need and saying that the request for pagination of every single possible user (the API that people use in v2) is not what we want to maintain in v3 because we are working to remove the need for the badly implemented (and not frequently used) store for user data. V2 will not stop it's support (except that it is frozen and will be deprecated in the near future). And v2 will pretty much just fail at pagination with ldap, but this is a long running behavior. The other argument I'm hearing here is that keystone is responsible for authentication and authorization, but not user management. I actually agree with this, but nobody's started a user management service and/or its delegation plugins, so now we have a rather large hole in horizon's features, late in a release cycle, and nobody has the resources to address it. What do you propose to do about it? As we move towards tools like FreeIPA Or something similar, we will be addressing the gap. I also assert horizon should not be managing users in the same way Keystone should not be managing users. Horizon should show what users have access to openstack (and this *can* be paginated) and allow for searching for a user that is visible to keystone but does not have access to openstack so that they can be grants access. The user management service would be FreeIPA in my previous example (or AD in the environments that deploy it, etc). So to distill out what we can do (and what is viable without having wilding different workflows based upon implementation details) and solve the immediate ask for pagination is: 1: you do not get to list every user visible to keystone via v3. This is where the problems lie. 2: you can search for users that are visible but do not have an active assignment (this may need some work - and is a reasonable ask). 2a: if the list of users is small the filter/search could return all users 3: you can list users with an active assignment (using the assignment apis) and this *can* be paginated (if it does not already support pagination) 4: (future view) leverage an already existing open source solution for user management such as FreeIPA and remove/deprecate the sql-based data store that is missing basic user management features Alternative to 4: find stake holders who want to take on the risk and effort of maintaining and supporting a full fledged identity provider and implement it in the SQL driver. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities
On Aug 15, 2015, at 10:58, Morgan Fainberg morgan.fainb...@gmail.com wrote: On Aug 15, 2015, at 10:15, Michael Krotscheck krotsch...@gmail.com wrote: On Fri, Aug 14, 2015 at 2:26 PM Adam Young ayo...@redhat.com wrote: On 08/14/2015 12:43 PM, Michael Krotscheck wrote: 1- Do users want to page through search results? Does not matter: in Federation, the User list is not available. Let's back up here for a sec: A user wants to page a list of data. This is something horizon needs, traditionally relying on keystone, and now keystone has broken backwards compatibility for horizon because of one use case, without taking responsibility for it and providing (with code) a good alternative. Furthermore, you and your team are saying You should go use a different service that's better at this, which is basically saying We live in this silo, we don't have to care about other silo's. You broke backwards compatibility. It's your responsibility to address it. Please do not construe a major api change as backwards incompatible. This pagination was never supported in v3 properly/at all. The v3 api was never claimed to be backwards compatible; this is a major api version. I am offering a different API that gets the information you need and saying that the request for pagination of every single possible user (the API that people use in v2) is not what we want to maintain in v3 because we are working to remove the need for the badly implemented (and not frequently used) store for user data. V2 will not stop it's support (except that it is frozen and will be deprecated in the near future). And v2 will pretty much just fail at pagination with ldap, but this is a long running behavior. The other argument I'm hearing here is that keystone is responsible for authentication and authorization, but not user management. I actually agree with this, but nobody's started a user management service and/or its delegation plugins, so now we have a rather large hole in horizon's features, late in a release cycle, and nobody has the resources to address it. What do you propose to do about it? As we move towards tools like FreeIPA Or something similar, we will be addressing the gap. I also assert horizon should not be managing users in the same way Keystone should not be managing users. Horizon should show what users have access to openstack (and this *can* be paginated) and allow for searching for a user that is visible to keystone but does not have access to openstack so that they can be grants access. The user management service would be FreeIPA in my previous example (or AD in the environments that deploy it, etc). So to distill out what we can do (and what is viable without having wilding different workflows based upon implementation details) and solve the immediate ask for pagination is: 1: you do not get to list every user visible to keystone via v3. This is where the problems lie. 2: you can search for users that are visible but do not have an active assignment (this may need some work - and is a reasonable ask). 2a: if the list of users is small the filter/search could return all users 3: you can list users with an active assignment (using the assignment apis) and this *can* be paginated (if it does not already support pagination) 4: (future view) leverage an already existing open source solution for user management such as FreeIPA and remove/deprecate the sql-based data store that is missing basic user management features Alternative to 4: find stake holders who want to take on the risk and effort of maintaining and supporting a full fledged identity provider and implement it in the SQL driver. Let me add that the above list does not differ from what I have already said in this thread previously. I have just broken it out explicitly so it is easier to see. I am trying to offer an alternative that meets the ask of the other teams without just saying no and shooting the ask down. We have some legitimate reasons for the choices that we have made and are working towards a solution that meets the needs of the community of developers and well as the real operational asks we consistently get from operators/deployers. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities
On Fri, Aug 14, 2015 at 2:26 PM Adam Young ayo...@redhat.com wrote: On 08/14/2015 12:43 PM, Michael Krotscheck wrote: 1- Do users want to page through search results? Does not matter: in Federation, the User list is not available. Let's back up here for a sec: A user wants to page a list of data. This is something horizon needs, traditionally relying on keystone, and now keystone has broken backwards compatibility for horizon because of one use case, without taking responsibility for it and providing (with code) a good alternative. Furthermore, you and your team are saying You should go use a different service that's better at this, which is basically saying We live in this silo, we don't have to care about other silo's. You broke backwards compatibility. It's your responsibility to address it. The other argument I'm hearing here is that keystone is responsible for authentication and authorization, but not user management. I actually agree with this, but nobody's started a user management service and/or its delegation plugins, so now we have a rather large hole in horizon's features, late in a release cycle, and nobody has the resources to address it. What do you propose to do about it? Michael __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities
I don't have a horse in the What should keystone support race. I do, however, need to point out that any UX argument made about how a UI should work should, at this point, ask the OpenStack UX program for help! Thus I've changed the topic of this email to make sure Piet and the UX teams get a chance to respond. It feels like we have four UX assumptions, which I feel should be converted into questions: 1- Do users want to page through search results? 2- Do users want to page through filter results? (do they use filter results?) 3- If they want to page, do they want to be able to go back a page and/or know their current page? 4- How much do users care about small data inconsistencies (i.e. ordering of record sets changing from page-to-page). Also, from personal experience, it is impossible to make a search implementation that users, especially enterprise users, trust. I personally blame Sharepoint for that. Michael On Fri, Aug 14, 2015 at 6:17 AM Morgan Fainberg morgan.fainb...@gmail.com wrote: As a quick note the api-ref you are linking to has some gaps/has not been kept in sync with the official api specifications. The official API specification is located at http://specs.openstack.org/openstack/keystone-specs/ (v2 and v3 sections at the top) and there is a known open bug to work with the docs team to get this in sync (somehow). Unfortunately there are a number of cases especially with the identity backend where pagination just does not work (or works completely unreliably) such as utilizing the ldap driver. Either a cursor must be maintained (problematic in REST) or the results could be wildly different every single request meaning each page is not really guaranteed to be the next page it could be the same/show inconsistent results. The second issue is that the pagination is not a good UX even where it works - the simple question is: if you can filter the results how many pages deep do you go before refining the query; think of your use of search engines. In light of these issues Keystone has opted for a filter / limit (config). If the results exceed the limit there is a truncation that occurs and it is recommended extra filtering be applied to reduce the total number of results. This discussion has gone around a few times, pagination in keystone is not currently on the roadmap. In addition to the above doc bug, we should work to better socialize this filter-over-paginate methodology. --Morgan Sent via mobile On Aug 14, 2015, at 05:46, Timur Sufiev tsuf...@mirantis.com wrote: Hello, Keystone folks! I've just discovered an unfortunate fact that Horizon pagination for Tenants/Projects list that worked with Keystone v2 doesn't work with Keysone v3 anymore - its API call simply lacks the 'marker' and 'limit' parameters [1] that Horizon is relying upon. Meanwhile having looked through [2] and [3] I'm a bit confused: while Keystone v3 API states it supports [2] pagination for every kind of entities (by using 'page' and 'per_page' parameters), the related blueprint [3] is not yet approved, meaning that most likely the API implementation did not make it into existing Keystone codebase. So I wonder whether there are some plans to implement pagination for Keystone API calls that list its entities? P.S. I'm aware of SearchLight project that tries to help Horizon with other OpenStack APIs (part of its mission), what I'm trying to understand here is are we going to have some fallback pagination mechanism for Horizon's Identity in a short-term/mid-term perspective. [1] http://developer.openstack.org/api-ref-identity-admin-v2.html [2] http://developer.openstack.org/api-ref-identity-v3.html [3] https://blueprints.launchpad.net/keystone/+spec/pagination __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities
On 08/14/2015 12:43 PM, Michael Krotscheck wrote: 1- Do users want to page through search results? Does not matter: in Federation, the User list is not available. 2- Do users want to page through filter results? (do they use filter results?) This is the only practical tool available for LDAP. 3- If they want to page, do they want to be able to go back a page and/or know their current page? 4- How much do users care about small data inconsistencies (i.e. ordering of record sets changing from page-to-page). So, yeah, we could support paging for SQL. That is becoming a repository for service users only. For the use cases requested, we do not have the ability to implement. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities
On Aug 14, 2015, at 15:10, Jay Pipes jaypi...@gmail.com wrote: On 08/14/2015 05:24 PM, Adam Young wrote: On 08/14/2015 12:43 PM, Michael Krotscheck wrote: 1- Do users want to page through search results? Does not matter: in Federation, the User list is not available. OK, so hobble the entire REST API for the deficiencies/architecture/reality of a single authentication/identity strategy. 2- Do users want to page through filter results? (do they use filter results?) This is the only practical tool available for LDAP. Again, hobble the entire API for the deficiencies and anachronisms of a single identity driver. The SQL driver is at best a toy and should go away. We are working diligently to provide the best solution for the small and large deployments and provide features we are regularly asked for (password lifecycle, complexity, better user management, etc). Again I will reiterate, asking for every user that could have an assignment is absurd. You should search for specific users if you need for find a user without an assignment (they can't access keystone or auth or act on OpenStack anyway). You should look at active assignments and we can implement pagination for that. It wont be a /users API call. 3- If they want to page, do they want to be able to go back a page and/or know their current page? 4- How much do users care about small data inconsistencies (i.e. ordering of record sets changing from page-to-page). So, yeah, we could support paging for SQL. That would be great. Double bonus if this pagination API followed the examples of all the other OpenStack API services and didn't invent its own terms (page, per_page...). I really do not want implementation details to be the cause of a massive UX change. Lets avoid doing that yet again in OpenStack. Asking horizon to have two completely different mechanisms based on what backend is following a poor design pattern we keep falling into where we expect the user to figure out/know what is different between deployments. That is becoming a repository for service users only. For the use cases requested, we do not have the ability to implement. Sure, but what you[1] *do* have the ability to implement is a capabilities discovery REST API that would allow tools like Horizon to determine if the only option available for them would be a filtering API, with no pagination capabilities. It would be super awesome if Keystone had such a capabilities discovery API. Best, -jay [1] I don't mean *you* personally, Adam :) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities
On 08/14/2015 06:30 PM, Morgan Fainberg wrote: On Aug 14, 2015, at 15:10, Jay Pipes jaypi...@gmail.com wrote: On 08/14/2015 05:24 PM, Adam Young wrote: On 08/14/2015 12:43 PM, Michael Krotscheck wrote: 1- Do users want to page through search results? Does not matter: in Federation, the User list is not available. OK, so hobble the entire REST API for the deficiencies/architecture/reality of a single authentication/identity strategy. 2- Do users want to page through filter results? (do they use filter results?) This is the only practical tool available for LDAP. Again, hobble the entire API for the deficiencies and anachronisms of a single identity driver. The SQL driver is at best a toy and should go away. And yet it is in deployments all over the world. People said the same about MySQL. It was a toy and should go away. And yet, here we are, 15 years later, and MySQL is running in some of the world's biggest and most complex web applications. Asking for something to go away because you consider it a toy (a toy with better capabilities than other things, I might add) doesn't just make it so... If anything, we should tell the anachronistic ActiveDirectory and LDAP solutions to go away ;) We are working diligently to provide the best solution for the small and large deployments and provide features we are regularly asked for (password lifecycle, complexity, better user management, etc). Aren't we talking about better user management here? Again I will reiterate, asking for every user that could have an assignment is absurd. Nobody is asking for that. We are asking for list me the first page of users in the system, ordered by their ID (or email, or whatever). This is hardly absurd. In fact, it's how all other OpenStack API services expose pagination functionality. And it is complementary to things like Searchlight, not anathema to it. You should search for specific users if you need for find a user without an assignment (they can't access keystone or auth or act on OpenStack anyway). You should look at active assignments and we can implement pagination for that. Where are you coming up with this find a user without an assignment use case? I've yet to hear anyone talking about this. The only use case that has been discussed is the (very common) one that Horizon needs to display a page of user account information, sorted by some key and filtered as needed. It wont be a /users API call. Why not? 3- If they want to page, do they want to be able to go back a page and/or know their current page? 4- How much do users care about small data inconsistencies (i.e. ordering of record sets changing from page-to-page). So, yeah, we could support paging for SQL. That would be great. Double bonus if this pagination API followed the examples of all the other OpenStack API services and didn't invent its own terms (page, per_page...). I really do not want implementation details to be the cause of a massive UX change. That is precisely the situation that Horizon finds itself in today: implementation details of Keystone's backend drivers causing Horizon to need to deal with Keystone like it's a special unicorn. Lets avoid doing that yet again in OpenStack. Asking horizon to have two completely different mechanisms based on what backend is following a poor design pattern we keep falling into where we expect the user to figure out/know what is different between deployments. No, that is not at all what I said. I said that there should be a discovery mechanism so that **Horizon** can figure out whether it can use a standard get me a page of listed results user experience, or whether it can ONLY use a filtering strategy for the user experience... Nobody has asked the end user to figure out what is different between deployments. We're talking about the dashboard here, and possibly the python-keystoneclient. Best, -jay That is becoming a repository for service users only. For the use cases requested, we do not have the ability to implement. Sure, but what you[1] *do* have the ability to implement is a capabilities discovery REST API that would allow tools like Horizon to determine if the only option available for them would be a filtering API, with no pagination capabilities. It would be super awesome if Keystone had such a capabilities discovery API. Best, -jay [1] I don't mean *you* personally, Adam :) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities
On 08/14/2015 05:24 PM, Adam Young wrote: On 08/14/2015 12:43 PM, Michael Krotscheck wrote: 1- Do users want to page through search results? Does not matter: in Federation, the User list is not available. OK, so hobble the entire REST API for the deficiencies/architecture/reality of a single authentication/identity strategy. 2- Do users want to page through filter results? (do they use filter results?) This is the only practical tool available for LDAP. Again, hobble the entire API for the deficiencies and anachronisms of a single identity driver. 3- If they want to page, do they want to be able to go back a page and/or know their current page? 4- How much do users care about small data inconsistencies (i.e. ordering of record sets changing from page-to-page). So, yeah, we could support paging for SQL. That would be great. Double bonus if this pagination API followed the examples of all the other OpenStack API services and didn't invent its own terms (page, per_page...). That is becoming a repository for service users only. For the use cases requested, we do not have the ability to implement. Sure, but what you[1] *do* have the ability to implement is a capabilities discovery REST API that would allow tools like Horizon to determine if the only option available for them would be a filtering API, with no pagination capabilities. It would be super awesome if Keystone had such a capabilities discovery API. Best, -jay [1] I don't mean *you* personally, Adam :) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [UX] [Keystone] [Horizon] Pagination support for Identity dashboard entities
On Fri, Aug 14, 2015 at 3:55 PM, Jay Pipes jaypi...@gmail.com wrote: On 08/14/2015 06:30 PM, Morgan Fainberg wrote: On Aug 14, 2015, at 15:10, Jay Pipes jaypi...@gmail.com wrote: On 08/14/2015 05:24 PM, Adam Young wrote: On 08/14/2015 12:43 PM, Michael Krotscheck wrote: 1- Do users want to page through search results? Does not matter: in Federation, the User list is not available. OK, so hobble the entire REST API for the deficiencies/architecture/reality of a single authentication/identity strategy. 2- Do users want to page through filter results? (do they use filter results?) This is the only practical tool available for LDAP. Again, hobble the entire API for the deficiencies and anachronisms of a single identity driver. The SQL driver is at best a toy and should go away. And yet it is in deployments all over the world. People said the same about MySQL. It was a toy and should go away. And yet, here we are, 15 years later, and MySQL is running in some of the world's biggest and most complex web applications. Asking for something to go away because you consider it a toy (a toy with better capabilities than other things, I might add) doesn't just make it so... The SQL identity backend is not providing comperable capabilities in a real way. It is providing a very limited store for a user data. It does not provide password complexity management, it does not allow for lockout based on failed logins, it does not provide limits on re-use of password, it does not provide clear user/group lifecycle management. It is not a real identity store, it is a very limited example implementation. These are all thing that have been regular requests of Keystone and provided for free with the most basic LDAP installation (or FreeIPA). I am inclined to say we should be moving towards LDAP being the identity store by default instead of continuing to use the SQL store and do the significant levels of enhancements needed (we have asked at midcycles and summits and have had no one interested in doing this enhancement work). The comparison to MySQL is incorrect, MySQL legitimately comparable featureset to the other options and the SQL data store we are using in openstack lacks basic identity management capabilities. If anything, we should tell the anachronistic ActiveDirectory and LDAP solutions to go away ;) We are working diligently to provide the best solution for the small and large deployments and provide features we are regularly asked for (password lifecycle, complexity, better user management, etc). Aren't we talking about better user management here? And Keystone's APIs are very poor user management. There are tools/systems out there that do it far better that we can leverage. FreeIPA is one such example. The Keystone User Management API is unused in many deployments and will not be considered for defcore because most all deployments use a read-only mode managed by an external service. With the new Tokenless Auth via x509 (Liberty Target) we now can eliminate service users needing to be in SQL as well. Again I will reiterate, asking for every user that could have an assignment is absurd. Nobody is asking for that. We are asking for list me the first page of users in the system, ordered by their ID (or email, or whatever). This is hardly absurd. In fact, it's how all other OpenStack API services expose pagination functionality. And it is complementary to things like Searchlight, not anathema to it. You are asking for it without being explicit about it (or may not be aware). Asking Keystone to list users is infact list all users the Keystone service can see assignments or not. What I've been advocating as the alternative is to use the query active assignments API calls (and enhance those) to show who has access to the OpenStack service. It wont be a /users call because /users is tied to the identity subsystem that (above) isn't part of defcore and wont be because of the aforementioned read-only and externally managed system. The exception may be the search for a user API I was alluding to that should be implemented/enhanced for the sake of finding a specific user (if the user in question doesn't have an active assignment). You should search for specific users if you need for find a user without an assignment (they can't access keystone or auth or act on OpenStack anyway). You should look at active assignments and we can implement pagination for that. Where are you coming up with this find a user without an assignment use case? I've yet to hear anyone talking about this. The only use case that has been discussed is the (very common) one that Horizon needs to display a page of user account information, sorted by some key and filtered as needed. It wont be a /users API call. Why not? See Above comment. 3- If they want to page, do they want to be able to go back a page and/or know their current page? 4- How much do users care about
