Re: [openstack-dev] [Cinder] cinder not support query volume/snapshot with regular expression
On 04/29/2014 03:34 PM, Steven Kaufer wrote: Jay Pipes wrote on 04/29/2014 02:26:42 PM: > From: Jay Pipes > To: openstack-dev@lists.openstack.org, > Date: 04/29/2014 02:27 PM > Subject: Re: [openstack-dev] [Cinder] cinder not support query > volume/snapshot with regular expression > > On 04/29/2014 02:16 AM, Zhangleiqiang (Trump) wrote: > > Currently, Nova API achieve this feature based on the database’s REGEX > > support. Do you have advice on alternative way to achieve it? > > Hi Trump, > > Unfortunately, REGEXP support in databases is almost always ridiculously > slow compared to prefix searches (WHERE col LIKE 'foo%'). > > Lately, I've been considering that a true tagging system for Nova would > allow for better-performing and more user-friendly search/winnow > functions in the Nova API. I'll post a blueprint specification for this > and hopefully have some time to implement in Juno... Jay, I am interested in this design, please add me as a reviewer when the blueprint is created. Even better, I listed you as a co-contributor :) https://review.openstack.org/91444 Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Cinder] cinder not support query volume/snapshot with regular expression
Jay Pipes wrote on 04/29/2014 02:26:42 PM: > From: Jay Pipes > To: openstack-dev@lists.openstack.org, > Date: 04/29/2014 02:27 PM > Subject: Re: [openstack-dev] [Cinder] cinder not support query > volume/snapshot with regular expression > > On 04/29/2014 02:16 AM, Zhangleiqiang (Trump) wrote: > > Currently, Nova API achieve this feature based on the database’s REGEX > > support. Do you have advice on alternative way to achieve it? > > Hi Trump, > > Unfortunately, REGEXP support in databases is almost always ridiculously > slow compared to prefix searches (WHERE col LIKE 'foo%'). > > Lately, I've been considering that a true tagging system for Nova would > allow for better-performing and more user-friendly search/winnow > functions in the Nova API. I'll post a blueprint specification for this > and hopefully have some time to implement in Juno... Jay, I am interested in this design, please add me as a reviewer when the blueprint is created. Thanks! Steven Kaufer > > Best, > -jay > > > zhangleiqiang (Trump) > > > > Best Regards > > > > *From:*laserjetyang [mailto:laserjety...@gmail.com] > > *Sent:* Tuesday, April 29, 2014 1:49 PM > > *To:* OpenStack Development Mailing List (not for usage questions) > > *Subject:* Re: [openstack-dev] [Cinder] cinder not support query > > volume/snapshot with regular expression > > > > It looks to me the Nova API will be dangerous source of DoS attacks due > > to the regexp? > > > > On Mon, Apr 28, 2014 at 7:04 PM, Duncan Thomas > <mailto:duncan.tho...@gmail.com>> wrote: > > > > Regex matching in APIs can be a dangerous source of DoS attacks - see > > http://en.wikipedia.org/wiki/ReDoS. Unless this is mitigated sensibly, > > I will continue to resist any cinder patch that adds them. > > > > Glob matches might be safer? > > > > > > On 26 April 2014 05:02, Zhangleiqiang (Trump) > <mailto:zhangleiqi...@huawei.com>> wrote: > >> Hi, all: > >> > >> I see Nova allows search instances by name, ip and ip6 > fields which can be normal string and regular expression: > >> > >> [stack@leiqzhang-stack cinder]$ nova help list > >> > >> List active servers. > >> > >> Optional arguments: > >> --ip Search with regular > expression match by IP address > >> (Admin only). > >> --ip6 Search with regular > expression match by IPv6 address > >> (Admin only). > >> --name Search with regular > expression match by name > >> --instance-name Search with regular > expression match by server name > >> (Admin only). > >> > >> I think it is also needed for Cinder when query the > volume/snapshot/backup by name. Any advice? > >> > >> -- > >> zhangleiqiang (Trump) > >> > >> Best Regards > >> > >> > >> ___ > >> OpenStack-dev mailing list > >>OpenStack-dev@lists.openstack.org <mailto:OpenStack-dev@lists.openstack.org> > >>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > -- > > Duncan Thomas > > > > > > ___ > > OpenStack-dev mailing list > > OpenStack-dev@lists.openstack.org <mailto:OpenStack-dev@lists.openstack.org> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > > > ___ > > OpenStack-dev mailing list > > OpenStack-dev@lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Cinder] cinder not support query volume/snapshot with regular expression
On 04/29/2014 02:16 AM, Zhangleiqiang (Trump) wrote: Currently, Nova API achieve this feature based on the database’s REGEX support. Do you have advice on alternative way to achieve it? Hi Trump, Unfortunately, REGEXP support in databases is almost always ridiculously slow compared to prefix searches (WHERE col LIKE 'foo%'). Lately, I've been considering that a true tagging system for Nova would allow for better-performing and more user-friendly search/winnow functions in the Nova API. I'll post a blueprint specification for this and hopefully have some time to implement in Juno... Best, -jay zhangleiqiang (Trump) Best Regards *From:*laserjetyang [mailto:laserjety...@gmail.com] *Sent:* Tuesday, April 29, 2014 1:49 PM *To:* OpenStack Development Mailing List (not for usage questions) *Subject:* Re: [openstack-dev] [Cinder] cinder not support query volume/snapshot with regular expression It looks to me the Nova API will be dangerous source of DoS attacks due to the regexp? On Mon, Apr 28, 2014 at 7:04 PM, Duncan Thomas mailto:duncan.tho...@gmail.com>> wrote: Regex matching in APIs can be a dangerous source of DoS attacks - see http://en.wikipedia.org/wiki/ReDoS. Unless this is mitigated sensibly, I will continue to resist any cinder patch that adds them. Glob matches might be safer? On 26 April 2014 05:02, Zhangleiqiang (Trump) mailto:zhangleiqi...@huawei.com>> wrote: Hi, all: I see Nova allows search instances by name, ip and ip6 fields which can be normal string and regular expression: [stack@leiqzhang-stack cinder]$ nova help list List active servers. Optional arguments: --ip Search with regular expression match by IP address (Admin only). --ip6 Search with regular expression match by IPv6 address (Admin only). --name Search with regular expression match by name --instance-name Search with regular expression match by server name (Admin only). I think it is also needed for Cinder when query the volume/snapshot/backup by name. Any advice? -- zhangleiqiang (Trump) Best Regards ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org <mailto:OpenStack-dev@lists.openstack.org> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Duncan Thomas ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org <mailto:OpenStack-dev@lists.openstack.org> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Cinder] cinder not support query volume/snapshot with regular expression
Currently, Nova API achieve this feature based on the database’s REGEX support. Do you have advice on alternative way to achieve it? -- zhangleiqiang (Trump) Best Regards From: laserjetyang [mailto:laserjety...@gmail.com] Sent: Tuesday, April 29, 2014 1:49 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [Cinder] cinder not support query volume/snapshot with regular expression It looks to me the Nova API will be dangerous source of DoS attacks due to the regexp? On Mon, Apr 28, 2014 at 7:04 PM, Duncan Thomas mailto:duncan.tho...@gmail.com>> wrote: Regex matching in APIs can be a dangerous source of DoS attacks - see http://en.wikipedia.org/wiki/ReDoS. Unless this is mitigated sensibly, I will continue to resist any cinder patch that adds them. Glob matches might be safer? On 26 April 2014 05:02, Zhangleiqiang (Trump) mailto:zhangleiqi...@huawei.com>> wrote: > Hi, all: > > I see Nova allows search instances by name, ip and ip6 fields which > can be normal string and regular expression: > > [stack@leiqzhang-stack cinder]$ nova help list > > List active servers. > > Optional arguments: > --ip Search with regular expression match by > IP address > (Admin only). > --ip6 Search with regular expression match by > IPv6 address > (Admin only). > --name Search with regular expression match by > name > --instance-name Search with regular expression > match by server name > (Admin only). > > I think it is also needed for Cinder when query the > volume/snapshot/backup by name. Any advice? > > -- > zhangleiqiang (Trump) > > Best Regards > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Duncan Thomas ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Cinder] cinder not support query volume/snapshot with regular expression
It looks to me the Nova API will be dangerous source of DoS attacks due to the regexp? On Mon, Apr 28, 2014 at 7:04 PM, Duncan Thomas wrote: > Regex matching in APIs can be a dangerous source of DoS attacks - see > http://en.wikipedia.org/wiki/ReDoS. Unless this is mitigated sensibly, > I will continue to resist any cinder patch that adds them. > > Glob matches might be safer? > > On 26 April 2014 05:02, Zhangleiqiang (Trump) > wrote: > > Hi, all: > > > > I see Nova allows search instances by name, ip and ip6 fields > which can be normal string and regular expression: > > > > [stack@leiqzhang-stack cinder]$ nova help list > > > > List active servers. > > > > Optional arguments: > > --ip Search with regular expression > match by IP address > > (Admin only). > > --ip6 Search with regular expression > match by IPv6 address > > (Admin only). > > --name Search with regular expression > match by name > > --instance-name Search with regular > expression match by server name > > (Admin only). > > > > I think it is also needed for Cinder when query the > volume/snapshot/backup by name. Any advice? > > > > -- > > zhangleiqiang (Trump) > > > > Best Regards > > > > > > ___ > > OpenStack-dev mailing list > > OpenStack-dev@lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > -- > Duncan Thomas > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Cinder] cinder not support query volume/snapshot with regular expression
Thanks for your reply. Regex matching can be implemented in Database, and glob matches may not work fine with "paginate_query". However, the ReDoS you mentioned will not be avoided when using regex matching. I will think of it again. Thanks. 2014-04-28 19:04 GMT+08:00 Duncan Thomas : > Regex matching in APIs can be a dangerous source of DoS attacks - see > http://en.wikipedia.org/wiki/ReDoS. Unless this is mitigated sensibly, > I will continue to resist any cinder patch that adds them. > > Glob matches might be safer? > > On 26 April 2014 05:02, Zhangleiqiang (Trump) > wrote: > > Hi, all: > > > > I see Nova allows search instances by name, ip and ip6 fields > which can be normal string and regular expression: > > > > [stack@leiqzhang-stack cinder]$ nova help list > > > > List active servers. > > > > Optional arguments: > > --ip Search with regular expression > match by IP address > > (Admin only). > > --ip6 Search with regular expression > match by IPv6 address > > (Admin only). > > --name Search with regular expression > match by name > > --instance-name Search with regular > expression match by server name > > (Admin only). > > > > I think it is also needed for Cinder when query the > volume/snapshot/backup by name. Any advice? > > > > -- > > zhangleiqiang (Trump) > > > > Best Regards > > > > > > ___ > > OpenStack-dev mailing list > > OpenStack-dev@lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > -- > Duncan Thomas > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- --- Best Regards Trump.Zhang ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Cinder] cinder not support query volume/snapshot with regular expression
Regex matching in APIs can be a dangerous source of DoS attacks - see http://en.wikipedia.org/wiki/ReDoS. Unless this is mitigated sensibly, I will continue to resist any cinder patch that adds them. Glob matches might be safer? On 26 April 2014 05:02, Zhangleiqiang (Trump) wrote: > Hi, all: > > I see Nova allows search instances by name, ip and ip6 fields which > can be normal string and regular expression: > > [stack@leiqzhang-stack cinder]$ nova help list > > List active servers. > > Optional arguments: > --ip Search with regular expression match by > IP address > (Admin only). > --ip6 Search with regular expression match by > IPv6 address > (Admin only). > --name Search with regular expression match by > name > --instance-name Search with regular expression > match by server name > (Admin only). > > I think it is also needed for Cinder when query the > volume/snapshot/backup by name. Any advice? > > -- > zhangleiqiang (Trump) > > Best Regards > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Duncan Thomas ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Cinder] cinder not support query volume/snapshot with regular expression
Hi, all: I see Nova allows search instances by name, ip and ip6 fields which can be normal string and regular expression: [stack@leiqzhang-stack cinder]$ nova help list List active servers. Optional arguments: --ip Search with regular expression match by IP address (Admin only). --ip6 Search with regular expression match by IPv6 address (Admin only). --name Search with regular expression match by name --instance-name Search with regular expression match by server name (Admin only). I think it is also needed for Cinder when query the volume/snapshot/backup by name. Any advice? -- zhangleiqiang (Trump) Best Regards ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev