Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-10-03 Thread Soren Hansen
2014-10-03 9:00 GMT+02:00 Michael Chapman :
> On Fri, Oct 3, 2014 at 4:05 AM, Soren Hansen 
> wrote:
>> That said, there will certainly be situations where there'll be a
>> need for some sort of anti-entropy mechanism. It just so happens that
>> those situations already exist. We're dealing with about a complex
>> distributed system.  We're kidding ourselves if we think that any
>> kind of consistency is guaranteed, just because our data store
>> favours consistency over availability.
> I apologize if I'm missing something, but doesn't denormalization to
> add join support put the same value in many places, such that an
> update to that value is no longer a single atomic transaction?

Yes.


> This would appear to counteract the requirement for strong
> consistency.

What requirement for strong consistency?


> If updating a single value is atomic (as in Riak's consistent mode)

Admittedly, I'm not 100% up-to-date on Riak, but last I looked, there
wasn't any "consistent mode". However, when writing a value, you can
specify that you want all (or a quorum of) replicas to be written to
disk before you get a succesful response. However, this does not imply
transactional support. In other words, if one of the writes fail, it
doesn't get rolled back on the other nodes. You just don't get a
succesful response.


> I also don't really see how a NoSQL system in strong consistency mode
> is any different from running MySQL with galera in its failure modes.

I agree. I never meant to imply that we should run anything in "strong
consistency mode". There might be a few operations that require strong
consistency, but they should be exceptional. Quotas sound like a good
example.


> The requirement for quorum makes the addition of nodes increase the
> potential latency of writes (and reads in some cases) so having large
> scale doesn't grant much benefit, if any.

I agree about the requirement for quorum having those effects (also for
e.g. Galera).  I think you are missing my point, though. My concerns are
not whether MySQL can handle the data volume of a large scale OpenStack
deployment.  I'm sure it can. Without even breaking a sweat. MySQL has
been used in countless deployments to handle data sets vastly bigger
than what we're dealing with.

My concern is reliability.

> Quorum will also prevent nodes on the wrong side of a partition from
> being able to access system state (or it will give them stale state,
> which is probably just as bad in our case).

This problem exists today.  Suppose you have a 5 node Galera cluster.
Would you refuse reads on the wrong side of the partition to avoid
providing stale data?

With e.g. Riak it's perfectly possible to accept both reads and writes
on both sides of the partition.

No matter what we do, we need accept the fact that when we handle the
data, it is by definition out of date. It can have changed the
millisecond after we read it from there and started using it.


> I think your goal of having state management that's able to handle
> network partitions is a good one, but I don't think the solution is as
> simple as swapping out where the state is stored.

It kinda is, and it kinda isn't. I never meant to suggest that just
replacing the datastore would solve everything. We need to carefully
look at our use of the data from the datastore and consider the impact
of eventual consistency on this use. On the other hand, as I just
mentioned above, this is a problem that exists right now, today. We're
just ignoring it, because we happen to have a consistent datastore.


> Maybe in some cases like split-racks the system needs to react to a
> network partition by forming its own independent cell with its own
> state storage, and when the network heals it then merges back into the
> other cluster cleanly?  That would be very difficult to implement, but
> fun (for some definition of fun).

Fun, but possible. Riak was designed for this. With an RDBMS I don't
even know how to begin solving something like that.


> As a thought experiment, a while ago I considered what would happen if
> instead of using a central store, I put a sqlite database behind every
> daemon and allowed them to query each other for the data they needed,
> and cluster if needed (using raft).

> Services like nova-scheduler need strong consistency

No, it doesn't. :)

> and would have to cluster to perform their role, but services like
> nova-compute would simply need to store the data concerning the
> resources they are responsible for. This follows the 'place state at
> the edge' kind of design principles that have been discussed in
> various circles.  It falls down in a number of pretty obvious ways,
> and ultimately it would require more work than I am able to put in,
> but I mention it because perhaps it provides you with food for
> thought.

Yeah, a million distributed consistent databases do not a single
distributed, eventually consistent database make :)

-- 
Soren Hansen | http://linux2go.dk/
Ubunt

Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-10-03 Thread Michael Chapman
On Fri, Oct 3, 2014 at 4:05 AM, Soren Hansen  wrote:

> I'm sorry about my slow responses. For some reason, gmail didn't think
> this was an important e-mail :(
>
> 2014-09-30 18:41 GMT+02:00 Jay Pipes :
> > On 09/30/2014 08:03 AM, Soren Hansen wrote:
> >> 2014-09-12 1:05 GMT+02:00 Jay Pipes :
> > How would I go about getting the associated fixed IPs for a network?
> > The query to get associated fixed IPs for a network [1] in Nova looks
> > like this:
> >
> > SELECT
> >  fip.address,
> >  fip.instance_uuid,
> [...]
> > AND fip.instance_uuid IS NOT NULL
> > AND i.host = :host
> >
> > would I have a Riak container for virtual_interfaces that would also
> > have instance information, network information, fixed_ip information?
> > How would I accomplish the query against a derived table that gets the
> > minimum virtual interface ID for each instance UUID?
>
> What's a minimum virtual interface ID?
>
> Anyway, I think Clint answered this quite well.
>
> >>> I've said it before, and I'll say it again. In Nova at least, the
> >>> SQL schema is complex because the problem domain is complex. That
> >>> means lots of relations, lots of JOINs, and that means the best way
> >>> to query for that data is via an RDBMS.
> [...]
> >> I don't think relying on a central data store is in any conceivable
> >> way appropriate for a project like OpenStack. Least of all Nova.
> >>
> >> I don't see how we can build a highly available, distributed service
> >> on top of a centralized data store like MySQL.
> [...]
> > I don't disagree with anything you say above. At all.
>
> Really? How can you agree that we can't "build a highly available,
> distributed service on top of a centralized data store like MySQL" while
> also saying that the best way to handle data in Nova is in an RDBMS?
>
> >>> For complex control plane software like Nova, though, an RDBMS is
> >>> the best tool for the job given the current lay of the land in open
> >>> source data storage solutions matched with Nova's complex query and
> >>> transactional requirements.
> >> What transactional requirements?
> >
> https://github.com/openstack/nova/blob/stable/icehouse/nova/db/sqlalchemy/api.py#L1654
> > When you delete an instance, you don't want the delete to just stop
> > half-way through the transaction and leave around a bunch of orphaned
> > children.  Similarly, when you reserve something, it helps to not have
> > a half-finished state change that you need to go clean up if something
> > goes boom.
>
> Looking at that particular example, it's about deleting an instance and
> all its associated metadata. As we established earlier, these are things
> that would just be in the same key as the instance itself, so it'd just
> be a single key that would get deleted. Easy.
>
> That said, there will certainly be situations where there'll be a need
> for some sort of anti-entropy mechanism. It just so happens that those
> situations already exist. We're dealing with about a complex distributed
> system.  We're kidding ourselves if we think that any kind of
> consistency is guaranteed, just because our data store favours
> consistency over availability.
>
>
I apologize if I'm missing something, but doesn't denormalization to add
join support put the same value in many places, such that an update to that
value is no longer a single atomic transaction? This would appear to
counteract the requirement for strong consistency. If updating a single
value is atomic (as in Riak's consistent mode) then it might be possible to
construct a way to make multiple updates appear atomic, but it would add
many more transactions and many more quorum checks, which would reduce
performance to a crawl.

I also don't really see how a NoSQL system in strong consistency mode is
any different from running MySQL with galera in its failure modes. The
requirement for quorum makes the addition of nodes increase the potential
latency of writes (and reads in some cases) so having large scale doesn't
grant much benefit, if any. Quorum will also prevent nodes on the wrong
side of a partition from being able to access system state (or it will give
them stale state, which is probably just as bad in our case).

I think your goal of having state management that's able to handle network
partitions is a good one, but I don't think the solution is as simple as
swapping out where the state is stored. Maybe in some cases like
split-racks the system needs to react to a network partition by forming its
own independent cell with its own state storage, and when the network heals
it then merges back into the other cluster cleanly? That would be very
difficult to implement, but fun (for some definition of fun).

As a thought experiment, a while ago I considered what would happen if
instead of using a central store, I put a sqlite database behind every
daemon and allowed them to query each other for the data they needed, and
cluster if needed (using raft). Services like nova-scheduler need strong
consistency and would

Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-10-02 Thread Soren Hansen
I'm sorry about my slow responses. For some reason, gmail didn't think
this was an important e-mail :(

2014-09-30 18:41 GMT+02:00 Jay Pipes :
> On 09/30/2014 08:03 AM, Soren Hansen wrote:
>> 2014-09-12 1:05 GMT+02:00 Jay Pipes :
> How would I go about getting the associated fixed IPs for a network?
> The query to get associated fixed IPs for a network [1] in Nova looks
> like this:
>
> SELECT
>  fip.address,
>  fip.instance_uuid,
[...]
> AND fip.instance_uuid IS NOT NULL
> AND i.host = :host
>
> would I have a Riak container for virtual_interfaces that would also
> have instance information, network information, fixed_ip information?
> How would I accomplish the query against a derived table that gets the
> minimum virtual interface ID for each instance UUID?

What's a minimum virtual interface ID?

Anyway, I think Clint answered this quite well.

>>> I've said it before, and I'll say it again. In Nova at least, the
>>> SQL schema is complex because the problem domain is complex. That
>>> means lots of relations, lots of JOINs, and that means the best way
>>> to query for that data is via an RDBMS.
[...]
>> I don't think relying on a central data store is in any conceivable
>> way appropriate for a project like OpenStack. Least of all Nova.
>>
>> I don't see how we can build a highly available, distributed service
>> on top of a centralized data store like MySQL.
[...]
> I don't disagree with anything you say above. At all.

Really? How can you agree that we can't "build a highly available,
distributed service on top of a centralized data store like MySQL" while
also saying that the best way to handle data in Nova is in an RDBMS?

>>> For complex control plane software like Nova, though, an RDBMS is
>>> the best tool for the job given the current lay of the land in open
>>> source data storage solutions matched with Nova's complex query and
>>> transactional requirements.
>> What transactional requirements?
> https://github.com/openstack/nova/blob/stable/icehouse/nova/db/sqlalchemy/api.py#L1654
> When you delete an instance, you don't want the delete to just stop
> half-way through the transaction and leave around a bunch of orphaned
> children.  Similarly, when you reserve something, it helps to not have
> a half-finished state change that you need to go clean up if something
> goes boom.

Looking at that particular example, it's about deleting an instance and
all its associated metadata. As we established earlier, these are things
that would just be in the same key as the instance itself, so it'd just
be a single key that would get deleted. Easy.

That said, there will certainly be situations where there'll be a need
for some sort of anti-entropy mechanism. It just so happens that those
situations already exist. We're dealing with about a complex distributed
system.  We're kidding ourselves if we think that any kind of
consistency is guaranteed, just because our data store favours
consistency over availability.

> https://github.com/openstack/nova/blob/stable/icehouse/nova/db/sqlalchemy/api.py#L3054

Sure, quotas will require stronger consistency. Any NoSQL data store
worth its salt gives you primitives to implement that.

>>> Folks in these other programs have actually, you know, thought about
>>> these kinds of things and had serious discussions about
>>> alternatives.  It would be nice to have someone acknowledge that
>>> instead of snarky comments implying everyone else "has it wrong".
>> I'm terribly sorry, but repeating over and over that an RDBMS is "the
>> best tool" without further qualification than "Nova's data model is
>> really complex" reads *exactly* like a snarky comment implying
>> everyone else "has it wrong".
> Sorry if I sound snarky. I thought your blog post was the definition
> of snark.

I don't see the relevance of the tone of my blog post?

You say it would be nice if people did something other than offer snarky
comments implying everyone else "has it wrong".  I'm just pointing out
that such requests ring really hollow when put forth in the very e-mail
where you snarkily tell everyone else that they have it wrong.

Since you did bring up my blog post, I really am astounded you find it
snarky.  It was intended to be constructive and forward looking. The
first one in the series, perhaps, but certainly not the one linked in
this thread.

Perhaps I need to take writing classes.

-- 
Soren Hansen | http://linux2go.dk/
Ubuntu Developer | http://www.ubuntu.com/
OpenStack Developer  | http://www.openstack.org/

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-10-01 Thread Dmitry Tantsur

On 09/30/2014 02:03 PM, Soren Hansen wrote:

2014-09-12 1:05 GMT+02:00 Jay Pipes :

If Nova was to take Soren's advice and implement its data-access layer
on top of Cassandra or Riak, we would just end up re-inventing SQL
Joins in Python-land.


I may very well be wrong(!), but this statement makes it sound like you've
never used e.g. Riak. Or, if you have, not done so in the way it's
supposed to be used.

If you embrace an alternative way of storing your data, you wouldn't just
blindly create a container for each table in your RDBMS.

For example: In Nova's SQL-based datastore we have a table for security
groups and another for security group rules. Rows in the security group
rules table have a foreign key referencing the security group to which
they belong. In a datastore like Riak, you could have a security group
container where each value contains not just the security group
information, but also all the security group rules. No joins in
Python-land necessary.


I've said it before, and I'll say it again. In Nova at least, the SQL
schema is complex because the problem domain is complex. That means
lots of relations, lots of JOINs, and that means the best way to query
for that data is via an RDBMS.


I was really hoping you could be more specific than "best"/"most
appropriate" so that we could have a focused discussion.

I don't think relying on a central data store is in any conceivable way
appropriate for a project like OpenStack. Least of all Nova.

I don't see how we can build a highly available, distributed service on
top of a centralized data store like MySQL.
Coming from Skype background I can assure your that you definitely can, 
depending on your needs (and our experiments with e.g. MongoDB ended 
very badly: it just died under IO loads, that our PostgreSQL treated 
like normal). I mean, that's complex topic and I see a lot of people 
switching to NoSQL and a lot of people switching from. NoSQL is not a 
silver bullet for scalability. Just my 0.5.


/me disappears again



Tens or hundreds of thousands of nodes, spread across many, many racks
and datacentre halls are going to experience connectivity problems[1].

This means that some percentage of your infrastructure (possibly many
thousands of nodes, affecting many, many thousands of customers) will
find certain functionality not working on account of your datastore not
being reachable from the part of the control plane they're attempting to
use (or possibly only being able to read from it).

I say over and over again that people should own their own uptime.
Expect things to fail all the time. Do whatever you need to do to ensure
your service keeps working even when something goes wrong. Of course
this applies to our customers too. Even if we take the greatest care to
avoid downtime, customers should spread their workloads across multiple
availability zones and/or regions and probably even multiple cloud
providers. Their service towards their users is their responsibility.

However, our service towards our users is our responsibility. We should
take the greatest care to avoid having internal problems affect our
users.  Building a massively distributed system like Nova on top of a
centralized data store is practically a guarantee of the opposite.


For complex control plane software like Nova, though, an RDBMS is the
best tool for the job given the current lay of the land in open source
data storage solutions matched with Nova's complex query and
transactional requirements.


What transactional requirements?


Folks in these other programs have actually, you know, thought about
these kinds of things and had serious discussions about alternatives.
It would be nice to have someone acknowledge that instead of snarky
comments implying everyone else "has it wrong".


I'm terribly sorry, but repeating over and over that an RDBMS is "the
best tool" without further qualification than "Nova's data model is
really complex" reads *exactly* like a snarky comment implying everyone
else "has it wrong".

[1]: http://aphyr.com/posts/288-the-network-is-reliable




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-30 Thread Clint Byrum
Excerpts from Jay Pipes's message of 2014-09-30 09:41:29 -0700:
> A relational database was built for the above types of queries, and 
> that's why I said it's the best tool for the job *in this specific case*.
> 
> Now... that said...
> 
> Is it possible to go through the Nova schema and identify mini-schemas 
> that could be pulled out of the RDBMS and placed into Riak or Cassandra? 
> Absolutely yes! The service group and compute node usage records are 
> good candidates for that, in my opinion. With the nova.objects work that 
> was completed over the last few cycles, we might actually now have the 
> foundation in place to make doing this a reality. I welcome your 
> contributions in this area.
> 

I forgot to say, +1 here. :)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-30 Thread Clint Byrum
Excerpts from Jay Pipes's message of 2014-09-30 09:41:29 -0700:
> On 09/30/2014 08:03 AM, Soren Hansen wrote:
> > 2014-09-12 1:05 GMT+02:00 Jay Pipes :
> >> If Nova was to take Soren's advice and implement its data-access layer
> >> on top of Cassandra or Riak, we would just end up re-inventing SQL
> >> Joins in Python-land.
> >
> > I may very well be wrong(!), but this statement makes it sound like you've
> > never used e.g. Riak. Or, if you have, not done so in the way it's
> > supposed to be used.
> >
> > If you embrace an alternative way of storing your data, you wouldn't just
> > blindly create a container for each table in your RDBMS.
> >
> > For example: In Nova's SQL-based datastore we have a table for security
> > groups and another for security group rules. Rows in the security group
> > rules table have a foreign key referencing the security group to which
> > they belong. In a datastore like Riak, you could have a security group
> > container where each value contains not just the security group
> > information, but also all the security group rules. No joins in
> > Python-land necessary.
> 
> OK, that's all fine for a simple one-to-many relation.
> 
> How would I go about getting the associated fixed IPs for a network? The 
> query to get associated fixed IPs for a network [1] in Nova looks like this:
> 
> SELECT
>   fip.address,
>   fip.instance_uuid,
>   fip.network_id,
>   fip.virtual_interface_id,
>   vif.address,
>   i.hostname,
>   i.updated_at,
>   i.created_at,
>   fip.allocated,
>   fip.leased,
>   vif2.id
> FROM fixed_ips fip
> LEFT JOIN virtual_interfaces vif
>   ON vif.id = fip.virtual_interface_id
>   AND vif.deleted = 0
> LEFT JOIN instances i
>   ON fip.instance_uuid = i.uuid
>   AND i.deleted = 0
> LEFT JOIN (
>   SELECT MIN(vi.id) AS id, vi.instance_uuid
>   FROM virtual_interfaces vi
>   GROUP BY instance_uuid
> ) as vif2
> WHERE fip.deleted = 0
> AND fip.network_id = :network_id
> AND fip.virtual_interface_id IS NOT NULL
> AND fip.instance_uuid IS NOT NULL
> AND i.host = :host
> 

You and I both know that this query is not something we want to be
running a lot. In the past when I've had systems where something like
the above needed to be run a lot, I created materialized views for it
and made access to this information asynchronous because relying on this
in real-time is _extremely expensive_.

This is where a massively scalable but somewhat dumb database shines
because you can materialize the view into it at a much faster pace and
with more scaled-out workers so that you now have fine grained resource
allocations for scaling the responsiveness of this view.

> would I have a Riak container for virtual_interfaces that would also 
> have instance information, network information, fixed_ip information? 
> How would I accomplish the query against a derived table that gets the 
> minimum virtual interface ID for each instance UUID?
> 

Yes you'd have a bunch of duplicated information everywhere.  Asynchronous
denormalizing is the new join. The only worry I'd have is what
accidental contracts have we made that require bits of information to
appear all at once. :-P

> More than likely, I would end up having to put a bunch of indexes and 
> relations into my Riak containers and structures just so I could do 
> queries like the above. Failing that, I'd need to do multiple queries to 
> multiple Riak containers and then join the resulting projection in 
> memory, in Python. And that is why I say you will just end up 
> implementing joins in Python.
> 

My experience has been that you have a map/reduce cluster somewhere
churning through updates to pre-compute things for slow queries and you
do any realtime duplication at write time. The idea is to create documents
that are themselves useful without the need of the other documents. Where
this approach suffers is when your documents all have so many copies of
data that you end up writing 10x more than you read. That is a rare thing,
and most certainly would not happen with Nova's data, which mostly would
have duplicates of lots of UUID's and IP addresses.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-30 Thread Jay Pipes

On 09/30/2014 08:03 AM, Soren Hansen wrote:

2014-09-12 1:05 GMT+02:00 Jay Pipes :

If Nova was to take Soren's advice and implement its data-access layer
on top of Cassandra or Riak, we would just end up re-inventing SQL
Joins in Python-land.


I may very well be wrong(!), but this statement makes it sound like you've
never used e.g. Riak. Or, if you have, not done so in the way it's
supposed to be used.

If you embrace an alternative way of storing your data, you wouldn't just
blindly create a container for each table in your RDBMS.

For example: In Nova's SQL-based datastore we have a table for security
groups and another for security group rules. Rows in the security group
rules table have a foreign key referencing the security group to which
they belong. In a datastore like Riak, you could have a security group
container where each value contains not just the security group
information, but also all the security group rules. No joins in
Python-land necessary.


OK, that's all fine for a simple one-to-many relation.

How would I go about getting the associated fixed IPs for a network? The 
query to get associated fixed IPs for a network [1] in Nova looks like this:


SELECT
 fip.address,
 fip.instance_uuid,
 fip.network_id,
 fip.virtual_interface_id,
 vif.address,
 i.hostname,
 i.updated_at,
 i.created_at,
 fip.allocated,
 fip.leased,
 vif2.id
FROM fixed_ips fip
LEFT JOIN virtual_interfaces vif
 ON vif.id = fip.virtual_interface_id
 AND vif.deleted = 0
LEFT JOIN instances i
 ON fip.instance_uuid = i.uuid
 AND i.deleted = 0
LEFT JOIN (
 SELECT MIN(vi.id) AS id, vi.instance_uuid
 FROM virtual_interfaces vi
 GROUP BY instance_uuid
) as vif2
WHERE fip.deleted = 0
AND fip.network_id = :network_id
AND fip.virtual_interface_id IS NOT NULL
AND fip.instance_uuid IS NOT NULL
AND i.host = :host

would I have a Riak container for virtual_interfaces that would also 
have instance information, network information, fixed_ip information? 
How would I accomplish the query against a derived table that gets the 
minimum virtual interface ID for each instance UUID?


More than likely, I would end up having to put a bunch of indexes and 
relations into my Riak containers and structures just so I could do 
queries like the above. Failing that, I'd need to do multiple queries to 
multiple Riak containers and then join the resulting projection in 
memory, in Python. And that is why I say you will just end up 
implementing joins in Python.


A relational database was built for the above types of queries, and 
that's why I said it's the best tool for the job *in this specific case*.


Now... that said...

Is it possible to go through the Nova schema and identify mini-schemas 
that could be pulled out of the RDBMS and placed into Riak or Cassandra? 
Absolutely yes! The service group and compute node usage records are 
good candidates for that, in my opinion. With the nova.objects work that 
was completed over the last few cycles, we might actually now have the 
foundation in place to make doing this a reality. I welcome your 
contributions in this area.


[1] 
https://github.com/openstack/nova/blob/stable/icehouse/nova/db/sqlalchemy/api.py#L2608



I've said it before, and I'll say it again. In Nova at least, the SQL
schema is complex because the problem domain is complex. That means
lots of relations, lots of JOINs, and that means the best way to query
for that data is via an RDBMS.


I was really hoping you could be more specific than "best"/"most
appropriate" so that we could have a focused discussion.

I don't think relying on a central data store is in any conceivable way
appropriate for a project like OpenStack. Least of all Nova.

I don't see how we can build a highly available, distributed service on
top of a centralized data store like MySQL.

Tens or hundreds of thousands of nodes, spread across many, many racks
and datacentre halls are going to experience connectivity problems[1].

This means that some percentage of your infrastructure (possibly many
thousands of nodes, affecting many, many thousands of customers) will
find certain functionality not working on account of your datastore not
being reachable from the part of the control plane they're attempting to
use (or possibly only being able to read from it).

I say over and over again that people should own their own uptime.
Expect things to fail all the time. Do whatever you need to do to ensure
your service keeps working even when something goes wrong. Of course
this applies to our customers too. Even if we take the greatest care to
avoid downtime, customers should spread their workloads across multiple
availability zones and/or regions and probably even multiple cloud
providers. Their service towards their users is their responsibility.

However, our service towards our users is our responsibility. We should
take the greatest care to avoid having internal problems affect our
users.  Building a massively distributed system like Nova on top of a
centralized data 

Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-30 Thread Soren Hansen
2014-09-12 1:05 GMT+02:00 Jay Pipes :
> If Nova was to take Soren's advice and implement its data-access layer
> on top of Cassandra or Riak, we would just end up re-inventing SQL
> Joins in Python-land.

I may very well be wrong(!), but this statement makes it sound like you've
never used e.g. Riak. Or, if you have, not done so in the way it's
supposed to be used.

If you embrace an alternative way of storing your data, you wouldn't just
blindly create a container for each table in your RDBMS.

For example: In Nova's SQL-based datastore we have a table for security
groups and another for security group rules. Rows in the security group
rules table have a foreign key referencing the security group to which
they belong. In a datastore like Riak, you could have a security group
container where each value contains not just the security group
information, but also all the security group rules. No joins in
Python-land necessary.

> I've said it before, and I'll say it again. In Nova at least, the SQL
> schema is complex because the problem domain is complex. That means
> lots of relations, lots of JOINs, and that means the best way to query
> for that data is via an RDBMS.

I was really hoping you could be more specific than "best"/"most
appropriate" so that we could have a focused discussion.

I don't think relying on a central data store is in any conceivable way
appropriate for a project like OpenStack. Least of all Nova.

I don't see how we can build a highly available, distributed service on
top of a centralized data store like MySQL.

Tens or hundreds of thousands of nodes, spread across many, many racks
and datacentre halls are going to experience connectivity problems[1].

This means that some percentage of your infrastructure (possibly many
thousands of nodes, affecting many, many thousands of customers) will
find certain functionality not working on account of your datastore not
being reachable from the part of the control plane they're attempting to
use (or possibly only being able to read from it).

I say over and over again that people should own their own uptime.
Expect things to fail all the time. Do whatever you need to do to ensure
your service keeps working even when something goes wrong. Of course
this applies to our customers too. Even if we take the greatest care to
avoid downtime, customers should spread their workloads across multiple
availability zones and/or regions and probably even multiple cloud
providers. Their service towards their users is their responsibility.

However, our service towards our users is our responsibility. We should
take the greatest care to avoid having internal problems affect our
users.  Building a massively distributed system like Nova on top of a
centralized data store is practically a guarantee of the opposite.

> For complex control plane software like Nova, though, an RDBMS is the
> best tool for the job given the current lay of the land in open source
> data storage solutions matched with Nova's complex query and
> transactional requirements.

What transactional requirements?

> Folks in these other programs have actually, you know, thought about
> these kinds of things and had serious discussions about alternatives.
> It would be nice to have someone acknowledge that instead of snarky
> comments implying everyone else "has it wrong".

I'm terribly sorry, but repeating over and over that an RDBMS is "the
best tool" without further qualification than "Nova's data model is
really complex" reads *exactly* like a snarky comment implying everyone
else "has it wrong".

[1]: http://aphyr.com/posts/288-the-network-is-reliable

-- 
Soren Hansen | http://linux2go.dk/
Ubuntu Developer | http://www.ubuntu.com/
OpenStack Developer  | http://www.openstack.org/

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-28 Thread Soren Hansen
2014-09-26 17:11 GMT+02:00 Jay Pipes :
> On 09/26/2014 06:45 AM, Soren Hansen wrote:
>> Define "best".
> best == most appropriate.

#copout

-- 
Soren Hansen | http://linux2go.dk/
Ubuntu Developer | http://www.ubuntu.com/
OpenStack Developer  | http://www.openstack.org/

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-26 Thread Jay Pipes



On 09/26/2014 06:45 AM, Soren Hansen wrote:

2014-09-12 1:05 GMT+02:00 Jay Pipes :

If Nova was to take Soren's advice and implement its data-access layer on
top of Cassandra or Riak, we would just end up re-inventing SQL Joins in
Python-land. I've said it before, and I'll say it again. In Nova at least,
the SQL schema is complex because the problem domain is complex. That means
lots of relations, lots of JOINs, and that means the best way to query for
that data is via an RDBMS.


Define "best".


best == most appropriate.

-jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-26 Thread Soren Hansen
2014-09-12 1:05 GMT+02:00 Jay Pipes :
> If Nova was to take Soren's advice and implement its data-access layer on
> top of Cassandra or Riak, we would just end up re-inventing SQL Joins in
> Python-land. I've said it before, and I'll say it again. In Nova at least,
> the SQL schema is complex because the problem domain is complex. That means
> lots of relations, lots of JOINs, and that means the best way to query for
> that data is via an RDBMS.

Define "best".

/Soren

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-12 Thread Clint Byrum
Excerpts from Flavio Percoco's message of 2014-09-12 00:22:35 -0700:
> On 09/12/2014 03:29 AM, Clint Byrum wrote:
> > Excerpts from Zane Bitter's message of 2014-09-11 15:21:26 -0700:
> >> On 09/09/14 19:56, Clint Byrum wrote:
> >>> Excerpts from Samuel Merritt's message of 2014-09-09 16:12:09 -0700:
>  On 9/9/14, 12:03 PM, Monty Taylor wrote:
> > On 09/04/2014 01:30 AM, Clint Byrum wrote:
> >> Excerpts from Flavio Percoco's message of 2014-09-04 00:08:47 -0700:
> >>> Greetings,
> >>>
> >>> Last Tuesday the TC held the first graduation review for Zaqar. During
> >>> the meeting some concerns arose. I've listed those concerns below with
> >>> some comments hoping that it will help starting a discussion before 
> >>> the
> >>> next meeting. In addition, I've added some comments about the project
> >>> stability at the bottom and an etherpad link pointing to a list of use
> >>> cases for Zaqar.
> >>>
> >>
> >> Hi Flavio. This was an interesting read. As somebody whose attention 
> >> has
> >> recently been drawn to Zaqar, I am quite interested in seeing it
> >> graduate.
> >>
> >>> # Concerns
> >>>
> >>> - Concern on operational burden of requiring NoSQL deploy expertise to
> >>> the mix of openstack operational skills
> >>>
> >>> For those of you not familiar with Zaqar, it currently supports 2 
> >>> nosql
> >>> drivers - MongoDB and Redis - and those are the only 2 drivers it
> >>> supports for now. This will require operators willing to use Zaqar to
> >>> maintain a new (?) NoSQL technology in their system. Before expressing
> >>> our thoughts on this matter, let me say that:
> >>>
> >>>   1. By removing the SQLAlchemy driver, we basically removed the
> >>> chance
> >>> for operators to use an already deployed "OpenStack-technology"
> >>>   2. Zaqar won't be backed by any AMQP based messaging technology 
> >>> for
> >>> now. Here's[0] a summary of the research the team (mostly done by
> >>> Victoria) did during Juno
> >>>   3. We (OpenStack) used to require Redis for the zmq matchmaker
> >>>   4. We (OpenStack) also use memcached for caching and as the oslo
> >>> caching lib becomes available - or a wrapper on top of dogpile.cache -
> >>> Redis may be used in place of memcached in more and more deployments.
> >>>   5. Ceilometer's recommended storage driver is still MongoDB,
> >>> although
> >>> Ceilometer has now support for sqlalchemy. (Please correct me if I'm
> >>> wrong).
> >>>
> >>> That being said, it's obvious we already, to some extent, promote some
> >>> NoSQL technologies. However, for the sake of the discussion, lets 
> >>> assume
> >>> we don't.
> >>>
> >>> I truly believe, with my OpenStack (not Zaqar's) hat on, that we can't
> >>> keep avoiding these technologies. NoSQL technologies have been around
> >>> for years and we should be prepared - including OpenStack operators - 
> >>> to
> >>> support these technologies. Not every tool is good for all tasks - one
> >>> of the reasons we removed the sqlalchemy driver in the first place -
> >>> therefore it's impossible to keep an homogeneous environment for all
> >>> services.
> >>>
> >>
> >> I whole heartedly agree that non traditional storage technologies that
> >> are becoming mainstream are good candidates for use cases where SQL
> >> based storage gets in the way. I wish there wasn't so much FUD
> >> (warranted or not) about MongoDB, but that is the reality we live in.
> >>
> >>> With this, I'm not suggesting to ignore the risks and the extra burden
> >>> this adds but, instead of attempting to avoid it completely by not
> >>> evolving the stack of services we provide, we should probably work on
> >>> defining a reasonable subset of NoSQL services we are OK with
> >>> supporting. This will help making the burden smaller and it'll give
> >>> operators the option to choose.
> >>>
> >>> [0] http://blog.flaper87.com/post/marconi-amqp-see-you-later/
> >>>
> >>>
> >>> - Concern on should we really reinvent a queue system rather than
> >>> piggyback on one
> >>>
> >>> As mentioned in the meeting on Tuesday, Zaqar is not reinventing 
> >>> message
> >>> brokers. Zaqar provides a service akin to SQS from AWS with an 
> >>> OpenStack
> >>> flavor on top. [0]
> >>>
> >>
> >> I think Zaqar is more like SMTP and IMAP than AMQP. You're not really
> >> trying to connect two processes in real time. You're trying to do fully
> >> asynchronous messaging with fully randomized access to any message.
> >>
> >> Perhaps somebody should explore whether the approaches taken by large
> >> scale IMAP providers could be applied to Zaqar.
> >>
> >> Anyway, I can't imagine writ

Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-12 Thread Flavio Percoco
On 09/12/2014 01:56 PM, Flavio Percoco wrote:
> On 09/12/2014 11:36 AM, Thierry Carrez wrote:
>> Flavio Percoco wrote:
>>> On 09/12/2014 12:14 AM, Zane Bitter wrote:
 The final question is the one of arbitrary access to messages in the
 queue (or "queue" if you prefer). Flavio indicated that this effectively
 came for free with their implementation of Pub-Sub. IMHO it is
 unnecessary and limits the choice of potential back ends in the future.
 I would personally be +1 on removing it from the v2 API, and also +1 on
 the v2 API shipping in Kilo so that as few new adopters as possible get
 stuck with the limited choices of back-end. I hope that would resolve
 Clint's concerns that we need a separate, light-weight queue system; I
 personally don't believe we need two projects, even though I agree that
 all of the use cases I personally care about could probably be satisfied
 without Pub-Sub.
>>>
>>> Right, being able to support other backends is one of the reasons we're
>>> looking forward to remove the support for arbitrary access to messages.
>>> As of now, the plan is to remove that endpoint unless a very good use
>>> case comes up that makes supporting other backends not worth it, which I
>>> doubt. The feedback from Zaqar's early adopters is that the endpoint is
>>> indeed not useful.
>>
>> Thanks Zane, that was indeed useful. I agree with you it would be better
>> to avoid needing 2 separate projects for such close use cases.
> 
> +1
> 
>> Let's assume we remove arbitrary access to messages in v2. When you say
>> it would remove limits on the choice of potential backends, does that
>> mean we could have a pure queue backend (like RabbitMQ), at least in
>> theory ? Would a ZaqarV2 address all of Clint and Devananda's concerns
>> about queue semantics ? If yes, then the graduation question becomes,
>> how likely is that work to be completed early enough in Kilo.
>>
>> If it's a no-brainer and takes a week to sort out, I think we could
>> approve Zaqar's Kilo graduation, even if that stretches the "no major
>> API rewrite planned" requirement.
> 
> Let me break the above down into several points so we can discuss them
> separately:
> 
> - Removing that endpoint won't take more than a week. It's an API change
> and it won't affect the existing storage drivers.

For the sake of discussion and to provide more info on this point, I've
done this (there are still some tests to clean-up but that's basically
all that's required):

https://review.openstack.org/#/c/121141/

Flavio

> 
> - Removing that endpoint will certainly make the adoption of other
> messaging technologies easier but there are other things to consider
> besides that specific endpoint (some of them were stated here[0]). In
> any case, removing the endpoint definitely makes it easier.
> 
> - Besides the random access to messages, I'm not clear what other
> concerns there are with regards the current semantics. It'd be nice if
> we could recollect them in this section and discuss them. I took a look
> at the other emails in this thread and it seems to me that the concerns
> that have been raised are more oriented to the project scope and
> use-cases. I also looked at the meeting logs again[1] and the only
> concern related to the semantics I found is about the
> `get-message-by-id` endpoint. Please, correct me if I'm wrong.
> 
> 
> [0] http://blog.flaper87.com/post/marconi-amqp-see-you-later/
> [1]
> http://eavesdrop.openstack.org/meetings/tc/2014/tc.2014-09-09-20.01.log.html
> 
> Flavio
> 
>> But if we think this needs careful discussion so that the v2 API design
>> (and backend support) satisfies the widest set of users, then incubating
>> for another cycle while v2 is implemented seems like the right course of
>> action. We shouldn't graduate if there is any risk we would end up with
>> ZaqarV1 in Kilo, and then have to deprecate it for n cycles just because
>> it was shipped in the official release and therefore inherits its API
>> deprecation rules.
>>
>> Regards,
>>
> 
> 


-- 
@flaper87
Flavio Percoco

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-12 Thread Gordon Sim

On 09/12/2014 09:50 AM, Flavio Percoco wrote:

Zaqar supports once and only once delivery.


For the transfer from Zaqar to consumers it does (providing the claim id 
can be recovered). For transfer from producers to Zaqar I believe it is 
more limited.


If the connection to Zaqar fails during a post, the sender can't tell 
whether the message was successfully enqueued or not.


It could try to determine this is by browsing the entire queue looking 
for a matching body. However thatt would be awkward and in any case the 
absence of the message could mean that it wasn't enqueued or that it was 
already consumed and deleted.


One way of handling this is to have the post return a unique url to 
which the message(s) are put or posted. The sender can then repost 
(re-put) to this in the event of failure and the server can determine 
whether it already processed the publications. Alternatively the client 
can be required to generate a unique id on which the server can 
de-deduplicate.


The ActiveMQ REST interface supports both of these approaches: 
http://activemq.apache.org/restful-queue.html


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-12 Thread Zane Bitter

On 12/09/14 04:50, Flavio Percoco wrote:

On 09/12/2014 12:14 AM, Zane Bitter wrote:

>However, Zaqar also supports the Pub-Sub model of messaging. I believe,
>but would like Flavio to confirm, that this is what is meant when the
>Zaqar team say that Zaqar is about messaging in general and not just
>queuing. That is to say, it is possible for multiple consumers to
>intentionally consume the same message, with each maintaining its own
>pointer in the queue. (Another way to think of this is that messages can
>be multicast to multiple virtual queues, with data de-duplication
>between them.) To a relative novice in the field like me, the difference
>between this and queuing sounds pretty academic :P. Call it what you
>will, it seems like a reasonable thing to implement to me.


Correct, this and other messaging patterns supported by Zaqar make it a
messaging service, which as Gordon mentioned in another email is just a
more generic term. Messages are the most important resource in Zaqar and
providing good, common and scalable patterns to access those messages is
what we strive for in Zaqar's API.


Thanks Flavio! I think we are more or less on the same page :)

Maybe you could clarify what the "other messaging patterns" are exactly, 
since that seems to be one of the points of confusion/contention.


cheers,
Zane.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-12 Thread Zane Bitter

On 11/09/14 19:05, Jay Pipes wrote:

On 09/11/2014 04:09 PM, Zane Bitter wrote:

Swift is the current exception here, but one could argue, and people
have[2], that Swift is also the only project that actually conforms to
our stated design tenets for OpenStack. I'd struggle to tell the Zaqar
folks they've done the Wrong Thing... especially when abandoning the
RDBMS driver was done largely at the direction of the TC iirc.




[2] http://blog.linux2go.dk/2013/08/30/openstack-design-tenets-part-2/


No offense to Soren, who wrote some interesting and poignant things, nor
to the Swift developers, who continue to produce excellent work, but
Swift is object storage. It is a data plane system with a small API
surface, a very limited functional domain, and a small, inflexible
storage schema (which is perfectly fine for its use cases). It's needs
for a relational database are nearly non-existent. It replicates a
SQLite database around using rsync [1]. Try doing that with a schema of
any complexity and you will quickly find the limitations of such a
strategy.

If Nova was to take Soren's advice and implement its data-access layer
on top of Cassandra or Riak, we would just end up re-inventing SQL Joins
in Python-land. I've said it before, and I'll say it again. In Nova at
least, the SQL schema is complex because the problem domain is complex.
That means lots of relations, lots of JOINs, and that means the best way
to query for that data is via an RDBMS.

And I say that knowing just how *poor* some of the queries are in Nova!


I wasn't trying to suggest that Nova should change (if there was another 
project I had in mind while reading that it would have been Heat, not 
Nova). My point was that it's understandable that Zaqar, which is *also* 
a data-plane service with a small API surface and a limited functional 
domain, doesn't have the same architecture as Nova (just as Swift 
doesn't) and that it's probably counter-productive to force it into that 
architecture purely because a bunch of other things use it.



For projects like Swift, Zaqar, even Keystone, Glance and Cinder, a
non-RDBMS solution might be a perfectly reasonable solution for the
underlying data storage and access layer (and for the record, I never
said that Zaqar should or should not use an RDBMS for its storage). For
complex control plane software like Nova, though, an RDBMS is the best
tool for the job given the current lay of the land in open source data
storage solutions matched with Nova's complex query and transactional
requirements.


+1


Folks in these other programs have actually, you know, thought about
these kinds of things and had serious discussions about alternatives. It
would be nice to have someone acknowledge that instead of snarky
comments implying everyone else "has it wrong".


I didn't mean to imply that anybody else has it wrong (although FWIW I 
do think that Heat probably has it wrong), and I apologise to anyone who 
interpreted it that way.



Going back in my hole,
-jay


No! Let's talk about Zaqar :)

cheers,
Zane.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-12 Thread Flavio Percoco
On 09/12/2014 11:36 AM, Thierry Carrez wrote:
> Flavio Percoco wrote:
>> On 09/12/2014 12:14 AM, Zane Bitter wrote:
>>> The final question is the one of arbitrary access to messages in the
>>> queue (or "queue" if you prefer). Flavio indicated that this effectively
>>> came for free with their implementation of Pub-Sub. IMHO it is
>>> unnecessary and limits the choice of potential back ends in the future.
>>> I would personally be +1 on removing it from the v2 API, and also +1 on
>>> the v2 API shipping in Kilo so that as few new adopters as possible get
>>> stuck with the limited choices of back-end. I hope that would resolve
>>> Clint's concerns that we need a separate, light-weight queue system; I
>>> personally don't believe we need two projects, even though I agree that
>>> all of the use cases I personally care about could probably be satisfied
>>> without Pub-Sub.
>>
>> Right, being able to support other backends is one of the reasons we're
>> looking forward to remove the support for arbitrary access to messages.
>> As of now, the plan is to remove that endpoint unless a very good use
>> case comes up that makes supporting other backends not worth it, which I
>> doubt. The feedback from Zaqar's early adopters is that the endpoint is
>> indeed not useful.
> 
> Thanks Zane, that was indeed useful. I agree with you it would be better
> to avoid needing 2 separate projects for such close use cases.

+1

> Let's assume we remove arbitrary access to messages in v2. When you say
> it would remove limits on the choice of potential backends, does that
> mean we could have a pure queue backend (like RabbitMQ), at least in
> theory ? Would a ZaqarV2 address all of Clint and Devananda's concerns
> about queue semantics ? If yes, then the graduation question becomes,
> how likely is that work to be completed early enough in Kilo.
> 
> If it's a no-brainer and takes a week to sort out, I think we could
> approve Zaqar's Kilo graduation, even if that stretches the "no major
> API rewrite planned" requirement.

Let me break the above down into several points so we can discuss them
separately:

- Removing that endpoint won't take more than a week. It's an API change
and it won't affect the existing storage drivers.

- Removing that endpoint will certainly make the adoption of other
messaging technologies easier but there are other things to consider
besides that specific endpoint (some of them were stated here[0]). In
any case, removing the endpoint definitely makes it easier.

- Besides the random access to messages, I'm not clear what other
concerns there are with regards the current semantics. It'd be nice if
we could recollect them in this section and discuss them. I took a look
at the other emails in this thread and it seems to me that the concerns
that have been raised are more oriented to the project scope and
use-cases. I also looked at the meeting logs again[1] and the only
concern related to the semantics I found is about the
`get-message-by-id` endpoint. Please, correct me if I'm wrong.


[0] http://blog.flaper87.com/post/marconi-amqp-see-you-later/
[1]
http://eavesdrop.openstack.org/meetings/tc/2014/tc.2014-09-09-20.01.log.html

Flavio

> But if we think this needs careful discussion so that the v2 API design
> (and backend support) satisfies the widest set of users, then incubating
> for another cycle while v2 is implemented seems like the right course of
> action. We shouldn't graduate if there is any risk we would end up with
> ZaqarV1 in Kilo, and then have to deprecate it for n cycles just because
> it was shipped in the official release and therefore inherits its API
> deprecation rules.
> 
> Regards,
> 


-- 
@flaper87
Flavio Percoco

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-12 Thread Mark McLoughlin
On Wed, 2014-09-10 at 12:46 -0700, Monty Taylor wrote:
> On 09/09/2014 07:04 PM, Samuel Merritt wrote:
> > On 9/9/14, 4:47 PM, Devananda van der Veen wrote:

> >> The questions now before us are:
> >> - should OpenStack include, in the integrated release, a
> >> messaging-as-a-service component?
> >
> > I certainly think so. I've worked on a few reasonable-scale web
> > applications, and they all followed the same pattern: HTTP app servers
> > serving requests quickly, background workers for long-running tasks, and
> > some sort of durable message-broker/queue-server thing for conveying
> > work from the first to the second.
> >
> > A quick straw poll of my nearby coworkers shows that every non-trivial
> > web application that they've worked on in the last decade follows the
> > same pattern.
> >
> > While not *every* application needs such a thing, web apps are quite
> > common these days, and Zaqar satisfies one of their big requirements.
> > Not only that, it does so in a way that requires much less babysitting
> > than run-your-own-broker does.
> 
> Right. But here's the thing.
> 
> What you just described is what we all thought zaqar was aiming to be in 
> the beginning. We did not think it was a GOOD implementation of that, so 
> while we agreed that it would be useful to have one of those, we were 
> not crazy about the implementation.

Those generalizations are uncomfortably sweeping.

What Samuel just described is one of the messaging patterns that Zaqar
implements and some (members of the TC?) believed that this messaging
pattern was the only pattern that Zaqar aimed to implement.

Some (members of the TC?) formed strong, negative opinions about how
this messaging pattern was implemented, but some/all of those same
people agreed a messaging API implementing those semantics would be a
useful thing to have.

Mark.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-12 Thread Thierry Carrez
Flavio Percoco wrote:
> On 09/12/2014 12:14 AM, Zane Bitter wrote:
>> The final question is the one of arbitrary access to messages in the
>> queue (or "queue" if you prefer). Flavio indicated that this effectively
>> came for free with their implementation of Pub-Sub. IMHO it is
>> unnecessary and limits the choice of potential back ends in the future.
>> I would personally be +1 on removing it from the v2 API, and also +1 on
>> the v2 API shipping in Kilo so that as few new adopters as possible get
>> stuck with the limited choices of back-end. I hope that would resolve
>> Clint's concerns that we need a separate, light-weight queue system; I
>> personally don't believe we need two projects, even though I agree that
>> all of the use cases I personally care about could probably be satisfied
>> without Pub-Sub.
> 
> Right, being able to support other backends is one of the reasons we're
> looking forward to remove the support for arbitrary access to messages.
> As of now, the plan is to remove that endpoint unless a very good use
> case comes up that makes supporting other backends not worth it, which I
> doubt. The feedback from Zaqar's early adopters is that the endpoint is
> indeed not useful.

Thanks Zane, that was indeed useful. I agree with you it would be better
to avoid needing 2 separate projects for such close use cases.

Let's assume we remove arbitrary access to messages in v2. When you say
it would remove limits on the choice of potential backends, does that
mean we could have a pure queue backend (like RabbitMQ), at least in
theory ? Would a ZaqarV2 address all of Clint and Devananda's concerns
about queue semantics ? If yes, then the graduation question becomes,
how likely is that work to be completed early enough in Kilo.

If it's a no-brainer and takes a week to sort out, I think we could
approve Zaqar's Kilo graduation, even if that stretches the "no major
API rewrite planned" requirement.

But if we think this needs careful discussion so that the v2 API design
(and backend support) satisfies the widest set of users, then incubating
for another cycle while v2 is implemented seems like the right course of
action. We shouldn't graduate if there is any risk we would end up with
ZaqarV1 in Kilo, and then have to deprecate it for n cycles just because
it was shipped in the official release and therefore inherits its API
deprecation rules.

Regards,

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-12 Thread Flavio Percoco
On 09/12/2014 12:14 AM, Zane Bitter wrote:
> On 09/09/14 15:03, Monty Taylor wrote:
>> On 09/04/2014 01:30 AM, Clint Byrum wrote:
>>> Excerpts from Flavio Percoco's message of 2014-09-04 00:08:47 -0700:
 Greetings,

 Last Tuesday the TC held the first graduation review for Zaqar. During
 the meeting some concerns arose. I've listed those concerns below with
 some comments hoping that it will help starting a discussion before the
 next meeting. In addition, I've added some comments about the project
 stability at the bottom and an etherpad link pointing to a list of use
 cases for Zaqar.

>>>
>>> Hi Flavio. This was an interesting read. As somebody whose attention has
>>> recently been drawn to Zaqar, I am quite interested in seeing it
>>> graduate.
>>>
 # Concerns

 - Concern on operational burden of requiring NoSQL deploy expertise to
 the mix of openstack operational skills

 For those of you not familiar with Zaqar, it currently supports 2 nosql
 drivers - MongoDB and Redis - and those are the only 2 drivers it
 supports for now. This will require operators willing to use Zaqar to
 maintain a new (?) NoSQL technology in their system. Before expressing
 our thoughts on this matter, let me say that:

  1. By removing the SQLAlchemy driver, we basically removed the
 chance
 for operators to use an already deployed "OpenStack-technology"
  2. Zaqar won't be backed by any AMQP based messaging technology
 for
 now. Here's[0] a summary of the research the team (mostly done by
 Victoria) did during Juno
  3. We (OpenStack) used to require Redis for the zmq matchmaker
  4. We (OpenStack) also use memcached for caching and as the oslo
 caching lib becomes available - or a wrapper on top of dogpile.cache -
 Redis may be used in place of memcached in more and more deployments.
  5. Ceilometer's recommended storage driver is still MongoDB,
 although
 Ceilometer has now support for sqlalchemy. (Please correct me if I'm
 wrong).

 That being said, it's obvious we already, to some extent, promote some
 NoSQL technologies. However, for the sake of the discussion, lets
 assume
 we don't.

 I truly believe, with my OpenStack (not Zaqar's) hat on, that we can't
 keep avoiding these technologies. NoSQL technologies have been around
 for years and we should be prepared - including OpenStack operators
 - to
 support these technologies. Not every tool is good for all tasks - one
 of the reasons we removed the sqlalchemy driver in the first place -
 therefore it's impossible to keep an homogeneous environment for all
 services.

>>>
>>> I whole heartedly agree that non traditional storage technologies that
>>> are becoming mainstream are good candidates for use cases where SQL
>>> based storage gets in the way. I wish there wasn't so much FUD
>>> (warranted or not) about MongoDB, but that is the reality we live in.
>>>
 With this, I'm not suggesting to ignore the risks and the extra burden
 this adds but, instead of attempting to avoid it completely by not
 evolving the stack of services we provide, we should probably work on
 defining a reasonable subset of NoSQL services we are OK with
 supporting. This will help making the burden smaller and it'll give
 operators the option to choose.

 [0] http://blog.flaper87.com/post/marconi-amqp-see-you-later/


 - Concern on should we really reinvent a queue system rather than
 piggyback on one

 As mentioned in the meeting on Tuesday, Zaqar is not reinventing
 message
 brokers. Zaqar provides a service akin to SQS from AWS with an
 OpenStack
 flavor on top. [0]

>>>
>>> I think Zaqar is more like SMTP and IMAP than AMQP. You're not really
>>> trying to connect two processes in real time. You're trying to do fully
>>> asynchronous messaging with fully randomized access to any message.
>>>
>>> Perhaps somebody should explore whether the approaches taken by large
>>> scale IMAP providers could be applied to Zaqar.
>>>
>>> Anyway, I can't imagine writing a system to intentionally use the
>>> semantics of IMAP and SMTP. I'd be very interested in seeing actual use
>>> cases for it, apologies if those have been posted before.
>>
>> It seems like you're EITHER describing something called XMPP that has at
>> least one open source scalable backend called ejabberd. OR, you've
>> actually hit the nail on the head with bringing up SMTP and IMAP but for
>> some reason that feels strange.
>>
>> SMTP and IMAP already implement every feature you've described, as well
>> as retries/failover/HA and a fully end to end secure transport (if
>> installed properly) If you don't actually set them up to run as a public
>> messaging interface but just as a cloud-local exchange, then you could
>> get by with very low overh

Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-12 Thread Flavio Percoco
On 09/12/2014 03:29 AM, Clint Byrum wrote:
> Excerpts from Zane Bitter's message of 2014-09-11 15:21:26 -0700:
>> On 09/09/14 19:56, Clint Byrum wrote:
>>> Excerpts from Samuel Merritt's message of 2014-09-09 16:12:09 -0700:
 On 9/9/14, 12:03 PM, Monty Taylor wrote:
> On 09/04/2014 01:30 AM, Clint Byrum wrote:
>> Excerpts from Flavio Percoco's message of 2014-09-04 00:08:47 -0700:
>>> Greetings,
>>>
>>> Last Tuesday the TC held the first graduation review for Zaqar. During
>>> the meeting some concerns arose. I've listed those concerns below with
>>> some comments hoping that it will help starting a discussion before the
>>> next meeting. In addition, I've added some comments about the project
>>> stability at the bottom and an etherpad link pointing to a list of use
>>> cases for Zaqar.
>>>
>>
>> Hi Flavio. This was an interesting read. As somebody whose attention has
>> recently been drawn to Zaqar, I am quite interested in seeing it
>> graduate.
>>
>>> # Concerns
>>>
>>> - Concern on operational burden of requiring NoSQL deploy expertise to
>>> the mix of openstack operational skills
>>>
>>> For those of you not familiar with Zaqar, it currently supports 2 nosql
>>> drivers - MongoDB and Redis - and those are the only 2 drivers it
>>> supports for now. This will require operators willing to use Zaqar to
>>> maintain a new (?) NoSQL technology in their system. Before expressing
>>> our thoughts on this matter, let me say that:
>>>
>>>   1. By removing the SQLAlchemy driver, we basically removed the
>>> chance
>>> for operators to use an already deployed "OpenStack-technology"
>>>   2. Zaqar won't be backed by any AMQP based messaging technology 
>>> for
>>> now. Here's[0] a summary of the research the team (mostly done by
>>> Victoria) did during Juno
>>>   3. We (OpenStack) used to require Redis for the zmq matchmaker
>>>   4. We (OpenStack) also use memcached for caching and as the oslo
>>> caching lib becomes available - or a wrapper on top of dogpile.cache -
>>> Redis may be used in place of memcached in more and more deployments.
>>>   5. Ceilometer's recommended storage driver is still MongoDB,
>>> although
>>> Ceilometer has now support for sqlalchemy. (Please correct me if I'm
>>> wrong).
>>>
>>> That being said, it's obvious we already, to some extent, promote some
>>> NoSQL technologies. However, for the sake of the discussion, lets assume
>>> we don't.
>>>
>>> I truly believe, with my OpenStack (not Zaqar's) hat on, that we can't
>>> keep avoiding these technologies. NoSQL technologies have been around
>>> for years and we should be prepared - including OpenStack operators - to
>>> support these technologies. Not every tool is good for all tasks - one
>>> of the reasons we removed the sqlalchemy driver in the first place -
>>> therefore it's impossible to keep an homogeneous environment for all
>>> services.
>>>
>>
>> I whole heartedly agree that non traditional storage technologies that
>> are becoming mainstream are good candidates for use cases where SQL
>> based storage gets in the way. I wish there wasn't so much FUD
>> (warranted or not) about MongoDB, but that is the reality we live in.
>>
>>> With this, I'm not suggesting to ignore the risks and the extra burden
>>> this adds but, instead of attempting to avoid it completely by not
>>> evolving the stack of services we provide, we should probably work on
>>> defining a reasonable subset of NoSQL services we are OK with
>>> supporting. This will help making the burden smaller and it'll give
>>> operators the option to choose.
>>>
>>> [0] http://blog.flaper87.com/post/marconi-amqp-see-you-later/
>>>
>>>
>>> - Concern on should we really reinvent a queue system rather than
>>> piggyback on one
>>>
>>> As mentioned in the meeting on Tuesday, Zaqar is not reinventing message
>>> brokers. Zaqar provides a service akin to SQS from AWS with an OpenStack
>>> flavor on top. [0]
>>>
>>
>> I think Zaqar is more like SMTP and IMAP than AMQP. You're not really
>> trying to connect two processes in real time. You're trying to do fully
>> asynchronous messaging with fully randomized access to any message.
>>
>> Perhaps somebody should explore whether the approaches taken by large
>> scale IMAP providers could be applied to Zaqar.
>>
>> Anyway, I can't imagine writing a system to intentionally use the
>> semantics of IMAP and SMTP. I'd be very interested in seeing actual use
>> cases for it, apologies if those have been posted before.
>
> It seems like you're EITHER describing something called XMPP that has at
> least one open source scalable backend called ejabberd. 

Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-11 Thread Clint Byrum
Excerpts from Zane Bitter's message of 2014-09-11 15:21:26 -0700:
> On 09/09/14 19:56, Clint Byrum wrote:
> > Excerpts from Samuel Merritt's message of 2014-09-09 16:12:09 -0700:
> >> On 9/9/14, 12:03 PM, Monty Taylor wrote:
> >>> On 09/04/2014 01:30 AM, Clint Byrum wrote:
>  Excerpts from Flavio Percoco's message of 2014-09-04 00:08:47 -0700:
> > Greetings,
> >
> > Last Tuesday the TC held the first graduation review for Zaqar. During
> > the meeting some concerns arose. I've listed those concerns below with
> > some comments hoping that it will help starting a discussion before the
> > next meeting. In addition, I've added some comments about the project
> > stability at the bottom and an etherpad link pointing to a list of use
> > cases for Zaqar.
> >
> 
>  Hi Flavio. This was an interesting read. As somebody whose attention has
>  recently been drawn to Zaqar, I am quite interested in seeing it
>  graduate.
> 
> > # Concerns
> >
> > - Concern on operational burden of requiring NoSQL deploy expertise to
> > the mix of openstack operational skills
> >
> > For those of you not familiar with Zaqar, it currently supports 2 nosql
> > drivers - MongoDB and Redis - and those are the only 2 drivers it
> > supports for now. This will require operators willing to use Zaqar to
> > maintain a new (?) NoSQL technology in their system. Before expressing
> > our thoughts on this matter, let me say that:
> >
> >   1. By removing the SQLAlchemy driver, we basically removed the
> > chance
> > for operators to use an already deployed "OpenStack-technology"
> >   2. Zaqar won't be backed by any AMQP based messaging technology 
> > for
> > now. Here's[0] a summary of the research the team (mostly done by
> > Victoria) did during Juno
> >   3. We (OpenStack) used to require Redis for the zmq matchmaker
> >   4. We (OpenStack) also use memcached for caching and as the oslo
> > caching lib becomes available - or a wrapper on top of dogpile.cache -
> > Redis may be used in place of memcached in more and more deployments.
> >   5. Ceilometer's recommended storage driver is still MongoDB,
> > although
> > Ceilometer has now support for sqlalchemy. (Please correct me if I'm
> > wrong).
> >
> > That being said, it's obvious we already, to some extent, promote some
> > NoSQL technologies. However, for the sake of the discussion, lets assume
> > we don't.
> >
> > I truly believe, with my OpenStack (not Zaqar's) hat on, that we can't
> > keep avoiding these technologies. NoSQL technologies have been around
> > for years and we should be prepared - including OpenStack operators - to
> > support these technologies. Not every tool is good for all tasks - one
> > of the reasons we removed the sqlalchemy driver in the first place -
> > therefore it's impossible to keep an homogeneous environment for all
> > services.
> >
> 
>  I whole heartedly agree that non traditional storage technologies that
>  are becoming mainstream are good candidates for use cases where SQL
>  based storage gets in the way. I wish there wasn't so much FUD
>  (warranted or not) about MongoDB, but that is the reality we live in.
> 
> > With this, I'm not suggesting to ignore the risks and the extra burden
> > this adds but, instead of attempting to avoid it completely by not
> > evolving the stack of services we provide, we should probably work on
> > defining a reasonable subset of NoSQL services we are OK with
> > supporting. This will help making the burden smaller and it'll give
> > operators the option to choose.
> >
> > [0] http://blog.flaper87.com/post/marconi-amqp-see-you-later/
> >
> >
> > - Concern on should we really reinvent a queue system rather than
> > piggyback on one
> >
> > As mentioned in the meeting on Tuesday, Zaqar is not reinventing message
> > brokers. Zaqar provides a service akin to SQS from AWS with an OpenStack
> > flavor on top. [0]
> >
> 
>  I think Zaqar is more like SMTP and IMAP than AMQP. You're not really
>  trying to connect two processes in real time. You're trying to do fully
>  asynchronous messaging with fully randomized access to any message.
> 
>  Perhaps somebody should explore whether the approaches taken by large
>  scale IMAP providers could be applied to Zaqar.
> 
>  Anyway, I can't imagine writing a system to intentionally use the
>  semantics of IMAP and SMTP. I'd be very interested in seeing actual use
>  cases for it, apologies if those have been posted before.
> >>>
> >>> It seems like you're EITHER describing something called XMPP that has at
> >>> least one open source scalable backend called ejabberd. OR, you've
> >>> actually hit the nail on the 

Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-11 Thread Jay Pipes

On 09/11/2014 04:09 PM, Zane Bitter wrote:

Swift is the current exception here, but one could argue, and people
have[2], that Swift is also the only project that actually conforms to
our stated design tenets for OpenStack. I'd struggle to tell the Zaqar
folks they've done the Wrong Thing... especially when abandoning the
RDBMS driver was done largely at the direction of the TC iirc.




[2] http://blog.linux2go.dk/2013/08/30/openstack-design-tenets-part-2/


No offense to Soren, who wrote some interesting and poignant things, nor 
to the Swift developers, who continue to produce excellent work, but 
Swift is object storage. It is a data plane system with a small API 
surface, a very limited functional domain, and a small, inflexible 
storage schema (which is perfectly fine for its use cases). It's needs 
for a relational database are nearly non-existent. It replicates a 
SQLite database around using rsync [1]. Try doing that with a schema of 
any complexity and you will quickly find the limitations of such a strategy.


If Nova was to take Soren's advice and implement its data-access layer 
on top of Cassandra or Riak, we would just end up re-inventing SQL Joins 
in Python-land. I've said it before, and I'll say it again. In Nova at 
least, the SQL schema is complex because the problem domain is complex. 
That means lots of relations, lots of JOINs, and that means the best way 
to query for that data is via an RDBMS.


And I say that knowing just how *poor* some of the queries are in Nova!

For projects like Swift, Zaqar, even Keystone, Glance and Cinder, a 
non-RDBMS solution might be a perfectly reasonable solution for the 
underlying data storage and access layer (and for the record, I never 
said that Zaqar should or should not use an RDBMS for its storage). For 
complex control plane software like Nova, though, an RDBMS is the best 
tool for the job given the current lay of the land in open source data 
storage solutions matched with Nova's complex query and transactional 
requirements.


Folks in these other programs have actually, you know, thought about 
these kinds of things and had serious discussions about alternatives. It 
would be nice to have someone acknowledge that instead of snarky 
comments implying everyone else "has it wrong".


Going back in my hole,
-jay

[1] 
https://github.com/openstack/swift/blob/master/swift/common/db_replicator.py#L232


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-11 Thread Zane Bitter

On 09/09/14 19:56, Clint Byrum wrote:

Excerpts from Samuel Merritt's message of 2014-09-09 16:12:09 -0700:

On 9/9/14, 12:03 PM, Monty Taylor wrote:

On 09/04/2014 01:30 AM, Clint Byrum wrote:

Excerpts from Flavio Percoco's message of 2014-09-04 00:08:47 -0700:

Greetings,

Last Tuesday the TC held the first graduation review for Zaqar. During
the meeting some concerns arose. I've listed those concerns below with
some comments hoping that it will help starting a discussion before the
next meeting. In addition, I've added some comments about the project
stability at the bottom and an etherpad link pointing to a list of use
cases for Zaqar.



Hi Flavio. This was an interesting read. As somebody whose attention has
recently been drawn to Zaqar, I am quite interested in seeing it
graduate.


# Concerns

- Concern on operational burden of requiring NoSQL deploy expertise to
the mix of openstack operational skills

For those of you not familiar with Zaqar, it currently supports 2 nosql
drivers - MongoDB and Redis - and those are the only 2 drivers it
supports for now. This will require operators willing to use Zaqar to
maintain a new (?) NoSQL technology in their system. Before expressing
our thoughts on this matter, let me say that:

  1. By removing the SQLAlchemy driver, we basically removed the
chance
for operators to use an already deployed "OpenStack-technology"
  2. Zaqar won't be backed by any AMQP based messaging technology for
now. Here's[0] a summary of the research the team (mostly done by
Victoria) did during Juno
  3. We (OpenStack) used to require Redis for the zmq matchmaker
  4. We (OpenStack) also use memcached for caching and as the oslo
caching lib becomes available - or a wrapper on top of dogpile.cache -
Redis may be used in place of memcached in more and more deployments.
  5. Ceilometer's recommended storage driver is still MongoDB,
although
Ceilometer has now support for sqlalchemy. (Please correct me if I'm
wrong).

That being said, it's obvious we already, to some extent, promote some
NoSQL technologies. However, for the sake of the discussion, lets assume
we don't.

I truly believe, with my OpenStack (not Zaqar's) hat on, that we can't
keep avoiding these technologies. NoSQL technologies have been around
for years and we should be prepared - including OpenStack operators - to
support these technologies. Not every tool is good for all tasks - one
of the reasons we removed the sqlalchemy driver in the first place -
therefore it's impossible to keep an homogeneous environment for all
services.



I whole heartedly agree that non traditional storage technologies that
are becoming mainstream are good candidates for use cases where SQL
based storage gets in the way. I wish there wasn't so much FUD
(warranted or not) about MongoDB, but that is the reality we live in.


With this, I'm not suggesting to ignore the risks and the extra burden
this adds but, instead of attempting to avoid it completely by not
evolving the stack of services we provide, we should probably work on
defining a reasonable subset of NoSQL services we are OK with
supporting. This will help making the burden smaller and it'll give
operators the option to choose.

[0] http://blog.flaper87.com/post/marconi-amqp-see-you-later/


- Concern on should we really reinvent a queue system rather than
piggyback on one

As mentioned in the meeting on Tuesday, Zaqar is not reinventing message
brokers. Zaqar provides a service akin to SQS from AWS with an OpenStack
flavor on top. [0]



I think Zaqar is more like SMTP and IMAP than AMQP. You're not really
trying to connect two processes in real time. You're trying to do fully
asynchronous messaging with fully randomized access to any message.

Perhaps somebody should explore whether the approaches taken by large
scale IMAP providers could be applied to Zaqar.

Anyway, I can't imagine writing a system to intentionally use the
semantics of IMAP and SMTP. I'd be very interested in seeing actual use
cases for it, apologies if those have been posted before.


It seems like you're EITHER describing something called XMPP that has at
least one open source scalable backend called ejabberd. OR, you've
actually hit the nail on the head with bringing up SMTP and IMAP but for
some reason that feels strange.

SMTP and IMAP already implement every feature you've described, as well
as retries/failover/HA and a fully end to end secure transport (if
installed properly) If you don't actually set them up to run as a public
messaging interface but just as a cloud-local exchange, then you could
get by with very low overhead for a massive throughput - it can very
easily be run on a single machine for Sean's simplicity, and could just
as easily be scaled out using well known techniques for public cloud
sized deployments?

So why not use existing daemons that do this? You could still use the
REST API you've got, but instead of writing it to a mongo backend and
trying to implement all of

Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-11 Thread Zane Bitter

On 09/09/14 15:03, Monty Taylor wrote:

On 09/04/2014 01:30 AM, Clint Byrum wrote:

Excerpts from Flavio Percoco's message of 2014-09-04 00:08:47 -0700:

Greetings,

Last Tuesday the TC held the first graduation review for Zaqar. During
the meeting some concerns arose. I've listed those concerns below with
some comments hoping that it will help starting a discussion before the
next meeting. In addition, I've added some comments about the project
stability at the bottom and an etherpad link pointing to a list of use
cases for Zaqar.



Hi Flavio. This was an interesting read. As somebody whose attention has
recently been drawn to Zaqar, I am quite interested in seeing it
graduate.


# Concerns

- Concern on operational burden of requiring NoSQL deploy expertise to
the mix of openstack operational skills

For those of you not familiar with Zaqar, it currently supports 2 nosql
drivers - MongoDB and Redis - and those are the only 2 drivers it
supports for now. This will require operators willing to use Zaqar to
maintain a new (?) NoSQL technology in their system. Before expressing
our thoughts on this matter, let me say that:

 1. By removing the SQLAlchemy driver, we basically removed the
chance
for operators to use an already deployed "OpenStack-technology"
 2. Zaqar won't be backed by any AMQP based messaging technology for
now. Here's[0] a summary of the research the team (mostly done by
Victoria) did during Juno
 3. We (OpenStack) used to require Redis for the zmq matchmaker
 4. We (OpenStack) also use memcached for caching and as the oslo
caching lib becomes available - or a wrapper on top of dogpile.cache -
Redis may be used in place of memcached in more and more deployments.
 5. Ceilometer's recommended storage driver is still MongoDB,
although
Ceilometer has now support for sqlalchemy. (Please correct me if I'm
wrong).

That being said, it's obvious we already, to some extent, promote some
NoSQL technologies. However, for the sake of the discussion, lets assume
we don't.

I truly believe, with my OpenStack (not Zaqar's) hat on, that we can't
keep avoiding these technologies. NoSQL technologies have been around
for years and we should be prepared - including OpenStack operators - to
support these technologies. Not every tool is good for all tasks - one
of the reasons we removed the sqlalchemy driver in the first place -
therefore it's impossible to keep an homogeneous environment for all
services.



I whole heartedly agree that non traditional storage technologies that
are becoming mainstream are good candidates for use cases where SQL
based storage gets in the way. I wish there wasn't so much FUD
(warranted or not) about MongoDB, but that is the reality we live in.


With this, I'm not suggesting to ignore the risks and the extra burden
this adds but, instead of attempting to avoid it completely by not
evolving the stack of services we provide, we should probably work on
defining a reasonable subset of NoSQL services we are OK with
supporting. This will help making the burden smaller and it'll give
operators the option to choose.

[0] http://blog.flaper87.com/post/marconi-amqp-see-you-later/


- Concern on should we really reinvent a queue system rather than
piggyback on one

As mentioned in the meeting on Tuesday, Zaqar is not reinventing message
brokers. Zaqar provides a service akin to SQS from AWS with an OpenStack
flavor on top. [0]



I think Zaqar is more like SMTP and IMAP than AMQP. You're not really
trying to connect two processes in real time. You're trying to do fully
asynchronous messaging with fully randomized access to any message.

Perhaps somebody should explore whether the approaches taken by large
scale IMAP providers could be applied to Zaqar.

Anyway, I can't imagine writing a system to intentionally use the
semantics of IMAP and SMTP. I'd be very interested in seeing actual use
cases for it, apologies if those have been posted before.


It seems like you're EITHER describing something called XMPP that has at
least one open source scalable backend called ejabberd. OR, you've
actually hit the nail on the head with bringing up SMTP and IMAP but for
some reason that feels strange.

SMTP and IMAP already implement every feature you've described, as well
as retries/failover/HA and a fully end to end secure transport (if
installed properly) If you don't actually set them up to run as a public
messaging interface but just as a cloud-local exchange, then you could
get by with very low overhead for a massive throughput - it can very
easily be run on a single machine for Sean's simplicity, and could just
as easily be scaled out using well known techniques for public cloud
sized deployments?

So why not use existing daemons that do this? You could still use the
REST API you've got, but instead of writing it to a mongo backend and
trying to implement all of the things that already exist in SMTP/IMAP -
you could just have them front to it. You could even bypass normal
del

Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-11 Thread Zane Bitter

On 04/09/14 08:14, Sean Dague wrote:


I've been one of the consistent voices concerned about a hard
requirement on adding NoSQL into the mix. So I'll explain that thinking
a bit more.

I feel like when the TC makes an integration decision previously this
has been about evaluating the project applying for integration, and if
they met some specific criteria they were told about some time in the
past. I think that's the wrong approach. It's a locally optimized
approach that fails to ask the more interesting question.

Is OpenStack better as a whole if this is a mandatory component of
OpenStack? Better being defined as technically better (more features,
less janky code work arounds, less unexpected behavior from the stack).
Better from the sense of easier or harder to run an actual cloud by our
Operators (taking into account what kinds of moving parts they are now
expected to manage). Better from the sense of a better user experience
in interacting with OpenStack as whole. Better from a sense that the
OpenStack release will experience less bugs, less unexpected cross
project interactions, an a greater overall feel of consistency so that
the OpenStack API feels like one thing.

https://dague.net/2014/08/26/openstack-as-layers/


I don't want to get off-topic here, but I want to state before this 
becomes the de-facto starting point for a layering discussion that I 
don't accept this model at all. It is not based on any analysis 
whatsoever but appears to be entirely arbitrary - a collection of 
individual prejudices arranged visually.


On a hopefully more constructive note, I believe there are at least two 
analyses that _would_ produce interesting data here:


1) Examine the dependencies, both hard and optional, between projects 
and enumerate the things you lose when ignoring each optional one.
2) Analyse projects based on the type of user consuming the service - 
e.g. Nova is mostly used (directly or indirectly via e.g. Heat and/or 
Horizon) by actual, corporeal persons, while Zaqar is used by both 
persons (to set up queues) and services (which actually send and receive 
messages) - of both OpenStack and applications. I believe, BTW that this 
analysis will uncover a lot of missing features in Keystone[1].


What you can _not_ produce is a linear model of the different types of 
clouds for different use cases, because different organisations have 
wildly differing needs.



One of the interesting qualities of Layers 1 & 2 is they all follow an
AMQP + RDBMS pattern (excepting swift). You can have a very effective
IaaS out of that stack. They are the things that you can provide pretty
solid integration testing on (and if you look at where everything stood
before the new TC mandates on testing / upgrade that was basically what
was getting integration tested). (Also note, I'll accept Barbican is
probably in the wrong layer, and should be a Layer 2 service.)


Swift is the current exception here, but one could argue, and people 
have[2], that Swift is also the only project that actually conforms to 
our stated design tenets for OpenStack. I'd struggle to tell the Zaqar 
folks they've done the Wrong Thing... especially when abandoning the 
RDBMS driver was done largely at the direction of the TC iirc.


Speaking of Swift, I would really love to see it investigated as a 
potential storage backend for Zaqar. If it proves to have the right 
guarantees (and durability is the crucial one, so it sounds promising) 
then that has the potential to smooth over a lot of the deployment problem.



While large shops can afford to have a dedicated team to figure out how
to make mongo or redis HA, provide monitoring, have a DR plan for when a
huricane requires them to flip datacenters, that basically means
OpenStack heads further down the path of "only for the big folks". I
don't want OpenStack to be only for the big folks, I want OpenStack to
be for all sized folks. I really do want to have all the local small
colleges around here have OpenStack clouds, because it's something that
people believe they can do and manage. I know the people that work in
this places, they all come out to the LUG I run. We've talked about
this. OpenStack is basically seen as too complex for them to use as it
stands, and that pains me a ton.


This is a great point, and one that we definitely have to keep in mind.

It's also worth noting that small organisations also get the most 
benefit. Rather than having to stand up a cluster of reliable message 
brokers (large organisations are much more likely to need this kind of 
flexibility anyway) - potentially one cluster per application - they can 
have their IT department deploy e.g. a single Redis cluster and have 
messaging handled for every application in their cloud with all the 
benefits of multitenancy.


Part of the move to the cloud is inevitably going to mean organisational 
changes in a lot of places, where the operations experts will 
increasingly focus on maintaining the cloud itself, rath

Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-10 Thread Stefano Maffulli
On 09/10/2014 12:56 PM, Monty Taylor wrote:
> I reject soundly and fundamentally the idea that Open Source projects
> NEED a commercial ecosystem to provide solid quality software. 

That's not what I said. I said that assuring the quality of code on a
public repository is not necessarily something that can be asked to the
maintainers of the code. A very simple example: a repository where
nobody committed anything for a long time.  If nobody seems to be
maintaining the code, how can a user get an assurance of the quality of
its code?

That's all I meant to say: "quality" of open source code (just like
non-free software code) is not necessarily something that can be left to
the maintainers of the code. It's like asking the waiter if the wine is
good... of course the answer is yes.

> That is a description of a thing called "Open Core" 

So not. Open Core is a different thing and is orthogonal to code
quality, quality and width of support, maintenance etc.

> Apache has been the dominant web server for forever, and if there is any
> human who has ever purchased "Commerical Apache" they are an absolute
> moron.

And yet, Red Hat sells quite a lot of software that is completely free
as in beer (in addition to being free as in freedom). This is what I'm
talking about, not 'open core'.

To go back in topic, I was explicitly addressing Tim's question
"How can the average deployer know whether a stackforge is [...]" and my
answer is that stackforge can have perfectly "fine" code while there
could be "bad" code in openstack/*.

And I simply don't think that the openstack project as a whole can
answer anything but "all our code is great", if asked. Therefore,
deciding whether code on stackforge or anywhere else for that matter,
needs to be properly evaluated at procurement time.

On 09/10/2014 10:45 AM, Tim Bell wrote:
> There is an impression that open source software does not have
> procurement issues.

I guarantee you I never had that impression, quite the contrary instead.
I've worked with Italian government and large corporations to teach them
how to deal with open source code long since there was no Red Hat in Italy.

Also, to clarify my answer to Tim, I reject the idea that code in
stackforge is a second class citizen. As simple as that.

/stef

-- 
Ask and answer questions on https://ask.openstack.org

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-10 Thread Monty Taylor

On 09/10/2014 10:29 AM, Stefano Maffulli wrote:

On 09/05/2014 12:36 PM, Tim Bell wrote:

How can the average deployer know whether a stackforge is

a.  An early prototype which has completed (such as some of the
early LBaaS packages)

b.  A project which has lost its initial steam and further
investment is not foreseen

c.  A reliable project where there has not been a significant need
to change recently but is a good long term bet


This pops up often and to me it looks like a software procurement issue,
something that the 'buyer' needs to be able to sort out either with the
help of their vendors (distributions or system integrators), or if they
go DIY, with other instruments to check code quality, level of support
by the community, professional support etc (there are a few EU funded
researches and tools, like http://qualipso.icmc.usp.br/OMM/).

Why do you think any open source project should be in the business of
providing such assurances? Isn't that a role more suitable for the
commercial ecosystem?


I reject soundly and fundamentally the idea that Open Source projects 
NEED a commercial ecosystem to provide solid quality software. That is a 
description of a thing called "Open Core" and every time it has come up 
in the context of OpenStack I and others have been quite successful in 
reiterating:


  OpenStack is NOT Open Core

bind has been running DNS for the world for 20 years and does it quite 
admirably without someone needing to buy "Commercial Bind"


Apache has been the dominant web server for forever, and if there is any 
human who has ever purchased "Commerical Apache" they are an absolute moron.


There is no such thing as "Commercial Python"

Open Source software has been running the entire internet, at scale, 
since before the Internet was wide spread, and since well before the 
term Open Source existed. In fact, Open Source software drove the world 
of computing before the idea of Commercial Software was invented.


Does this mean that our commercial vendors are bad? No. Absolutely not. 
I love each and every one of them and want all of them to make tons of 
money. But if the thing I produce isn't good in the first place, I 
expect them to show up and help me fix it. Here. In the Open Source 
project. Otherwise, they are a leech and should be mocked.


So, no, I do not believe that it should be the purview of the commercial 
ecosystem to provide quality assurances about the Open Source software 
that I work on. And I'm highly offended that as a community manager for 
an Open Source project, that you seem to think that the output of our 
project should not be world class all by itself.


Monty


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-10 Thread Monty Taylor

On 09/09/2014 07:04 PM, Samuel Merritt wrote:

On 9/9/14, 4:47 PM, Devananda van der Veen wrote:

On Tue, Sep 9, 2014 at 4:12 PM, Samuel Merritt 
wrote:

On 9/9/14, 12:03 PM, Monty Taylor wrote:

[snip]

So which is it? Because it sounds like to me it's a thing that actually
does NOT need to diverge in technology in any way, but that I've been
told that it needs to diverge because it's delivering a different
set of
features - and I'm pretty sure if it _is_ the thing that needs to
diverge in technology because of its feature set, then it's a thing I
don't think we should be implementing in python in OpenStack because it
already exists and it's called AMQP.



Whether Zaqar is more like AMQP or more like email is a really strange
metric to use for considering its inclusion.



I don't find this strange at all -- I had been judging the technical
merits of Zaqar (ex-Marconi) for the last ~18 months based on the
understanding that it aimed to provide Queueing-as-a-Service, and
found its delivery of that to be lacking on technical grounds. The
implementation did not meet my view of what a queue service should
provide; it is based on some serious antipatterns (storing a queue in
an RDBMS is probably the most obvious); and in fact, it isn't even
queue-like in the access patterns enabled by the REST API (random
access to a set != a queue). That was the basis for a large part of my
objections to the project over time, and a source of frustration for
me as the developers justified many of their positions rather than
accepted feedback and changed course during the incubation period. The
reason for this seems clear now...

As was pointed out in the TC meeting today, Zaqar is (was?) actually
aiming to provide Messaging-as-a-Service -- not queueing as a service!
This is another way of saying "it's more like email and less like
AMQP", which means my but-its-not-a-queue objection to the project's
graduation is irrelevant, and I need to rethink about all my previous
assessments of the project.

The questions now before us are:
- should OpenStack include, in the integrated release, a
messaging-as-a-service component?


I certainly think so. I've worked on a few reasonable-scale web
applications, and they all followed the same pattern: HTTP app servers
serving requests quickly, background workers for long-running tasks, and
some sort of durable message-broker/queue-server thing for conveying
work from the first to the second.

A quick straw poll of my nearby coworkers shows that every non-trivial
web application that they've worked on in the last decade follows the
same pattern.

While not *every* application needs such a thing, web apps are quite
common these days, and Zaqar satisfies one of their big requirements.
Not only that, it does so in a way that requires much less babysitting
than run-your-own-broker does.


Right. But here's the thing.

What you just described is what we all thought zaqar was aiming to be in 
the beginning. We did not think it was a GOOD implementation of that, so 
while we agreed that it would be useful to have one of those, we were 
not crazy about the implementation.


But that is what we have recently learned that zaqar is NOT. It is not a 
message queue in the way that you describe. It is something else. Now, I 
tend to believe that that something else is also potentially interesting 
- but the question of whether that thing is a thing that we want, and 
whether _this_ implementation of that this is a good implementation are 
the questions at hand.




- is Zaqar a technically sound implementation of such a service?

As an aside, there are still references to Zaqar as a queue in both
the wiki [0], in the governance repo [1], and on launchpad [2].

Regards,
Devananda


[0] "Multi-tenant queues based on Keystone project IDs"
   https://wiki.openstack.org/wiki/Zaqar#Key_features

[1] "Queue service" is even the official OpenStack Program name, and
the mission statement starts with "To produce an OpenStack message
queueing API and service."

http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml#n315


[2] "Zaqar is a new OpenStack project to create a multi-tenant cloud
queuing service"
   https://launchpad.net/zaqar



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-10 Thread Tim Bell



> -Original Message-
> From: Stefano Maffulli [mailto:stef...@openstack.org]
> Sent: 10 September 2014 19:29
> To: openstack-dev@lists.openstack.org
> Subject: Re: [openstack-dev] [Zaqar] Comments on the concerns arose during
> the TC meeting
> 
> On 09/05/2014 12:36 PM, Tim Bell wrote:
> > How can the average deployer know whether a stackforge is
> >
> > a.  An early prototype which has completed (such as some of the
> > early LBaaS packages)
> >
> > b.  A project which has lost its initial steam and further
> > investment is not foreseen
> >
> > c.  A reliable project where there has not been a significant need
> > to change recently but is a good long term bet
> 
> This pops up often and to me it looks like a software procurement issue,
> something that the 'buyer' needs to be able to sort out either with the help 
> of
> their vendors (distributions or system integrators), or if they go DIY, with 
> other
> instruments to check code quality, level of support by the community,
> professional support etc (there are a few EU funded researches and tools, like
> http://qualipso.icmc.usp.br/OMM/).
> 
> Why do you think any open source project should be in the business of 
> providing
> such assurances? Isn't that a role more suitable for the commercial ecosystem?
>

There is an impression that open source software does not have procurement 
issues. Where we choose to invest manpower in testing, documentation,
bug reporting, scaling is as much as commitment as those who choose to buy 
software. CERN does not have resources on its own to keep a project alive. 
Combined with others, 
I'm OK to assist. 

If there is significant doubt, I will not invest at all, no additional function 
for my users and the overall community loses.

Tim

> /stef
> 
> --
> Ask and answer questions on https://ask.openstack.org
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-10 Thread Stefano Maffulli
On 09/05/2014 12:36 PM, Tim Bell wrote:
> How can the average deployer know whether a stackforge is
> 
> a.  An early prototype which has completed (such as some of the
> early LBaaS packages)
> 
> b.  A project which has lost its initial steam and further
> investment is not foreseen
> 
> c.  A reliable project where there has not been a significant need
> to change recently but is a good long term bet

This pops up often and to me it looks like a software procurement issue,
something that the 'buyer' needs to be able to sort out either with the
help of their vendors (distributions or system integrators), or if they
go DIY, with other instruments to check code quality, level of support
by the community, professional support etc (there are a few EU funded
researches and tools, like http://qualipso.icmc.usp.br/OMM/).

Why do you think any open source project should be in the business of
providing such assurances? Isn't that a role more suitable for the
commercial ecosystem?

/stef

-- 
Ask and answer questions on https://ask.openstack.org

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-10 Thread Gordon Sim

On 09/10/2014 12:47 AM, Devananda van der Veen wrote:

As was pointed out in the TC meeting today, Zaqar is (was?) actually
aiming to provide Messaging-as-a-Service -- not queueing as a service!
This is another way of saying "it's more like email and less like
AMQP"


The glossary[1] describes a model that is much more like 
messaging-oriented-middleware (AMQP and other things like it) than email.


To me, messaging-as-a-service is perhaps a more general purpose term. 
The concept of a 'queue' might not be exposed in such a system (e.g. 
MQTT), whereas the term queueing-as-a-service implies that perhaps it 
is. I don't see them as necessarily distinct things though. The model 
exposed by Zaqar certainly has 'queues' featured prominently.


[1] http://docs.openstack.org/developer/marconi/glossary.html



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-10 Thread Chris Dent

On Tue, 9 Sep 2014, Samuel Merritt wrote:

On 9/9/14, 4:47 PM, Devananda van der Veen wrote:

The questions now before us are:
- should OpenStack include, in the integrated release, a
messaging-as-a-service component?


I certainly think so. I've worked on a few reasonable-scale web applications, 
and they all followed the same pattern: HTTP app servers serving requests 
quickly, background workers for long-running tasks, and some sort of durable 
message-broker/queue-server thing for conveying work from the first to the 
second.


A quick straw poll of my nearby coworkers shows that every non-trivial web 
application that they've worked on in the last decade follows the same 
pattern.


While not *every* application needs such a thing, web apps are quite common 
these days, and Zaqar satisfies one of their big requirements. Not only that, 
it does so in a way that requires much less babysitting than 
run-your-own-broker does.


I don't think there's any question about the value of a "durable
message-broke/queue-server thing" in the general case.

The question is whether OpenStack is in the business of satisfying
that case.

Which leads inevitably to the existential questions of What is
OpenStack in the business of? and How do we stay sane while being in
that business?

Every long thread over the last couple of months has trended towards
those questions. It's getting pretty tiresome. We'd all be a lot
more focused if we knew the answer.

--
Chris Dent tw:@anticdent freenode:cdent
https://tank.peermore.com/tanks/cdent

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-10 Thread Flavio Percoco
On 09/10/2014 01:47 AM, Devananda van der Veen wrote:
> On Tue, Sep 9, 2014 at 4:12 PM, Samuel Merritt  wrote:
>> On 9/9/14, 12:03 PM, Monty Taylor wrote:
> [snip]
>>> So which is it? Because it sounds like to me it's a thing that actually
>>> does NOT need to diverge in technology in any way, but that I've been
>>> told that it needs to diverge because it's delivering a different set of
>>> features - and I'm pretty sure if it _is_ the thing that needs to
>>> diverge in technology because of its feature set, then it's a thing I
>>> don't think we should be implementing in python in OpenStack because it
>>> already exists and it's called AMQP.
>>
>>
>> Whether Zaqar is more like AMQP or more like email is a really strange
>> metric to use for considering its inclusion.
>>
> 
> I don't find this strange at all -- I had been judging the technical
> merits of Zaqar (ex-Marconi) for the last ~18 months based on the
> understanding that it aimed to provide Queueing-as-a-Service, and
> found its delivery of that to be lacking on technical grounds. The
> implementation did not meet my view of what a queue service should
> provide; it is based on some serious antipatterns (storing a queue in
> an RDBMS is probably the most obvious); and in fact, it isn't even
> queue-like in the access patterns enabled by the REST API (random
> access to a set != a queue). That was the basis for a large part of my
> objections to the project over time, and a source of frustration for
> me as the developers justified many of their positions rather than
> accepted feedback and changed course during the incubation period. The
> reason for this seems clear now...

Let me clarify some points, again:

1. We have never said that our recommended driver is an RBDMS. That
driver was added for reasons that were already explained in this thread.[0]

2. The get-message-by-id feature/bug was not added as 'queue' feature.
When we added it, we were working on messages pagination, that is, a way
to iterate and keep getting messages out of the queue by following the
lead of the last message. When the pagination thing was completed,
adding the get-message-by-id endpoint was easy and cheap, hence we added
it. Our current API version is 1.1 and we *can't* remove that endpoint
until the next major version.

3. You saying that we just justified our positions rather than accepting
the feedback is what frustrates me the most and it just means, with all
due respect, that you haven't followed the project close enough. We've
addressed feedback from *EVERYONE* whenever that was possible. As far as
I can see, your frustration is mostly related to the fact that we
haven't removed the `get-message-by-id` endpoint but as I've mentioned,
we can't do it right now. However, we have talked about it several times
in meetings and a couple of times on the mailing list.[1][2]


[0]
http://lists.openstack.org/pipermail/openstack-dev/2014-March/030367.html
[1] http://lists.openstack.org/pipermail/openstack-dev/2014-May/036131.html
[2]
http://lists.openstack.org/pipermail/openstack-dev/2014-August/044213.html

> 
> As was pointed out in the TC meeting today, Zaqar is (was?) actually
> aiming to provide Messaging-as-a-Service -- not queueing as a service!
> This is another way of saying "it's more like email and less like
> AMQP", which means my but-its-not-a-queue objection to the project's
> graduation is irrelevant, and I need to rethink about all my previous
> assessments of the project.

Zaqar is a messaging service, we changed that almost right before the
Juno summit to help clarify the project goals. [0]

[0] https://wiki.openstack.org/wiki/Zaqar#Overview

> 
> The questions now before us are:
> - should OpenStack include, in the integrated release, a
> messaging-as-a-service component?
> - is Zaqar a technically sound implementation of such a service?

I believe there is. I've been collecting use-cases from integrated
projects and other users. I've put them here[0]

[0] https://etherpad.openstack.org/p/zaqar-integrated-projects-use-cases

> 
> As an aside, there are still references to Zaqar as a queue in both
> the wiki [0], in the governance repo [1], and on launchpad [2].

I'm sorry about this, apparently the change was not spread enough
throughout our community, again.


Thanks for your feedback,
Flavio

> 
> 
> [0] "Multi-tenant queues based on Keystone project IDs"
>   https://wiki.openstack.org/wiki/Zaqar#Key_features
> 
> [1] "Queue service" is even the official OpenStack Program name, and
> the mission statement starts with "To produce an OpenStack message
> queueing API and service."
>   
> http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml#n315
> 
> [2] "Zaqar is a new OpenStack project to create a multi-tenant cloud
> queuing service"
>   https://launchpad.net/zaqar
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mail

Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-09 Thread Clint Byrum
Excerpts from Samuel Merritt's message of 2014-09-09 19:04:58 -0700:
> On 9/9/14, 4:47 PM, Devananda van der Veen wrote:
> > On Tue, Sep 9, 2014 at 4:12 PM, Samuel Merritt  wrote:
> >> On 9/9/14, 12:03 PM, Monty Taylor wrote:
> > [snip]
> >>> So which is it? Because it sounds like to me it's a thing that actually
> >>> does NOT need to diverge in technology in any way, but that I've been
> >>> told that it needs to diverge because it's delivering a different set of
> >>> features - and I'm pretty sure if it _is_ the thing that needs to
> >>> diverge in technology because of its feature set, then it's a thing I
> >>> don't think we should be implementing in python in OpenStack because it
> >>> already exists and it's called AMQP.
> >>
> >>
> >> Whether Zaqar is more like AMQP or more like email is a really strange
> >> metric to use for considering its inclusion.
> >>
> >
> > I don't find this strange at all -- I had been judging the technical
> > merits of Zaqar (ex-Marconi) for the last ~18 months based on the
> > understanding that it aimed to provide Queueing-as-a-Service, and
> > found its delivery of that to be lacking on technical grounds. The
> > implementation did not meet my view of what a queue service should
> > provide; it is based on some serious antipatterns (storing a queue in
> > an RDBMS is probably the most obvious); and in fact, it isn't even
> > queue-like in the access patterns enabled by the REST API (random
> > access to a set != a queue). That was the basis for a large part of my
> > objections to the project over time, and a source of frustration for
> > me as the developers justified many of their positions rather than
> > accepted feedback and changed course during the incubation period. The
> > reason for this seems clear now...
> >
> > As was pointed out in the TC meeting today, Zaqar is (was?) actually
> > aiming to provide Messaging-as-a-Service -- not queueing as a service!
> > This is another way of saying "it's more like email and less like
> > AMQP", which means my but-its-not-a-queue objection to the project's
> > graduation is irrelevant, and I need to rethink about all my previous
> > assessments of the project.
> >
> > The questions now before us are:
> > - should OpenStack include, in the integrated release, a
> > messaging-as-a-service component?
> 
> I certainly think so. I've worked on a few reasonable-scale web 
> applications, and they all followed the same pattern: HTTP app servers 
> serving requests quickly, background workers for long-running tasks, and 
> some sort of durable message-broker/queue-server thing for conveying 
> work from the first to the second.
> 
> A quick straw poll of my nearby coworkers shows that every non-trivial 
> web application that they've worked on in the last decade follows the 
> same pattern.
> 
> While not *every* application needs such a thing, web apps are quite 
> common these days, and Zaqar satisfies one of their big requirements. 
> Not only that, it does so in a way that requires much less babysitting 
> than run-your-own-broker does.
> 

I think you missed the distinction.

What you describe is _message queueing_. Not messaging. The difference
being the durability and addressability of each message.

As Devananda pointed out, a queue doesn't allow addressing the items in
the queue directly. You can generally only send, receive, ACK, or NACK.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-09 Thread Samuel Merritt

On 9/9/14, 4:47 PM, Devananda van der Veen wrote:

On Tue, Sep 9, 2014 at 4:12 PM, Samuel Merritt  wrote:

On 9/9/14, 12:03 PM, Monty Taylor wrote:

[snip]

So which is it? Because it sounds like to me it's a thing that actually
does NOT need to diverge in technology in any way, but that I've been
told that it needs to diverge because it's delivering a different set of
features - and I'm pretty sure if it _is_ the thing that needs to
diverge in technology because of its feature set, then it's a thing I
don't think we should be implementing in python in OpenStack because it
already exists and it's called AMQP.



Whether Zaqar is more like AMQP or more like email is a really strange
metric to use for considering its inclusion.



I don't find this strange at all -- I had been judging the technical
merits of Zaqar (ex-Marconi) for the last ~18 months based on the
understanding that it aimed to provide Queueing-as-a-Service, and
found its delivery of that to be lacking on technical grounds. The
implementation did not meet my view of what a queue service should
provide; it is based on some serious antipatterns (storing a queue in
an RDBMS is probably the most obvious); and in fact, it isn't even
queue-like in the access patterns enabled by the REST API (random
access to a set != a queue). That was the basis for a large part of my
objections to the project over time, and a source of frustration for
me as the developers justified many of their positions rather than
accepted feedback and changed course during the incubation period. The
reason for this seems clear now...

As was pointed out in the TC meeting today, Zaqar is (was?) actually
aiming to provide Messaging-as-a-Service -- not queueing as a service!
This is another way of saying "it's more like email and less like
AMQP", which means my but-its-not-a-queue objection to the project's
graduation is irrelevant, and I need to rethink about all my previous
assessments of the project.

The questions now before us are:
- should OpenStack include, in the integrated release, a
messaging-as-a-service component?


I certainly think so. I've worked on a few reasonable-scale web 
applications, and they all followed the same pattern: HTTP app servers 
serving requests quickly, background workers for long-running tasks, and 
some sort of durable message-broker/queue-server thing for conveying 
work from the first to the second.


A quick straw poll of my nearby coworkers shows that every non-trivial 
web application that they've worked on in the last decade follows the 
same pattern.


While not *every* application needs such a thing, web apps are quite 
common these days, and Zaqar satisfies one of their big requirements. 
Not only that, it does so in a way that requires much less babysitting 
than run-your-own-broker does.



- is Zaqar a technically sound implementation of such a service?

As an aside, there are still references to Zaqar as a queue in both
the wiki [0], in the governance repo [1], and on launchpad [2].

Regards,
Devananda


[0] "Multi-tenant queues based on Keystone project IDs"
   https://wiki.openstack.org/wiki/Zaqar#Key_features

[1] "Queue service" is even the official OpenStack Program name, and
the mission statement starts with "To produce an OpenStack message
queueing API and service."
   
http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml#n315

[2] "Zaqar is a new OpenStack project to create a multi-tenant cloud
queuing service"
   https://launchpad.net/zaqar



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-09 Thread Devananda van der Veen
On Tue, Sep 9, 2014 at 5:31 PM, Boris Pavlovic  wrote:
>
> Devananda,
>>
>>
>> While that is de rigueur today, it's actually at the core of the
>> current problem space. Blessing a project by integrating it is not a
>> scalable long-term solution. We don't have a model to integrate >1
>> project for the same space // of the same type, or to bless the
>> stability of a non-integrated project. You won't see two messaging
>> services, or two compute services, in the integrated release. In fact,
>> integration is supposed to occur only *after* the community has sorted
>> out "a winner" within a given space. In my view, it should also happen
>> only after the community has proven a project to be stable and
>> scalable in production.
>
>
> After looking at such profiles:
> http://boris-42.github.io/ngk.html
> And getting 150 DB requests (without neutron) to create one single VM, I 
> don't believe that set of current integrated OpenStack projects is scalable 
> well. (I mean without customization)

I'm not going to defend the DB performance of Nova or other services.
This thread isn't the place for that discussion.

>
> So I would like to say 2 things:
>
> - Rules should be the same for all projects (including incubated/integrated)

Yup. This is why the TC revisits integrated projects once per cycle now, too.

>
> - Nothing should be incubated/integrated.

This is a blatant straw-man. If you're suggesting we stop all
integration testing, release management, etc -- the very things which
the integrated release process coordinates... well, I don't think
that's what you're saying. Except it is.

> Cause projects have to evolve, to evolve they need competition. In other 
> words, monopoly sux in any moment of time (even after community decided to 
> chose project A and not project B)
>

In order for a project to evolve, a project needs people contributing
to it. More often than not, that is because someone is using the
project, and it doesn't do what they want, so they improve it in some
way. Incubation was intended to be a signal to early adopters to begin
using (and thus, hopefully, contributing to) a project, encouraging
collaboration and reducing NIH friction between corporations within
the ecosystem. It hasn't gone exactly as planned, but it's also worked
fairly well for _this_ purpose, in my opinion.

However, adding more and more projects into the integrated release,
and thus increasing the testing complexity and imposing greater
requirements on operators -- this is an imminent scaling problem, as
Sean has eloquently pointed out before in several long email threads
which I won't recount here.

All of this is to say that Kurt's statement:
  "[You don't get] broad exposure and usage... as a non-integrated
project in the OpenStack ecosystem."
is an accurate representation of one problem facing OpenStack today. I
don't think we solve that problem by following the established norm -
we solve it by creating a mechanism for non-integrated projects to get
the exposure and usage they need _without_ becoming a burden on our
QA, docs, and release teams, and without forcing that project upon
operators.

But as I said earlier, we shouldn't hold Zaqar hostage while we sort
out what that solution looks like...


Anyhow, my apologies for the bike shed. I felt it was worth voicing my
disagreement with Kurt's statement that graduation should not be
viewed as an official blessing of Zaqar as OpenStack's Messaging
Service. Today, I believe that's exactly what it is. With that
blessing comes an additional burden on the community to support it.

Perhaps that will change in the future.

-Devananda

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-09 Thread Boris Pavlovic
Devananda,



> While that is de rigueur today, it's actually at the core of the
> current problem space. Blessing a project by integrating it is not a
> scalable long-term solution. We don't have a model to integrate >1
> project for the same space // of the same type, or to bless the
> stability of a non-integrated project. You won't see two messaging
> services, or two compute services, in the integrated release. In fact,
> integration is supposed to occur only *after* the community has sorted
> out "a winner" within a given space. In my view, it should also happen
> only after the community has proven a project to be stable and
> scalable in production.


After looking at such profiles:
http://boris-42.github.io/ngk.html
And getting 150 DB requests (without neutron) to create one single VM, I
don't believe that set of current integrated OpenStack projects is scalable
well. (I mean without customization)

So I would like to say 2 things:

- Rules should be the same for all projects (including
incubated/integrated)
- Nothing should be incubated/integrated. Cause projects have to evolve, to
evolve they need competition. In other words, monopoly sux in any moment of
time (even after community decided to chose project A and not project B)


Best regards,
Boris Pavlovic



On Wed, Sep 10, 2014 at 4:18 AM, Adam Lawson  wrote:

> *"should OpenStack include, in the integrated release,
> a messaging-as-a-service component"*
>
> Assuming this is truly a question that represents where we are and not
> exploratory of what we might want to address, I would say the answer is a
> resounding no, as queuing is within the scope of what Openstack is and has
> always been. If we get into integrated messaging, I'm struggling to
> understand what value it adds to the IaaS goal. We might as well start
> integrating office and productivity applications while we're at it.
>
> Sorry if i sound cheeky but considering this seems rather odd to me.
>
>
> *Adam Lawson*
> *CEO, Principal Architect*
>
> AQORN, Inc.
> 427 North Tatnall Street
> Ste. 58461
> Wilmington, Delaware 19801-2230
> Toll-free: (844) 4-AQORN-NOW ext. 101
> International: +1 302-387-4660
> Direct: +1 916-246-2072
>
>
> On Tue, Sep 9, 2014 at 5:03 PM, Clint Byrum  wrote:
>
>> Excerpts from Devananda van der Veen's message of 2014-09-09 16:47:27
>> -0700:
>> > On Tue, Sep 9, 2014 at 4:12 PM, Samuel Merritt 
>> wrote:
>> > > On 9/9/14, 12:03 PM, Monty Taylor wrote:
>> > [snip]
>> > >> So which is it? Because it sounds like to me it's a thing that
>> actually
>> > >> does NOT need to diverge in technology in any way, but that I've been
>> > >> told that it needs to diverge because it's delivering a different
>> set of
>> > >> features - and I'm pretty sure if it _is_ the thing that needs to
>> > >> diverge in technology because of its feature set, then it's a thing I
>> > >> don't think we should be implementing in python in OpenStack because
>> it
>> > >> already exists and it's called AMQP.
>> > >
>> > >
>> > > Whether Zaqar is more like AMQP or more like email is a really strange
>> > > metric to use for considering its inclusion.
>> > >
>> >
>> > I don't find this strange at all -- I had been judging the technical
>> > merits of Zaqar (ex-Marconi) for the last ~18 months based on the
>> > understanding that it aimed to provide Queueing-as-a-Service, and
>> > found its delivery of that to be lacking on technical grounds. The
>> > implementation did not meet my view of what a queue service should
>> > provide; it is based on some serious antipatterns (storing a queue in
>> > an RDBMS is probably the most obvious); and in fact, it isn't even
>> > queue-like in the access patterns enabled by the REST API (random
>> > access to a set != a queue). That was the basis for a large part of my
>> > objections to the project over time, and a source of frustration for
>> > me as the developers justified many of their positions rather than
>> > accepted feedback and changed course during the incubation period. The
>> > reason for this seems clear now...
>> >
>> > As was pointed out in the TC meeting today, Zaqar is (was?) actually
>> > aiming to provide Messaging-as-a-Service -- not queueing as a service!
>> > This is another way of saying "it's more like email and less like
>> > AMQP", which means my but-its-not-a-queue objection to the project's
>> > graduation is irrelevant, and I need to rethink about all my previous
>> > assessments of the project.
>>
>> Well said.
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.o

Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-09 Thread Adam Lawson
*"should OpenStack include, in the integrated release,
a messaging-as-a-service component"*

Assuming this is truly a question that represents where we are and not
exploratory of what we might want to address, I would say the answer is a
resounding no, as queuing is within the scope of what Openstack is and has
always been. If we get into integrated messaging, I'm struggling to
understand what value it adds to the IaaS goal. We might as well start
integrating office and productivity applications while we're at it.

Sorry if i sound cheeky but considering this seems rather odd to me.


*Adam Lawson*
*CEO, Principal Architect*

AQORN, Inc.
427 North Tatnall Street
Ste. 58461
Wilmington, Delaware 19801-2230
Toll-free: (844) 4-AQORN-NOW ext. 101
International: +1 302-387-4660
Direct: +1 916-246-2072


On Tue, Sep 9, 2014 at 5:03 PM, Clint Byrum  wrote:

> Excerpts from Devananda van der Veen's message of 2014-09-09 16:47:27
> -0700:
> > On Tue, Sep 9, 2014 at 4:12 PM, Samuel Merritt 
> wrote:
> > > On 9/9/14, 12:03 PM, Monty Taylor wrote:
> > [snip]
> > >> So which is it? Because it sounds like to me it's a thing that
> actually
> > >> does NOT need to diverge in technology in any way, but that I've been
> > >> told that it needs to diverge because it's delivering a different set
> of
> > >> features - and I'm pretty sure if it _is_ the thing that needs to
> > >> diverge in technology because of its feature set, then it's a thing I
> > >> don't think we should be implementing in python in OpenStack because
> it
> > >> already exists and it's called AMQP.
> > >
> > >
> > > Whether Zaqar is more like AMQP or more like email is a really strange
> > > metric to use for considering its inclusion.
> > >
> >
> > I don't find this strange at all -- I had been judging the technical
> > merits of Zaqar (ex-Marconi) for the last ~18 months based on the
> > understanding that it aimed to provide Queueing-as-a-Service, and
> > found its delivery of that to be lacking on technical grounds. The
> > implementation did not meet my view of what a queue service should
> > provide; it is based on some serious antipatterns (storing a queue in
> > an RDBMS is probably the most obvious); and in fact, it isn't even
> > queue-like in the access patterns enabled by the REST API (random
> > access to a set != a queue). That was the basis for a large part of my
> > objections to the project over time, and a source of frustration for
> > me as the developers justified many of their positions rather than
> > accepted feedback and changed course during the incubation period. The
> > reason for this seems clear now...
> >
> > As was pointed out in the TC meeting today, Zaqar is (was?) actually
> > aiming to provide Messaging-as-a-Service -- not queueing as a service!
> > This is another way of saying "it's more like email and less like
> > AMQP", which means my but-its-not-a-queue objection to the project's
> > graduation is irrelevant, and I need to rethink about all my previous
> > assessments of the project.
>
> Well said.
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-09 Thread Clint Byrum
Excerpts from Devananda van der Veen's message of 2014-09-09 16:47:27 -0700:
> On Tue, Sep 9, 2014 at 4:12 PM, Samuel Merritt  wrote:
> > On 9/9/14, 12:03 PM, Monty Taylor wrote:
> [snip]
> >> So which is it? Because it sounds like to me it's a thing that actually
> >> does NOT need to diverge in technology in any way, but that I've been
> >> told that it needs to diverge because it's delivering a different set of
> >> features - and I'm pretty sure if it _is_ the thing that needs to
> >> diverge in technology because of its feature set, then it's a thing I
> >> don't think we should be implementing in python in OpenStack because it
> >> already exists and it's called AMQP.
> >
> >
> > Whether Zaqar is more like AMQP or more like email is a really strange
> > metric to use for considering its inclusion.
> >
> 
> I don't find this strange at all -- I had been judging the technical
> merits of Zaqar (ex-Marconi) for the last ~18 months based on the
> understanding that it aimed to provide Queueing-as-a-Service, and
> found its delivery of that to be lacking on technical grounds. The
> implementation did not meet my view of what a queue service should
> provide; it is based on some serious antipatterns (storing a queue in
> an RDBMS is probably the most obvious); and in fact, it isn't even
> queue-like in the access patterns enabled by the REST API (random
> access to a set != a queue). That was the basis for a large part of my
> objections to the project over time, and a source of frustration for
> me as the developers justified many of their positions rather than
> accepted feedback and changed course during the incubation period. The
> reason for this seems clear now...
> 
> As was pointed out in the TC meeting today, Zaqar is (was?) actually
> aiming to provide Messaging-as-a-Service -- not queueing as a service!
> This is another way of saying "it's more like email and less like
> AMQP", which means my but-its-not-a-queue objection to the project's
> graduation is irrelevant, and I need to rethink about all my previous
> assessments of the project.

Well said.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-09 Thread Devananda van der Veen
On Thu, Sep 4, 2014 at 1:44 PM, Kurt Griffiths
 wrote:
[snip]
> Does a Qpid/Rabbit/Kafka provisioning service make sense? Probably. Would
> such a service totally overlap in terms of use-cases with Zaqar? Community
> feedback suggests otherwise. Will there be some other kind of thing that
> comes out of the woodwork? Possibly. (Heck, if something better comes
> along I for one have no qualms in shifting resources to the more elegant
> solution--again, use the best tool for the job.) This process happens all
> the time in the broader open-source world. But this process takes a
> healthy amount of time, plus broad exposure and usage, which is something
> that you simply don’t get as a non-integrated project in the OpenStack
> ecosystem.

While that is de rigueur today, it's actually at the core of the
current problem space. Blessing a project by integrating it is not a
scalable long-term solution. We don't have a model to integrate >1
project for the same space // of the same type, or to bless the
stability of a non-integrated project. You won't see two messaging
services, or two compute services, in the integrated release. In fact,
integration is supposed to occur only *after* the community has sorted
out "a winner" within a given space. In my view, it should also happen
only after the community has proven a project to be stable and
scalable in production.

It should be self-evident that, for a large and healthy ecosystem of
production-quality projects to be created and flourish, we can not
pick a winner and shut down competition by integrating a project
*prior* to that project getting "broad exposure and usage". A practice
of integrating projects merely to get them exposure and contributors
is self-defeating.


> In any case, it’s pretty clear to me that Zaqar graduating should not be
> viewed as making it "the officially blessed messaging service for the
> cloud”

That's exactly what graduation does, though. Your statement in the
previous paragraph - that non-integrated projects don't get adoption -
only furthers this point.

> and nobody is allowed to have any other ideas, ever.

Of course other people can have other ideas -- but we don't have a
precedent for handling it inside the community. Look at Ceilometer -
there are at least two other projects which attempted to fill that
space, but we haven't any means to accept them into OpenStack without
either removing Ceilometer or encouraging those projects to merge into
Ceilometer.

> If that
> happens, it’s only a symptom of a deeper perception/process problem that
> is far from unique to Zaqar. In fact, I think it touches on all
> non-integrated projects, and many integrated ones as well.
>

Yup.

I agree that we shouldn't hold Zaqar hostage while the community sorts
out the small-tent-big-camp questions. But I also feel like we _must_
sort that out soon, because the current system (integrate all the
things!) doesn't appear to be sustainable for much longer.


-Devananda

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-09 Thread Clint Byrum
Excerpts from Samuel Merritt's message of 2014-09-09 16:12:09 -0700:
> On 9/9/14, 12:03 PM, Monty Taylor wrote:
> > On 09/04/2014 01:30 AM, Clint Byrum wrote:
> >> Excerpts from Flavio Percoco's message of 2014-09-04 00:08:47 -0700:
> >>> Greetings,
> >>>
> >>> Last Tuesday the TC held the first graduation review for Zaqar. During
> >>> the meeting some concerns arose. I've listed those concerns below with
> >>> some comments hoping that it will help starting a discussion before the
> >>> next meeting. In addition, I've added some comments about the project
> >>> stability at the bottom and an etherpad link pointing to a list of use
> >>> cases for Zaqar.
> >>>
> >>
> >> Hi Flavio. This was an interesting read. As somebody whose attention has
> >> recently been drawn to Zaqar, I am quite interested in seeing it
> >> graduate.
> >>
> >>> # Concerns
> >>>
> >>> - Concern on operational burden of requiring NoSQL deploy expertise to
> >>> the mix of openstack operational skills
> >>>
> >>> For those of you not familiar with Zaqar, it currently supports 2 nosql
> >>> drivers - MongoDB and Redis - and those are the only 2 drivers it
> >>> supports for now. This will require operators willing to use Zaqar to
> >>> maintain a new (?) NoSQL technology in their system. Before expressing
> >>> our thoughts on this matter, let me say that:
> >>>
> >>>  1. By removing the SQLAlchemy driver, we basically removed the
> >>> chance
> >>> for operators to use an already deployed "OpenStack-technology"
> >>>  2. Zaqar won't be backed by any AMQP based messaging technology for
> >>> now. Here's[0] a summary of the research the team (mostly done by
> >>> Victoria) did during Juno
> >>>  3. We (OpenStack) used to require Redis for the zmq matchmaker
> >>>  4. We (OpenStack) also use memcached for caching and as the oslo
> >>> caching lib becomes available - or a wrapper on top of dogpile.cache -
> >>> Redis may be used in place of memcached in more and more deployments.
> >>>  5. Ceilometer's recommended storage driver is still MongoDB,
> >>> although
> >>> Ceilometer has now support for sqlalchemy. (Please correct me if I'm
> >>> wrong).
> >>>
> >>> That being said, it's obvious we already, to some extent, promote some
> >>> NoSQL technologies. However, for the sake of the discussion, lets assume
> >>> we don't.
> >>>
> >>> I truly believe, with my OpenStack (not Zaqar's) hat on, that we can't
> >>> keep avoiding these technologies. NoSQL technologies have been around
> >>> for years and we should be prepared - including OpenStack operators - to
> >>> support these technologies. Not every tool is good for all tasks - one
> >>> of the reasons we removed the sqlalchemy driver in the first place -
> >>> therefore it's impossible to keep an homogeneous environment for all
> >>> services.
> >>>
> >>
> >> I whole heartedly agree that non traditional storage technologies that
> >> are becoming mainstream are good candidates for use cases where SQL
> >> based storage gets in the way. I wish there wasn't so much FUD
> >> (warranted or not) about MongoDB, but that is the reality we live in.
> >>
> >>> With this, I'm not suggesting to ignore the risks and the extra burden
> >>> this adds but, instead of attempting to avoid it completely by not
> >>> evolving the stack of services we provide, we should probably work on
> >>> defining a reasonable subset of NoSQL services we are OK with
> >>> supporting. This will help making the burden smaller and it'll give
> >>> operators the option to choose.
> >>>
> >>> [0] http://blog.flaper87.com/post/marconi-amqp-see-you-later/
> >>>
> >>>
> >>> - Concern on should we really reinvent a queue system rather than
> >>> piggyback on one
> >>>
> >>> As mentioned in the meeting on Tuesday, Zaqar is not reinventing message
> >>> brokers. Zaqar provides a service akin to SQS from AWS with an OpenStack
> >>> flavor on top. [0]
> >>>
> >>
> >> I think Zaqar is more like SMTP and IMAP than AMQP. You're not really
> >> trying to connect two processes in real time. You're trying to do fully
> >> asynchronous messaging with fully randomized access to any message.
> >>
> >> Perhaps somebody should explore whether the approaches taken by large
> >> scale IMAP providers could be applied to Zaqar.
> >>
> >> Anyway, I can't imagine writing a system to intentionally use the
> >> semantics of IMAP and SMTP. I'd be very interested in seeing actual use
> >> cases for it, apologies if those have been posted before.
> >
> > It seems like you're EITHER describing something called XMPP that has at
> > least one open source scalable backend called ejabberd. OR, you've
> > actually hit the nail on the head with bringing up SMTP and IMAP but for
> > some reason that feels strange.
> >
> > SMTP and IMAP already implement every feature you've described, as well
> > as retries/failover/HA and a fully end to end secure transport (if
> > installed properly) If you don't actually set them up to run as a public
> >

Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-09 Thread Devananda van der Veen
On Tue, Sep 9, 2014 at 4:12 PM, Samuel Merritt  wrote:
> On 9/9/14, 12:03 PM, Monty Taylor wrote:
[snip]
>> So which is it? Because it sounds like to me it's a thing that actually
>> does NOT need to diverge in technology in any way, but that I've been
>> told that it needs to diverge because it's delivering a different set of
>> features - and I'm pretty sure if it _is_ the thing that needs to
>> diverge in technology because of its feature set, then it's a thing I
>> don't think we should be implementing in python in OpenStack because it
>> already exists and it's called AMQP.
>
>
> Whether Zaqar is more like AMQP or more like email is a really strange
> metric to use for considering its inclusion.
>

I don't find this strange at all -- I had been judging the technical
merits of Zaqar (ex-Marconi) for the last ~18 months based on the
understanding that it aimed to provide Queueing-as-a-Service, and
found its delivery of that to be lacking on technical grounds. The
implementation did not meet my view of what a queue service should
provide; it is based on some serious antipatterns (storing a queue in
an RDBMS is probably the most obvious); and in fact, it isn't even
queue-like in the access patterns enabled by the REST API (random
access to a set != a queue). That was the basis for a large part of my
objections to the project over time, and a source of frustration for
me as the developers justified many of their positions rather than
accepted feedback and changed course during the incubation period. The
reason for this seems clear now...

As was pointed out in the TC meeting today, Zaqar is (was?) actually
aiming to provide Messaging-as-a-Service -- not queueing as a service!
This is another way of saying "it's more like email and less like
AMQP", which means my but-its-not-a-queue objection to the project's
graduation is irrelevant, and I need to rethink about all my previous
assessments of the project.

The questions now before us are:
- should OpenStack include, in the integrated release, a
messaging-as-a-service component?
- is Zaqar a technically sound implementation of such a service?

As an aside, there are still references to Zaqar as a queue in both
the wiki [0], in the governance repo [1], and on launchpad [2].

Regards,
Devananda


[0] "Multi-tenant queues based on Keystone project IDs"
  https://wiki.openstack.org/wiki/Zaqar#Key_features

[1] "Queue service" is even the official OpenStack Program name, and
the mission statement starts with "To produce an OpenStack message
queueing API and service."
  
http://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml#n315

[2] "Zaqar is a new OpenStack project to create a multi-tenant cloud
queuing service"
  https://launchpad.net/zaqar

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-09 Thread Samuel Merritt

On 9/9/14, 12:03 PM, Monty Taylor wrote:

On 09/04/2014 01:30 AM, Clint Byrum wrote:

Excerpts from Flavio Percoco's message of 2014-09-04 00:08:47 -0700:

Greetings,

Last Tuesday the TC held the first graduation review for Zaqar. During
the meeting some concerns arose. I've listed those concerns below with
some comments hoping that it will help starting a discussion before the
next meeting. In addition, I've added some comments about the project
stability at the bottom and an etherpad link pointing to a list of use
cases for Zaqar.



Hi Flavio. This was an interesting read. As somebody whose attention has
recently been drawn to Zaqar, I am quite interested in seeing it
graduate.


# Concerns

- Concern on operational burden of requiring NoSQL deploy expertise to
the mix of openstack operational skills

For those of you not familiar with Zaqar, it currently supports 2 nosql
drivers - MongoDB and Redis - and those are the only 2 drivers it
supports for now. This will require operators willing to use Zaqar to
maintain a new (?) NoSQL technology in their system. Before expressing
our thoughts on this matter, let me say that:

 1. By removing the SQLAlchemy driver, we basically removed the
chance
for operators to use an already deployed "OpenStack-technology"
 2. Zaqar won't be backed by any AMQP based messaging technology for
now. Here's[0] a summary of the research the team (mostly done by
Victoria) did during Juno
 3. We (OpenStack) used to require Redis for the zmq matchmaker
 4. We (OpenStack) also use memcached for caching and as the oslo
caching lib becomes available - or a wrapper on top of dogpile.cache -
Redis may be used in place of memcached in more and more deployments.
 5. Ceilometer's recommended storage driver is still MongoDB,
although
Ceilometer has now support for sqlalchemy. (Please correct me if I'm
wrong).

That being said, it's obvious we already, to some extent, promote some
NoSQL technologies. However, for the sake of the discussion, lets assume
we don't.

I truly believe, with my OpenStack (not Zaqar's) hat on, that we can't
keep avoiding these technologies. NoSQL technologies have been around
for years and we should be prepared - including OpenStack operators - to
support these technologies. Not every tool is good for all tasks - one
of the reasons we removed the sqlalchemy driver in the first place -
therefore it's impossible to keep an homogeneous environment for all
services.



I whole heartedly agree that non traditional storage technologies that
are becoming mainstream are good candidates for use cases where SQL
based storage gets in the way. I wish there wasn't so much FUD
(warranted or not) about MongoDB, but that is the reality we live in.


With this, I'm not suggesting to ignore the risks and the extra burden
this adds but, instead of attempting to avoid it completely by not
evolving the stack of services we provide, we should probably work on
defining a reasonable subset of NoSQL services we are OK with
supporting. This will help making the burden smaller and it'll give
operators the option to choose.

[0] http://blog.flaper87.com/post/marconi-amqp-see-you-later/


- Concern on should we really reinvent a queue system rather than
piggyback on one

As mentioned in the meeting on Tuesday, Zaqar is not reinventing message
brokers. Zaqar provides a service akin to SQS from AWS with an OpenStack
flavor on top. [0]



I think Zaqar is more like SMTP and IMAP than AMQP. You're not really
trying to connect two processes in real time. You're trying to do fully
asynchronous messaging with fully randomized access to any message.

Perhaps somebody should explore whether the approaches taken by large
scale IMAP providers could be applied to Zaqar.

Anyway, I can't imagine writing a system to intentionally use the
semantics of IMAP and SMTP. I'd be very interested in seeing actual use
cases for it, apologies if those have been posted before.


It seems like you're EITHER describing something called XMPP that has at
least one open source scalable backend called ejabberd. OR, you've
actually hit the nail on the head with bringing up SMTP and IMAP but for
some reason that feels strange.

SMTP and IMAP already implement every feature you've described, as well
as retries/failover/HA and a fully end to end secure transport (if
installed properly) If you don't actually set them up to run as a public
messaging interface but just as a cloud-local exchange, then you could
get by with very low overhead for a massive throughput - it can very
easily be run on a single machine for Sean's simplicity, and could just
as easily be scaled out using well known techniques for public cloud
sized deployments?

So why not use existing daemons that do this? You could still use the
REST API you've got, but instead of writing it to a mongo backend and
trying to implement all of the things that already exist in SMTP/IMAP -
you could just have them front to it. You could even bypass normal
d

Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-09 Thread Monty Taylor

On 09/04/2014 01:30 AM, Clint Byrum wrote:

Excerpts from Flavio Percoco's message of 2014-09-04 00:08:47 -0700:

Greetings,

Last Tuesday the TC held the first graduation review for Zaqar. During
the meeting some concerns arose. I've listed those concerns below with
some comments hoping that it will help starting a discussion before the
next meeting. In addition, I've added some comments about the project
stability at the bottom and an etherpad link pointing to a list of use
cases for Zaqar.



Hi Flavio. This was an interesting read. As somebody whose attention has
recently been drawn to Zaqar, I am quite interested in seeing it
graduate.


# Concerns

- Concern on operational burden of requiring NoSQL deploy expertise to
the mix of openstack operational skills

For those of you not familiar with Zaqar, it currently supports 2 nosql
drivers - MongoDB and Redis - and those are the only 2 drivers it
supports for now. This will require operators willing to use Zaqar to
maintain a new (?) NoSQL technology in their system. Before expressing
our thoughts on this matter, let me say that:

 1. By removing the SQLAlchemy driver, we basically removed the chance
for operators to use an already deployed "OpenStack-technology"
 2. Zaqar won't be backed by any AMQP based messaging technology for
now. Here's[0] a summary of the research the team (mostly done by
Victoria) did during Juno
 3. We (OpenStack) used to require Redis for the zmq matchmaker
 4. We (OpenStack) also use memcached for caching and as the oslo
caching lib becomes available - or a wrapper on top of dogpile.cache -
Redis may be used in place of memcached in more and more deployments.
 5. Ceilometer's recommended storage driver is still MongoDB, although
Ceilometer has now support for sqlalchemy. (Please correct me if I'm wrong).

That being said, it's obvious we already, to some extent, promote some
NoSQL technologies. However, for the sake of the discussion, lets assume
we don't.

I truly believe, with my OpenStack (not Zaqar's) hat on, that we can't
keep avoiding these technologies. NoSQL technologies have been around
for years and we should be prepared - including OpenStack operators - to
support these technologies. Not every tool is good for all tasks - one
of the reasons we removed the sqlalchemy driver in the first place -
therefore it's impossible to keep an homogeneous environment for all
services.



I whole heartedly agree that non traditional storage technologies that
are becoming mainstream are good candidates for use cases where SQL
based storage gets in the way. I wish there wasn't so much FUD
(warranted or not) about MongoDB, but that is the reality we live in.


With this, I'm not suggesting to ignore the risks and the extra burden
this adds but, instead of attempting to avoid it completely by not
evolving the stack of services we provide, we should probably work on
defining a reasonable subset of NoSQL services we are OK with
supporting. This will help making the burden smaller and it'll give
operators the option to choose.

[0] http://blog.flaper87.com/post/marconi-amqp-see-you-later/


- Concern on should we really reinvent a queue system rather than
piggyback on one

As mentioned in the meeting on Tuesday, Zaqar is not reinventing message
brokers. Zaqar provides a service akin to SQS from AWS with an OpenStack
flavor on top. [0]



I think Zaqar is more like SMTP and IMAP than AMQP. You're not really
trying to connect two processes in real time. You're trying to do fully
asynchronous messaging with fully randomized access to any message.

Perhaps somebody should explore whether the approaches taken by large
scale IMAP providers could be applied to Zaqar.

Anyway, I can't imagine writing a system to intentionally use the
semantics of IMAP and SMTP. I'd be very interested in seeing actual use
cases for it, apologies if those have been posted before.


It seems like you're EITHER describing something called XMPP that has at 
least one open source scalable backend called ejabberd. OR, you've 
actually hit the nail on the head with bringing up SMTP and IMAP but for 
some reason that feels strange.


SMTP and IMAP already implement every feature you've described, as well 
as retries/failover/HA and a fully end to end secure transport (if 
installed properly) If you don't actually set them up to run as a public 
messaging interface but just as a cloud-local exchange, then you could 
get by with very low overhead for a massive throughput - it can very 
easily be run on a single machine for Sean's simplicity, and could just 
as easily be scaled out using well known techniques for public cloud 
sized deployments?


So why not use existing daemons that do this? You could still use the 
REST API you've got, but instead of writing it to a mongo backend and 
trying to implement all of the things that already exist in SMTP/IMAP - 
you could just have them front to it. You could even bypass normal 
delivery mechanisms and do 

Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-05 Thread Tim Bell

If I take a recent case we had, we’re looking for ways to simplify backup for 
our users. They should not have to write crons to do snapshots but leave that 
to a service.

Raksha seems promising… a wiki page on openstack.org, code in github etc.

How can the average deployer know whether a stackforge is


a.  An early prototype which has completed (such as some of the early LBaaS 
packages)

b.  A project which has lost its initial steam and further investment is 
not foreseen

c.  A reliable project where there has not been a significant need to 
change recently but is a good long term bet

The end result is that deployers hold back waiting for an indication that this 
is more than a prototype in whatever form that would take. This means a missed 
opportunity as if something is interesting to many deployers, that could create 
the community needed to keep a project going.

The statement of “this is good and mature, even if it's not in the inner 
circle" is exactly what is needed. When I offer something to my end-users, 
there is an implied promise of this (and a corresponding credibility issue if 
that is not met).

Tim



From: Dean Troyer [mailto:dtro...@gmail.com]
Sent: 05 September 2014 19:11
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the 
TC meeting

On Fri, Sep 5, 2014 at 4:27 AM, Thierry Carrez 
mailto:thie...@openstack.org>> wrote:
Tim Bell wrote:
> The one concern I have with a small core is that there is not an easy way to 
> assess the maturity of a project on stackforge. The stackforge projects may 
> be missing packaging, Red Hat testing, puppet modules, install/admin 
> documentation etc. Thus, I need to have some indication that a project is 
> deployable before looking at it with my user community to see if it meets a 
> need that is sustainable.
>
> Do you see the "optional layer" services being blessed / validated in some 
> way and therefore being easy to identify ?
Yes, I think whatever exact shape this takes, it should convey some
assertion of stability to be able to distinguish itself from random
projects. Some way of saying "this is good and mature, even if it's not
in the inner circle".

Being in The "integrated release" has been seen as a sign of stability
forever, while it was only ensuring "integration" with other projects
and OpenStack processes. We are getting better at requiring "maturity"
there, but if we set up layers, we'll have to get even better at that.

The layers are (or originally were) a purely technical organization, 
intentionally avoiding association with defcore and other groupings, and on 
reflection, maturity too.  The problem that repeatedly bubbles up is that 
"people" (mostly outside the community) want a simple tag for maturity or 
blessedness and have been using the integrated/incubated status for that.  
Maturity has nothing to do with technical relationships between projects 
(required/optional layers).

The "good and mature" blessing should be an independent attribute that is set 
on projects as a result of nomination by TC members or PTLs or existing core 
members or whatever trusted group we choose.  I'd say for starters that 
anything from Stackforge that we use in integrated/incubated projects is on the 
short list for that status, as it already has that implicitly by our use.

dt

--

Dean Troyer
dtro...@gmail.com<mailto:dtro...@gmail.com>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-05 Thread Dean Troyer
On Fri, Sep 5, 2014 at 4:27 AM, Thierry Carrez 
wrote:

> Tim Bell wrote:
> > The one concern I have with a small core is that there is not an easy
> way to assess the maturity of a project on stackforge. The stackforge
> projects may be missing packaging, Red Hat testing, puppet modules,
> install/admin documentation etc. Thus, I need to have some indication that
> a project is deployable before looking at it with my user community to see
> if it meets a need that is sustainable.
> >
> > Do you see the "optional layer" services being blessed / validated in
> some way and therefore being easy to identify ?
>
> Yes, I think whatever exact shape this takes, it should convey some
> assertion of stability to be able to distinguish itself from random
> projects. Some way of saying "this is good and mature, even if it's not
> in the inner circle".
>
> Being in The "integrated release" has been seen as a sign of stability
> forever, while it was only ensuring "integration" with other projects
> and OpenStack processes. We are getting better at requiring "maturity"
> there, but if we set up layers, we'll have to get even better at that.


The layers are (or originally were) a purely technical organization,
intentionally avoiding association with defcore and other groupings, and on
reflection, maturity too.  The problem that repeatedly bubbles up is that
"people" (mostly outside the community) want a simple tag for maturity or
blessedness and have been using the integrated/incubated status for that.
 Maturity has nothing to do with technical relationships between projects
(required/optional layers).

The "good and mature" blessing should be an independent attribute that is
set on projects as a result of nomination by TC members or PTLs or existing
core members or whatever trusted group we choose.  I'd say for starters
that anything from Stackforge that we use in integrated/incubated projects
is on the short list for that status, as it already has that implicitly by
our use.

dt

-- 

Dean Troyer
dtro...@gmail.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-05 Thread Victoria Martínez de la Cruz
2014-09-05 9:09 GMT-03:00 Sean Dague :

> On 09/05/2014 07:39 AM, Robert Collins wrote:
> > On 5 September 2014 23:33, Sean Dague  wrote:
> >
> >> I think realistically a self certification process that would have
> >> artifacts in a discoverable place. I was thinking something along the
> >> lines of a baseball card interface with a short description of the
> >> project, a list of the requirements to deploy (native and python), a
> >> link the the API docs, a link to current test coverage, as well as some
> >> statement on the frequency of testing, stats on open bugs and current
> >> trending and review backlog, current user testimonials. Basically the
> >> kind of first stage analysis that a deployer would do before pushing
> >> this out to their users.
> >
> > Add into that their deployment support - e.g. do they have TripleO
> > support // Chef // Fuel // Puppet etc etc etc.
>
> ACK, good points. I expect packaging might go on that list as well.
>
> I won't pretend I've got the whole baseball card self cert thing worked
> out, just trying to sketch an idea that might bear fruit.
>
> -Sean
>
> --
> Sean Dague
> http://dague.net
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

Hi all,

Thanks for bringing up this topic Flavio and everyone for the feedback!

>From my humble junior developer point of view I would like to share some
comments about some of the concerns mentioned.

- Added complexity on adding Zaqar to the ecosystem and the NoSQL concern

I won't deny that adding Zaqar, if operators need it, will make OpenStack a
little more harder to deploy. But this will happen with any extra project
added.

Complexity is part of OpenStack though. We, as OpenStack, are trying to
provide a software solution for a really complex need. And that is what
makes us great.

IMO operators interested in using Zaqar will consider the pros and cons of
adding it and will make their choice based on that.

And it's not about the tools we use. NoSQL was chosen to make the job for
Zaqar because it was proven to do a better job. And, in the whole family of
NoSQL solutions, we choose the ones that were considered easier to deploy.

It's a tecnology that has been in use for a long time now and it fits
perfectly the requirements of Zaqar. In this regard, I think there is a
long way to go and is something that Zaqar care about everyday. Zaqar will
keep on researching for the best solutions for the users and working on
adding support for them, as every other project does.

- Reinventing or not reinventing a messaging system

In the last couple of months I has been working on adding AMQP as a
storage/transport backend for Zaqar. During that period I managed to learn
a lot from other messaging systems, including the ones that has been
discussed from now.

With that basis I can say that Zaqar is covering other, different, uses
cases that the mentioned technologies are not meant to cover. And the best
of all, it's being specialized for the cloud.

Most widely used cloud services already has they message and notification
systems solution, and we should be up to the game with that. Right now it
doesn't seems like that need is being filled.

- Integration as a sign of stability

Right now we are in a situation in which Zaqar is a mature project with
many features to provide, but in order to keep on growing and getting
better it needs to integrate with other projects and start showing what is
capable of.

Zaqar is robust, well documented and with a team willing to keep on
enhancing it.

It doesn't matter what places it takes on the stack, what is matter is that
it's on the stack.

Hope this all makes sense and please correct me if I'm wrong. I want the
best for both OpenStack and Zaqar, as you do.

All the best,

Victoria
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-05 Thread Gordon Sim

On 09/04/2014 01:14 PM, Sean Dague wrote:

https://dague.net/2014/08/26/openstack-as-layers/


Just wanted to say that I found this article very useful indeed and 
agree with the points you make in it.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-05 Thread Gordon Sim

On 09/04/2014 09:44 PM, Kurt Griffiths wrote:

Does a Qpid/Rabbit/Kafka provisioning service make sense? Probably.


I think something like that would be valuable, especially in conjunction 
with some application layer proxying and mapping between 'virtual' 
addresses/endpoints and specific queues/virtual hosts/brokers.


That would allow people to use the brokers, protocols and semantics they 
are already familiar with, combine different systems even, and have 
self-provisioning and scalability on top of it, with varying degrees of 
isolation for tenants.



Would
such a service totally overlap in terms of use-cases with Zaqar? Community
feedback suggests otherwise.


I'd love to read the feedback if you have any links. While I agree 
overlap is rarely total, I suspect there will be quite a bit and that it 
will grow as each approach evolves. That emphatically does not mean I 
think Zaqar should in any way be held back though, just that I think the 
openstack community should anticipate and indeed encourage other 
approaches also.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-05 Thread Sean Dague
On 09/05/2014 07:39 AM, Robert Collins wrote:
> On 5 September 2014 23:33, Sean Dague  wrote:
> 
>> I think realistically a self certification process that would have
>> artifacts in a discoverable place. I was thinking something along the
>> lines of a baseball card interface with a short description of the
>> project, a list of the requirements to deploy (native and python), a
>> link the the API docs, a link to current test coverage, as well as some
>> statement on the frequency of testing, stats on open bugs and current
>> trending and review backlog, current user testimonials. Basically the
>> kind of first stage analysis that a deployer would do before pushing
>> this out to their users.
> 
> Add into that their deployment support - e.g. do they have TripleO
> support // Chef // Fuel // Puppet etc etc etc.

ACK, good points. I expect packaging might go on that list as well.

I won't pretend I've got the whole baseball card self cert thing worked
out, just trying to sketch an idea that might bear fruit.

-Sean

-- 
Sean Dague
http://dague.net

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-05 Thread Robert Collins
On 5 September 2014 23:33, Sean Dague  wrote:

> I think realistically a self certification process that would have
> artifacts in a discoverable place. I was thinking something along the
> lines of a baseball card interface with a short description of the
> project, a list of the requirements to deploy (native and python), a
> link the the API docs, a link to current test coverage, as well as some
> statement on the frequency of testing, stats on open bugs and current
> trending and review backlog, current user testimonials. Basically the
> kind of first stage analysis that a deployer would do before pushing
> this out to their users.

Add into that their deployment support - e.g. do they have TripleO
support // Chef // Fuel // Puppet etc etc etc.

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-05 Thread Robert Collins
On 5 September 2014 08:44, Kurt Griffiths  wrote:

> In fact, I personally have long been a proponent of using the best tool
> for the job. The Zaqar project was kicked off at an unconference session
> several summits ago because the community saw a need that was not covered
> by other messaging systems. Does that mean those other systems are “bad”
> or “wrong”? Of course not. It simply means that there are some cases where
> those other systems aren’t the best tool for the job, and another tool is
> needed (and vice versa).

Apropos of this - I read
http://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying
with plenty of 'well yes' and 'great way to put it' moments - have you
considered that perhaps kafka (or a matching implementation in
Python/whatever) might make a great bridge between the mailbox needs
your current API offers, and the realtime delivery needs that
oslo.messaging serves.

I don't know if that would look like a different backend, or perhaps a
migration to just offering multi-tenant kafka, but I thought I'd ask
since I don't recall seeing it discussed :).

One reason in particular for me asking is that we have really only a
couple of experiments in pushing events cross-API-server today, and
they aren't super great answers. If we had a clear API that was
scalable and fast enough to replace both those cross-API use cases and
the plumbing use case, we might be able to simplify our stack
substantially - and that would pay off vs the new deployment overhead
of a new component.

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-05 Thread Sean Dague
On 09/05/2014 05:27 AM, Thierry Carrez wrote:
> Tim Bell wrote:
>>> -Original Message-
>>> From: Thierry Carrez [mailto:thie...@openstack.org]
>>> Sent: 04 September 2014 16:59
>>> To: openstack-dev@lists.openstack.org
>>> Subject: Re: [openstack-dev] [Zaqar] Comments on the concerns arose during
>>> the TC meeting
>>>
>>> Sean Dague wrote:
>>>> [...]
>>>> So, honestly, I'll probably remain -1 on the final integration vote,
>>>> not because Zaqar is bad, but because I'm feeling more firmly that for
>>>> OpenStack to not leave the small deployers behind we need to redefine
>>>> the tightly integrated piece of OpenStack to basically the Layer 1 & 2
>>>> parts of my diagram, and consider the rest of the layers exciting
>>>> parts of our ecosystem that more advanced users may choose to deploy
>>>> to meet their needs. Smaller tent, big ecosystem, easier on ramp.
>>>>
>>>> I realize that largely means Zaqar would be caught up in a definition
>>>> discussion outside of it's control, and that's kind of unfortunate, as
>>>> Flavio and team have been doing a bang up job of late. But we need to
>>>> stop considering "integration" as the end game of all interesting
>>>> software in the OpenStack ecosystem, and I think it's better to have
>>>> that conversation sooner rather than later.
>>>
>>> I think it's pretty clear at this point that:
>>>
>>> (1) we need to have a discussion about layers (base nucleus, optional extra
>>> services at the very least) and the level of support we grant to each -- the
>>> current binary approach is not working very well
>>>
>>> (2) If we accept Zaqar next week, it's pretty clear it would not fall in 
>>> the base
>>> nucleus layer but more in an optional extra services layer, together with 
>>> at the
>>> very least Trove and Sahara
>>>
>>> There are two ways of doing this: follow Sean's approach and -1 integration
>>> (and have zaqar apply to that "optional layer" when we create it), or +1
>>> integration now (and have zaqar follow whichever other integrated projects 
>>> we
>>> place in that layer when we create it).
>>>
>>> I'm still hesitating on the best approach. I think they yield the same end 
>>> result,
>>> but the -1 approach seems to be a bit more unfair, since it would be purely 
>>> for
>>> reasons we don't (yet) apply to currently-integrated projects...
>>>
>>
>> The one concern I have with a small core is that there is not an easy way to 
>> assess the maturity of a project on stackforge. The stackforge projects may 
>> be missing packaging, Red Hat testing, puppet modules, install/admin 
>> documentation etc. Thus, I need to have some indication that a project is 
>> deployable before looking at it with my user community to see if it meets a 
>> need that is sustainable.
>>
>> Do you see the "optional layer" services being blessed / validated in some 
>> way and therefore being easy to identify ?
> 
> Yes, I think whatever exact shape this takes, it should convey some
> assertion of stability to be able to distinguish itself from random
> projects. Some way of saying "this is good and mature, even if it's not
> in the inner circle".
> 
> Being in The "integrated release" has been seen as a sign of stability
> forever, while it was only ensuring "integration" with other projects
> and OpenStack processes. We are getting better at requiring "maturity"
> there, but if we set up layers, we'll have to get even better at that.

(This is definitely drifting, sorry to the zaqar team for that)

Honestly, I really think the TC needs to get out of the gold star
granting business for the larger ecosystem. It's going to be hard enough
to keep focus and attention on a small nucleus, and I think that has to
be the primary responsibility of the group.

When I say big ecosystem, I mean *big*. I think the goal is 100s of
production ready projects in the ecosystem. All solving interesting
problems, all adding value to the whole.

I think realistically a self certification process that would have
artifacts in a discoverable place. I was thinking something along the
lines of a baseball card interface with a short description of the
project, a list of the requirements to deploy (native and python), a
link the the API docs, a link to current test coverage, as well as some
statement on the frequency of testing, stats on open bugs and current
trending and review backlog, current user testimonials. Basically the
kind of first stage analysis that a deployer would do before pushing
this out to their users.

-Sean

-- 
Sean Dague
http://dague.net

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-05 Thread Gordon Sim

On 09/04/2014 09:44 PM, Kurt Griffiths wrote:

Thanks for your comments Gordon. I appreciate where you are coming from
and I think we are actually in agreement on a lot of things.

I just want to make it clear that from the very beginning of the project
the team has tried to communicate (but perhaps could have done a better
job at it) that we aren’t trying to displace other messaging systems that
are clearly delivering a lot of value today.

In fact, I personally have long been a proponent of using the best tool
for the job. The Zaqar project was kicked off at an unconference session
several summits ago because the community saw a need that was not covered
by other messaging systems. Does that mean those other systems are “bad”
or “wrong”? Of course not. It simply means that there are some cases where
those other systems aren’t the best tool for the job, and another tool is
needed (and vice versa).


I think communicating that unmet need, those use-cases not best served 
by other systems, would help a lot in clarifying Zaqar's intended role.



Does that other tool look *exactly* like Zaqar? Probably not. But a lot of
people have told us Zaqar--in its current form--already delivers a lot of
value that they can’t get from other messaging systems that are available
to them. Zaqar, like any open source project, is a manifestation of lots
of peoples' ideas, and will evolve over time to meet the needs of the
community.

Does a Qpid/Rabbit/Kafka provisioning service make sense? Probably. Would
such a service totally overlap in terms of use-cases with Zaqar? Community
feedback suggests otherwise. Will there be some other kind of thing that
comes out of the woodwork? Possibly. (Heck, if something better comes
along I for one have no qualms in shifting resources to the more elegant
solution--again, use the best tool for the job.) This process happens all
the time in the broader open-source world. But this process takes a
healthy amount of time, plus broad exposure and usage, which is something
that you simply don’t get as a non-integrated project in the OpenStack
ecosystem.

In any case, it’s pretty clear to me that Zaqar graduating should not be
viewed as making it "the officially blessed messaging service for the
cloud” and nobody is allowed to have any other ideas, ever.


Indeed, and to be clear, I wasn't really commenting on the graduation at 
all. I was really just responding to the statements on scope and 
differentiation; 'messaging service for the cloud' is a very broad 
problem space and as you rightly point out there may be different tools 
that best serve different parts of that problem space.



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-05 Thread Thierry Carrez
Tim Bell wrote:
>> -Original Message-
>> From: Thierry Carrez [mailto:thie...@openstack.org]
>> Sent: 04 September 2014 16:59
>> To: openstack-dev@lists.openstack.org
>> Subject: Re: [openstack-dev] [Zaqar] Comments on the concerns arose during
>> the TC meeting
>>
>> Sean Dague wrote:
>>> [...]
>>> So, honestly, I'll probably remain -1 on the final integration vote,
>>> not because Zaqar is bad, but because I'm feeling more firmly that for
>>> OpenStack to not leave the small deployers behind we need to redefine
>>> the tightly integrated piece of OpenStack to basically the Layer 1 & 2
>>> parts of my diagram, and consider the rest of the layers exciting
>>> parts of our ecosystem that more advanced users may choose to deploy
>>> to meet their needs. Smaller tent, big ecosystem, easier on ramp.
>>>
>>> I realize that largely means Zaqar would be caught up in a definition
>>> discussion outside of it's control, and that's kind of unfortunate, as
>>> Flavio and team have been doing a bang up job of late. But we need to
>>> stop considering "integration" as the end game of all interesting
>>> software in the OpenStack ecosystem, and I think it's better to have
>>> that conversation sooner rather than later.
>>
>> I think it's pretty clear at this point that:
>>
>> (1) we need to have a discussion about layers (base nucleus, optional extra
>> services at the very least) and the level of support we grant to each -- the
>> current binary approach is not working very well
>>
>> (2) If we accept Zaqar next week, it's pretty clear it would not fall in the 
>> base
>> nucleus layer but more in an optional extra services layer, together with at 
>> the
>> very least Trove and Sahara
>>
>> There are two ways of doing this: follow Sean's approach and -1 integration
>> (and have zaqar apply to that "optional layer" when we create it), or +1
>> integration now (and have zaqar follow whichever other integrated projects we
>> place in that layer when we create it).
>>
>> I'm still hesitating on the best approach. I think they yield the same end 
>> result,
>> but the -1 approach seems to be a bit more unfair, since it would be purely 
>> for
>> reasons we don't (yet) apply to currently-integrated projects...
>>
> 
> The one concern I have with a small core is that there is not an easy way to 
> assess the maturity of a project on stackforge. The stackforge projects may 
> be missing packaging, Red Hat testing, puppet modules, install/admin 
> documentation etc. Thus, I need to have some indication that a project is 
> deployable before looking at it with my user community to see if it meets a 
> need that is sustainable.
> 
> Do you see the "optional layer" services being blessed / validated in some 
> way and therefore being easy to identify ?

Yes, I think whatever exact shape this takes, it should convey some
assertion of stability to be able to distinguish itself from random
projects. Some way of saying "this is good and mature, even if it's not
in the inner circle".

Being in The "integrated release" has been seen as a sign of stability
forever, while it was only ensuring "integration" with other projects
and OpenStack processes. We are getting better at requiring "maturity"
there, but if we set up layers, we'll have to get even better at that.

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-05 Thread Flavio Percoco
On 09/04/2014 07:08 PM, Clint Byrum wrote:
> Excerpts from Flavio Percoco's message of 2014-09-04 06:01:45 -0700:
>> On 09/04/2014 02:14 PM, Sean Dague wrote:
>>> On 09/04/2014 03:08 AM, Flavio Percoco wrote:
 Greetings,

 Last Tuesday the TC held the first graduation review for Zaqar. During
 the meeting some concerns arose. I've listed those concerns below with
 some comments hoping that it will help starting a discussion before the
 next meeting. In addition, I've added some comments about the project
 stability at the bottom and an etherpad link pointing to a list of use
 cases for Zaqar.

 # Concerns

 - Concern on operational burden of requiring NoSQL deploy expertise to
 the mix of openstack operational skills

 For those of you not familiar with Zaqar, it currently supports 2 nosql
 drivers - MongoDB and Redis - and those are the only 2 drivers it
 supports for now. This will require operators willing to use Zaqar to
 maintain a new (?) NoSQL technology in their system. Before expressing
 our thoughts on this matter, let me say that:

 1. By removing the SQLAlchemy driver, we basically removed the chance
 for operators to use an already deployed "OpenStack-technology"
 2. Zaqar won't be backed by any AMQP based messaging technology for
 now. Here's[0] a summary of the research the team (mostly done by
 Victoria) did during Juno
 3. We (OpenStack) used to require Redis for the zmq matchmaker
 4. We (OpenStack) also use memcached for caching and as the oslo
 caching lib becomes available - or a wrapper on top of dogpile.cache -
 Redis may be used in place of memcached in more and more deployments.
 5. Ceilometer's recommended storage driver is still MongoDB, although
 Ceilometer has now support for sqlalchemy. (Please correct me if I'm 
 wrong).

 That being said, it's obvious we already, to some extent, promote some
 NoSQL technologies. However, for the sake of the discussion, lets assume
 we don't.

 I truly believe, with my OpenStack (not Zaqar's) hat on, that we can't
 keep avoiding these technologies. NoSQL technologies have been around
 for years and we should be prepared - including OpenStack operators - to
 support these technologies. Not every tool is good for all tasks - one
 of the reasons we removed the sqlalchemy driver in the first place -
 therefore it's impossible to keep an homogeneous environment for all
 services.

 With this, I'm not suggesting to ignore the risks and the extra burden
 this adds but, instead of attempting to avoid it completely by not
 evolving the stack of services we provide, we should probably work on
 defining a reasonable subset of NoSQL services we are OK with
 supporting. This will help making the burden smaller and it'll give
 operators the option to choose.

 [0] http://blog.flaper87.com/post/marconi-amqp-see-you-later/
>>>
>>> I've been one of the consistent voices concerned about a hard
>>> requirement on adding NoSQL into the mix. So I'll explain that thinking
>>> a bit more.
>>>
>>> I feel like when the TC makes an integration decision previously this
>>> has been about evaluating the project applying for integration, and if
>>> they met some specific criteria they were told about some time in the
>>> past. I think that's the wrong approach. It's a locally optimized
>>> approach that fails to ask the more interesting question.
>>>
>>> Is OpenStack better as a whole if this is a mandatory component of
>>> OpenStack? Better being defined as technically better (more features,
>>> less janky code work arounds, less unexpected behavior from the stack).
>>> Better from the sense of easier or harder to run an actual cloud by our
>>> Operators (taking into account what kinds of moving parts they are now
>>> expected to manage). Better from the sense of a better user experience
>>> in interacting with OpenStack as whole. Better from a sense that the
>>> OpenStack release will experience less bugs, less unexpected cross
>>> project interactions, an a greater overall feel of consistency so that
>>> the OpenStack API feels like one thing.
>>>
>>> https://dague.net/2014/08/26/openstack-as-layers/
>>>
>>> One of the interesting qualities of Layers 1 & 2 is they all follow an
>>> AMQP + RDBMS pattern (excepting swift). You can have a very effective
>>> IaaS out of that stack. They are the things that you can provide pretty
>>> solid integration testing on (and if you look at where everything stood
>>> before the new TC mandates on testing / upgrade that was basically what
>>> was getting integration tested). (Also note, I'll accept Barbican is
>>> probably in the wrong layer, and should be a Layer 2 service.)
>>>
>>> While large shops can afford to have a dedicated team to figure out how
>>> to make mongo or redis HA, provide monitoring, 

Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-04 Thread Kurt Griffiths
Thanks for your comments Gordon. I appreciate where you are coming from
and I think we are actually in agreement on a lot of things.

I just want to make it clear that from the very beginning of the project
the team has tried to communicate (but perhaps could have done a better
job at it) that we aren’t trying to displace other messaging systems that
are clearly delivering a lot of value today.

In fact, I personally have long been a proponent of using the best tool
for the job. The Zaqar project was kicked off at an unconference session
several summits ago because the community saw a need that was not covered
by other messaging systems. Does that mean those other systems are “bad”
or “wrong”? Of course not. It simply means that there are some cases where
those other systems aren’t the best tool for the job, and another tool is
needed (and vice versa).

Does that other tool look *exactly* like Zaqar? Probably not. But a lot of
people have told us Zaqar--in its current form--already delivers a lot of
value that they can’t get from other messaging systems that are available
to them. Zaqar, like any open source project, is a manifestation of lots
of peoples' ideas, and will evolve over time to meet the needs of the
community. 

Does a Qpid/Rabbit/Kafka provisioning service make sense? Probably. Would
such a service totally overlap in terms of use-cases with Zaqar? Community
feedback suggests otherwise. Will there be some other kind of thing that
comes out of the woodwork? Possibly. (Heck, if something better comes
along I for one have no qualms in shifting resources to the more elegant
solution--again, use the best tool for the job.) This process happens all
the time in the broader open-source world. But this process takes a
healthy amount of time, plus broad exposure and usage, which is something
that you simply don’t get as a non-integrated project in the OpenStack
ecosystem.

In any case, it’s pretty clear to me that Zaqar graduating should not be
viewed as making it "the officially blessed messaging service for the
cloud” and nobody is allowed to have any other ideas, ever. If that
happens, it’s only a symptom of a deeper perception/process problem that
is far from unique to Zaqar. In fact, I think it touches on all
non-integrated projects, and many integrated ones as well.

--Kurt

On 9/4/14, 12:38 PM, "Gordon Sim"  wrote:
>
>This is not intended as criticism of Zaqar in anyway. In my opinion,
>'reinvention' is not necessarily a bad thing. It's just another way of
>saying innovative approach and/or original thinking, both of which are
>good and both of which I think are important in the context of
>communicating in the cloud.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-04 Thread Anne Gentle
On Thu, Sep 4, 2014 at 12:39 PM, Tim Bell  wrote:

> > -Original Message-
> > From: Thierry Carrez [mailto:thie...@openstack.org]
> > Sent: 04 September 2014 16:59
> > To: openstack-dev@lists.openstack.org
> > Subject: Re: [openstack-dev] [Zaqar] Comments on the concerns arose
> during
> > the TC meeting
> >
> > Sean Dague wrote:
> > > [...]
> > > So, honestly, I'll probably remain -1 on the final integration vote,
> > > not because Zaqar is bad, but because I'm feeling more firmly that for
> > > OpenStack to not leave the small deployers behind we need to redefine
> > > the tightly integrated piece of OpenStack to basically the Layer 1 & 2
> > > parts of my diagram, and consider the rest of the layers exciting
> > > parts of our ecosystem that more advanced users may choose to deploy
> > > to meet their needs. Smaller tent, big ecosystem, easier on ramp.
> > >
> > > I realize that largely means Zaqar would be caught up in a definition
> > > discussion outside of it's control, and that's kind of unfortunate, as
> > > Flavio and team have been doing a bang up job of late. But we need to
> > > stop considering "integration" as the end game of all interesting
> > > software in the OpenStack ecosystem, and I think it's better to have
> > > that conversation sooner rather than later.
> >
> > I think it's pretty clear at this point that:
> >
> > (1) we need to have a discussion about layers (base nucleus, optional
> extra
> > services at the very least) and the level of support we grant to each --
> the
> > current binary approach is not working very well
> >
> > (2) If we accept Zaqar next week, it's pretty clear it would not fall in
> the base
> > nucleus layer but more in an optional extra services layer, together
> with at the
> > very least Trove and Sahara
> >
> > There are two ways of doing this: follow Sean's approach and -1
> integration
> > (and have zaqar apply to that "optional layer" when we create it), or +1
> > integration now (and have zaqar follow whichever other integrated
> projects we
> > place in that layer when we create it).
> >
> > I'm still hesitating on the best approach. I think they yield the same
> end result,
> > but the -1 approach seems to be a bit more unfair, since it would be
> purely for
> > reasons we don't (yet) apply to currently-integrated projects...
> >
>
> The one concern I have with a small core is that there is not an easy way
> to assess the maturity of a project on stackforge. The stackforge projects
> may be missing packaging, Red Hat testing, puppet modules, install/admin
> documentation etc. Thus, I need to have some indication that a project is
> deployable before looking at it with my user community to see if it meets a
> need that is sustainable.
>
>
I identify with this concern as Docs PTL. That said, it's very difficult
already even if a project is under the "openstack" github org to know how
well documented it is.


> Do you see the "optional layer" services being blessed / validated in some
> way and therefore being easy to identify ?
>
>
I've already started to work on doc requirements for incubating and
integrating projects [1], which may help with some checklist creation. My
thinking is, if you can't document it, it's not well-validated. Is that
reasonable, to adopt a "docs or it didn't happen" stance here?

Thanks for thinking on this -- I do like the layers approach and I think we
just need to define expectations for layers so all the community can do a
quick sanity check.
Anne

1. https://wiki.openstack.org/wiki/Documentation/IncubateIntegrate

 Tim
> > --
> > Thierry Carrez (ttx)
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-04 Thread Tim Bell
> -Original Message-
> From: Thierry Carrez [mailto:thie...@openstack.org]
> Sent: 04 September 2014 16:59
> To: openstack-dev@lists.openstack.org
> Subject: Re: [openstack-dev] [Zaqar] Comments on the concerns arose during
> the TC meeting
> 
> Sean Dague wrote:
> > [...]
> > So, honestly, I'll probably remain -1 on the final integration vote,
> > not because Zaqar is bad, but because I'm feeling more firmly that for
> > OpenStack to not leave the small deployers behind we need to redefine
> > the tightly integrated piece of OpenStack to basically the Layer 1 & 2
> > parts of my diagram, and consider the rest of the layers exciting
> > parts of our ecosystem that more advanced users may choose to deploy
> > to meet their needs. Smaller tent, big ecosystem, easier on ramp.
> >
> > I realize that largely means Zaqar would be caught up in a definition
> > discussion outside of it's control, and that's kind of unfortunate, as
> > Flavio and team have been doing a bang up job of late. But we need to
> > stop considering "integration" as the end game of all interesting
> > software in the OpenStack ecosystem, and I think it's better to have
> > that conversation sooner rather than later.
> 
> I think it's pretty clear at this point that:
> 
> (1) we need to have a discussion about layers (base nucleus, optional extra
> services at the very least) and the level of support we grant to each -- the
> current binary approach is not working very well
> 
> (2) If we accept Zaqar next week, it's pretty clear it would not fall in the 
> base
> nucleus layer but more in an optional extra services layer, together with at 
> the
> very least Trove and Sahara
> 
> There are two ways of doing this: follow Sean's approach and -1 integration
> (and have zaqar apply to that "optional layer" when we create it), or +1
> integration now (and have zaqar follow whichever other integrated projects we
> place in that layer when we create it).
> 
> I'm still hesitating on the best approach. I think they yield the same end 
> result,
> but the -1 approach seems to be a bit more unfair, since it would be purely 
> for
> reasons we don't (yet) apply to currently-integrated projects...
> 

The one concern I have with a small core is that there is not an easy way to 
assess the maturity of a project on stackforge. The stackforge projects may be 
missing packaging, Red Hat testing, puppet modules, install/admin documentation 
etc. Thus, I need to have some indication that a project is deployable before 
looking at it with my user community to see if it meets a need that is 
sustainable.

Do you see the "optional layer" services being blessed / validated in some way 
and therefore being easy to identify ?

Tim
> --
> Thierry Carrez (ttx)
> 
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-04 Thread Gordon Sim

Hi Flavio,

On 09/04/2014 08:08 AM, Flavio Percoco wrote:

- Concern on should we really reinvent a queue system rather than
piggyback on one

As mentioned in the meeting on Tuesday, Zaqar is not reinventing message
brokers. Zaqar provides a service akin to SQS from AWS with an OpenStack
flavor on top. [0]


On 09/04/2014 01:13 PM, Flavio Percoco wrote:

If we put both features, multi-tenancy and multi-protocol, aside for a
bit, we can simplify Zaqars goal down to a "messaging service for the
cloud".  I believe this is exactly where the line between Zaqar and other
*queuing*  technologies should be drawn. Zaqar is, at the very end, a
messaging service thought for the cloud whereas existing queuing
technologies were not designed for it.


Isn't this the real answer to the concern above? Zaqar *is* a 
reimplementation of a messaging service, but you believe it is 
*necessary* to do this because existing implementations would not lend 
themselves to being as well integrated into the 'cloud', or do not 
provide sufficient support for multi-tenancy.


I suspect if you asked, you would find that many existing projects 
implementing a messaging service would see their solution as very much 
suitable for the cloud. You disagree and probably with good reason. I'm 
just pointing out the dividing line is I think not as clear to everyone 
as it may be to you. If you add support for STOMP, MQTT, AMQP or any 
other protocol the overlap with other messaging services will be ever 
greater (and will bring in new requirements and new expectations around 
performance).


This is not intended as criticism of Zaqar in anyway. In my opinion, 
'reinvention' is not necessarily a bad thing. It's just another way of 
saying innovative approach and/or original thinking, both of which are 
good and both of which I think are important in the context of 
communicating in the cloud.


My concern would be more that Zaqar's privileged status as the 
officially 'blessed' "messaging service for the cloud" - which is a 
broad space, much broader than just an SQS equivalent - would make it 
harder for other approaches to gain traction.


--Gordon.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-04 Thread Clint Byrum
Excerpts from Flavio Percoco's message of 2014-09-04 02:11:15 -0700:
> Hey Clint,
> 
> Thanks for reading, some comments in-line:
> 
> On 09/04/2014 10:30 AM, Clint Byrum wrote:
> > Excerpts from Flavio Percoco's message of 2014-09-04 00:08:47 -0700:
> 
> [snip]
> 
> >> - Concern on should we really reinvent a queue system rather than
> >> piggyback on one
> >>
> >> As mentioned in the meeting on Tuesday, Zaqar is not reinventing message
> >> brokers. Zaqar provides a service akin to SQS from AWS with an OpenStack
> >> flavor on top. [0]
> >>
> > 
> > I think Zaqar is more like SMTP and IMAP than AMQP. You're not really
> > trying to connect two processes in real time. You're trying to do fully
> > asynchronous messaging with fully randomized access to any message.
> > 
> > Perhaps somebody should explore whether the approaches taken by large
> > scale IMAP providers could be applied to Zaqar.
> > 
> > Anyway, I can't imagine writing a system to intentionally use the
> > semantics of IMAP and SMTP. I'd be very interested in seeing actual use
> > cases for it, apologies if those have been posted before.
> > 
> >> Some things that differentiate Zaqar from SQS is it's capability for
> >> supporting different protocols without sacrificing multi-tenantcy and
> >> other intrinsic features it provides. Some protocols you may consider
> >> for Zaqar are: STOMP, MQTT.
> >>
> >> As far as the backend goes, Zaqar is not re-inventing it either. It sits
> >> on top of existing storage technologies that have proven to be fast and
> >> reliable for this task. The choice of using NoSQL technologies has a lot
> >> to do with this particular thing and the fact that Zaqar needs a storage
> >> capable of scaling, replicating and good support for failover.
> >>
> > 
> > What's odd to me is that other systems like Cassandra and Riak are not
> > being discussed. There are well documented large scale message storage
> > systems on both, and neither is encumbered by the same licensing FUD
> > as MongoDB.
> 
> FWIW, they both have been discussed. As far as Cassandra goes, we raised
> the red flag after reading reading this post[0]. The post itself may be
> obsolete already but I don't think I have enough knowledge about
> Cassandra to actually figure this out. Some folks have come to us asking
> for a Cassandra driver and they were interested in contributing/working
> on one. I really hope that will happen someday, although it'll certainly
> happen as an external driver. Riak, on the other hand, was certainly a
> good candidate. What made us go with MongoDB and Redis is they're both
> good for the job, they are both likely already deployed in OpenStack
> clouds and we have enough knowledge to provide support and maintenance
> for both drivers.

It seems like Cassandra is good for when you're going to be writing all
the time but only reading once. I would agree that this makes it less
attractive for a generalized messaging platform, since you won't know
how users will consume the messages, and if they are constantly
reading then you'll have terrible performance.

> >> # Use Cases
> >>
> >> In addition to the aforementioned concerns and comments, I also would
> >> like to share an etherpad that contains some use cases that other
> >> integrated projects have for Zaqar[0]. The list is not exhaustive and
> >> it'll contain more information before the next meeting.
> >>
> >> [0] https://etherpad.openstack.org/p/zaqar-integrated-projects-use-cases
> >>
> > 
> > Just taking a look, there are two basic applications needed:
> > 
> > 1) An inbox. Horizon wants to know when snapshots are done. Heat wants
> > to know what happened during a stack action. Etc.
> > 
> > 2) A user-focused message queue. Heat wants to push data to agents.
> > Swift wants to synchronize processes when things happen.
> > 
> > To me, #1 is Zaqar as it is today. #2 is the one that I worry may not
> > be served best by bending #1 onto it.
> 
> Push semantics are being developed. We've had enough discussions that
> have helped preparing the ground for it. However, I believe both use
> cases could be covered by Zaqar as-is.
> 
> Could you elaborate a bit more on #2? Especially on why you think Zaqar
> as is can't serve this specific case?

The difference between 1 and 2 is that 2 is a true queueing problem. The
message should go away when it has been consumed, and the volume may be
rather high. With 1, you have a storage problem, and a database makes a
lot more sense. If users can stick to type 2 problems, they'll be able
to stay much more lightweight because they won't need a large data store
that supports random access.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-04 Thread Clint Byrum
Excerpts from Flavio Percoco's message of 2014-09-04 06:01:45 -0700:
> On 09/04/2014 02:14 PM, Sean Dague wrote:
> > On 09/04/2014 03:08 AM, Flavio Percoco wrote:
> >> Greetings,
> >>
> >> Last Tuesday the TC held the first graduation review for Zaqar. During
> >> the meeting some concerns arose. I've listed those concerns below with
> >> some comments hoping that it will help starting a discussion before the
> >> next meeting. In addition, I've added some comments about the project
> >> stability at the bottom and an etherpad link pointing to a list of use
> >> cases for Zaqar.
> >>
> >> # Concerns
> >>
> >> - Concern on operational burden of requiring NoSQL deploy expertise to
> >> the mix of openstack operational skills
> >>
> >> For those of you not familiar with Zaqar, it currently supports 2 nosql
> >> drivers - MongoDB and Redis - and those are the only 2 drivers it
> >> supports for now. This will require operators willing to use Zaqar to
> >> maintain a new (?) NoSQL technology in their system. Before expressing
> >> our thoughts on this matter, let me say that:
> >>
> >> 1. By removing the SQLAlchemy driver, we basically removed the chance
> >> for operators to use an already deployed "OpenStack-technology"
> >> 2. Zaqar won't be backed by any AMQP based messaging technology for
> >> now. Here's[0] a summary of the research the team (mostly done by
> >> Victoria) did during Juno
> >> 3. We (OpenStack) used to require Redis for the zmq matchmaker
> >> 4. We (OpenStack) also use memcached for caching and as the oslo
> >> caching lib becomes available - or a wrapper on top of dogpile.cache -
> >> Redis may be used in place of memcached in more and more deployments.
> >> 5. Ceilometer's recommended storage driver is still MongoDB, although
> >> Ceilometer has now support for sqlalchemy. (Please correct me if I'm 
> >> wrong).
> >>
> >> That being said, it's obvious we already, to some extent, promote some
> >> NoSQL technologies. However, for the sake of the discussion, lets assume
> >> we don't.
> >>
> >> I truly believe, with my OpenStack (not Zaqar's) hat on, that we can't
> >> keep avoiding these technologies. NoSQL technologies have been around
> >> for years and we should be prepared - including OpenStack operators - to
> >> support these technologies. Not every tool is good for all tasks - one
> >> of the reasons we removed the sqlalchemy driver in the first place -
> >> therefore it's impossible to keep an homogeneous environment for all
> >> services.
> >>
> >> With this, I'm not suggesting to ignore the risks and the extra burden
> >> this adds but, instead of attempting to avoid it completely by not
> >> evolving the stack of services we provide, we should probably work on
> >> defining a reasonable subset of NoSQL services we are OK with
> >> supporting. This will help making the burden smaller and it'll give
> >> operators the option to choose.
> >>
> >> [0] http://blog.flaper87.com/post/marconi-amqp-see-you-later/
> > 
> > I've been one of the consistent voices concerned about a hard
> > requirement on adding NoSQL into the mix. So I'll explain that thinking
> > a bit more.
> > 
> > I feel like when the TC makes an integration decision previously this
> > has been about evaluating the project applying for integration, and if
> > they met some specific criteria they were told about some time in the
> > past. I think that's the wrong approach. It's a locally optimized
> > approach that fails to ask the more interesting question.
> > 
> > Is OpenStack better as a whole if this is a mandatory component of
> > OpenStack? Better being defined as technically better (more features,
> > less janky code work arounds, less unexpected behavior from the stack).
> > Better from the sense of easier or harder to run an actual cloud by our
> > Operators (taking into account what kinds of moving parts they are now
> > expected to manage). Better from the sense of a better user experience
> > in interacting with OpenStack as whole. Better from a sense that the
> > OpenStack release will experience less bugs, less unexpected cross
> > project interactions, an a greater overall feel of consistency so that
> > the OpenStack API feels like one thing.
> > 
> > https://dague.net/2014/08/26/openstack-as-layers/
> > 
> > One of the interesting qualities of Layers 1 & 2 is they all follow an
> > AMQP + RDBMS pattern (excepting swift). You can have a very effective
> > IaaS out of that stack. They are the things that you can provide pretty
> > solid integration testing on (and if you look at where everything stood
> > before the new TC mandates on testing / upgrade that was basically what
> > was getting integration tested). (Also note, I'll accept Barbican is
> > probably in the wrong layer, and should be a Layer 2 service.)
> > 
> > While large shops can afford to have a dedicated team to figure out how
> > to make mongo or redis HA, provide monitoring, have a DR plan for when a
> > huricane r

Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-04 Thread Flavio Percoco
On 09/04/2014 04:59 PM, Thierry Carrez wrote:
> Sean Dague wrote:
>> [...]
>> So, honestly, I'll probably remain -1 on the final integration vote, not
>> because Zaqar is bad, but because I'm feeling more firmly that for
>> OpenStack to not leave the small deployers behind we need to redefine
>> the tightly integrated piece of OpenStack to basically the Layer 1 & 2
>> parts of my diagram, and consider the rest of the layers exciting parts
>> of our ecosystem that more advanced users may choose to deploy to meet
>> their needs. Smaller tent, big ecosystem, easier on ramp.
>>
>> I realize that largely means Zaqar would be caught up in a definition
>> discussion outside of it's control, and that's kind of unfortunate, as
>> Flavio and team have been doing a bang up job of late. But we need to
>> stop considering "integration" as the end game of all interesting
>> software in the OpenStack ecosystem, and I think it's better to have
>> that conversation sooner rather than later.
> 
> I think it's pretty clear at this point that:
> 
> (1) we need to have a discussion about layers (base nucleus, optional
> extra services at the very least) and the level of support we grant to
> each -- the current binary approach is not working very well
> 
> (2) If we accept Zaqar next week, it's pretty clear it would not fall in
> the base nucleus layer but more in an optional extra services layer,
> together with at the very least Trove and Sahara
> 
> There are two ways of doing this: follow Sean's approach and -1
> integration (and have zaqar apply to that "optional layer" when we
> create it), or +1 integration now (and have zaqar follow whichever other
> integrated projects we place in that layer when we create it).

As I mentioned in my reply to Sean's email, I believe +1 integration is
the correct thing to do. I know it's hard to believe that I'm saying
this with my OpenStack hat on and not Zaqar's but that's the truth.

I truly believe we can't stop OpenStack's growth on this. We'll manage
these growth details later on as we've done so far. Growing is as
important as managing the growth. Though, in this case we're not growing
without any clue of what will happen. We've a well known path that all
integrated projects have followed and, in this specific case, Zaqar is
following.

Re-evaluating projects is something that has happened - and should
happen - every once in a while. Once we have a place for this "optional"
services, we will have to re-evaluate all the integrated projects and
move those that fit into that category.

> 
> I'm still hesitating on the best approach. I think they yield the same
> end result, but the -1 approach seems to be a bit more unfair, since it
> would be purely for reasons we don't (yet) apply to currently-integrated
> projects...

+1

Cheers,
Flavio


-- 
@flaper87
Flavio Percoco

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-04 Thread Thierry Carrez
Sean Dague wrote:
> [...]
> So, honestly, I'll probably remain -1 on the final integration vote, not
> because Zaqar is bad, but because I'm feeling more firmly that for
> OpenStack to not leave the small deployers behind we need to redefine
> the tightly integrated piece of OpenStack to basically the Layer 1 & 2
> parts of my diagram, and consider the rest of the layers exciting parts
> of our ecosystem that more advanced users may choose to deploy to meet
> their needs. Smaller tent, big ecosystem, easier on ramp.
> 
> I realize that largely means Zaqar would be caught up in a definition
> discussion outside of it's control, and that's kind of unfortunate, as
> Flavio and team have been doing a bang up job of late. But we need to
> stop considering "integration" as the end game of all interesting
> software in the OpenStack ecosystem, and I think it's better to have
> that conversation sooner rather than later.

I think it's pretty clear at this point that:

(1) we need to have a discussion about layers (base nucleus, optional
extra services at the very least) and the level of support we grant to
each -- the current binary approach is not working very well

(2) If we accept Zaqar next week, it's pretty clear it would not fall in
the base nucleus layer but more in an optional extra services layer,
together with at the very least Trove and Sahara

There are two ways of doing this: follow Sean's approach and -1
integration (and have zaqar apply to that "optional layer" when we
create it), or +1 integration now (and have zaqar follow whichever other
integrated projects we place in that layer when we create it).

I'm still hesitating on the best approach. I think they yield the same
end result, but the -1 approach seems to be a bit more unfair, since it
would be purely for reasons we don't (yet) apply to currently-integrated
projects...

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-04 Thread Flavio Percoco
On 09/04/2014 02:14 PM, Sean Dague wrote:
> On 09/04/2014 03:08 AM, Flavio Percoco wrote:
>> Greetings,
>>
>> Last Tuesday the TC held the first graduation review for Zaqar. During
>> the meeting some concerns arose. I've listed those concerns below with
>> some comments hoping that it will help starting a discussion before the
>> next meeting. In addition, I've added some comments about the project
>> stability at the bottom and an etherpad link pointing to a list of use
>> cases for Zaqar.
>>
>> # Concerns
>>
>> - Concern on operational burden of requiring NoSQL deploy expertise to
>> the mix of openstack operational skills
>>
>> For those of you not familiar with Zaqar, it currently supports 2 nosql
>> drivers - MongoDB and Redis - and those are the only 2 drivers it
>> supports for now. This will require operators willing to use Zaqar to
>> maintain a new (?) NoSQL technology in their system. Before expressing
>> our thoughts on this matter, let me say that:
>>
>>  1. By removing the SQLAlchemy driver, we basically removed the chance
>> for operators to use an already deployed "OpenStack-technology"
>>  2. Zaqar won't be backed by any AMQP based messaging technology for
>> now. Here's[0] a summary of the research the team (mostly done by
>> Victoria) did during Juno
>>  3. We (OpenStack) used to require Redis for the zmq matchmaker
>>  4. We (OpenStack) also use memcached for caching and as the oslo
>> caching lib becomes available - or a wrapper on top of dogpile.cache -
>> Redis may be used in place of memcached in more and more deployments.
>>  5. Ceilometer's recommended storage driver is still MongoDB, although
>> Ceilometer has now support for sqlalchemy. (Please correct me if I'm wrong).
>>
>> That being said, it's obvious we already, to some extent, promote some
>> NoSQL technologies. However, for the sake of the discussion, lets assume
>> we don't.
>>
>> I truly believe, with my OpenStack (not Zaqar's) hat on, that we can't
>> keep avoiding these technologies. NoSQL technologies have been around
>> for years and we should be prepared - including OpenStack operators - to
>> support these technologies. Not every tool is good for all tasks - one
>> of the reasons we removed the sqlalchemy driver in the first place -
>> therefore it's impossible to keep an homogeneous environment for all
>> services.
>>
>> With this, I'm not suggesting to ignore the risks and the extra burden
>> this adds but, instead of attempting to avoid it completely by not
>> evolving the stack of services we provide, we should probably work on
>> defining a reasonable subset of NoSQL services we are OK with
>> supporting. This will help making the burden smaller and it'll give
>> operators the option to choose.
>>
>> [0] http://blog.flaper87.com/post/marconi-amqp-see-you-later/
> 
> I've been one of the consistent voices concerned about a hard
> requirement on adding NoSQL into the mix. So I'll explain that thinking
> a bit more.
> 
> I feel like when the TC makes an integration decision previously this
> has been about evaluating the project applying for integration, and if
> they met some specific criteria they were told about some time in the
> past. I think that's the wrong approach. It's a locally optimized
> approach that fails to ask the more interesting question.
> 
> Is OpenStack better as a whole if this is a mandatory component of
> OpenStack? Better being defined as technically better (more features,
> less janky code work arounds, less unexpected behavior from the stack).
> Better from the sense of easier or harder to run an actual cloud by our
> Operators (taking into account what kinds of moving parts they are now
> expected to manage). Better from the sense of a better user experience
> in interacting with OpenStack as whole. Better from a sense that the
> OpenStack release will experience less bugs, less unexpected cross
> project interactions, an a greater overall feel of consistency so that
> the OpenStack API feels like one thing.
> 
> https://dague.net/2014/08/26/openstack-as-layers/
> 
> One of the interesting qualities of Layers 1 & 2 is they all follow an
> AMQP + RDBMS pattern (excepting swift). You can have a very effective
> IaaS out of that stack. They are the things that you can provide pretty
> solid integration testing on (and if you look at where everything stood
> before the new TC mandates on testing / upgrade that was basically what
> was getting integration tested). (Also note, I'll accept Barbican is
> probably in the wrong layer, and should be a Layer 2 service.)
> 
> While large shops can afford to have a dedicated team to figure out how
> to make mongo or redis HA, provide monitoring, have a DR plan for when a
> huricane requires them to flip datacenters, that basically means
> OpenStack heads further down the path of "only for the big folks". I
> don't want OpenStack to be only for the big folks, I want OpenStack to
> be for all sized folks. I really do want to hav

Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-04 Thread Flavio Percoco
On 09/04/2014 01:15 PM, Chris Dent wrote:
> On Thu, 4 Sep 2014, Flavio Percoco wrote:
> 
> Thanks for writing this up, interesting read.
> 

Thank you for your feedback :)

Some comments in-line.

>> 5. Ceilometer's recommended storage driver is still MongoDB, although
>> Ceilometer has now support for sqlalchemy. (Please correct me if I'm
>> wrong).
> 
> For sake of reference: Yes, MongoDB is currently the recommended
> store and yes, sqlalchemy support is present. Until recently only
> sqlalchemy support was tested in the gate. Two big changes being
> developed in Juno related to storage:
> 
> * Improved read and write performance in the sqlalchemy setup.
> * time series storage and Gnocchi:
>  
> https://julien.danjou.info/blog/2014/openstack-ceilometer-the-gnocchi-experiment

Awesome, thanks for clarifying this.

[snip]

>> As mentioned in the meeting on Tuesday, Zaqar is not reinventing message
>> brokers. Zaqar provides a service akin to SQS from AWS with an OpenStack
>> flavor on top. [0]
> 
> In my efforts to track this stuff I remain confused on the points in
> these two questions:
> 
> https://wiki.openstack.org/wiki/Zaqar/Frequently_asked_questions#How_does_Zaqar_compare_to_oslo.messaging.3F
> 
> https://wiki.openstack.org/wiki/Zaqar/Frequently_asked_questions#Is_Zaqar_an_under-cloud_or_an_over-cloud_service.3F
> 
> 
> What or where is the boundary between Zaqar and existing messaging
> infrastructure? Not just in terms of technology but also use cases?
> The answers above suggest its not super solid on the use case side,
> notably: "In addition, several projects have expressed interest in
> integrating with Zaqar in order to surface events..."
> 
> Instead of Zaqar doing what it does and instead of oslo.messaging
> abstracting RPC, why isn't the end goal a multi-tenant, multi-protocol
> event pool? Wouldn't that have the most flexibility in terms of
> ecosystem and scalability?

If we put both features, multi-tenancy and multi-protocol, aside for a
bit, we can simplify Zaqars goal down to a "messaging service for the
cloud". I believe this is exactly where the line between Zaqar and other
*queuing* technologies should be drawn. Zaqar is, at the very end, a
messaging service thought for the cloud whereas existing queuing
technologies were not designed for it. By cloud I don't mean
performance, scalability nor anything like that. I'm talking about
providing a service that end-users of the cloud can consume.

The fact that Zaqar is also ideal for the under-cloud is a plus. The
service has been designed to suffice a set of messaging features that
serve perfectly use-cases in both the under-cloud and over-cloud.

If we add to that a multi-protocol transport layer with support for
multi-tenancy, you'll get a queuing service that fits the need of cloud
providers and covers a broader set of use cases like, say, IoT.

I forgot to add this link[0] to my previous email. Does the overview of
the service, the key features and scope help clearing things out a bit?

Please, let me know if they don't. I'm happy to provide more info if needed.

[0] https://wiki.openstack.org/wiki/Zaqar#Overview

> 
>> In addition to the aforementioned concerns and comments, I also would
>> like to share an etherpad that contains some use cases that other
>> integrated projects have for Zaqar[0]. The list is not exhaustive and
>> it'll contain more information before the next meeting.
>>
>> [0] https://etherpad.openstack.org/p/zaqar-integrated-projects-use-cases
> 
> For these, what is Zaqar providing that oslo.messaging (and its
> still extant antecedents) does not? I'm not asking to naysay Zaqar,
> but to understand more clearly what's going on. My interest here comes
> from a general interest in now events and notifications are handled
> throughout OpenStack.

One of the reasons you would want to use Zaqar instead of oslo.messaging
for, say, guest-agents is that you don't want guest-agents to talk to
your main messaging layer. Zaqar will help guest-agents to communicate
with the main service in a more secure, authenticated and isolated way.

If you were going to do that with oslo.messaging, you'd need to have
separate virtual_hosts, exchanges and probably even users. This things
cannot be easily configured without manual intervention. With Zaqar you
can easily rely on your deployed cloud services - keystone, Barbican and
Zaqar, for example - to achieve such isolation and security.

There are also other aspects that are worrisome of relying on the main
messaging infrastructure for the use cases mentioned in that etherpad.
For example, using OpenStack's main rabbitmq instance to communicate
with guest-agents would increase the workload on the infrastructure,
which would require a better scaling strategy for it.

I hope the above clears your doubts. Thanks a lot for your feedback,
it's useful to keep the discussion going and helps everyone to keep
re-evaluating the goals and scopes of the project.

I hope other folks from the te

Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-04 Thread Sean Dague
On 09/04/2014 03:08 AM, Flavio Percoco wrote:
> Greetings,
> 
> Last Tuesday the TC held the first graduation review for Zaqar. During
> the meeting some concerns arose. I've listed those concerns below with
> some comments hoping that it will help starting a discussion before the
> next meeting. In addition, I've added some comments about the project
> stability at the bottom and an etherpad link pointing to a list of use
> cases for Zaqar.
> 
> # Concerns
> 
> - Concern on operational burden of requiring NoSQL deploy expertise to
> the mix of openstack operational skills
> 
> For those of you not familiar with Zaqar, it currently supports 2 nosql
> drivers - MongoDB and Redis - and those are the only 2 drivers it
> supports for now. This will require operators willing to use Zaqar to
> maintain a new (?) NoSQL technology in their system. Before expressing
> our thoughts on this matter, let me say that:
> 
>   1. By removing the SQLAlchemy driver, we basically removed the chance
> for operators to use an already deployed "OpenStack-technology"
>   2. Zaqar won't be backed by any AMQP based messaging technology for
> now. Here's[0] a summary of the research the team (mostly done by
> Victoria) did during Juno
>   3. We (OpenStack) used to require Redis for the zmq matchmaker
>   4. We (OpenStack) also use memcached for caching and as the oslo
> caching lib becomes available - or a wrapper on top of dogpile.cache -
> Redis may be used in place of memcached in more and more deployments.
>   5. Ceilometer's recommended storage driver is still MongoDB, although
> Ceilometer has now support for sqlalchemy. (Please correct me if I'm wrong).
> 
> That being said, it's obvious we already, to some extent, promote some
> NoSQL technologies. However, for the sake of the discussion, lets assume
> we don't.
> 
> I truly believe, with my OpenStack (not Zaqar's) hat on, that we can't
> keep avoiding these technologies. NoSQL technologies have been around
> for years and we should be prepared - including OpenStack operators - to
> support these technologies. Not every tool is good for all tasks - one
> of the reasons we removed the sqlalchemy driver in the first place -
> therefore it's impossible to keep an homogeneous environment for all
> services.
> 
> With this, I'm not suggesting to ignore the risks and the extra burden
> this adds but, instead of attempting to avoid it completely by not
> evolving the stack of services we provide, we should probably work on
> defining a reasonable subset of NoSQL services we are OK with
> supporting. This will help making the burden smaller and it'll give
> operators the option to choose.
> 
> [0] http://blog.flaper87.com/post/marconi-amqp-see-you-later/

I've been one of the consistent voices concerned about a hard
requirement on adding NoSQL into the mix. So I'll explain that thinking
a bit more.

I feel like when the TC makes an integration decision previously this
has been about evaluating the project applying for integration, and if
they met some specific criteria they were told about some time in the
past. I think that's the wrong approach. It's a locally optimized
approach that fails to ask the more interesting question.

Is OpenStack better as a whole if this is a mandatory component of
OpenStack? Better being defined as technically better (more features,
less janky code work arounds, less unexpected behavior from the stack).
Better from the sense of easier or harder to run an actual cloud by our
Operators (taking into account what kinds of moving parts they are now
expected to manage). Better from the sense of a better user experience
in interacting with OpenStack as whole. Better from a sense that the
OpenStack release will experience less bugs, less unexpected cross
project interactions, an a greater overall feel of consistency so that
the OpenStack API feels like one thing.

https://dague.net/2014/08/26/openstack-as-layers/

One of the interesting qualities of Layers 1 & 2 is they all follow an
AMQP + RDBMS pattern (excepting swift). You can have a very effective
IaaS out of that stack. They are the things that you can provide pretty
solid integration testing on (and if you look at where everything stood
before the new TC mandates on testing / upgrade that was basically what
was getting integration tested). (Also note, I'll accept Barbican is
probably in the wrong layer, and should be a Layer 2 service.)

While large shops can afford to have a dedicated team to figure out how
to make mongo or redis HA, provide monitoring, have a DR plan for when a
huricane requires them to flip datacenters, that basically means
OpenStack heads further down the path of "only for the big folks". I
don't want OpenStack to be only for the big folks, I want OpenStack to
be for all sized folks. I really do want to have all the local small
colleges around here have OpenStack clouds, because it's something that
people believe they can do and manage. I know the people that wo

Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-04 Thread Chris Dent

On Thu, 4 Sep 2014, Flavio Percoco wrote:

Thanks for writing this up, interesting read.


5. Ceilometer's recommended storage driver is still MongoDB, although
Ceilometer has now support for sqlalchemy. (Please correct me if I'm wrong).


For sake of reference: Yes, MongoDB is currently the recommended
store and yes, sqlalchemy support is present. Until recently only
sqlalchemy support was tested in the gate. Two big changes being
developed in Juno related to storage:

* Improved read and write performance in the sqlalchemy setup.
* time series storage and Gnocchi:
  
https://julien.danjou.info/blog/2014/openstack-ceilometer-the-gnocchi-experiment


I truly believe, with my OpenStack (not Zaqar's) hat on, that we can't
keep avoiding these technologies. NoSQL technologies have been around
for years and we should be prepared - including OpenStack operators - to
support these technologies. Not every tool is good for all tasks - one
of the reasons we removed the sqlalchemy driver in the first place -
therefore it's impossible to keep an homogeneous environment for all
services.


+1. Ain't that the truth.


As mentioned in the meeting on Tuesday, Zaqar is not reinventing message
brokers. Zaqar provides a service akin to SQS from AWS with an OpenStack
flavor on top. [0]


In my efforts to track this stuff I remain confused on the points in
these two questions:

https://wiki.openstack.org/wiki/Zaqar/Frequently_asked_questions#How_does_Zaqar_compare_to_oslo.messaging.3F
https://wiki.openstack.org/wiki/Zaqar/Frequently_asked_questions#Is_Zaqar_an_under-cloud_or_an_over-cloud_service.3F

What or where is the boundary between Zaqar and existing messaging
infrastructure? Not just in terms of technology but also use cases?
The answers above suggest its not super solid on the use case side,
notably: "In addition, several projects have expressed interest in
integrating with Zaqar in order to surface events..."

Instead of Zaqar doing what it does and instead of oslo.messaging
abstracting RPC, why isn't the end goal a multi-tenant, multi-protocol
event pool? Wouldn't that have the most flexibility in terms of
ecosystem and scalability?


In addition to the aforementioned concerns and comments, I also would
like to share an etherpad that contains some use cases that other
integrated projects have for Zaqar[0]. The list is not exhaustive and
it'll contain more information before the next meeting.

[0] https://etherpad.openstack.org/p/zaqar-integrated-projects-use-cases


For these, what is Zaqar providing that oslo.messaging (and its
still extant antecedents) does not? I'm not asking to naysay Zaqar,
but to understand more clearly what's going on. My interest here comes
from a general interest in now events and notifications are handled
throughout OpenStack.

Thanks.

--
Chris Dent tw:@anticdent freenode:cdent
https://tank.peermore.com/tanks/cdent

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-04 Thread Flavio Percoco
Hey Clint,

Thanks for reading, some comments in-line:

On 09/04/2014 10:30 AM, Clint Byrum wrote:
> Excerpts from Flavio Percoco's message of 2014-09-04 00:08:47 -0700:

[snip]

>> - Concern on should we really reinvent a queue system rather than
>> piggyback on one
>>
>> As mentioned in the meeting on Tuesday, Zaqar is not reinventing message
>> brokers. Zaqar provides a service akin to SQS from AWS with an OpenStack
>> flavor on top. [0]
>>
> 
> I think Zaqar is more like SMTP and IMAP than AMQP. You're not really
> trying to connect two processes in real time. You're trying to do fully
> asynchronous messaging with fully randomized access to any message.
> 
> Perhaps somebody should explore whether the approaches taken by large
> scale IMAP providers could be applied to Zaqar.
> 
> Anyway, I can't imagine writing a system to intentionally use the
> semantics of IMAP and SMTP. I'd be very interested in seeing actual use
> cases for it, apologies if those have been posted before.
> 
>> Some things that differentiate Zaqar from SQS is it's capability for
>> supporting different protocols without sacrificing multi-tenantcy and
>> other intrinsic features it provides. Some protocols you may consider
>> for Zaqar are: STOMP, MQTT.
>>
>> As far as the backend goes, Zaqar is not re-inventing it either. It sits
>> on top of existing storage technologies that have proven to be fast and
>> reliable for this task. The choice of using NoSQL technologies has a lot
>> to do with this particular thing and the fact that Zaqar needs a storage
>> capable of scaling, replicating and good support for failover.
>>
> 
> What's odd to me is that other systems like Cassandra and Riak are not
> being discussed. There are well documented large scale message storage
> systems on both, and neither is encumbered by the same licensing FUD
> as MongoDB.

FWIW, they both have been discussed. As far as Cassandra goes, we raised
the red flag after reading reading this post[0]. The post itself may be
obsolete already but I don't think I have enough knowledge about
Cassandra to actually figure this out. Some folks have come to us asking
for a Cassandra driver and they were interested in contributing/working
on one. I really hope that will happen someday, although it'll certainly
happen as an external driver. Riak, on the other hand, was certainly a
good candidate. What made us go with MongoDB and Redis is they're both
good for the job, they are both likely already deployed in OpenStack
clouds and we have enough knowledge to provide support and maintenance
for both drivers.

As a curious note, ElasticSearch and Swift have also been brought up
several times as valid stores for Zaqar. I haven't thought about this
throughly but I think they'd do a good job.

[0]
http://www.datastax.com/dev/blog/cassandra-anti-patterns-queues-and-queue-like-datasets

> Anyway, again if we look at this as a place to storage and retrieve
> messages, and not as a queue, then talking about databases, instead of
> message brokers, makes a lot more sense.
> 
>>
>> - concern on the maturity of the NoQSL not AGPL backend (Redis)
>>
>> Redis backend just landed and I've been working on a gate job for it
>> today. Although it hasn't been tested in production, if Zaqar graduates,
>> it still has a full development cycle to be tested and improved before
>> the first integrated release happens.
>>
> 
> I'd be quite interested to see how it is expected to scale. From my very
> quick reading of the driver, it only supports a single redis server. No
> consistent hash ring or anything like that.

Indeed, support for redis-cluster is in our roadmap[0]. As of now, it
can be scaled by using Zaqar pools. You can create several pools of
redis nodes, that you can balance between queues.

The next series of benchmarks will be done on this new Redis driver. I
hope those will be ready soon.

[0] https://blueprints.launchpad.net/zaqar/+spec/redis-pool


>> # Use Cases
>>
>> In addition to the aforementioned concerns and comments, I also would
>> like to share an etherpad that contains some use cases that other
>> integrated projects have for Zaqar[0]. The list is not exhaustive and
>> it'll contain more information before the next meeting.
>>
>> [0] https://etherpad.openstack.org/p/zaqar-integrated-projects-use-cases
>>
> 
> Just taking a look, there are two basic applications needed:
> 
> 1) An inbox. Horizon wants to know when snapshots are done. Heat wants
> to know what happened during a stack action. Etc.
> 
> 2) A user-focused message queue. Heat wants to push data to agents.
> Swift wants to synchronize processes when things happen.
> 
> To me, #1 is Zaqar as it is today. #2 is the one that I worry may not
> be served best by bending #1 onto it.

Push semantics are being developed. We've had enough discussions that
have helped preparing the ground for it. However, I believe both use
cases could be covered by Zaqar as-is.

Could you elaborate a bit more on #2? Especially on why yo

Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-04 Thread Clint Byrum
Excerpts from Flavio Percoco's message of 2014-09-04 00:08:47 -0700:
> Greetings,
> 
> Last Tuesday the TC held the first graduation review for Zaqar. During
> the meeting some concerns arose. I've listed those concerns below with
> some comments hoping that it will help starting a discussion before the
> next meeting. In addition, I've added some comments about the project
> stability at the bottom and an etherpad link pointing to a list of use
> cases for Zaqar.
> 

Hi Flavio. This was an interesting read. As somebody whose attention has
recently been drawn to Zaqar, I am quite interested in seeing it
graduate.

> # Concerns
> 
> - Concern on operational burden of requiring NoSQL deploy expertise to
> the mix of openstack operational skills
> 
> For those of you not familiar with Zaqar, it currently supports 2 nosql
> drivers - MongoDB and Redis - and those are the only 2 drivers it
> supports for now. This will require operators willing to use Zaqar to
> maintain a new (?) NoSQL technology in their system. Before expressing
> our thoughts on this matter, let me say that:
> 
> 1. By removing the SQLAlchemy driver, we basically removed the chance
> for operators to use an already deployed "OpenStack-technology"
> 2. Zaqar won't be backed by any AMQP based messaging technology for
> now. Here's[0] a summary of the research the team (mostly done by
> Victoria) did during Juno
> 3. We (OpenStack) used to require Redis for the zmq matchmaker
> 4. We (OpenStack) also use memcached for caching and as the oslo
> caching lib becomes available - or a wrapper on top of dogpile.cache -
> Redis may be used in place of memcached in more and more deployments.
> 5. Ceilometer's recommended storage driver is still MongoDB, although
> Ceilometer has now support for sqlalchemy. (Please correct me if I'm wrong).
> 
> That being said, it's obvious we already, to some extent, promote some
> NoSQL technologies. However, for the sake of the discussion, lets assume
> we don't.
> 
> I truly believe, with my OpenStack (not Zaqar's) hat on, that we can't
> keep avoiding these technologies. NoSQL technologies have been around
> for years and we should be prepared - including OpenStack operators - to
> support these technologies. Not every tool is good for all tasks - one
> of the reasons we removed the sqlalchemy driver in the first place -
> therefore it's impossible to keep an homogeneous environment for all
> services.
> 

I whole heartedly agree that non traditional storage technologies that
are becoming mainstream are good candidates for use cases where SQL
based storage gets in the way. I wish there wasn't so much FUD
(warranted or not) about MongoDB, but that is the reality we live in.

> With this, I'm not suggesting to ignore the risks and the extra burden
> this adds but, instead of attempting to avoid it completely by not
> evolving the stack of services we provide, we should probably work on
> defining a reasonable subset of NoSQL services we are OK with
> supporting. This will help making the burden smaller and it'll give
> operators the option to choose.
> 
> [0] http://blog.flaper87.com/post/marconi-amqp-see-you-later/
> 
> 
> - Concern on should we really reinvent a queue system rather than
> piggyback on one
> 
> As mentioned in the meeting on Tuesday, Zaqar is not reinventing message
> brokers. Zaqar provides a service akin to SQS from AWS with an OpenStack
> flavor on top. [0]
> 

I think Zaqar is more like SMTP and IMAP than AMQP. You're not really
trying to connect two processes in real time. You're trying to do fully
asynchronous messaging with fully randomized access to any message.

Perhaps somebody should explore whether the approaches taken by large
scale IMAP providers could be applied to Zaqar.

Anyway, I can't imagine writing a system to intentionally use the
semantics of IMAP and SMTP. I'd be very interested in seeing actual use
cases for it, apologies if those have been posted before.

> Some things that differentiate Zaqar from SQS is it's capability for
> supporting different protocols without sacrificing multi-tenantcy and
> other intrinsic features it provides. Some protocols you may consider
> for Zaqar are: STOMP, MQTT.
> 
> As far as the backend goes, Zaqar is not re-inventing it either. It sits
> on top of existing storage technologies that have proven to be fast and
> reliable for this task. The choice of using NoSQL technologies has a lot
> to do with this particular thing and the fact that Zaqar needs a storage
> capable of scaling, replicating and good support for failover.
> 

What's odd to me is that other systems like Cassandra and Riak are not
being discussed. There are well documented large scale message storage
systems on both, and neither is encumbered by the same licensing FUD
as MongoDB.

Anyway, again if we look at this as a place to storage and retrieve
messages, and not as a queue, then talking about databases, instead of
message brokers, makes a lot more sen

[openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting

2014-09-04 Thread Flavio Percoco
Greetings,

Last Tuesday the TC held the first graduation review for Zaqar. During
the meeting some concerns arose. I've listed those concerns below with
some comments hoping that it will help starting a discussion before the
next meeting. In addition, I've added some comments about the project
stability at the bottom and an etherpad link pointing to a list of use
cases for Zaqar.

# Concerns

- Concern on operational burden of requiring NoSQL deploy expertise to
the mix of openstack operational skills

For those of you not familiar with Zaqar, it currently supports 2 nosql
drivers - MongoDB and Redis - and those are the only 2 drivers it
supports for now. This will require operators willing to use Zaqar to
maintain a new (?) NoSQL technology in their system. Before expressing
our thoughts on this matter, let me say that:

1. By removing the SQLAlchemy driver, we basically removed the chance
for operators to use an already deployed "OpenStack-technology"
2. Zaqar won't be backed by any AMQP based messaging technology for
now. Here's[0] a summary of the research the team (mostly done by
Victoria) did during Juno
3. We (OpenStack) used to require Redis for the zmq matchmaker
4. We (OpenStack) also use memcached for caching and as the oslo
caching lib becomes available - or a wrapper on top of dogpile.cache -
Redis may be used in place of memcached in more and more deployments.
5. Ceilometer's recommended storage driver is still MongoDB, although
Ceilometer has now support for sqlalchemy. (Please correct me if I'm wrong).

That being said, it's obvious we already, to some extent, promote some
NoSQL technologies. However, for the sake of the discussion, lets assume
we don't.

I truly believe, with my OpenStack (not Zaqar's) hat on, that we can't
keep avoiding these technologies. NoSQL technologies have been around
for years and we should be prepared - including OpenStack operators - to
support these technologies. Not every tool is good for all tasks - one
of the reasons we removed the sqlalchemy driver in the first place -
therefore it's impossible to keep an homogeneous environment for all
services.

With this, I'm not suggesting to ignore the risks and the extra burden
this adds but, instead of attempting to avoid it completely by not
evolving the stack of services we provide, we should probably work on
defining a reasonable subset of NoSQL services we are OK with
supporting. This will help making the burden smaller and it'll give
operators the option to choose.

[0] http://blog.flaper87.com/post/marconi-amqp-see-you-later/


- Concern on should we really reinvent a queue system rather than
piggyback on one

As mentioned in the meeting on Tuesday, Zaqar is not reinventing message
brokers. Zaqar provides a service akin to SQS from AWS with an OpenStack
flavor on top. [0]

Some things that differentiate Zaqar from SQS is it's capability for
supporting different protocols without sacrificing multi-tenantcy and
other intrinsic features it provides. Some protocols you may consider
for Zaqar are: STOMP, MQTT.

As far as the backend goes, Zaqar is not re-inventing it either. It sits
on top of existing storage technologies that have proven to be fast and
reliable for this task. The choice of using NoSQL technologies has a lot
to do with this particular thing and the fact that Zaqar needs a storage
capable of scaling, replicating and good support for failover.

[0]
https://wiki.openstack.org/wiki/Zaqar/Frequently_asked_questions#Is_Zaqar_a_provisioning_service_or_a_data_API.3F

- concern on dataplane vs. controlplane, should we add more dataplane
things in the integrated release

I'm really not sure I understand the arguments against dataplane
services. What concerns do people have about these services?
As far as I can tell, we already have several services - some in the
lower layers - that provide a data plane API. For example:


* keystone (service catalogs and tokens)
* glance (image management)
* swift (object storage)
* ceilometer (metrics)
* heat (provisioning)
* barbican (key management)

Are the concerns specific to Zaqar's dataplane API?


- concern on API v2 being already planned

At the meeting, we discussed a bit about Zaqar's API and more
importantly how stable it is. During that discussion I mentioned an
hypothetical v2 of the API. I'd like to clarify that a v2 is not being
planned for Kilo, what we would like to do is to gather feedback from
the community and services consuming Zaqar about the existing API and
use that feedback to design a new version of the API if necessary.

All this has yet to be discussed but most importantly, we would first
like to get more feedback from the community. We have already gotten
some feedback, but it has been fairly limited because most people are
waiting for us to graduate before kicking the tires.

We do have some endpoints that will go away in the API v2 - getting
messages by id