Re: [openstack-dev] Reasoning behind my vote on the Go topic

[Ben Meyer]

On 06/08/2016 11:05 PM, Chris Friesen wrote:
> On 06/07/2016 04:26 PM, Ben Meyer wrote:
>> On 06/07/2016 06:09 PM, Samuel Merritt wrote:
>>> On 6/7/16 12:00 PM, Monty Taylor wrote:
>>>> [snip]
>>>>
>>>> I'd rather see us focus energy on Python3, asyncio and its pluggable
>>>> event loops. The work in:
>>>>
>>>> http://magic.io/blog/uvloop-blazing-fast-python-networking/
>>>>
>>>> is a great indication in an actual apples-to-apples comparison of what
>>>> can be accomplished in python doing IO-bound activities by using
>>>> modern
>>>> Python techniques. I think that comparing python2+eventlet to a fresh
>>>> rewrite in Go isn't 100% of the story. A TON of work has gone in to
>>>> Python that we're not taking advantage of because we're still
>>>> supporting
>>>> Python2. So what I've love to see in the realm of comparative
>>>> experimentation is to see if the existing Python we already have
>>>> can be
>>>> leveraged as we adopt newer and more modern things.
>>>
>>> Asyncio, eventlet, and other similar libraries are all very good for
>>> performing asynchronous IO on sockets and pipes. However, none of them
>>> help for filesystem IO. That's why Swift needs a golang object server:
>>> the go runtime will keep some goroutines running even though some
>>> other goroutines are performing filesystem IO, whereas filesystem IO
>>> in Python blocks the entire process, asyncio or no asyncio.
>>
>> That can be modified. gevent has a tool
>> (http://www.gevent.org/gevent.fileobject.html) that enables the File IO
>> to be async  as well by putting the file into non-blocking mode. I've
>> used it, and it works and scales well.
>
> Arguably non-blocking isn't really async when it comes to reads.  I
> suspect what we really want is full-async where you issue a request
> and then get notified when it's done.

So when it comes to Swift or Glance where you have to transfer large
amounts of data between the HTTP client and HTTP serverunder WSGI,
the only way to make it truly cooperative for eventlet, gevent, etc is
to use non-blocking File I/O. These situations also reveal how
uncooperative green threads are too since if you can keep the data
pipeline full (e.g the read is continuous because the OS is able to
quickly service it) then one greed thread will take over and block the
others. Non-blocking I/O is only part of the solution, not the entire
solution.

Furthermore, the constraint of the web head having to provide the data
transfer makes the full-async offload with a notification impossible for
data transfer services like Swift. Tools like uvloop make great strides
in providing better environments for doing cooperative tasks than what
existed prior, and some of the articles comparing it to the various
tools - both alone and in combination - make for some fascinating
possibilities and solutions. Most likely the best solution here is going
to be going to Py3's asyncio+uvloop+non-blocking I/O on the files in
order to hit the throughput for Swift.

uvloop (first commit 2015-11-01) is newer than Swift's hummingbird
(2015-04-20, based on
https://github.com/openstack/swift/commit/a0e300df180f7f4ca64fc1eaf3601a1a73fc68cb
and github network graph) so it would not have been part of the
consideration.

$0.02

Ben

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Reasoning behind my vote on the Go topic

[Ben Meyer]

On 06/07/2016 06:09 PM, Samuel Merritt wrote:
> On 6/7/16 12:00 PM, Monty Taylor wrote:
>> [snip]
> >
>> I'd rather see us focus energy on Python3, asyncio and its pluggable
>> event loops. The work in:
>>
>> http://magic.io/blog/uvloop-blazing-fast-python-networking/
>>
>> is a great indication in an actual apples-to-apples comparison of what
>> can be accomplished in python doing IO-bound activities by using modern
>> Python techniques. I think that comparing python2+eventlet to a fresh
>> rewrite in Go isn't 100% of the story. A TON of work has gone in to
>> Python that we're not taking advantage of because we're still supporting
>> Python2. So what I've love to see in the realm of comparative
>> experimentation is to see if the existing Python we already have can be
>> leveraged as we adopt newer and more modern things.
>
> Asyncio, eventlet, and other similar libraries are all very good for
> performing asynchronous IO on sockets and pipes. However, none of them
> help for filesystem IO. That's why Swift needs a golang object server:
> the go runtime will keep some goroutines running even though some
> other goroutines are performing filesystem IO, whereas filesystem IO
> in Python blocks the entire process, asyncio or no asyncio.

That can be modified. gevent has a tool
(http://www.gevent.org/gevent.fileobject.html) that enables the File IO
to be async  as well by putting the file into non-blocking mode. I've
used it, and it works and scales well.

Sadly, Python doesn't offer this by default; perhaps OpenStack can get
that changed.

$0.02

Ben

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"

[Ben Meyer]

On 05/24/2016 11:13 AM, Dean Troyer wrote:
> On Tue, May 24, 2016 at 8:20 AM, Flavio Percoco  > wrote:
>
> So, just to make sure I'm making myself clear, I believe we should
> go with
> option #2 in Thierry's comment from May 23 11:3 on this[0] review.
> While I'm not
> entirely opposed to #1 I think #2 is better for us at this point
> in time. Here's
> a quote of Thierry's comment:
>
>   "To summarize my view on this, I think our only two options
> here are (1)
>   approve the addition of golang (with caveats on where it
> should be used
>   to try to minimize useless churn), or (2) precise the line
> between
>   'openstack projects' and 'dependencies of openstack
> projects' in a way
>   that makes it obvious that components requiring such
> optimization as to
>   require golang (or any other such language) should be
> developed as
>   dependencies"
>
> My main motivation is that I still believe option #1 will have a
> negative impact
> on the community and, perhaps more importantly, I don't think
> it'll help
> reaching the goal we've been talking about in this thread. Many
> people have been
> asking for focus and I think #2 will do that, whereas #1 will open
> the doors to
> a different set of problems and complexities that won't help with
> keeping the focus.
>
>
> Option #2 without the followup of actually evaluating and removing
> things that do not fit is really Option #3, do nothing. Which is what
> I am afraid will happen.  No renewed focus, no growth, no goal.
>
> On the language front, since we want focus, the exiting decisions re
> languages should also be part of that re-evaluation for focus.  It
> sure feels like JavaScript is in exactly the same boat as folks fear
> Golang will be here (a special case, domain-specific, division of
> community (ask Horizon devs)).  And Bash, well, that isn't even a
> language.

Just $0.02 - if you want to support a language, then it would seem like
having a full SDK for that language would be a first step so that people
inside and outside the community can use the language in a supported
manner. With an SDK, it seems like everyone will just reinvent the
wheel. That would also seem to further the goal of using the language as
the community intends - whether for services, clients, or UI - since the
SDK would be targeted appropriately. If no SDK, then special casing
would seem to the proper place.

Again, $0.02

Ben
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc] supporting Go

[Ben Meyer]

On 05/18/2016 06:42 PM, Eric Larson wrote:
>
> Dmitry Tantsur writes:
>
>> This is pretty subjective, I would say. I personally don't feel Go
>> (especially its approach to error handling) any natural (at least no
>> more than Rust or Scala, for example). If familiarity for Python
>> developers is an argument here, mastering Cython or making OpenStack
>> run on PyPy must be much easier for a random Python developer out
>> there to seriously bump the performance. And it would not require
>> introducing a completely new language to the picture.
>
> In one sense you are correct. It is easier for a Pythonista to pick up
> Cython and use that for performance specific areas of code. At the
> same time, I'd argue that OpenStack as a community is not the same as
> Python at large. There are packaging requirements and cross project
> standards that also come into play, not to mention operators that end
> up bearing the brunt of those decisions. For example, Debian will
> likely not package a PyPy only version of Designate along with all its
> requirements. Similarly, while 50% of operators use packaged
> versioned, that means 50% work from source control to build, test, and
> release OpenStack projects.
>
> You are correct that my position is subjective, but it is based on my
> experiences trying to operate and deploy OpenStack in addition to
> writing code. The draw of Go, in my experience, has been easily
> deploying a single binary I've been able to build and test
> consistently. The target system has doesn't require Go installed at
> all and it works on old distros. And it has been much faster.
True as long as your Kernel is new enough.

There are some issues with Go on older kernels, so you can only go so
far back.

For a product I'm working on (outside of OpenStack) I chose Go for an
updater so we could move forward against Python3 (our Python2-based
updater wasn't portable to Python-3, and no hope of making it so).
I had kept it simple, but after I finished discovered Go didn't
officially support CentOS/RHEL5 due to kernel issues - example
https://github.com/golang/go/issues/8445 (closed as "unfortunate"), see
also http://dave.cheney.net/2013/06/18/how-to-install-go-1-1-on-centos-5
which *might* help get things working, but is not guaranteed.

That said, my CentOS/RHEL binary was built on CentOS6, it ran on CentOS5
but I also didn't do anything very spectacular - just shell calls and an
IPC for the daemon, all very simple and not timing related. Something
more complex, like what has been discussed in this thread, will likely
run into a lot more issues.

$0.02

Ben

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc] supporting Go

[Ben Meyer]

On 05/11/2016 03:23 PM, Samuel Merritt wrote:
> On 5/11/16 7:09 AM, Thomas Goirand wrote:
>> On 05/10/2016 09:56 PM, Samuel Merritt wrote:
>>> On 5/9/16 5:21 PM, Robert Collins wrote:
 On 10 May 2016 at 10:54, John Dickinson  wrote:
> On 9 May 2016, at 13:16, Gregory Haynes wrote:
>>
>> This is a bit of an aside but I am sure others are wondering the
>> same
>> thing - Is there some info (specs/etherpad/ML thread/etc) that
>> has more
>> details on the bottleneck you're running in to? Given that the only
>> clients of your service are the public facing DNS servers I am
>> now even
>> more surprised that you're hitting a python-inherent bottleneck.
>
> In Swift's case, the summary is that it's hard[0] to write a network
> service in Python that shuffles data between the network and a block
> device (hard drive) and effectively utilizes all of the hardware
> available. So far, we've done very well by fork()'ing child
> processes,
 ...
> Initial results from a golang reimplementation of the object
> server in
> Python are very positive[1]. We're not proposing to rewrite Swift
> entirely in Golang. Specifically, we're looking at improving object
> replication time in Swift. This service must discover what data is on
> a drive, talk to other servers in the cluster about what they have,
> and coordinate any data sync process that's needed.
>
> [0] Hard, not impossible. Of course, given enough time, we can do
>  anything in a Turing-complete language, right? But we're not talking
>  about possible, we're talking about efficient tools for the job at
>  hand.
 ...

 I'm glad you're finding you can get good results in (presumably)
 clean, understandable code.

 Given go's historically poor perfornance with multiple cores
 (https://golang.org/doc/faq#Why_GOMAXPROCS) I'm going to presume the
 major advantage is in the CSP programming model - something that
 Twisted does very well: and frustratingly we've had numerous
 discussions from folk in the Twisted world who see the pain we have
 and want to help, but as a community we've consistently stayed with
 eventlet, which has a threaded programming model - and threaded models
 are poorly suited for the case here.
>>>
>>> At its core, the problem is that filesystem IO can take a surprisingly
>>> long time, during which the calling thread/process is blocked, and
>>> there's no good asynchronous alternative.
>>>
>>> Some background:
>>>
>>> With Eventlet, when your greenthread tries to read from a socket and
>>> the
>>> socket is not readable, then recvfrom() returns -1/EWOULDBLOCK; then,
>>> the Eventlet hub steps in, unschedules your greenthread, finds an
>>> unblocked one, and lets it proceed. It's pretty good at servicing a
>>> bunch of concurrent connections and keeping the CPU busy.
>>>
>>> On the other hand, when the socket is readable, then recvfrom() returns
>>> quickly (a few microseconds). The calling process was technically
>>> blocked, but the syscall is so fast that it hardly matters.
>>>
>>> Now, when your greenthread tries to read from a file, that read() call
>>> doesn't return until the data is in your process's memory. This can
>>> take
>>> a surprisingly long time. If the data isn't in buffer cache and the
>>> kernel has to go fetch it from a spinning disk, then you're looking
>>> at a
>>> seek time of ~7 ms, and that's assuming there are no other pending
>>> requests for the disk.
>>>
>>> There's no EWOULDBLOCK when reading from a plain file, either. If the
>>> file pointer isn't at EOF, then the calling process blocks until the
>>> kernel fetches data for it.
>>>
>>> Back to Swift:
>>>
>>> The Swift object server basically does two things: it either reads from
>>> a disk and writes to a socket or vice versa. There's a little HTTP
>>> parsing in there, but the vast majority of the work is shuffling bytes
>>> between network and disk. One Swift object server can service many
>>> clients simultaneously.
>>>
>>> The problem is those pauses due to read(). If your process is servicing
>>> hundreds of clients reading from and writing to dozens of disks (in,
>>> say, a 48-disk 4U server), then all those little 7 ms waits are pretty
>>> bad for throughput. Now, a lot of the time, the kernel does some
>>> readahead so your read() calls can quickly return data from buffer
>>> cache, but there are still lots of little hitches.
>>>
>>> But wait: it gets worse. Sometimes a disk gets slow. Maybe it's got a
>>> lot of pending IO requests, maybe its filesystem is getting close to
>>> full, or maybe the disk hardware is just starting to get flaky. For
>>> whatever reason, IO to this disk starts taking a lot longer than 7
>>> ms on
>>> average; think dozens or hundreds of milliseconds. Now, every time your
>>> process tries to read from this disk, all other work stops for quite a
>>> long 

Re: [openstack-dev] [all] re-introducing twisted to global-requirements

[Ben Meyer]

On 01/08/2016 12:56 PM, Jim Rollenhagen wrote:
> On Fri, Jan 08, 2016 at 11:00:35AM +0100, Thierry Carrez wrote:
>> Jim Rollenhagen wrote:
>>> [...]
>>> Here's the catch - mimic is built on twisted. I know twisted was
>>> previously removed from OpenStack (or at least people said "pls no", I
>>> don't know the full history). We didn't intend to stealth-introduce
>>> twisted back into g-r, but it was pointed out to me that it may appear
>>> this way, so here I am letting everyone know. lifeless pointed out that
>>> when tests are failing, people may end up digging into mimic or twisted
>>> code, which most people in this community aren't familiar with AFAIK,
>>> which is a valid point though I hope it isn't required often.
>> A bit of history with Twisted.
>>
>> Back in 2010 we decided we could not afford asking OpenStack developers to
>> be familiar with multiple service architecture frameworks, and eventlet was
>> chosen as the simplest framework to learn and debug. The best reference I
>> found on this is still visible in the wiki:
>>
>> https://wiki.openstack.org/wiki/UnifiedServiceArchitecture
>>
>>> So, the primary question here is: do folks have a problem with adding
>>> twisted here? We're holding off on Ironic changes that depend on this
>>> until this discussion has happened, but aren't reverting the g-r change
>>> until we decide one way or another.
>> The only friction I see is how many developers would be expected to need to
>> learn Twisted in order to complete their jobs. My understanding is that
>> Twisted expertise could be needed to debug python-ironicclient functional
>> tests, which makes the cost relatively limited. So if Mimic brings in a
>> clear and significant benefit, I don't think its Twisted dependence should
>> play that much against it.
>>
>> However, I agree with Sean and Jay that the benefit is unclear -- the few
>> features that Mimic brings seem to be outweighed by the increased risk of
>> introducing a delta between the implementation and the mock. If the main
>> benefit is that it's used in other Rackspace projects for testing (like Ben
>> said), I'm not sure that makes a compelling argument for the rest of the
>> community...
> No, that is not the main benefit, at all. Ben isn't involved in Ironic
> and until now has had nothing to do with this work to add mimic here, so
> I'm not sure where he got that impression from, or why he's speaking on
> our behalf as to the goals here.
I never claimed involvement in Ironic and from the outset noted my
status on this list. I've only commented in so far as trying to show the
value of mimic and similar tools.

I'm sorry if there was any confusion in that with respect to ironic.
That was certainly not my intent.

> As pointed out before, the risk of a delta between the mocks in mimic
> and reality is identical to the risk of a delta between the mocks in a
> client's unit tests and reality, so I don't see a particular downside
> there.
Agreed.

Ben

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] re-introducing twisted to global-requirements

[Ben Meyer]

On 01/07/2016 06:28 PM, Jay Pipes wrote:
> On 01/07/2016 06:12 PM, Ben Meyer wrote:
>> On 01/07/2016 03:32 PM, Jay Pipes wrote:
>>> On 01/07/2016 03:01 PM, Jim Rollenhagen wrote:
>>>> On Thu, Jan 07, 2016 at 02:41:12PM -0500, Sean Dague wrote:
>>>>> On 01/07/2016 02:09 PM, Jim Rollenhagen wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> A change to global-requirements[1] introduces mimic, which is an
>>>>>> http
>>>>>> server that can mock various APIs, including nova and ironic,
>>>>>> including
>>>>>> control of error codes and timeouts. The ironic team plans to use
>>>>>> this
>>>>>> for testing python-ironicclient without standing up a full ironic
>>>>>> environment.
>>>>>>
>>>>>> Here's the catch - mimic is built on twisted. I know twisted was
>>>>>> previously removed from OpenStack (or at least people said "pls
>>>>>> no", I
>>>>>> don't know the full history). We didn't intend to stealth-introduce
>>>>>> twisted back into g-r, but it was pointed out to me that it may
>>>>>> appear
>>>>>> this way, so here I am letting everyone know. lifeless pointed out
>>>>>> that
>>>>>> when tests are failing, people may end up digging into mimic or
>>>>>> twisted
>>>>>> code, which most people in this community aren't familiar with
>>>>>> AFAIK,
>>>>>> which is a valid point though I hope it isn't required often.
>>>>>>
>>>>>> So, the primary question here is: do folks have a problem with
>>>>>> adding
>>>>>> twisted here? We're holding off on Ironic changes that depend on
>>>>>> this
>>>>>> until this discussion has happened, but aren't reverting the g-r
>>>>>> change
>>>>>> until we decide one way or another.
>>>>>>
>>>>>> // jim
>>>>>>
>>>>>> [1] https://review.openstack.org/#/c/220268/
>>>>>
>>>>> What is the advantage of running another server like this over using
>>>>> requests-mock (which is used by other OpenStack projects for testing
>>>>> today)? The only difference here seems to be that you actually
>>>>> execute
>>>>> requests code in one case and not in the other.
>>>>>
>>>>> Requests-mock debugging when things go wrong seems a bit simpler.
>>>>>
>>>>> This is less about twisted and more about trying to not introduce yet
>>>>> another way to mock code in the tree that people need to understand.
>>>>>
>>>>>  -Sean
>>>>
>>>> We'd be using this for functional tests, not unit, where we can't
>>>> really
>>>> inject mocks. The idea is that we could run a full functional suite
>>>> against either mimic or a full ironic environment, just by changing a
>>>> test setting.
>>>
>>> I don't really see the point of a separate project like Mimic that has
>>> a whole bunch of reimplementations (mocked out) of all sorts of
>>> OpenStack (and RAX-specific) API services. It's just a great way to
>>> introduce a larger surface area for bugs to creep in -- since you have
>>> to keep the Mimic interfaces up to date with the real interfaces.
>>> Better to keep something like this -- if it is TRULY needed -- in-tree
>>> with the API service itself, so that the chances of divergence are
>>> reduced. This is similar to the fakevirt driver in Nova. It's in tree
>>> for good reason: when someone changes the virt driver interface, the
>>> fakevirt driver goes boom and needs to be changed in a corresponding
>>> fashion in the same patch.
>> A tool like my OpenStackInABox could certainly benefit from the models
>> or services being provided by each project - aside from the complexity
>> that that adds to the installation of installing all the dependencies
>> related instead of just implementing a simple model with a file and
>> sqlite backend that has minimal dependencies...but I digress as I
>> haven't looked at how nova's fakevirt driver installs so may be that's
>> not as big an issue. I'll certainly look at it for use in
>> OpenStackInABox, but even there I'm aiming for a more complete scenario
>> where you can interact w

Re: [openstack-dev] [all] re-introducing twisted to global-requirements

[Ben Meyer]

On 01/07/2016 03:32 PM, Jay Pipes wrote:
> On 01/07/2016 03:01 PM, Jim Rollenhagen wrote:
>> On Thu, Jan 07, 2016 at 02:41:12PM -0500, Sean Dague wrote:
>>> On 01/07/2016 02:09 PM, Jim Rollenhagen wrote:
 Hi all,

 A change to global-requirements[1] introduces mimic, which is an http
 server that can mock various APIs, including nova and ironic,
 including
 control of error codes and timeouts. The ironic team plans to use this
 for testing python-ironicclient without standing up a full ironic
 environment.

 Here's the catch - mimic is built on twisted. I know twisted was
 previously removed from OpenStack (or at least people said "pls no", I
 don't know the full history). We didn't intend to stealth-introduce
 twisted back into g-r, but it was pointed out to me that it may appear
 this way, so here I am letting everyone know. lifeless pointed out
 that
 when tests are failing, people may end up digging into mimic or
 twisted
 code, which most people in this community aren't familiar with AFAIK,
 which is a valid point though I hope it isn't required often.

 So, the primary question here is: do folks have a problem with adding
 twisted here? We're holding off on Ironic changes that depend on this
 until this discussion has happened, but aren't reverting the g-r
 change
 until we decide one way or another.

 // jim

 [1] https://review.openstack.org/#/c/220268/
>>>
>>> What is the advantage of running another server like this over using
>>> requests-mock (which is used by other OpenStack projects for testing
>>> today)? The only difference here seems to be that you actually execute
>>> requests code in one case and not in the other.
>>>
>>> Requests-mock debugging when things go wrong seems a bit simpler.
>>>
>>> This is less about twisted and more about trying to not introduce yet
>>> another way to mock code in the tree that people need to understand.
>>>
>>> -Sean
>>
>> We'd be using this for functional tests, not unit, where we can't really
>> inject mocks. The idea is that we could run a full functional suite
>> against either mimic or a full ironic environment, just by changing a
>> test setting.
>
> I don't really see the point of a separate project like Mimic that has
> a whole bunch of reimplementations (mocked out) of all sorts of
> OpenStack (and RAX-specific) API services. It's just a great way to
> introduce a larger surface area for bugs to creep in -- since you have
> to keep the Mimic interfaces up to date with the real interfaces.
> Better to keep something like this -- if it is TRULY needed -- in-tree
> with the API service itself, so that the chances of divergence are
> reduced. This is similar to the fakevirt driver in Nova. It's in tree
> for good reason: when someone changes the virt driver interface, the
> fakevirt driver goes boom and needs to be changed in a corresponding
> fashion in the same patch.
A tool like my OpenStackInABox could certainly benefit from the models
or services being provided by each project - aside from the complexity
that that adds to the installation of installing all the dependencies
related instead of just implementing a simple model with a file and
sqlite backend that has minimal dependencies...but I digress as I
haven't looked at how nova's fakevirt driver installs so may be that's
not as big an issue. I'll certainly look at it for use in
OpenStackInABox, but even there I'm aiming for a more complete scenario
where you can interact with Keystone, Nova, Swift, etc...(e.g auth
against a fake Keystone, use the token with the fake nova which
validates it against the same fake Keystone instance, same with a fake
Swift...).

> What value does a functional test against an HTTP API service that
> does nothing (other than introduce greater surface area for bugs)
> actually offer over unit tests anyway?

So to use Nova's fakevirt your project is limited to the same language
as nova - python, correct?

The main advantage of mimic is that it is language agnostic and multiple
moving parts (f.e taskflow) can interact with it.

$0.02

Ben


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] re-introducing twisted to global-requirements

[Ben Meyer]

On 01/07/2016 02:41 PM, Sean Dague wrote:
> On 01/07/2016 02:09 PM, Jim Rollenhagen wrote:
>> Hi all,
>>
>> A change to global-requirements[1] introduces mimic, which is an http
>> server that can mock various APIs, including nova and ironic, including
>> control of error codes and timeouts. The ironic team plans to use this
>> for testing python-ironicclient without standing up a full ironic
>> environment.
>>
>> Here's the catch - mimic is built on twisted. I know twisted was
>> previously removed from OpenStack (or at least people said "pls no", I
>> don't know the full history). We didn't intend to stealth-introduce
>> twisted back into g-r, but it was pointed out to me that it may appear
>> this way, so here I am letting everyone know. lifeless pointed out that
>> when tests are failing, people may end up digging into mimic or twisted
>> code, which most people in this community aren't familiar with AFAIK,
>> which is a valid point though I hope it isn't required often.
>>
>> So, the primary question here is: do folks have a problem with adding
>> twisted here? We're holding off on Ironic changes that depend on this
>> until this discussion has happened, but aren't reverting the g-r change
>> until we decide one way or another.
>>
>> // jim
>>
>> [1] https://review.openstack.org/#/c/220268/
> What is the advantage of running another server like this over using
> requests-mock (which is used by other OpenStack projects for testing
> today)? The only difference here seems to be that you actually execute
> requests code in one case and not in the other.
>
> Requests-mock debugging when things go wrong seems a bit simpler.
>
> This is less about twisted and more about trying to not introduce yet
> another way to mock code in the tree that people need to understand.

As an outsider and lurker...this is a fair point. I also maintain
another project - StackInABox (and OpenStackInABox) - that resolves the
issue in a slightly different way that utilizes requests-mock to do
something similar, but closer to true unit tests.

The main advantage of mimic (https://pypi.python.org/pypi/mimic) is that
it can be independent of language framework (e.g you can use it for C++
or Java or Python2, Python3, or...) as its an independent process from
your testing, and the tasks can be handled very asynchronously so it can
mock out the responses for various tasks in a timely manner, and several
parts of a system can interact together during a test. You're also
utilizing a common mock of the entire system so multiple projects can
benefit from the same code base. It's kind of integration-test but not
quite - a good fit for projects where unit tests need multiple other
components working together, e.g services that use taskflow would
certainly benefit from using mimic.

The disadvantage of mimic from what I can tell is how the tests are
setup. But I'll leave that more to the mimic users and maintainers
(OpenStack Poppy, Rackspace Autoscale, etc) to explain.

For comparison: My StackInABox project provides a framework upon which
OpenStackInABox builds to provide a common model so multiple projects
benefit. I've used StackInABox on several of my own (non-OpenStack
projects) and my unit tests (run by tox) have been able to be nearly
complete integration tests. Aside from the slight use-case difference
(as exemplified by Rackspace AutoScale),  the main advantage of mimic
over my OpenStackInABox is the momentum behind it due to its use by
Rackspace AutoScale and OpenStack Poppy; I just don't have the resources
to build out the models in OpenStackInABox efficiently or quickly
(though I'd certainly welcome the help to do so). Advantage of the
StackInABox approach is every unit test is self-contained and you don't
have to change code to run the test in most cases (where you do, it's
minimal) - just like with using requests-mock; but you can't have two
services independently hit it - like with Rackspace AutoScale.

HTH,

Ben

P.S here's my two projects:
https://github.com/BenjamenMeyer/stackInABox ~
https://pypi.python.org/pypi/stackinabox
https://github.com/BenjamenMeyer/openstackinabox ~
https://pypi.python.org/pypi/openstackinabox

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] Question regarding Service Catalog and Identity entries...

[Ben Meyer]

On 10/20/2014 08:12 AM, Jamie Lennox wrote:
 - Original Message -
 From: Ben Meyer ben.me...@rackspace.com
 To: openstack-dev@lists.openstack.org
 Cc: Jamie Painter jamie.pain...@rackspace.com
 Sent: Tuesday, October 7, 2014 4:31:16 PM
 Subject: [openstack-dev] [Keystone] Question regarding Service Catalog and   
 Identity entries...

 I am trying to use the Python Keystone client to integration
 authentication functionality into a project I am contributing to
 (https://github.com/rackerlabs/deuce-client).
 However, I ran into a situation where if I do the following:

 c = keystoneclient.v2_0.client.Client(username='username',
 password='password',
 auth_url=https://keystone-compatible-service.example.com/v2.0/;)
 Failed to retrieve management_url from token

 I traced it through the Python Keystoneclient code and it fails due to
 not finding the identity service entry in the Service Catalog. The
 authentication otherwise happens in that it has already received a
 successful response and a full Service Catalog, aside from the
 missing identity service. This happens with both version 0.10 and 0.11
 python keystone clients; I did not try older clients.

 Talking with the service provider, their version does not include itself
 in the Service Catalog, and they learned the Keystone itself inserts
 itself into the Service Catalog.
 I can certainly understand why it having the identity service entry be
 part of the Service Catalog, but for them it is (at least for now) not
 desirable to do so.

 Questions:
 - Is it now a standard that Keystone inserts itself into the Service
 Catalog?
 It's not a standard that keystone inserts itself into the catalog, the cloud 
 operator should maintain the list of endpoints for their deployment and the 
 'identity' service should be amongst those endpoints. I'm unclear as to why 
 it would be undesirable to list the identity endpoint in the service catalog. 
 How would this addition change their deployment? 
The argument is that the Service Catalog is too big so they are hesitant
to add new entries to it; and 'identity' in the catalog is redundant
since you have to know the 'identity' end-point to even get the service
catalog in the first place.

Not saying I agree, just that's the argument being made. If it is
required by Keystone to be self-referential then they are likely to
add it.

 The problem with the code that you provided is that the token that is being 
 returned from your code is unscoped. Which means that it is not associated 
 with a project and therefore it doesn't have a service catalog because the 
 catalog can be project specific. Thus when you go to perform an operation the 
 client will look for the URL it is supposed to talk to in an empty list and 
 fail to find the identity endpoint. This message really needs to be improved. 
 If you add a project_id or project_name to the client information then you 
 should get back a token with a catalog. 

In my normal case I'm using the project_id field; but have found that it
didn't really matter what was used for the credentials in this case
since they simply don't have the 'identity' end-points in the Service
Catalog.

 - Or is the Python Keystone Client broken because it is forcing it to be so?
 I wouldn't say that it is broken because having an identity endpoint in your 
 catalog is a required part of a deployment, in the same way that having a 
 'compute' endpoint is required if you want to talk to nova. I would be 
 surprised by any decision to purposefully omit the 'identity' endpoint from 
 the service catalog. 

See above; but from what you are presenting here it sounds like the
deployment is broken so it is in fact required by Keystone, even if
only a required part of a deployment.

Thanks

Ben

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Keystone] Question regarding Service Catalog and Identity entries...

[Ben Meyer]

I am trying to use the Python Keystone client to integration
authentication functionality into a project I am contributing to
(https://github.com/rackerlabs/deuce-client).
However, I ran into a situation where if I do the following:

 c = keystoneclient.v2_0.client.Client(username='username',
password='password',
auth_url=https://keystone-compatible-service.example.com/v2.0/;)
Failed to retrieve management_url from token

I traced it through the Python Keystoneclient code and it fails due to
not finding the identity service entry in the Service Catalog. The
authentication otherwise happens in that it has already received a
successful response and a full Service Catalog, aside from the
missing identity service. This happens with both version 0.10 and 0.11
python keystone clients; I did not try older clients.

Talking with the service provider, their version does not include itself
in the Service Catalog, and they learned the Keystone itself inserts
itself into the Service Catalog.
I can certainly understand why it having the identity service entry be
part of the Service Catalog, but for them it is (at least for now) not
desirable to do so.

Questions:
- Is it now a standard that Keystone inserts itself into the Service
Catalog?
- Or is the Python Keystone Client broken because it is forcing it to be so?

Thanks,

Benjamen R. Meyer

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev