Re: [openstack-dev] [qa][ceilometer] swapping the roles of mongodb and sqlalchemy for ceilometer in Tempest

2014-08-09 Thread Joshua Harlow
+2 from me, More mongodb adoption (as stated) when it's questionable legally doesn't seem like a good long term strategy (I know it will/does impact yahoo adopting or using ceilometer...). Is this another one of those tactical changes that we keep on making that ends up being yet another piece

Re: [openstack-dev] [qa][ceilometer] swapping the roles of mongodb and sqlalchemy for ceilometer in Tempest

2014-08-09 Thread Joshua Harlow
Agreed, testing options is good; and should likely be disjoint from the legal questions around mongodb... Although if there is really only one viable scalable option and that option has legal usage questions surrounding it then it makes me wonder how much we are kidding ourselves on there

Re: [openstack-dev] [all] The future of the integrated release

2014-08-09 Thread Joshua Harlow
+1 to what john said, I would also like to wait and see on this... I'd rather be honest with ourselves, and to contributors new and old, and admit core reviewers are air traffic controllers (directing the project vs reviewing changes, two very different things IMHO) and reflecting that in what

Re: [openstack-dev] [oslo] oslo.concurrency repo review

2014-08-10 Thread Joshua Harlow
One question from me: Will there be later fixes to remove oslo.config dependency/usage from oslo.concurrency? I still don't understand how oslo.concurrency can be used as a library with the configuration being set in a static manner via oslo.config (let's use the example of `lock_path` @

Re: [openstack-dev] [oslo] oslo.concurrency repo review

2014-08-11 Thread Joshua Harlow
On Mon, Aug 11, 2014 at 11:39 AM, Ben Nemec openst...@nemebean.com wrote: On 08/11/2014 01:02 PM, Yuriy Taraday wrote: On Mon, Aug 11, 2014 at 5:44 AM, Joshua Harlow harlo...@outlook.com wrote: One question from me: Will there be later fixes to remove oslo.config dependency/usage

Re: [openstack-dev] [oslo] oslo.concurrency repo review

2014-08-11 Thread Joshua Harlow
On Mon, Aug 11, 2014 at 11:02 AM, Yuriy Taraday yorik@gmail.com wrote: On Mon, Aug 11, 2014 at 5:44 AM, Joshua Harlow harlo...@outlook.com wrote: One question from me: Will there be later fixes to remove oslo.config dependency/usage from oslo.concurrency? I still don't understand how

Re: [openstack-dev] [oslo] oslo.concurrency repo review

2014-08-11 Thread Joshua Harlow
On Mon, Aug 11, 2014 at 12:47 PM, Doug Hellmann d...@doughellmann.com wrote: On Aug 11, 2014, at 3:26 PM, Joshua Harlow harlo...@outlook.com wrote: On Mon, Aug 11, 2014 at 11:02 AM, Yuriy Taraday yorik@gmail.com wrote: On Mon, Aug 11, 2014 at 5:44 AM, Joshua Harlow harlo

Re: [openstack-dev] [oslo] oslo.concurrency repo review

2014-08-12 Thread Joshua Harlow
tooz if taskflow starts to use tooz by the very nature of taskflow using tooz as a library). Thoughts? -Josh On Tue, Aug 12, 2014 at 6:12 AM, Julien Danjou jul...@danjou.info wrote: On Mon, Aug 11 2014, Joshua Harlow wrote: Sounds great, I've been wondering why https://github.com/stackforge

Re: [openstack-dev] [Ceilometer] Way to get wrapped method's name/class using Pecan secure decorators?

2014-08-12 Thread Joshua Harlow
Do u know if ceilometer is using six.wraps? If so, that helper adds in the `__wrapped__` attribute to decorated methods (which can be used to find the original decorated function). If just plain functools are used (and python3.x isn't used) then it will be pretty hard afaik to find the

Re: [openstack-dev] [nova][core] Expectations of core reviewers

2014-08-13 Thread Joshua Harlow
A big +1 to what daniel said, If f2f events are becoming so important the only way to get things done, IMHO we should really start to do some reflection on how our community operates and start thinking about what we are doing wrong. Expecting every company to send developers (core or

Re: [openstack-dev] [all] The future of the integrated release

2014-08-13 Thread Joshua Harlow
On Wed, Aug 13, 2014 at 5:37 AM, Mark McLoughlin mar...@redhat.com wrote: On Fri, 2014-08-08 at 15:36 -0700, Devananda van der Veen wrote: On Tue, Aug 5, 2014 at 10:02 AM, Monty Taylor mord...@inaugust.com wrote: Yes. Additionally, and I think we've been getting better at this in

Re: [openstack-dev] [all] The future of the integrated release

2014-08-22 Thread Joshua Harlow
Comment inline. On Aug 22, 2014, at 10:13 AM, Michael Chapman wop...@gmail.com wrote: On Fri, Aug 22, 2014 at 9:51 PM, Sean Dague s...@dague.net wrote: On 08/22/2014 01:30 AM, Michael Chapman wrote: On Fri, Aug 22, 2014 at 2:57 AM, Jay Pipes jaypi...@gmail.com

Re: [openstack-dev] What does NASA not using OpenStack mean to OS's future

2014-08-25 Thread Joshua Harlow
So to see if we can get something useful from this thread. What was your internal analysis, can it be published? Even negative analysis is useful to make openstack better... It'd be nice to have some details on what you found, what u didn't find, so that we can all improve... After all that

Re: [openstack-dev] Announcing Gertty 1.0.0: A console interface to Gerrit

2014-09-04 Thread Joshua Harlow
Congrats to making it to 1.0! May there be many more :) Sent from my really tiny device... On Sep 4, 2014, at 4:18 PM, James E. Blair cor...@inaugust.com wrote: Announcing Gertty 1.0.0 Gertty is a console-based interface to the Gerrit Code Review system. If that doesn't sound

Re: [openstack-dev] how to provide tests environments for python things that require C extensions

2014-09-06 Thread Joshua Harlow
There was a review once that tried to move the evzookeeper to using kazoo, perhaps that can get reprioritzed?? https://review.openstack.org/#/c/28951/ Sent from my really tiny device... On Sep 5, 2014, at 6:36 AM, Sean Dague s...@dague.net wrote: While reviewing this zookeeper service

Re: [openstack-dev] Kilo Cycle Goals Exercise

2014-09-08 Thread Joshua Harlow
Amen to this! I've always felt bad that before yahoo tries to include a new feature in its openstack cloud/s we have to figure out how much the feature is a land-mine, how much of it works, how much of it doesn't and so-on. That type of investigation imho shouldn't really be needed and the

[openstack-dev] Standard virtualenv for docs building

2014-09-08 Thread Joshua Harlow
Hi all, I just wanted to get some feedback on a change that I think will make the docs building process more understood, Currently there is a script @ https://github.com/openstack-infra/config/blob/master/modules/openstack_project/files/slave_scripts/run-docs.sh (this is the github mirror

[openstack-dev] [TaskFlow][Oslo] TaskFlow 0.4.0 Released!

2014-09-08 Thread Joshua Harlow
Howdy all, Just wanted to send a mini-announcement about the new taskflow 0.4.0 release on behalf of the oslo and taskflow teams, The details about what is new and what is changed and all that goodness can be found @ https://etherpad.openstack.org/p/TaskFlow-0.4 Updated developer docs can be

Re: [openstack-dev] [Nova] What's holding nova development back?

2014-09-16 Thread Joshua Harlow
And if u can also prove NP = P u get 1 million dollars[1, 2] Let me know when u got the proof, Thanks much, [1] http://www.claymath.org/millenium-problems/p-vs-np-problem [2] http://www.claymath.org/millennium-problems/millennium-prize-problems -Josh On Mon, Sep 15, 2014 at 5:07 PM, Jeremy

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: retrying)

2014-09-17 Thread Joshua Harlow
On a related and slightly less problematic case is another one like this... https://github.com/rholder/retrying/issues/11 On Sep 17, 2014, at 8:15 AM, Thomas Goirand z...@debian.org wrote: Hi, I'm horrified by what I just found. I have just found out this in glanceclient: File

Re: [openstack-dev] PostgreSQL jobs slow in the gate

2014-09-17 Thread Joshua Harlow
Same seen @ http://logs.openstack.org/80/121280/32/check/check-tempest-dsvm-postgres-full/395a05d/logs/postgres.txt.gz Although I'm not sure if those are abnormal or not (seems likely they wouldn't be). Was there a postgres release? On Sep 17, 2014, at 5:10 PM, Joe Gordon

Re: [openstack-dev] [neutron] [infra] Python 2.6 tests can't possibly be passing in neutron

2014-09-22 Thread Joshua Harlow
Just as an update to what exactly is RHEL python 2.6... This is the expanded source rpm: http://paste.ubuntu.com/8405074/ The main one here appears to be: - python-2.6.6-ordereddict-backport.patch Full changelog @ http://paste.ubuntu.com/8405082/ Overall I'd personally like to get rid of

Re: [openstack-dev] [all] In defence of faking

2014-09-22 Thread Joshua Harlow
I would have to agree, fakes certainly have there benefits. IMHO, sometimes when you are testing a complex set of interactions you don't want to have to mock the full set of interactions when you just want to determine if the end result worked or did not work. For example, lets say I have a

Re: [openstack-dev] [Heat] Convergence: Backing up template instead of stack

2014-09-23 Thread Joshua Harlow
I believe heat has its own dependency graph implementation but if that was switched to networkx[1] that library has a bunch of nice read/write capabilities. See: https://github.com/networkx/networkx/tree/master/networkx/readwrite And one made for sqlalchemy @

Re: [openstack-dev] [all][tripleo] New Project - Kolla: Deploy and Manage OpenStack using Kubernetes and Docker

2014-09-24 Thread Joshua Harlow
Sounds like an interesting project/goal and will be interesting to see where this goes. A few questions/comments: How much golang will people be exposed to with this addition? Seeing that this could be the first 'go' using project it will be interesting to see where this goes (since afaik

Re: [openstack-dev] FW: [Bug 1257788] Re: nova-network doesn't handle large amount of concurrent booting instances well

2013-12-10 Thread Joshua Harlow
Yup, IMHO this is where the rally[1] project, and others that attempt to push the boundaries are really helping openstack become much better around these types of issues. At yahoo! in a short-time it will not be that uncommon to boot 100+ VMs concurrently (traffic spikes, other random QA testing

[openstack-dev] [State-Management] Agenda for tomorrow meeting at 2000 UTC

2013-12-11 Thread Joshua Harlow
topics are welcome :-) See you all soon! -- Joshua Harlow It's openstack, relax... | harlo...@yahoo-inc.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] [Neutron] How do we know a host is ready to have servers scheduled onto it?

2013-12-12 Thread Joshua Harlow
Maybe time to revive something like: https://review.openstack.org/#/c/12759/ From experience, all sites (and those internal to yahoo) provide a /status (or equivalent) that is used for all sorts of things (from basic load-balancing up/down) to other things like actually introspecting the state

Re: [openstack-dev] Re-initializing or dynamically configuring cinder driver

2013-12-15 Thread Joshua Harlow
this is the hard way of achieving the objective and SIGHUP approach will handle it more gracefully. As you mentioned it is a major change, tentatively can we use SIGTERM to achieve the objective? On Mon, Dec 16, 2013 at 9:50 AM, Joshua Harlow harlo...@yahoo-inc.commailto:harlo...@yahoo-inc.com wrote

Re: [openstack-dev] Re-initializing or dynamically configuring cinder driver

2013-12-16 Thread Joshua Harlow
to child processes and then starting them? I know this is the hard way of achieving the objective and SIGHUP approach will handle it more gracefully. As you mentioned it is a major change, tentatively can we use SIGTERM to achieve the objective? On Mon, Dec 16, 2013 at 9:50 AM, Joshua Harlow

[openstack-dev] [State-Management] Agenda for today meeting at 2000 UTC

2013-12-19 Thread Joshua Harlow
/TaskFlow ## Agenda (30-60 mins): - Discuss any action items from last meeting. - Discuss about any other potential new use-cases for said library. - Discuss about any other ideas, questions and answers (and more!). Any other topics are welcome :-) See you all soon! -- Joshua Harlow

[openstack-dev] [State-Management] No meeting this week

2013-12-23 Thread Joshua Harlow
Since its xmas in most of the world lets skip the IRC meeting this week (normally on thursdays). See you all soon and have a great vacation! P.S. #openstack-state-management if u feel the need to chat :-) -- Joshua Harlow It's openstack, relax... | harlo...@yahoo-inc.com

Re: [openstack-dev] [trove][mistral] scheduled tasks

2013-12-30 Thread Joshua Harlow
Any reason to not use taskflow (https://wiki.openstack.org/wiki/TaskFlow) to help u here?? I think it could be easily adapted to do what u want, and would save u from having to recreate the same task execution logic that everyone seems to rebuild… -Josh From: Greg Hill

Re: [openstack-dev] [trove][mistral] scheduled tasks

2013-12-31 Thread Joshua Harlow
Agreed taskflow doesn't currently provide scheduling as it was thought that reliable execution that can be restarted and resumed is the foundation that someone using taskflow can easily provide scheduling ontop of... Better IMHO to have a project doing this foundation well (since openstack

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Joshua Harlow
So taskflow was tested with the version of sqlalchemy that was available and in the requirements at the time of its 0.1 release (taskflow syncs it's requirements from the same global requirements). From what I remember this is the same requirement that everyone else is bound to:

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Joshua Harlow
even more of these types of new 3rd party libraries to appear on pypi (and therefore causing similar issues of transitive dependencies as taskflow). Will bug u on #openstack-infra soon :-) On 1/3/14, 9:05 AM, Sean Dague s...@dague.net wrote: On 01/03/2014 11:37 AM, Joshua Harlow wrote: So

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Joshua Harlow
Sounds good to me. Talked on #openstack-infra with some folks there and just awaiting next steps. Doesn't seem like should be anything to hard to adjust/move/... -Josh On 1/3/14, 11:27 AM, Sean Dague s...@dague.net wrote: On 01/03/2014 12:45 PM, Joshua Harlow wrote: Ok, I think I'm fine

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Joshua Harlow
really need to I can push out a 0.1.2 with the unpinned version (one of the above reviews). -Josh From: Doug Hellmann doug.hellm...@dreamhost.commailto:doug.hellm...@dreamhost.com Date: Friday, January 3, 2014 at 11:44 AM To: Joshua Harlow harlo...@yahoo-inc.commailto:harlo...@yahoo-inc.com Cc

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Joshua Harlow
Since the library model is what most everyone else uses outside of openstack (I assume?) what can we do to get there so that this model works as well? Expanding dependencies recursively seems like it could help? This could then detect transitive dependency issues (and doesn't seem so hard to do).

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-03 Thread Joshua Harlow
, but sometimes u just work with what u got :-P On 1/3/14, 3:32 PM, Sean Dague s...@dague.net wrote: On 01/03/2014 06:14 PM, Joshua Harlow wrote: Since the library model is what most everyone else uses outside of openstack (I assume?) what can we do to get there so that this model works as well

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-04 Thread Joshua Harlow
Such a bad state seems like FUD. Taskflow was just syncing its requirements with the same requirements that everyone else is... Those global requirements have 0.7.99 in them as we speak (which is why taskflow picked up that version). The issue here will be worked through and fixed, it won't

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-04 Thread Joshua Harlow
I was more of referring to general dependency issues, sqlalchemy hopefully never again but one never knows :P Sent from my really tiny device... On Jan 4, 2014, at 8:40 AM, Thomas Goirand z...@debian.org wrote: On 01/05/2014 12:12 AM, Joshua Harlow wrote: it won't be the last time

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-05 Thread Joshua Harlow
With regards to the futures module it should just work fine with the packaging of https://pypi.python.org/pypi/futures which is a backport of the 3.2 concurrent futures package into 2.6,2.7, so that's the package there. Feel free to bug me on irc if u want any other help with dependencies, the

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-05 Thread Joshua Harlow
Agreed, we are going to expand it and work on figuring out how to test against multiple versions. It does work with 0.8 and it seems even like 0.9 works fine also. But all compatible also means I can't guarantee 0.10 (if it comes out) will work since afaik semver means sqlalchemy could still

Re: [openstack-dev] [Gantt] Looking for some answers...

2014-01-06 Thread Joshua Harlow
Was the history filtered out using something like http://git-scm.com/docs/git-filter-branch?? There seems to be a lot of commit history that isn't related to gantt files (baremetal…) Was the plan to figure out which files to keep, then cleanup that commit history? I wouldn't expect

Re: [openstack-dev] [requirements] - taskflow preventing sqla 0.8 upgrade

2014-01-06 Thread Joshua Harlow
...@openstack.orgmailto:jebl...@openstack.org wrote: Joshua Harlow harlo...@yahoo-inc.commailto:harlo...@yahoo-inc.com writes: It seems simple to have variations of venvs (or something similar) that taskflow tox.ini can have that specify the different 0.7, 0.8, 0.9, when sqlalchemy 1.0 comes out then this should

[openstack-dev] [State-Management] Agenda for tommorow meeting at 2000 UTC

2014-01-08 Thread Joshua Harlow
are welcome :-) See you all soon! -- Joshua Harlow It's openstack, relax... | harlo...@yahoo-inc.commailto:harlo...@yahoo-inc.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo

Re: [openstack-dev] [all] Organizing a Gate Blocking Bug Fix Day

2014-01-09 Thread Joshua Harlow
And my ring, my precious. Count me in! On 1/9/14, 6:06 AM, Julien Danjou jul...@danjou.info wrote: On Thu, Jan 09 2014, Sean Dague wrote: I'm hopefully that if we can get everyone looking at this one a single day, we can start to dislodge the log jam that exists. I will help you bear

Re: [openstack-dev] [oslo] dependency analysis for graduating more oslo libraries

2014-01-15 Thread Joshua Harlow
Very nice write up +1 A question, many modules are pulling in oslo.log as seen from the dependency graph which itself then pulls in oslo.config. Is the plan to just have all these modules use the regular python logging and have oslo.log be a plugin/formatter/adapter to python logging? Likely

[openstack-dev] [State-Management] Agenda for tommorow meeting at 2000 UTC

2014-01-15 Thread Joshua Harlow
, feedback...) - Any other ongoing taskflow work (questions, comments, feedback...) - Discuss about any other potential new use-cases for said library. - Discuss about any other ideas, questions and answers (and more!). Any other topics are welcome :-) See you all soon! -- Joshua Harlow

Re: [openstack-dev] sqla 0.8 ... and sqla 0.9

2014-01-16 Thread Joshua Harlow
Also with: https://review.openstack.org/#/c/66051/ On 1/16/14, 10:40 AM, Jeremy Stanley fu...@yuggoth.org wrote: On 2014-01-12 07:27:11 -0500 (-0500), Sean Dague wrote: With the taskflow update, the only thing between upping our sqla requirement to 0.8.99 is pbr's requirements integration

[openstack-dev] [State-Management] Proposal to add Changbin Liu to taskflow-core

2014-01-16 Thread Joshua Harlow
Greetings all stackers, I propose that we add Changbin Liu[1] to the taskflow-core team[2]. Changbin has been actively contributing to taskflow for a while now, both in helping develop code and helping with the review load. He has provided quality reviews and is doing an awesome job with the

[openstack-dev] [State-Management] Agenda for tommorow meeting at 2000 UTC

2014-01-22 Thread Joshua Harlow
you all soon! -- Joshua Harlow It's openstack, relax... | harlo...@yahoo-inc.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo] memoizer aka cache

2014-01-23 Thread Joshua Harlow
So to me memoizing is typically a premature optimization in a lot of cases. And doing it incorrectly leads to overfilling the python processes memory (your global dict will have objects in it that can't be garbage collected, and with enough keys+values being stored will act just like a memory

Re: [openstack-dev] [oslo] memoizer aka cache

2014-01-23 Thread Joshua Harlow
at any type of scale). -Josh On 1/23/14, 1:35 PM, Renat Akhmerov rakhme...@mirantis.com wrote: On 23 Jan 2014, at 08:41, Joshua Harlow harlo...@yahoo-inc.com wrote: So to me memoizing is typically a premature optimization in a lot of cases. And doing it incorrectly leads to overfilling

Re: [openstack-dev] [oslo] memoizer aka cache

2014-01-23 Thread Joshua Harlow
for all to use. --Morgan On Thu, Jan 23, 2014 at 2:54 PM, Joshua Harlow harlo...@yahoo-inc.commailto:harlo...@yahoo-inc.com wrote: Sure, no cancelling cases of conscious usage, but we need to be careful here and make sure its really appropriate. Caching and invalidation techniques are right up

Re: [openstack-dev] [Ironic] File Injection (and the lack thereof)

2014-01-24 Thread Joshua Harlow
Cloud-init 0.7.5 (not yet released) will have the ability to read from an ec2-metadata server using SSL. In a recent change I did we now use requests which correctly does SSL for the ec2-metadata/ec2-userdata reading. - http://bazaar.launchpad.net/~cloud-init-dev/cloud-init/trunk/revision/910

Re: [openstack-dev] [Ironic] File Injection (and the lack thereof)

2014-01-24 Thread Joshua Harlow
Also just to note; file-injection seems unneeded when cloud-init can use this: http://cloudinit.readthedocs.org/en/latest/topics/examples.html#writing-out -arbitrary-files That I believe is in most modern versions of cloud-init (forgot when I implemented that). Just FYI :) -Josh On 1/24/14,

Re: [openstack-dev] Fwd: [TripleO] mid-cycle meetup?

2014-01-24 Thread Joshua Harlow
Yahoo! is right in the middle of sunnyvale; and probably has enough free space to handle all u folks (some rooms are quite big really). I can talk with some folks here about hosting all of this here if that’s desired (depending on how big). -Josh From: Roman Alekseenkov

Re: [openstack-dev] Fwd: [TripleO] mid-cycle meetup?

2014-01-25 Thread Joshua Harlow
good to know we have options if we overflow available meeting rooms or suchlike :) Cheers, -- Chris Jones On 25 Jan 2014, at 03:22, Joshua Harlow harlo...@yahoo-inc.commailto:harlo...@yahoo-inc.com wrote: Yahoo! is right in the middle of sunnyvale; and probably has enough free space to handle

Re: [openstack-dev] [Ironic] Node groups and multi-node operations

2014-01-26 Thread Joshua Harlow
Doesn't nova already have logic for creating N virtual machines (similar to a group) in the same request? I thought it did (maybe it doesn't anymore in the v3 API), creating N bare metal machines seems like it would comply to that api? Sent from my really tiny device... On Jan 22, 2014, at

Re: [openstack-dev] [Ironic] Node groups and multi-node operations

2014-01-26 Thread Joshua Harlow
, Robert Collins robe...@robertcollins.net wrote: On 27 January 2014 08:04, Joshua Harlow harlo...@yahoo-inc.com wrote: Doesn't nova already have logic for creating N virtual machines (similar to a group) in the same request? I thought it did (maybe it doesn't anymore in the v3 API

[openstack-dev] Mistral + taskflow mini-meetup

2014-01-27 Thread Joshua Harlow
Hi all, In order to encourage further discussion off IRC and more in public I'd like to share a etherpad that was worked on during a 'meetup' with some of the mistral folks and me. https://etherpad.openstack.org/p/taskflow-mistral-jan-meetup It was more of a (mini) in-person meetup but I

Re: [openstack-dev] [oslo] log message translations

2014-01-27 Thread Joshua Harlow
+1 I've never understood this either personally. From what I know most all (correct me if I am wrong) open source projects don't translate log messages; so it seems odd to be the special snowflake project/s. Do people find this type of translation useful? It'd be nice to know how many people

Re: [openstack-dev] Mistral + taskflow mini-meetup

2014-01-27 Thread Joshua Harlow
PoC is not using TaskFlow but moving forward we we’d like to try to marry these two technologies to be more aligned in terms of APIs and feature sets. Renat Akhmerov @ Mirantis Inc. On 27 Jan 2014, at 13:21, Joshua Harlow harlo...@yahoo-inc.commailto:harlo...@yahoo-inc.com wrote: Hi all

[openstack-dev] Hacking repair scripts

2014-01-27 Thread Joshua Harlow
Hi all, I have had in my ~/bin for a while a little script that I finally got around to tuning up and I thought others might be interested in it/find it useful. The concept is similar to https://pypi.python.org/pypi/autopep8 but does a really simple action to start. As many of u know the

Re: [openstack-dev] [Nova][Neutron] nova-network in Icehouse and beyond

2014-01-29 Thread Joshua Harlow
I think depending on use-case it can be accomplished. The steps we have been thinking at Y! 1. Take offline APIs nova-compute (so new/existing VMs can't be scheduled/modified) -- existing running VMs will not be affected. 2. Save/dump nova database. 3. Translate nova database entries into

[openstack-dev] [State-Management] Agenda for tommorow meeting at 2000 UTC

2014-01-29 Thread Joshua Harlow
library. - Discuss about any other ideas, questions and answers (and more!). Any other topics are welcome :-) See you all soon! -- Joshua Harlow It's openstack, relax... | harlo...@yahoo-inc.com ___ OpenStack-dev mailing list OpenStack-dev

[openstack-dev] Cinder + taskflow

2014-02-03 Thread Joshua Harlow
Hi all, After talking with john g. about taskflow in cinder and seeing more and more reviews showing up I wanted to start a thread to gather all our lessons learned and how we can improve a little before continuing to add too many more refactoring and more reviews (making sure everyone is

Re: [openstack-dev] Cinder + taskflow

2014-02-03 Thread Joshua Harlow
Development Mailing List (not for usage questions) openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] Cinder + taskflow On Mon, Feb 3, 2014 at 1:53 PM, Joshua Harlow harlo...@yahoo-inc.com wrote: Hi all, After talking with john g. about taskflow in cinder and seeing more and more

Re: [openstack-dev] Asynchrounous programming: replace eventlet with asyncio

2014-02-04 Thread Joshua Harlow
A follow up kind of question, Why not just create a eventlet micro version that underneath uses asyncio? Then code isn't riddled with yield from or Return() or similar. It seems possible (maybe?) to replace the eventlet hub with one that uses asyncio? This is then a nice change for all those

Re: [openstack-dev] Asynchrounous programming: replace eventlet with asyncio

2014-02-04 Thread Joshua Harlow
I was just thinking that openstack would be better to slowly get off eventlet (avoids the large amount of changes this new mostly untested/unproven tulip/trollius framework introduces). A first step could be to replace the greenlet layer with asyncio (or similar). Then the eventlet layer still

Re: [openstack-dev] Asynchrounous programming: replace eventlet with asyncio

2014-02-04 Thread Joshua Harlow
of the same feature, one for eventlet and another for asyncio. For example, the Oslo Messaging project has an abstraction of the asynchronous framework called executor. So you can use a blocking executor, eventlet, trollius or something else. Today, a patch was proposed by Joshua Harlow

[openstack-dev] [State-Management] Agenda for meeting (tommorow) at 2000 UTC

2014-02-05 Thread Joshua Harlow
library. - Discuss about any other ideas, reviews needing help, questions and answers (and more!). Any other topics are welcome :-) See you all soon! -- Joshua Harlow It's openstack, relax... | harlo...@yahoo-inc.com ___ OpenStack-dev mailing list

Re: [openstack-dev] Asynchrounous programming: replace eventlet with asyncio

2014-02-05 Thread Joshua Harlow
Any mysql DB drivers (I think the majority of openstack deployments use mysql?). How about sqlalchemy (what would possibly need to change there for it to work)? The pain that I see is that to connect all these libraries into asyncio they have to invert how they work (sqlalchemy would have to

Re: [openstack-dev] Asynchrounous programming: replace eventlet with asyncio

2014-02-06 Thread Joshua Harlow
Has there been any investigation into heat. Heat has already used parts of the coroutine approach (for better or worse). An example: https://github.com/openstack/heat/blob/master/heat/engine/scheduler.py#L230 Decorator for a task that needs to drive a subtask. This is essentially a

Re: [openstack-dev] Asynchrounous programming: replace eventlet with asyncio

2014-02-06 Thread Joshua Harlow
Its a good question, I see openstack as mostly like the following 2 groups of applications. Group 1: API entrypoints using [apache/nginx]+wsgi (nova-api, glance-api…) In this group we can just let the underlying framework/app deal with the scaling and just use native wsgi as it was intended.

Re: [openstack-dev] [nova][glance] bp: glance-snapshot-tasks

2014-02-06 Thread Joshua Harlow
Hi alex, I think u are referring to the following: https://blueprints.launchpad.net/nova/+spec/glance-snapshot-tasks Can u describe the #2 part in more detail. Do some of the drivers already implement these new steps? The goal I think u are having is to make the snapshot functionality resume

Re: [openstack-dev] Asynchrounous programming: replace eventlet with asyncio

2014-02-06 Thread Joshua Harlow
+1 lots of respect for zane in doing this :) I'm still very much interested in seeing how we can connect taskflow in to your model. I think the features that you guys were wanting (remote workers) are showing up and hopefully will be all they can be! It helps (imho) that taskflow doesn't

[openstack-dev] Taskflow 0.1.3 released

2014-02-06 Thread Joshua Harlow
Hi all, Taskflow 0.1.3 we released today (0.2.0 to arrive soon). It was a minor patch release to help fix the unicode/str() issue we discovered in cinder. It also includes a few other changes around testing, comments, utility functions and zookeeper backends. Full git changes: $ git log

Re: [openstack-dev] Asynchrounous programming: replace eventlet with asyncio

2014-02-06 Thread Joshua Harlow
+1 To give an example as to why eventlet implicit monkey patch the world isn't especially great (although it's what we are currently using throughout openstack). The way I think about how it works is to think about what libraries that a single piece of code calls and how it is very hard to

Re: [openstack-dev] several oslo library repositories have moved

2014-02-07 Thread Joshua Harlow
+1 thanks guys! From: Doug Hellmann doug.hellm...@dreamhost.commailto:doug.hellm...@dreamhost.com Reply-To: OpenStack Development Mailing List (not for usage questions) openstack-dev@lists.openstack.orgmailto:openstack-dev@lists.openstack.org Date: Friday, February 7, 2014 at 1:02 PM To:

Re: [openstack-dev] [Murano] Can we migrate to oslo.messaging?

2014-02-11 Thread Joshua Harlow
Is murano python3.x compatible, from what I understand oslo.messaging isn't (yet). If murano is supporting python3.x then brining in oslo.messaging might make it hard for murano to be 3.x compatible. Maybe not a problem (I'm not sure of muranos python version support). From: Serg Melikyan

Re: [openstack-dev] [All]Optional dependencies and requirements.txt

2014-02-12 Thread Joshua Harlow
In taskflow we've done something like the following. Its not perfect, but it is what currently works (willing to take suggestions on better). We have three different requirements files. 1. Required to work in any manner @ https://github.com/openstack/taskflow/blob/master/requirements.txt 2.

Re: [openstack-dev] [All]Optional dependencies and requirements.txt

2014-02-12 Thread Joshua Harlow
seems to do what you want: SQLAlchemy=0.7.99,=0.9.99 [SQLAlchemyPersistence] alembic=0.4.1 [SQLAlchemyPersistence] ... [1] http://www.pip-installer.org/en/1.1/requirements.html -- Koo On Thu, 13 Feb 2014 02:37:44 + Joshua Harlow harlo...@yahoo-inc.com wrote: In taskflow we've

[openstack-dev] [State-Management] Agenda for meeting (today) at 2000 UTC

2014-02-13 Thread Joshua Harlow
!! See you all soon! -- Joshua Harlow It's openstack, relax... | harlo...@yahoo-inc.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Gamification and on-boarding ...

2014-02-14 Thread Joshua Harlow
+1 Mentoring and devoted mentors and not demotivating new folks (but instead growing and fostering them) is IMHO 10x more important than a badge program. Badges seem nice and all but I think it's not the biggest win for the buck. Sent from my really tiny device... On Feb 13, 2014, at 6:06

Re: [openstack-dev] [Murano] Need a new DSL for Murano

2014-02-14 Thread Joshua Harlow
An honest question, U are mentioning what appears to be the basis for a full programming language (variables, calling other workflows - similar to functions) but then u mention this is being stuffed into yaml. Why? It appears like u might as well spend the effort and define a grammar and

Re: [openstack-dev] Storing license information in openstack/requirements

2014-02-17 Thread Joshua Harlow
+1 for yaml instead of shoving all kinds of package metadata in comments. Sent from my really tiny device... On Feb 17, 2014, at 4:12 PM, Sean Dague s...@dague.net wrote: Honestly, if we are going to track this, we should probably do the set of things that reviewers tend to do when running

Re: [openstack-dev] [Nova][glance] Question about evacuate with no shared storage..

2014-02-21 Thread Joshua Harlow
That requires ssh (or some tunnel/other RPC?) connections from-to all hypervisors to work correct?? Is that allowed in your organization (headless ssh keys from-to all hypervisors)? Isn't that a huge security problem if someone manages to break out of a VM and get access to those keys? If I was

Re: [openstack-dev] [Mistral] Porting executor and engine to oslo.messaging

2014-02-26 Thread Joshua Harlow
So this design is starting to look pretty familiar to a what we have in taskflow. Any reason why it can't just be used instead? https://etherpad.openstack.org/p/TaskFlowWorkerBasedEngine This code is in a functional state right now, using kombu (for the moment, until oslo.messaging becomes

[openstack-dev] [State-Management] Agenda for meeting (tommorow) at 2000 UTC

2014-02-26 Thread Joshua Harlow
/desirable/useful for 0.3. - Discuss about any other potential new use-cases for said library. - Discuss about any other ideas, reviews needing help, questions and answers (and more!). Any other topics are welcome :-) See you all soon! -- Joshua Harlow It's openstack, relax... | harlo...@yahoo

Re: [openstack-dev] [nova] Future of the Nova API

2014-02-27 Thread Joshua Harlow
+1 I'd rather have good and supported clients that satisfy peoples needs rather than nit-pick or remove v3 or something else... Forward progress is good so I'll throw my hat for v3, but I doubt many people that use the API's care (as long as their friendly client library still works). Make it

Re: [openstack-dev] [Mistral] Porting executor and engine to oslo.messaging

2014-02-28 Thread Joshua Harlow
for your feedback and this info. We’ll get in touch with you soon. Renat Akhmerov @ Mirantis Inc. On 27 Feb 2014, at 03:22, Joshua Harlow harlo...@yahoo-inc.commailto:harlo...@yahoo-inc.com wrote: So this design is starting to look pretty familiar to a what we have in taskflow. Any reason why

Re: [openstack-dev] [Mistral] Porting executor and engine to oslo.messaging

2014-02-28 Thread Joshua Harlow
consumers like Mistral? 3) Does TaskFlow have its own DSL? The examples provided are mostly code based. Thanks. Winson On Fri, Feb 28, 2014 at 10:54 AM, Joshua Harlow harlo...@yahoo-inc.commailto:harlo...@yahoo-inc.com wrote: Sounds good, Lets connect, the value of central oslo connected

[openstack-dev] [State-Management] Agenda for meeting (tommorow) at 2000 UTC

2014-03-05 Thread Joshua Harlow
discovery. - Discuss about any other potential new use-cases for said library. - Discuss about any other ideas, reviews needing help, questions and answers (and more!). Any other topics are welcome :-) See you all soon! -- Joshua Harlow It's openstack, relax... | harlo...@yahoo-inc.com

Re: [openstack-dev] [Mistral] Crack at a Real life workflow

2014-03-06 Thread Joshua Harlow
That sounds a little similar to what taskflow is trying to do (I am of course biased). I agree with letting the native language implement the basics (expressions, assignment...) and then building the domain ontop of that. Just seems more natural IMHO, and is similar to what linq (in c#) has

Re: [openstack-dev] [nova][vmware] Locking: A can of worms

2014-03-07 Thread Joshua Harlow
Tooz folks has been thinking about this problem (as well as myself) for a little while. I've started something like: https://review.openstack.org/#/c/71167/ Also: https://wiki.openstack.org/wiki/StructuredWorkflowLocks Perhaps we can get more movement on that (sorry I haven't had tons of time

Re: [openstack-dev] [Mistral] Crack at a Real life workflow

2014-03-07 Thread Joshua Harlow
engines. -S Renat Akhmerov @ Mirantis Inc. On 06 Mar 2014, at 22:26, Joshua Harlow harlo...@yahoo-inc.commailto:harlo...@yahoo-inc.com wrote: That sounds a little similar to what taskflow is trying to do (I am of course biased). I agree with letting the native language implement the basics

Re: [openstack-dev] [Mistral] Crack at a Real life workflow

2014-03-07 Thread Joshua Harlow
. On 06 Mar 2014, at 22:26, Joshua Harlow harlo...@yahoo-inc.commailto:harlo...@yahoo-inc.com wrote: That sounds a little similar to what taskflow is trying to do (I am of course biased). I agree with letting the native language implement the basics (expressions, assignment...) and then building

  1   2   3   4   5   6   7   8   9   10   >